Problem with webservice that runs threaded [modified]
-
Sorry if my question is a bit out of the normal and my approach is less briliant. Im under education and currently on internship at a company being the only programmer on a website at the moment. To make things even better its also my first experience with ASP.NET untill now ive only made C# applications. I have a problem with a webservice that runs threaded. The webservice it self is rather simple. All code used to start the webservice thread is shown here
public virtual OverwriteCollectionResponse OverwriteCollection([System.Xml.Serialization.XmlElementAttribute(Namespace="http://XXXX/MAP\_INHOUSE\_OUT/COLLECTION:MESSAGES:V1", ElementName="OverwriteCollectionRequest")] OverwriteCollectionRequest overwriteCollectionRequest){
//Creation of Class that should run
WebserviceThread webservice = new WebserviceThread();//Saving data to the class
webservice.overwriteCollectionRequest = overwriteCollectionRequest;//Create a thread based on the class
Thread thread = new Thread(new ThreadStart(webservice.OverwriteCollection));//Start the thread
thread.Start();
return new OverwriteCollectionResponse(1);The real method takes up to two hours to run depending on amount of data. I have read about asyncrone webservices after making this but im unsure if thats the right choice to solve my problem. The webservice is running on a IIS6 server configured to have session timeout to 300 min. The configuration for timeout in the config file is the following The problem is the following: After a random amount of time (can be all from 15 min to 25 min havent seen it below or above that) the Thread stops. Only prof that it stops is that the log file isnt updated anymore. (Using NLog as logger). Also the data in the database isnt updated more. Anyone that have an idea to why the thread suddenly stops? Hope my question is clear else dont hessitate to ask for more info :) Thanks for any help in advance HE
modified on Thursday, May 20, 2010 9:32 AM
-
Sorry if my question is a bit out of the normal and my approach is less briliant. Im under education and currently on internship at a company being the only programmer on a website at the moment. To make things even better its also my first experience with ASP.NET untill now ive only made C# applications. I have a problem with a webservice that runs threaded. The webservice it self is rather simple. All code used to start the webservice thread is shown here
public virtual OverwriteCollectionResponse OverwriteCollection([System.Xml.Serialization.XmlElementAttribute(Namespace="http://XXXX/MAP\_INHOUSE\_OUT/COLLECTION:MESSAGES:V1", ElementName="OverwriteCollectionRequest")] OverwriteCollectionRequest overwriteCollectionRequest){
//Creation of Class that should run
WebserviceThread webservice = new WebserviceThread();//Saving data to the class
webservice.overwriteCollectionRequest = overwriteCollectionRequest;//Create a thread based on the class
Thread thread = new Thread(new ThreadStart(webservice.OverwriteCollection));//Start the thread
thread.Start();
return new OverwriteCollectionResponse(1);The real method takes up to two hours to run depending on amount of data. I have read about asyncrone webservices after making this but im unsure if thats the right choice to solve my problem. The webservice is running on a IIS6 server configured to have session timeout to 300 min. The configuration for timeout in the config file is the following The problem is the following: After a random amount of time (can be all from 15 min to 25 min havent seen it below or above that) the Thread stops. Only prof that it stops is that the log file isnt updated anymore. (Using NLog as logger). Also the data in the database isnt updated more. Anyone that have an idea to why the thread suddenly stops? Hope my question is clear else dont hessitate to ask for more info :) Thanks for any help in advance HE
modified on Thursday, May 20, 2010 9:32 AM
It sounds like the request is being killed for taking too long. ASP.Net will kill long running requests after a set amount of time ~20 minutes I think. But if your process takes that long maybe a webservice isn't the place to do the work. Is there any scope for you to do the work in say a windows service? Otherwise, I think we'll need a bit more information about what you're trying to achieve so we can suggest a better way - what you're doing now just looks wrong.
-
It sounds like the request is being killed for taking too long. ASP.Net will kill long running requests after a set amount of time ~20 minutes I think. But if your process takes that long maybe a webservice isn't the place to do the work. Is there any scope for you to do the work in say a windows service? Otherwise, I think we'll need a bit more information about what you're trying to achieve so we can suggest a better way - what you're doing now just looks wrong.
Any way I can make it allow to run longer? A windows service might do it but I have no idea how to make one that will fulfill the demands. I will try to describe what im trying to do as good as I can. There is an excisting SAP system with information about products. Im working on a system that recives these informations from SAP with a webservice and stores them in a database to then show them on a webpage and let the user create catalogues based on the informations. In the long run the idea is that every time there is made a change in SAP its send to the webservice (around 100kb of data). Now there is made some changeings to the system and we need to send all data from SAP to the webservice from scratch. Therefor its needed to send alot of data to the webservice. If there is a way to change the alowed running time it might help. Best Regards HE
-
Any way I can make it allow to run longer? A windows service might do it but I have no idea how to make one that will fulfill the demands. I will try to describe what im trying to do as good as I can. There is an excisting SAP system with information about products. Im working on a system that recives these informations from SAP with a webservice and stores them in a database to then show them on a webpage and let the user create catalogues based on the informations. In the long run the idea is that every time there is made a change in SAP its send to the webservice (around 100kb of data). Now there is made some changeings to the system and we need to send all data from SAP to the webservice from scratch. Therefor its needed to send alot of data to the webservice. If there is a way to change the alowed running time it might help. Best Regards HE
Yes there is, but you'll need to investigate the IIS configuration and ASP.Net settings yourself as I don't have time. My experience with this type of thing is extending the request lifetime isn't the best way and is problematic and difficult to debug. As you know, the process just dies with no notification of which timeout barrier caused the problem. I strongly suggest you look at another way to solve this, maybe many smaller requests? If this mechanism needs to reliable you may also want to look at some sort of queuing mechanic (MSMQ). WCF offers nice way's to handle both and it's well worth educating yourself in the technology.
-
Sorry if my question is a bit out of the normal and my approach is less briliant. Im under education and currently on internship at a company being the only programmer on a website at the moment. To make things even better its also my first experience with ASP.NET untill now ive only made C# applications. I have a problem with a webservice that runs threaded. The webservice it self is rather simple. All code used to start the webservice thread is shown here
public virtual OverwriteCollectionResponse OverwriteCollection([System.Xml.Serialization.XmlElementAttribute(Namespace="http://XXXX/MAP\_INHOUSE\_OUT/COLLECTION:MESSAGES:V1", ElementName="OverwriteCollectionRequest")] OverwriteCollectionRequest overwriteCollectionRequest){
//Creation of Class that should run
WebserviceThread webservice = new WebserviceThread();//Saving data to the class
webservice.overwriteCollectionRequest = overwriteCollectionRequest;//Create a thread based on the class
Thread thread = new Thread(new ThreadStart(webservice.OverwriteCollection));//Start the thread
thread.Start();
return new OverwriteCollectionResponse(1);The real method takes up to two hours to run depending on amount of data. I have read about asyncrone webservices after making this but im unsure if thats the right choice to solve my problem. The webservice is running on a IIS6 server configured to have session timeout to 300 min. The configuration for timeout in the config file is the following The problem is the following: After a random amount of time (can be all from 15 min to 25 min havent seen it below or above that) the Thread stops. Only prof that it stops is that the log file isnt updated anymore. (Using NLog as logger). Also the data in the database isnt updated more. Anyone that have an idea to why the thread suddenly stops? Hope my question is clear else dont hessitate to ask for more info :) Thanks for any help in advance HE
modified on Thursday, May 20, 2010 9:32 AM
If we have any calls longer than about a second we tend to hand them off to another process in SQL. The Webservice submits the request to SQL which stores the request, we then use SQL Service Broker to trigger processing of that request, which is then on another SQL thread and independent of the webservice which has long since returned. For really long SQL jobs (over 20 seconds) we have a scheduled job pick up the stored request for processing. If your webservice call takes longer than a few seconds you are using the wrong technology. Having said that, we once had a task that processed over 16 million financial transactions and ran for several weeks. Migrating to SQL (it was FoxPro) brought it down to hours, and fixing the crap SQL brought it down to under a minute. So the problem may be in the processing not the technology.
If you have knowledge, let others light their candles at it. Margaret Fuller (1810 - 1850) [My Articles] [My Website]
-
Yes there is, but you'll need to investigate the IIS configuration and ASP.Net settings yourself as I don't have time. My experience with this type of thing is extending the request lifetime isn't the best way and is problematic and difficult to debug. As you know, the process just dies with no notification of which timeout barrier caused the problem. I strongly suggest you look at another way to solve this, maybe many smaller requests? If this mechanism needs to reliable you may also want to look at some sort of queuing mechanic (MSMQ). WCF offers nice way's to handle both and it's well worth educating yourself in the technology.
-
If we have any calls longer than about a second we tend to hand them off to another process in SQL. The Webservice submits the request to SQL which stores the request, we then use SQL Service Broker to trigger processing of that request, which is then on another SQL thread and independent of the webservice which has long since returned. For really long SQL jobs (over 20 seconds) we have a scheduled job pick up the stored request for processing. If your webservice call takes longer than a few seconds you are using the wrong technology. Having said that, we once had a task that processed over 16 million financial transactions and ran for several weeks. Migrating to SQL (it was FoxPro) brought it down to hours, and fixing the crap SQL brought it down to under a minute. So the problem may be in the processing not the technology.
If you have knowledge, let others light their candles at it. Margaret Fuller (1810 - 1850) [My Articles] [My Website]
Well dont get me started on this... The amount of WTFs in this code is enough to fill thedailywtf for a few weeks... (It was made by an external consult that have programming skills below what I have after 3 years of school) But im almost certain there is a wrong technology in use here. The site is using a ORM tool that have been put in over the webservice as well. I have previously considerd to remake the entire webservice to not use the ORM tool (that is really database heavy) and instead go directly to the database but havent got around to it yet. There is no doubt that it takes to long to process but since its still posible to do it during the night the ppl prioritising the assignment just dont think it important how long it takes (It processes around 20 items each min so its not "that" slow. Its just there is alot of items). Best Regards HE
-
Well dont get me started on this... The amount of WTFs in this code is enough to fill thedailywtf for a few weeks... (It was made by an external consult that have programming skills below what I have after 3 years of school) But im almost certain there is a wrong technology in use here. The site is using a ORM tool that have been put in over the webservice as well. I have previously considerd to remake the entire webservice to not use the ORM tool (that is really database heavy) and instead go directly to the database but havent got around to it yet. There is no doubt that it takes to long to process but since its still posible to do it during the night the ppl prioritising the assignment just dont think it important how long it takes (It processes around 20 items each min so its not "that" slow. Its just there is alot of items). Best Regards HE
So 3 Seconds an item, via a web service. That consultant should be taken out and shot. I think your safest bet would be to post a handful of items at a time then. Don't even attempt to throw the lot over in one shot. And forget the threading, unless its only purpose is to keep a progress indicator ticking. Oh, a final thought. Pepper your code with comments along the
/* this is terrible but I have no choice */style so any future developer is forewarned to run away fast.If you have knowledge, let others light their candles at it. Margaret Fuller (1810 - 1850) [My Articles] [My Website]
-
So 3 Seconds an item, via a web service. That consultant should be taken out and shot. I think your safest bet would be to post a handful of items at a time then. Don't even attempt to throw the lot over in one shot. And forget the threading, unless its only purpose is to keep a progress indicator ticking. Oh, a final thought. Pepper your code with comments along the
/* this is terrible but I have no choice */style so any future developer is forewarned to run away fast.If you have knowledge, let others light their candles at it. Margaret Fuller (1810 - 1850) [My Articles] [My Website]
Shooting is to good a punishment for him... we need something more evil not sure what thou... (Did I mention that I started with a webservice that took around 10 sec. an item...) And well Ill try talking with them if we could limit the amount of items. The threaded way actually works like it should and helps make it not time out (great improvement from a webservice that timed out when it had moved some data but not all) I will try to see what we can come up with. And well im handing it over to a programmer that have been sitting next to me for the last 5 months he knows how crappy it is... maybe I should just let him have a look at it as an introduction to the system... so that he dont have his hopes to high when he comes to the crappy parts... And its more giving to just write
/* Consultants code */
everywhere something is crappy ;) Best Regards HE
-
Any way I can make it allow to run longer? A windows service might do it but I have no idea how to make one that will fulfill the demands. I will try to describe what im trying to do as good as I can. There is an excisting SAP system with information about products. Im working on a system that recives these informations from SAP with a webservice and stores them in a database to then show them on a webpage and let the user create catalogues based on the informations. In the long run the idea is that every time there is made a change in SAP its send to the webservice (around 100kb of data). Now there is made some changeings to the system and we need to send all data from SAP to the webservice from scratch. Therefor its needed to send alot of data to the webservice. If there is a way to change the alowed running time it might help. Best Regards HE
How can a company have SAP and not have ETL tools in place? Anyway, you might look into the SQLBulkInsert object rather than using an ORM. Something like this should be done in a transaction which it obviously isn't since you get partial updates when it times out. SQLBulkInsert will both be much faster and transactional.
-
How can a company have SAP and not have ETL tools in place? Anyway, you might look into the SQLBulkInsert object rather than using an ORM. Something like this should be done in a transaction which it obviously isn't since you get partial updates when it times out. SQLBulkInsert will both be much faster and transactional.
The problem is not that its not working from the SAP part. That part of it works great all data arrives at the webservice from where its going to get bad... Anything thats not ORM tool might do the trick... only issue is to make sure that the information is correct in form of what the ORM tool want it in when its reading it. Maybe I just should let it go and ask the guy next to me thats good with SQL and ASP.NET <-> SQL to make a new webservice that saves the informations in the database. After all hes the one going to do mantinance on it for the next few years.. (Im happy im not him)