scraping C#
-
I would like to know how the HTML source of ajax based sites can be read using HttpWebRequest / HttpWebResponse (That is reading the contents of a website at server side). The problem that I'm facing is that I'm unable to read parts of the webpage which uses Ajax or stuffs like UpdatePanel. http://stackoverflow.com/questions/20097175/how-to-web-scrape-ajax-update-panel-with-c[^]
-
I would like to know how the HTML source of ajax based sites can be read using HttpWebRequest / HttpWebResponse (That is reading the contents of a website at server side). The problem that I'm facing is that I'm unable to read parts of the webpage which uses Ajax or stuffs like UpdatePanel. http://stackoverflow.com/questions/20097175/how-to-web-scrape-ajax-update-panel-with-c[^]
You can't. The problem is that the remaining HTML is generated by javascript code at "run-time" of the HTML page, using data retrieved over separate HTTP requests. Unless you're code can execute the javascript itself and figure out what it's outputting, you'll never get this to work. I don't know of any implementations that can scrape this.
A guide to posting questions on CodeProject
Click this: Asking questions is a skill. Seriously, do it.
Dave Kreskowiak -
I would like to know how the HTML source of ajax based sites can be read using HttpWebRequest / HttpWebResponse (That is reading the contents of a website at server side). The problem that I'm facing is that I'm unable to read parts of the webpage which uses Ajax or stuffs like UpdatePanel. http://stackoverflow.com/questions/20097175/how-to-web-scrape-ajax-update-panel-with-c[^]