Download File From Website With Login
-
I am attempting to automate the downloading of a file from a website that requires user login. At this point i want to get clarification of the steps i need to follow to accomplish this. Currently i am doing the following (in pseudo code) without success: --Make POST HttpWebRequest/HttpWebResponse of "Login.php" that posts login username and password; --Get response cookie collection and put into CookieContainer variable; --Make HttpWebRequest/HttpWebResponse of "Content.php"; assign CookieContainer variable to request; --Get GetResponseStream of response and put HTML content into String; --Parse HTML content for download link; --Download link to disk; --Process downloaded file.... With this code, the HTML retrieved from Content.php is for a webpage shown to users that have not loged on and so the download link is not shown. Below is test code for the above. // ################################################################## public void TEST() { // login // forms authenticate string loginUri = AppGlobal.LoginUrl; string requestString = "username=" + AppGlobal.UserName + "&password=" + AppGlobal.Password + "&autologin=checked"; byte[] requestData = Encoding.UTF8.GetBytes(requestString); HttpWebRequest request = null; Stream stream = null; HttpWebResponse response = null; // set up request _cookies = new CookieContainer(); request = (HttpWebRequest)WebRequest.Create(loginUri); request.Proxy = null; request.CookieContainer = _cookies; request.Method = "POST"; request.ContentType = "application/x-www-form-urlencoded"; request.ContentLength = requestData.Length; // make form post stream = request.GetRequestStream(); stream.Write(requestData, 0, requestData.Length); // get response response = (HttpWebResponse)request.GetResponse(); // update cookies foreach (Cookie returnCookie in response.Cookies) { // debug help Debug.WriteLine("COOKIE: " + returnCookie.Name + " = " + returnCookie.Value); // compare new cookie with stored cookie // update cookie container bool cookieFound = false; foreach (Cookie oldCookie in _cookies.GetCookies(new Uri(AppGlobal.CookieUrl))) { if (returnCookie.Name == oldCookie.Name) { oldCookie.Value = returnCookie.Value; cookieFound = true; } } if (cookieFound == false) { _cookies.Add(new Uri(AppGlobal.CookieUrl), returnCookie); } } // get webpage request = (HttpWebRequest)WebRequest.Create(new Uri(AppGlobal.Start
-
I am attempting to automate the downloading of a file from a website that requires user login. At this point i want to get clarification of the steps i need to follow to accomplish this. Currently i am doing the following (in pseudo code) without success: --Make POST HttpWebRequest/HttpWebResponse of "Login.php" that posts login username and password; --Get response cookie collection and put into CookieContainer variable; --Make HttpWebRequest/HttpWebResponse of "Content.php"; assign CookieContainer variable to request; --Get GetResponseStream of response and put HTML content into String; --Parse HTML content for download link; --Download link to disk; --Process downloaded file.... With this code, the HTML retrieved from Content.php is for a webpage shown to users that have not loged on and so the download link is not shown. Below is test code for the above. // ################################################################## public void TEST() { // login // forms authenticate string loginUri = AppGlobal.LoginUrl; string requestString = "username=" + AppGlobal.UserName + "&password=" + AppGlobal.Password + "&autologin=checked"; byte[] requestData = Encoding.UTF8.GetBytes(requestString); HttpWebRequest request = null; Stream stream = null; HttpWebResponse response = null; // set up request _cookies = new CookieContainer(); request = (HttpWebRequest)WebRequest.Create(loginUri); request.Proxy = null; request.CookieContainer = _cookies; request.Method = "POST"; request.ContentType = "application/x-www-form-urlencoded"; request.ContentLength = requestData.Length; // make form post stream = request.GetRequestStream(); stream.Write(requestData, 0, requestData.Length); // get response response = (HttpWebResponse)request.GetResponse(); // update cookies foreach (Cookie returnCookie in response.Cookies) { // debug help Debug.WriteLine("COOKIE: " + returnCookie.Name + " = " + returnCookie.Value); // compare new cookie with stored cookie // update cookie container bool cookieFound = false; foreach (Cookie oldCookie in _cookies.GetCookies(new Uri(AppGlobal.CookieUrl))) { if (returnCookie.Name == oldCookie.Name) { oldCookie.Value = returnCookie.Value; cookieFound = true; } } if (cookieFound == false) { _cookies.Add(new Uri(AppGlobal.CookieUrl), returnCookie); } } // get webpage request = (HttpWebRequest)WebRequest.Create(new Uri(AppGlobal.Start
Have you already searched on Code Project? There are several articles that talk about how to do this, including this[^] one.
Scott.
—In just two days, tomorrow will be yesterday. [Forum Guidelines] [Articles] [Blog]
-
Have you already searched on Code Project? There are several articles that talk about how to do this, including this[^] one.
Scott.
—In just two days, tomorrow will be yesterday. [Forum Guidelines] [Articles] [Blog]
Yes, i have done search of the internet. I got code examples for downloading files using credentials. In order to download the file in my problem, user has to login to website using a POST webform, then go to webpage with link on it, then download link. One cannot go to webpage direct as webpage will not show download link without login cookie. One cannot go to download link directly as link changes name and so one must parse HTML for it first. My problem is that after i do POST to the login page and get login cookies back, i cannot seem to use these cookies to retrieve the desired webpage. I keep getting the webpage version shown to users not logged in (which does not contain the download link). Since i am new at this, i am not sure what i am doing wrong.
-
Yes, i have done search of the internet. I got code examples for downloading files using credentials. In order to download the file in my problem, user has to login to website using a POST webform, then go to webpage with link on it, then download link. One cannot go to webpage direct as webpage will not show download link without login cookie. One cannot go to download link directly as link changes name and so one must parse HTML for it first. My problem is that after i do POST to the login page and get login cookies back, i cannot seem to use these cookies to retrieve the desired webpage. I keep getting the webpage version shown to users not logged in (which does not contain the download link). Since i am new at this, i am not sure what i am doing wrong.
Hi dude, I've got the same problem here. I have figured out that the problem raises from the fact that the cookie returned to you from the first request is not persisted on the server, so you are not recognized when you request the second page. I am trying to solve this problem though. Please tell me if you've figured out the problem nearly after a year. Thanks