Still Stuck: How to intercept rapidshare download in WebBrowser control using Http POST

Anindya Chatterjee

I am developing a C# app which will download Rapidshare files for Free users. The program would work as follows: 1. Rapidshare webpage would open in WebBrowser control. 2. Captcha would be input in the webpage 3. File download will occur using the download manager of the app, instead of the default download manager. I have designed the download section in the following way. private void rapidWebBrowser_Navigating(object sender, WebBrowserNavigatingEventArgs e) { string filename = @"C:\test.rar"; Regex pattern = new Regex(@"http://([0-9a-zA-Z]+)\.rapidshare\.(com|de)/files/([0-9]+)/([0-9]+)/([^\r\n]+)", RegexOptions.IgnoreCase); string match = e.Url.AbsoluteUri; if (pattern.IsMatch(match)) { e.Cancel = true; WebClient client = new WebClient(); Uri uri = new Uri(match); client.DownloadFileAsync(uri, filename); } } While running, the app downloads a 10KB file instead of the real one. If I use the followsing code: private void rapidWebBrowser_FileDownload(object sender, EventArgs e) { Regex pattern = new Regex(@"http://([0-9a-zA-Z]+)\.rapidshare\.(com|de)/files/([0-9]+)/([0-9]+)/([^\r\n]+)", RegexOptions.IgnoreCase); if (pattern.IsMatch(match)) // match is assigned to the required navigating url { WebClient client = new WebClient(); Uri uri = new Uri(match); client.DownloadFileAsync(uri, filename); } } the app downloads a 10KB file and pops up IE's default download manager. How to solve the problem? I want to intercept the original file download by WebBrowser control, and pass the url to my app's custom downloader suppressing the IE's downloader. How to do this? I am eagerly waiting for the solution. Thank you in advance.

Anindya Chatterjee

led mike

Anindya Chatterjee wrote:

While running, the app downloads a 10KB file instead of the real one.

10kb file containing what?

led mike

Spacix One

Sorry this has nothing todo with the OP's questions but.... Why is it called "rapidWebBrowser" when it prevents downloading rapidshare files?? :confused:

-Spacix All your skynet questions[^] belong to solved

I dislike the black-and-white voting system on questions/answers. X|

Anindya Chatterjee

The 10KB file has the same extension as the real one. But when I change its extension to "html" and open it into IE, a rapidshare page opens telling: Your IP address ***.***.***.*** is already downloading a file. You have to wait until it is finished. Premium-accounts can download many files at the same time. Get your own premium-account now. Instant download-access! [Original IP is replaced by *, it is confirmed that no ther RS download is going on at that time. I am not using any kind of proxy. This IP is for my own.] I don't know why this is happenning. More thing to notice during Rapidshare happy hour time the file is downloaded successfully, but when CAPTCHA is used this thing happens and downloads never get successful. Help plz..

Anindya Chatterjee

That Asian Guy

He can call it anything he wants :).

Anthony Mushrow

Like "Fatty McCheese's Big Browser"?

My current favourite word is: Bacon!

-SK Genius

Spacix One

or someone having public enum myBool{ TRUE, FALSE, FILE_NOT_FOUND } (yes people do this!, one example was on thedailywtf.com awhile back) Mine was more of a retorical question... Let make a bool and call it "bSaveData" but only save the data when it is false! I mean why do people do these things :( :mad:

-Spacix All your skynet questions[^] belong to solved

I dislike the black-and-white voting system on questions/answers. X|

led mike

Spacix One wrote:

I mean why do people do these things

I know that one. Because, clowns to the left of me monkeys to the right, here I am stuck in the middle with you. :-D

led mike

led mike

Anindya Chatterjee wrote:

Your IP address ***.***.***.*** is already downloading a file. You have to wait until it is finished.

First you cannot believe that message.

Anindya Chatterjee wrote:

I don't know why this is happenning.

Second there are any number of HTTP headers and/or sections they might use to deny your request. Have you sniffed successful downloads using a browser to see what Headers / Sections are present? Unless RapidShare documents an API that is probably the only way you are going to figure it out.

led mike

Anindya Chatterjee

I have seen a Orbit downloader log while downloading a rapidshare file: 2008-05-06 02:17:07 Connecting rs266gc.rapidshare.com:80 2008-05-06 02:17:08 Connected 2008-05-06 02:17:08 POST /files/96986359/1517356/MyPet.rar HTTP/1.1 2008-05-06 02:17:08 Host: rs266gc.rapidshare.com 2008-05-06 02:17:08 Referer: http://rs266.rapidshare.com/files/96986359/MyPet.rar 2008-05-06 02:17:08 User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0) 2008-05-06 02:17:08 Content-Length: 66 2008-05-06 02:17:08 Content-Type: application/x-www-form-urlencoded 2008-05-06 02:17:08 Accept: */* 2008-05-06 02:17:08 Pragma: no-cache 2008-05-06 02:17:08 Cache-Control: no-cache 2008-05-06 02:17:08 Connection: close 2008-05-06 02:17:08 HTTP/1.1 200 OK 2008-05-06 02:17:08 Date: Mon, 05 May 2008 20:46:37 GMT 2008-05-06 02:17:08 Connection: close 2008-05-06 02:17:08 Content-Type: application/octet-stream 2008-05-06 02:17:08 Accept-Ranges: bytes 2008-05-06 02:17:08 Content-Disposition: Attachment; filename=MyPet.rar 2008-05-06 02:17:08 Content-Length: 5518769 2008-05-06 02:17:08 Start receiving data How to use this in my C# App? I don't know. Help plz.

Anindya Chatterjee

Spacix One

The lines in the log look like a DateTime stamp followed by HTTP headers. If you use HttpWebRequest you can access/change HTTP header information. The problem you are having is this. You open a WebBrowser control, launch the page and the user enters the catchup info and clicks download. When this finishes you capture the URL of the file once it is suppose to start downloading, that is the Navigating event you captured. You cancel that request (ending the server connection) and then try to launch a web client to download from the now expired URL. I seem to recall sending e.Cancel=true; is the same as pressing the "Stop" button inside IE. What you might try doing, without getting into the HTTP headers is just move the e.Cancel=true; below your web client download (since you never opened the page, it might let you download it)

-Spacix All your skynet questions[^] belong to solved

I dislike the black-and-white voting system on questions/answers. X|

Spacix One

if(B2||~(B2))
{
:rose:
}
else
{
X|
}

-Spacix All your skynet questions[^] belong to solved

I dislike the black-and-white voting system on questions/answers. X|

Anindya Chatterjee

Yes, Ofcourse it will download, but with the default download manager of IE. I don't want this. I want to download it by the app itself. Problem is that catching the direct link of the file on Rapidshare server, I am not able to download it. I think you get my point and that is here I am stuck.

Anindya Chatterjee

Spacix One

No, it won't download in IE until the Navigating event exits. I'm asking you to do the following:

    private void rapidWebBrowser\_Navigating(object sender, WebBrowserNavigatingEventArgs e)
    {
        string filename = @"C:\\test.rar";
        Regex pattern = new Regex(@"http://(\[0-9a-zA-Z\]+)\\.rapidshare\\.(com|de)/files/(\[0-9\]+)/(\[0-9\]+)/(\[^\\r\\n\]+)", RegexOptions.IgnoreCase);
        string match = e.Url.AbsoluteUri;
        if (pattern.IsMatch(match))
        {
            WebClient client = new WebClient();
            Uri uri = new Uri(match);
            client.DownloadFileAsync(uri, filename);
            e.Cancel = true;
        }
    }

and if this doesn't work, you can use download and use WireShark[^] and WinPCap to record a successful request from within IE and reverse engineer the "good" request header requirements and you're set. You might also be able to use FireFox with live HTTP headers plugin that allows you to view HTTP data... Yet if not canceling the event until AFTER your file download fixes the problem; none of the header stuff would be required.

-Spacix All your skynet questions[^] belong to solved

I dislike the black-and-white voting system on questions/answers. X|