Hi, I have csv file which contains 1 million records and with column 160. I have to insert(update the csv file) more 60 columns in the same file so the total column finally will generated 220. 60 columns and there data will get update like - for first Row I have to select values from column 2,3,4 and generate the url and execute the url so that I will get xml data. Then I have to parse that data and fill 60 columns for that perticular row. Like same I have to do for 2 row and 3row....upto 1 million. I have to process this file within 3 hrs. How I can improve the performance of the reading and writing to csv file. Regards, sjs
sjs4u
Posts
-
Increase performance of csv -
Get and Update the CSV File cellsHi, I have csv file which contains 1 million records and with column 160. I have to insert(update the csv file) more 60 columns in the same file so the total column finally will generated 220. 60 columns and there data will get update like - for first Row I have to select values from column 2,3,4 and generate the url and execute the url so that I will get xml data. Then I have to parse that data and fill 60 columns for that perticular row. Like same I have to do for 2 row and 3row....upto 1 million. I have processed this file within 3 hrs. How I can improve the performance of the reading and writing to csv file. Regards, sjs
-
Upload and Download using SFTPHi, I have "sftp://ftp4.abc.com" url with username and password for FTP Details. I have port no 22 and with this details I am able to connect to the server using FileZila I am using FTPWebRequest to upload and download the files from server and my code is working for my existing FTP details which is not secure. But When I use the above details it showing the error The underlying connection was closed: The connection was closed unexpectedly. Here is my code to execute the url Uri uriObj = new Uri(uri); var r = (FtpWebRequest)WebRequest.Create(uriObj); r.Credentials = new NetworkCredential(ftpUsername, ftpPassword); r.Method = method; r.UsePassive = true; r.EnableSsl=true; r.UseBinary = true; r.KeepAlive = true; return r; let me know how can i use sftp with existing code or any other solution for the same. Regards, sjs
-
Unable to set registry valueHi, I am using MSI to install my c# winform application. I am using custom action at the start of the installer. This is my latest version where I have added custom action. When I install latest version (fresh installation) then it works properly. But when I install latest version on previous version my custom action related registry key(e.g. ModeSelection Key) is not creating in the registry but some basic and common key(Filepath,language etc) are created successfully. What is the reason behind this? I wanted to create that registry key depeding upon the custom action selection. Thanks sjs
-
Install software and menus depending upon radion button selection option used in MSIHi Eddy, Thanks for your reply. Yes I am passing tech/eng mode values as commadline argument and checked that in Main Method in program.cs. Default is tech mode. In Installer Project - I set the radio button values and can able to get these values using string myInput1 = this.Context.Parameters["BUTTON1"]; string myInput2 = this.Context.Parameters["BUTTON2"]; Now when I start installer it is showing 2 radio button and in that i want selected radion button value to be passed to exe so that only that exe get installed. Regards, sjs4u
-
Install software and menus depending upon radion button selection option used in MSIHi, I made simple winform application in C#. I am showing menu options depending upon the mode selection. .i.e. if mode is eng then I am showing all the menus in MDI and if mode is tech then I am hiding some menus of MDI.This is handled in Code. I am using MSI to create setup.I am using custom dialog box i.e. 2 radio button for mode selection. Now I want install software depeding upon the mode selection. i.e. if I select Eng mode radio button then installer install software with all menus and if I select tech mode then installer install only few menus which is applicable to tech mode. I dont know how to use custom action so that i can pass the selection option to the exe. Is there any other solution just let me know. Thanks, sjs4u
-
MSI not taking/Installing .NET frameworkHi, I am using MSI to create setup. I already added prerequisite (.NET framwork 3.5) into MSI. I have selected option i.e. Download prerequisite fromthe same location as my application While installing setup on windows XP SP3 its showing error .NET framework is not installed on the machine. Setup is not installing .NET framework from MSI. Even I added all framework files and folders into application folder and program files folder. Is I am doing any wrong? can pls let me know the steps how can i used my prerequisite .NET framework 3.5 on windows XP SP3 even not framework is available. Thanks sjs
-
How to call C++ parameterised function from C# using threadHi, I want to call parameterised function which is written in c++ from c# by using thread and also how will c# know or will get return value from c++ function, so that c# function will proceed further. If you have any example then let me know. Thanks sjs
-
Page.PreviousePage ProblemHi I am having problem for Page.PreviousPage where I am getting all the values from next page to previous page. But I want to close the next page and I want to fully transfer the control to the previous page. I tried with response.redirect and server.transfer but in the same(next) page i am getting the updated result. Thanks Sjs
-
Updateprogress barHi, I want to use progress bar in my webpage. I dont want third party control. How can i use AJAX updateprogress. Thanks sjs4u
-
want to access same page using different user at a same timeHi, I want to access same page same time from different users. Is it possible using Session. Also I don't want to use database once i create users.(I dont want to use flag if someone is login etc.) Also if I want to restrict user not to access page same time. what can i do for this also. Thanks sjs
-
How to Get FTP username and passwordHi, I am using Filezilla Software for uploading web pages. Now I have domain name and their username and password. That uses by me and my friend. Now we want such method that any one changes these ftp username or password from hosting email will b send. Is that possible through ASP.NET? If yes then How can I get the details? Is there any other method for that? Thanks sjs
-
Error while uploading large filewebclient.uploadfile() :- for This should I have to write credential? Right Now my path is http://loacalhost/server/file.wav. as i m uploading .wav file. Why I am saying that I want to use same code only for uploading the file. I will check..Thanks
-
Error while uploading large fileOk. Let me specify what I am actually doing. I am using windows application for uploading file on server. My client applications were using by our various CLIENT's. In that I am using one folder where there will be more files in that and once CLIENT's finish there work they save it to that folder. then by using threading concept I am fetching all the files and uploaded on the server. Also in that I need to check the size of files that is being uploaded on the server. In that I dont want to set some credential like username and password. By using your solution webclient.uploadfile() is this will work. How can i use Webclient.UploadfileAsync? Thanks sjs
modified on Saturday, April 10, 2010 7:38 AM
-
Error while uploading large fileHi, I am using webclient while uploading large file(135MB) using windows application. My Code works but when it reaches to stream.close() it showing following error Error:- The CLR has been unable to transition from COM context 0x1ff030 to COM context 0x1ff1a0 for 60 seconds. The thread that owns the destination context/apartment is most likely either doing a non pumping wait or processing a very long running operation without pumping Windows messages. This situation generally has a negative performance impact and may even lead to the application becoming non responsive or memory usage accumulating continually over time. To avoid this problem, all single threaded apartment (STA) threads should use pumping wait primitives (such as CoWaitForMultipleHandles) and routinely pump messages during long running operations. So because of that my file is not uploaded. what is the problem? My Code is:- WebClient client = new WebClient(); Stream stream = client.OpenWrite(remoteFilename, "PUT"); // The buffer size is set to 2kb const int buffLength = 2048; byte[] buff = new byte[buffLength]; int contentLen; // Opens a file stream (System.IO.FileStream) to read the file // to be uploaded FileStream fs = fileInf.OpenRead(); int doneSoFar = 0; // Read from the file stream 2kb at a time contentLen = fs.Read(buff, 0, buffLength); //added for accurate progress doneSoFar += contentLen; setItemStatus("Uploading"); while (contentLen != 0) { setItemProgress(doneSoFar, fileInf.Length); // Write Content from the file stream to the FTP Upload Stream stream.Write(buff, 0, contentLen); contentLen = fs.Read(buff, 0, buffLength); doneSoFar += contentLen; } // Close the file stream and the Request Stream stream.Close(); fs.Close(); Thanks sjs
-
Error While Large File UploadingHi, I am using webclient while uploading large file(135MB) using windows application. My Code works but when it reaches to stream.close() it showing following error Error:- The CLR has been unable to transition from COM context 0x1ff030 to COM context 0x1ff1a0 for 60 seconds. The thread that owns the destination context/apartment is most likely either doing a non pumping wait or processing a very long running operation without pumping Windows messages. This situation generally has a negative performance impact and may even lead to the application becoming non responsive or memory usage accumulating continually over time. To avoid this problem, all single threaded apartment (STA) threads should use pumping wait primitives (such as CoWaitForMultipleHandles) and routinely pump messages during long running operations. So because of that my file is not uploaded. what is the problem? My Code is:- WebClient client = new WebClient(); Stream stream = client.OpenWrite(remoteFilename, "PUT"); // The buffer size is set to 2kb const int buffLength = 2048; byte[] buff = new byte[buffLength]; int contentLen; // Opens a file stream (System.IO.FileStream) to read the file // to be uploaded FileStream fs = fileInf.OpenRead(); int doneSoFar = 0; // Read from the file stream 2kb at a time contentLen = fs.Read(buff, 0, buffLength); //added for accurate progress doneSoFar += contentLen; setItemStatus("Uploading"); while (contentLen != 0) { setItemProgress(doneSoFar, fileInf.Length); // Write Content from the file stream to the FTP Upload Stream stream.Write(buff, 0, contentLen); contentLen = fs.Read(buff, 0, buffLength); doneSoFar += contentLen; } // Close the file stream and the Request Stream stream.Close(); fs.Close(); for Small file size it works fine. Thanks sjs
-
Passing value to confirmation boxHi, I am using gridview in that i used button as Item Template field. Now I am using gridviews rowcommand event for some database calculations. I want confirmation box where I want to pass one database value to show in the message of confirmation box. now i want to show my value in the confirmation boxs message. How can I do this? Thanks sjs
-
GetChanges() & AcceptChanges() not working....ok thanks... I will try it
-
GetChanges() & AcceptChanges() not working....I write it after adpt.Fill(ds) It is showing in the dt that value is updated but when i click on 2nd button still it is not showing that perticular updated row.. Can you show it with the above example
-
GetChanges() & AcceptChanges() not working....hi, let me clarify now. string mySelectQuery = "SELECT * from Table1"; SqlConnection myConnection = new SqlConnection(connectionString); SqlDataAdapter adpt = new SqlDataAdapter(mySelectQuery, myConnection); dt = new DataTable(); myConnection.Open(); adpt.Fill(ds); dt = ds.Tables[0]; dataGridView1.DataSource = dt; myConnection.Close(); This is written on Refresh button. Now there is another button where i written code to get only updated row from above datatable. DataSet ds3 = new DataSet(); dt.AcceptChanges(); // dt delcared globally ds3 = dt.GetChanges(); If I change one of the field in table1 directly from database and if i clicked on refresh button then it is showing the updated field. Now if i clicked on second button then I want only that updated row and want to store in dataset ds3. Just let me know if i m doing any wrong from above code. Thanks sjs