Avoiding memory problems
-
Hello, I am writing an email application in VC++ .Net. I have come across a problem, when downloading mail into a CString, that I run out of memory, especially if there are any large file attachments. I was wondering if there was any way I could save these large emails gradually as I am downloading, in other words transferring out of memory and onto disk, as I go. NB the emails are stored in an Access database, if it makes any difference. I have no idea how to do the above, and would appreciate advice. Thanks, Trimtrom
-
Hello, I am writing an email application in VC++ .Net. I have come across a problem, when downloading mail into a CString, that I run out of memory, especially if there are any large file attachments. I was wondering if there was any way I could save these large emails gradually as I am downloading, in other words transferring out of memory and onto disk, as I go. NB the emails are stored in an Access database, if it makes any difference. I have no idea how to do the above, and would appreciate advice. Thanks, Trimtrom
Using a CString sounds like a bad idea... a fixed buffer and advancing reads should be just fine. Hard to give you a better answer since I have no clue HOW you download the mail. And as far as I can remember, Access allows access to blobs by a stream interface so storing shouldn't pose a problem. "was wir auch tun, wohin wir gehen die illuminaten sind im system sie kontrollieren überall und 23 ist ihre zahl!" 23, welle: erdball
-
Hello, I am writing an email application in VC++ .Net. I have come across a problem, when downloading mail into a CString, that I run out of memory, especially if there are any large file attachments. I was wondering if there was any way I could save these large emails gradually as I am downloading, in other words transferring out of memory and onto disk, as I go. NB the emails are stored in an Access database, if it makes any difference. I have no idea how to do the above, and would appreciate advice. Thanks, Trimtrom
Wow! I am very supprised. First of all if you are copying every thing into a CString including the attachment(binaray format, or other), I would probubly be using a byte array. Second CString and CByteArray has a limit of 2147483647 bytes of data (Win32) and thats why virtual memory is so important (aka: swaping memory to disk behind the sceens). As for copying it to a file, if disk space is realy available, it is simple, (1) open a file (2) copy data bytes into file (3) close file (or rewind to start and process data) (4) if data file is no longer needed then remove/delete file from disk. Question: What am I missing since other than using CString instead of using CByteArray I do not see why you are having a problem? If you are compiling a unicode version of your software then using CSting is a major misstake, since it will expect all data to be using UNICODE. ---------------------------------------- Trust in the code Luke. Yea right!
-
Wow! I am very supprised. First of all if you are copying every thing into a CString including the attachment(binaray format, or other), I would probubly be using a byte array. Second CString and CByteArray has a limit of 2147483647 bytes of data (Win32) and thats why virtual memory is so important (aka: swaping memory to disk behind the sceens). As for copying it to a file, if disk space is realy available, it is simple, (1) open a file (2) copy data bytes into file (3) close file (or rewind to start and process data) (4) if data file is no longer needed then remove/delete file from disk. Question: What am I missing since other than using CString instead of using CByteArray I do not see why you are having a problem? If you are compiling a unicode version of your software then using CSting is a major misstake, since it will expect all data to be using UNICODE. ---------------------------------------- Trust in the code Luke. Yea right!
Actually, real allocatable memory less than 2147483647, especially when you use CString or CByteArray classes. CString and CByteArray allocate memory in process heap, and exception will rise when you allocate more than aproximately 256 Mb. So better allocate memory in 10 heaps by 100 Mb than in 1 Heap by 1Gb... WBR NB
-
Actually, real allocatable memory less than 2147483647, especially when you use CString or CByteArray classes. CString and CByteArray allocate memory in process heap, and exception will rise when you allocate more than aproximately 256 Mb. So better allocate memory in 10 heaps by 100 Mb than in 1 Heap by 1Gb... WBR NB
Your are correct! I would probubly use a multi-buffer sceam too. Just out of curiosity I ran a test using GlobalAlloc() and found that the most I could allocate for one buffer was 1,326,055,381 bytes. Trust in the code Luke. Yea right!
-
Wow! I am very supprised. First of all if you are copying every thing into a CString including the attachment(binaray format, or other), I would probubly be using a byte array. Second CString and CByteArray has a limit of 2147483647 bytes of data (Win32) and thats why virtual memory is so important (aka: swaping memory to disk behind the sceens). As for copying it to a file, if disk space is realy available, it is simple, (1) open a file (2) copy data bytes into file (3) close file (or rewind to start and process data) (4) if data file is no longer needed then remove/delete file from disk. Question: What am I missing since other than using CString instead of using CByteArray I do not see why you are having a problem? If you are compiling a unicode version of your software then using CSting is a major misstake, since it will expect all data to be using UNICODE. ---------------------------------------- Trust in the code Luke. Yea right!
John R. Shaw wrote: and thats why virtual memory is so important (aka: swaping memory to disk behind the sceens). Thanks for your reply. I really am interested about this question of virtual memory: I am not sure how I would code the swapping memory to disk. Is it possible to have a bit of code showing how to download a message into a diskfile via virtual memory?? Many thanks, Trimtrom
-
John R. Shaw wrote: and thats why virtual memory is so important (aka: swaping memory to disk behind the sceens). Thanks for your reply. I really am interested about this question of virtual memory: I am not sure how I would code the swapping memory to disk. Is it possible to have a bit of code showing how to download a message into a diskfile via virtual memory?? Many thanks, Trimtrom
You do not download into a diskfile via virtual memory, you just copy the memory contents to a file in the normal way. Actualy what I was refering to by virtual memory was GlobalAlloc() which uses virtual memory via the heap. Depending on the flags used, if the handle is not locked its memory contents may be swaped to the disk in the background untill needed (this is done by the system not you). When you need access to the memory contents you simply lock the handle and the system will swap it back into normal memory for you. This is all transparent from our perspective and most of us do not need to know how it works, it is enough that is does. Note: You treat the pointer return my GlobalLock() the same as you would a pointer returned by malloc() or calloc(). If you want more control of the memory, and a head ack, you could use VirtualAlloc()/VirtualFree() directly instead. Well I am off to go refesh my memory on all this virtual memory stuff, since I have not studied the subject in years. Trust in the code Luke. Yea right!