OutOfMemoryException
-
/*
I HAVE A LARGE FILE WHICH I AM READING AND CONVERTING FROM BYTE FORM TO DECIMAL. I THEN CONVERT THE FILE BACK TO BYTE FORM AND WRITE IT TO ANOTHER FOLDER ON MY DESKTOP. I AM GETTING "OUT OF MEMORY EXCEPTION". IS THERE A WAY TO SPECIFY TO THE MEMORY TO MOVE THE DATA IN SMALLER CHUNKS? WHY IS TRYING TO MOVE IT ALL AT ONCE ANYWAY, ISN'T THIS VERY INEFFICIENT? MY OUTPUT IS BELOW.Total Bytes = 821903722 bytes
Unhandled Exception: OutOfMemoryException.
Press any key to continue . . .
*/using System;
using System.IO;
using System.Collections.Generic;
using System.Linq;
using System.Text;namespace Applica
{
class Program
{
static void Main(string[] args)
{
DirectoryInfo da = new DirectoryInfo("C:\\Folder");
FileInfo[] Arr = da.GetFiles();
if (Arr.Length == 0)
{
throw new InvalidOperationException("No files found.");
}
// No need to loop through the array just to get the last item:
FileInfo ap = Arr[Arr.Length - 1];
long Totbyte = ap.Length;
string filePath = ap.FullName;
Console.WriteLine("Total Bytes = {0} bytes", Totbyte);
// GetTempFileName *creates* the file, so it always exists:
string temPath = Path.GetTempFileName();
byte[] data = File.ReadAllBytes(filePath);
File.WriteAllBytes(temPath, data);
// Convert the bytes to decimals:
decimal[] arry = new decimal[Totbyte];
for (int count = 0; count < data.Length; count++)
{
arry[count] = data[count];
}
// Convert the decimals back to bytes:
byte[] data2 = new byte[Totbyte];
for (int count = 0; count < arry.Length; count++)
{
data2[count] = (byte)arry[count];
}
// Just to prove they're the same:
if (data2.Length != data.Length)
{
throw new InvalidOperationException("Wrong length!");
}
for (int index = 0; index < data.Length; index++)
{
if (data[index] != data2[index])
{
throw new InvalidOperationException("Data has changed at index " + index);
}
}
// Write the bytes back to the file:
string filePath2 = Path.Combine("C:\\check", Path.GetThe
decimaltype[^] is 128 bits, or 16 bytes long. So 821903722 * 16, will take up 13,150,459,552 bytes, which is rather more memory than you have available. You could simplify this by reading your file in blocks and converting a block at a time. But, a more important issue is what are you actually trying to achieve, since all your program is doing, when it works, is to create a copy of the file? The conversion to decimals is totally redundant. -
/*
I HAVE A LARGE FILE WHICH I AM READING AND CONVERTING FROM BYTE FORM TO DECIMAL. I THEN CONVERT THE FILE BACK TO BYTE FORM AND WRITE IT TO ANOTHER FOLDER ON MY DESKTOP. I AM GETTING "OUT OF MEMORY EXCEPTION". IS THERE A WAY TO SPECIFY TO THE MEMORY TO MOVE THE DATA IN SMALLER CHUNKS? WHY IS TRYING TO MOVE IT ALL AT ONCE ANYWAY, ISN'T THIS VERY INEFFICIENT? MY OUTPUT IS BELOW.Total Bytes = 821903722 bytes
Unhandled Exception: OutOfMemoryException.
Press any key to continue . . .
*/using System;
using System.IO;
using System.Collections.Generic;
using System.Linq;
using System.Text;namespace Applica
{
class Program
{
static void Main(string[] args)
{
DirectoryInfo da = new DirectoryInfo("C:\\Folder");
FileInfo[] Arr = da.GetFiles();
if (Arr.Length == 0)
{
throw new InvalidOperationException("No files found.");
}
// No need to loop through the array just to get the last item:
FileInfo ap = Arr[Arr.Length - 1];
long Totbyte = ap.Length;
string filePath = ap.FullName;
Console.WriteLine("Total Bytes = {0} bytes", Totbyte);
// GetTempFileName *creates* the file, so it always exists:
string temPath = Path.GetTempFileName();
byte[] data = File.ReadAllBytes(filePath);
File.WriteAllBytes(temPath, data);
// Convert the bytes to decimals:
decimal[] arry = new decimal[Totbyte];
for (int count = 0; count < data.Length; count++)
{
arry[count] = data[count];
}
// Convert the decimals back to bytes:
byte[] data2 = new byte[Totbyte];
for (int count = 0; count < arry.Length; count++)
{
data2[count] = (byte)arry[count];
}
// Just to prove they're the same:
if (data2.Length != data.Length)
{
throw new InvalidOperationException("Wrong length!");
}
for (int index = 0; index < data.Length; index++)
{
if (data[index] != data2[index])
{
throw new InvalidOperationException("Data has changed at index " + index);
}
}
// Write the bytes back to the file:
string filePath2 = Path.Combine("C:\\check", Path.GetSee the BinaryReader[^] and BinaryWriter[^] classes. They can read/write files in chuncks. If your file is a text file, see the StreamReader and StreamWriter classes instead. .Net is limited to 2Gb for an object if I'm not mistaken. That might be the reason. ReadAllBytes does what it says: It reads ALL the bytes, so for large files, this could be an issue. Hope this helps.
V.
(MQOTD rules and previous solutions)OriginalGriff wrote:
V is absolutely right
-
See the BinaryReader[^] and BinaryWriter[^] classes. They can read/write files in chuncks. If your file is a text file, see the StreamReader and StreamWriter classes instead. .Net is limited to 2Gb for an object if I'm not mistaken. That might be the reason. ReadAllBytes does what it says: It reads ALL the bytes, so for large files, this could be an issue. Hope this helps.
V.
(MQOTD rules and previous solutions)OriginalGriff wrote:
V is absolutely right
It is not redundant if you want to see a file in a way that looks simple. Decimal is very simple. It is kindergarten simple, which is my level in programming currently. I want to treat all files like binary. I don't want to specify type such as text. The streamwriter class is reading line by line, but that is not what I want. Is there anyway to do with binary chunks for one file?
-
It is not redundant if you want to see a file in a way that looks simple. Decimal is very simple. It is kindergarten simple, which is my level in programming currently. I want to treat all files like binary. I don't want to specify type such as text. The streamwriter class is reading line by line, but that is not what I want. Is there anyway to do with binary chunks for one file?
1. I'm not saying anything about redundancy, perhaps you meant to reply to the other member? 2. I'm not saying you should StreamReader or StreamWriter, I was merely giving an alternative for the BINARY Reader/Writer classes in case it was an option. 3. If you would open the links the members indicate several methods for reading (and writing) chunks of data.
V.
(MQOTD rules and previous solutions)OriginalGriff wrote:
V is absolutely right
-
1. I'm not saying anything about redundancy, perhaps you meant to reply to the other member? 2. I'm not saying you should StreamReader or StreamWriter, I was merely giving an alternative for the BINARY Reader/Writer classes in case it was an option. 3. If you would open the links the members indicate several methods for reading (and writing) chunks of data.
V.
(MQOTD rules and previous solutions)OriginalGriff wrote:
V is absolutely right
Yes, I was commenting what another member wrote about "redundant." I did look up the streamwriter and saw that it would not work for me. Thank you.
-
Yes, I was commenting what another member wrote about "redundant." I did look up the streamwriter and saw that it would not work for me. Thank you.
-
It is not redundant if you want to see a file in a way that looks simple. Decimal is very simple. It is kindergarten simple, which is my level in programming currently. I want to treat all files like binary. I don't want to specify type such as text. The streamwriter class is reading line by line, but that is not what I want. Is there anyway to do with binary chunks for one file?
computerpublic wrote:
It is not redundant if you want to see a file in a way that looks simple.
It still is redundant.
computerpublic wrote:
Decimal is very simple.
A byte is a small integer number, without any decimals. What good does it do to convert it to a decimal? How does it make things simpeler?
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^]
-
It is not redundant if you want to see a file in a way that looks simple. Decimal is very simple. It is kindergarten simple, which is my level in programming currently. I want to treat all files like binary. I don't want to specify type such as text. The streamwriter class is reading line by line, but that is not what I want. Is there anyway to do with binary chunks for one file?
computerpublic wrote:
It is not redundant if you want to see a file in a way that looks simple. Decimal is very simple.
Just a guess, but are you trying to view the bytes as decimal (base-10) numbers? You don't need to convert to the
decimaltype for that.
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
-
It is not redundant if you want to see a file in a way that looks simple. Decimal is very simple. It is kindergarten simple, which is my level in programming currently. I want to treat all files like binary. I don't want to specify type such as text. The streamwriter class is reading line by line, but that is not what I want. Is there anyway to do with binary chunks for one file?
computerpublic wrote:
t is not redundant
Of course it is, you convert bytes to decimal and then immediately convert them back to bytes, so it serves no purpose.
computerpublic wrote:
Is there anyway to do with binary chunks for one file?
Yes, use the
Readmethod[^] of theBinaryReaderclass[^], and read in blocks as I suggested previously. More importantly, you still have not explained why you want to convert these bytes to decimal in the first place. Perhaps if you did, we would be able to make some more useful suggestions. -
It is not redundant if you want to see a file in a way that looks simple. Decimal is very simple. It is kindergarten simple, which is my level in programming currently. I want to treat all files like binary. I don't want to specify type such as text. The streamwriter class is reading line by line, but that is not what I want. Is there anyway to do with binary chunks for one file?
Honestly, how are decimals easier to understand? Note, a byte is a value from 0 to 255. You convert it to decimal and it is a value from 0 to 255. Maybe you are getting confused by the term "binary". Binary means 2 things: A value that can be only 0 or 1. So, instead of seeing 255 you see 11111111. A representation of data that's not text-only. So, a byte is usually used to read binary data. This doesn't mean it only reads 0 or 1. It can still be seen as 0, 1, 2, 3, 100 etc... limited to 255. It is actually not limited to visible characters (so, it is not text). Your conversion from byte to decimal is only converting 8 bit values to 128 bit values that contain the same important data (a value from 0 to 255)... and then you are converting it back. So, it is not redundant. It is completely useless. Redundant is for those cases that it actually generates a result that could be easily obtained. In this case you are doing completely useless work. If what you want is to read a file that has lines like this: 01010101 11111111 00010001 And you want to understand those values, you need to: Read each line as string. Convert each string to a byte. Then, you convert the numeric value back to a string. So, 11111111 (binary) will become 255 (decimal). This is a decimal representation, not the decimal type.
