Speed difference between two functions with unknown reason
-
Hi, I have two methods, which are esentially doing similar jobs but have drastic speed differences. I don't know what is causing this speed difference. Here are the methods. The first one is like 100 times faster.
private void SaveNativeBinary(string file, long origin, long length, string opt)
{
SetSSVisible(StatusStrip, pbSmallPerc, true);FileStream fs = null; FileStream writer = null; try { fs = new FileStream(file, FileMode.Open, FileAccess.Read); } catch { throw new Exception("ERR"); } try { writer = new FileStream(opt, FileMode.Create, FileAccess.ReadWrite); } catch { throw new Exception("ERR"); } fs.Seek(origin, SeekOrigin.Begin); for (long pos = 0; pos < length; pos++) { if (pos % 204800 == 0) // 200KB'da bir güncelle { SetSSValue(StatusStrip, pbSmallPerc, Convert.ToInt32(PercentageCalc(pos, length)) ); SetSSText(StatusStrip, lblStatus, "Aktarılıyor: " + BtoKB(pos).ToString() + " / " + BtoKB(length).ToString() + " KB (%" + PercentageCalc(pos, length) + ")"); } writer.WriteByte((byte)fs.ReadByte()); } fs.Close(); fs.Dispose(); writer.Close(); writer.Dispose(); SetSSVisible(StatusStrip, pbSmallPerc, false); }private void AppendBinToBin(string Container, string toAppend)
{
SetSSVisible(StatusStrip, pbSmallPerc, true);
FileStream fsA, fsB = null;
long tempLength = GetFileLength(toAppend);try { fsA = new FileStream(Container, FileMode.Open, FileAccess.ReadWrite); } catch { throw new Exception("ERR"); } try { fsB = new FileStream(toAppend, FileMode.Open, FileAccess.Read); } catch { throw new Exception("RRR"); } try { fsA.Seek(0, SeekOrigin.End); for (long pos = 0; pos < fsB.Length; pos++) { if (pos % 204800 == 0) // 200KB'da bir güncelle { SetSSValue( StatusStrip, pbSmallPerc, Convert.ToInt32(PercentageCalc(pos, tempLength)) ); SetSSText( StatusStrip, lblStatus, "Aktarılıyor: " + BtoKB(pos).ToString() + " / " + BtoKB(tempLength).ToString() + " KB (%" + PercentageCalc(pos, tempLength) + ")" ); -
I am using the Process Explorer from SysInternals and the first method creates an IO traffic with the magnitude of approx. 32MB, but the second one can't even exceed 1MB.
-
Hi, I have two methods, which are esentially doing similar jobs but have drastic speed differences. I don't know what is causing this speed difference. Here are the methods. The first one is like 100 times faster.
private void SaveNativeBinary(string file, long origin, long length, string opt)
{
SetSSVisible(StatusStrip, pbSmallPerc, true);FileStream fs = null; FileStream writer = null; try { fs = new FileStream(file, FileMode.Open, FileAccess.Read); } catch { throw new Exception("ERR"); } try { writer = new FileStream(opt, FileMode.Create, FileAccess.ReadWrite); } catch { throw new Exception("ERR"); } fs.Seek(origin, SeekOrigin.Begin); for (long pos = 0; pos < length; pos++) { if (pos % 204800 == 0) // 200KB'da bir güncelle { SetSSValue(StatusStrip, pbSmallPerc, Convert.ToInt32(PercentageCalc(pos, length)) ); SetSSText(StatusStrip, lblStatus, "Aktarılıyor: " + BtoKB(pos).ToString() + " / " + BtoKB(length).ToString() + " KB (%" + PercentageCalc(pos, length) + ")"); } writer.WriteByte((byte)fs.ReadByte()); } fs.Close(); fs.Dispose(); writer.Close(); writer.Dispose(); SetSSVisible(StatusStrip, pbSmallPerc, false); }private void AppendBinToBin(string Container, string toAppend)
{
SetSSVisible(StatusStrip, pbSmallPerc, true);
FileStream fsA, fsB = null;
long tempLength = GetFileLength(toAppend);try { fsA = new FileStream(Container, FileMode.Open, FileAccess.ReadWrite); } catch { throw new Exception("ERR"); } try { fsB = new FileStream(toAppend, FileMode.Open, FileAccess.Read); } catch { throw new Exception("RRR"); } try { fsA.Seek(0, SeekOrigin.End); for (long pos = 0; pos < fsB.Length; pos++) { if (pos % 204800 == 0) // 200KB'da bir güncelle { SetSSValue( StatusStrip, pbSmallPerc, Convert.ToInt32(PercentageCalc(pos, tempLength)) ); SetSSText( StatusStrip, lblStatus, "Aktarılıyor: " + BtoKB(pos).ToString() + " / " + BtoKB(tempLength).ToString() + " KB (%" + PercentageCalc(pos, tempLength) + ")" );Hi, IMO the major suspect is the test in
for (long pos = 0; pos < fsB.Length; pos++)which goes out and reads a property for each byte available. It may, or may not, be much better when running a release build; anyhow, I always make sure loop termination tests are as simple as possible, a local variable often does wonders. :)Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]
I only read code that is properly formatted, adding PRE tags is the easiest way to obtain that.
-
Hi, IMO the major suspect is the test in
for (long pos = 0; pos < fsB.Length; pos++)which goes out and reads a property for each byte available. It may, or may not, be much better when running a release build; anyhow, I always make sure loop termination tests are as simple as possible, a local variable often does wonders. :)Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]
I only read code that is properly formatted, adding PRE tags is the easiest way to obtain that.
Hi, I am not sure I understood your point exactly. if(pos % 204800 == 0) block is used in both cases but they still have speed differences.
-
Hi, I am not sure I understood your point exactly. if(pos % 204800 == 0) block is used in both cases but they still have speed differences.
The point, as I understand it, was that calling
fsB.get_Length(which is what you're doing when you ask the value offsB.Length) may secretly be doing something that takes more than a reasonable amount of time, which is true. As proof, here is the code forFileStream.Lengthpublic override long get_Length()
{
if (this._handle.IsClosed)
{
__Error.FileNotOpen();
}
if (!this.CanSeek)
{
__Error.SeekNotSupported();
}
int highSize = 0;
int fileSize = 0;
fileSize = Win32Native.GetFileSize(this._handle, out highSize);
if (fileSize == -1)
{
int errorCode = Marshal.GetLastWin32Error();
if (errorCode != 0)
{
__Error.WinIOError(errorCode, string.Empty);
}
}
long num4 = (highSize << 0x20) | ((long) ((ulong) fileSize));
if ((this._writePos > 0) && ((this._pos + this._writePos) > num4))
{
num4 = this._writePos + this._pos;
}
return num4;
}Win32Native.GetFileSizeuses P/Invoke to GetFileSize in kernel32.dll Solution: store the length in a local variable (like Luc said) -
Hi, I am not sure I understood your point exactly. if(pos % 204800 == 0) block is used in both cases but they still have speed differences.
Luc is talking about your use of
fsB.Length, which is a property that you get every time. He's suggesting that you should store this and then work with that instead offsB.Lengthin the loop."WPF has many lovers. It's a veritable porn star!" - Josh Smith
As Braveheart once said, "You can take our freedom but you'll never take our Hobnobs!" - Martin Hughes.
-
The point, as I understand it, was that calling
fsB.get_Length(which is what you're doing when you ask the value offsB.Length) may secretly be doing something that takes more than a reasonable amount of time, which is true. As proof, here is the code forFileStream.Lengthpublic override long get_Length()
{
if (this._handle.IsClosed)
{
__Error.FileNotOpen();
}
if (!this.CanSeek)
{
__Error.SeekNotSupported();
}
int highSize = 0;
int fileSize = 0;
fileSize = Win32Native.GetFileSize(this._handle, out highSize);
if (fileSize == -1)
{
int errorCode = Marshal.GetLastWin32Error();
if (errorCode != 0)
{
__Error.WinIOError(errorCode, string.Empty);
}
}
long num4 = (highSize << 0x20) | ((long) ((ulong) fileSize));
if ((this._writePos > 0) && ((this._pos + this._writePos) > num4))
{
num4 = this._writePos + this._pos;
}
return num4;
}Win32Native.GetFileSizeuses P/Invoke to GetFileSize in kernel32.dll Solution: store the length in a local variable (like Luc said)Right. I didn't know it was that bad. And why don't they use GetFileSizeEx, that one at least returns a long right away. :doh:
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]
I only read code that is properly formatted, adding PRE tags is the easiest way to obtain that.
-
Right. I didn't know it was that bad. And why don't they use GetFileSizeEx, that one at least returns a long right away. :doh:
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]
I only read code that is properly formatted, adding PRE tags is the easiest way to obtain that.
It is, I think, not even correct for sizes above 4GB. I mean see this:
(highSize << 0x20)highSizeis anint, and if you try to shift anintby 32, nothing happens, and it's only upcast after the shift. There is just no way anyone they'd let something like this in after all the patches, so something else is going on. I went back to the reflector to check the MSIL:L\_004a: conv.i8 <-- L\_004b: ldc.i4.s 0x20 L\_004d: shledit: this must be the "offending" part of the code, it's the only place in the code where there is a shift by 0x20 So the actual code is right after all, that means it's a bug in Reflector, right? Or am I missing something?
-
It is, I think, not even correct for sizes above 4GB. I mean see this:
(highSize << 0x20)highSizeis anint, and if you try to shift anintby 32, nothing happens, and it's only upcast after the shift. There is just no way anyone they'd let something like this in after all the patches, so something else is going on. I went back to the reflector to check the MSIL:L\_004a: conv.i8 <-- L\_004b: ldc.i4.s 0x20 L\_004d: shledit: this must be the "offending" part of the code, it's the only place in the code where there is a shift by 0x20 So the actual code is right after all, that means it's a bug in Reflector, right? Or am I missing something?
- Well spotted 2) I looked into it, and IL is OK, C# "disassembly" is wrong. Not the first bug in Reflector. 3) See, I would have used GetFileSizeEx and not have had the opportunity to make that mistake. But then it only exists since Windows Server 2000, so it would never have worked on Win98/Me . :)
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]
I only read code that is properly formatted, adding PRE tags is the easiest way to obtain that.
-
The point, as I understand it, was that calling
fsB.get_Length(which is what you're doing when you ask the value offsB.Length) may secretly be doing something that takes more than a reasonable amount of time, which is true. As proof, here is the code forFileStream.Lengthpublic override long get_Length()
{
if (this._handle.IsClosed)
{
__Error.FileNotOpen();
}
if (!this.CanSeek)
{
__Error.SeekNotSupported();
}
int highSize = 0;
int fileSize = 0;
fileSize = Win32Native.GetFileSize(this._handle, out highSize);
if (fileSize == -1)
{
int errorCode = Marshal.GetLastWin32Error();
if (errorCode != 0)
{
__Error.WinIOError(errorCode, string.Empty);
}
}
long num4 = (highSize << 0x20) | ((long) ((ulong) fileSize));
if ((this._writePos > 0) && ((this._pos + this._writePos) > num4))
{
num4 = this._writePos + this._pos;
}
return num4;
}Win32Native.GetFileSizeuses P/Invoke to GetFileSize in kernel32.dll Solution: store the length in a local variable (like Luc said)Now I got the idea. Thank you for your answers, this was a really hard thing to spot, for me. :)
-
- Well spotted 2) I looked into it, and IL is OK, C# "disassembly" is wrong. Not the first bug in Reflector. 3) See, I would have used GetFileSizeEx and not have had the opportunity to make that mistake. But then it only exists since Windows Server 2000, so it would never have worked on Win98/Me . :)
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]
I only read code that is properly formatted, adding PRE tags is the easiest way to obtain that.
Thank you all for your help. Now I got the idea and solved the problem. It would take me much longer to spot the cause by myself.