.NET 6.0 is Slower than .NET Framework In Some String Operations
-
Eddy Vluggen wrote:
Note the #1 in all tests.
Thank you. I will test it it. And C Language seems to be a great choice. Or Golang. Or Rust.
C because it close to assembly. Golang or rust aren't competing there. If it is a core function of what you do, then it'd make sense; eliminate the dependency.
Bastard Programmer from Hell :suss: "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
-
Some tests to check some functions not tested in link below: Performance Improvements in .NET 6 - .NET Blog (arrays-strings-spans) .NET 6.0 is (comparing its performance to .NET Framework 4.8): • 3x faster on String.Replace operations • 16x slower on String.IndexOf operations, • 1.4x faster on String.Substring operations • The same on String.Remove operations Copy paste the code below and compile with .NET 6.0 and .NET Framework 4.8 and see by yourself.
//Simple Benchmark test for working with Strings in different versions of .NET Framework
string test = "Lorem Ipsum is simply dummy text" +
" of the printing and typesetting industry. " +
"Lorem Ipsum has been the industry's " +
"standard dummy text ever since the 1500s, " +
"when an unknown printer took a galley " +
"of type and scrambled it to make a type specimen book. " +
"It has survived not only " +
"five centuries, but also the leap into electronic typesetting," +
" remaining essentially unchanged." +
" It was popularised in the 1960s with the release" +
" of Letraset sheets containing Lorem Ipsum passages," +
" and more recently with desktop publishing software like " +
"Aldus PageMaker including versions of Lorem Ipsum.";System.Diagnostics.Stopwatch K = new System.Diagnostics.Stopwatch();
K.Reset(); K.Start();
for (var v = 1; v <= 10000000; v++)
{
test = test.Replace("a", "bla bla bla bla");
test = test.Replace("bla bla bla bla", "a");
}
K.Stop();
System.Console.WriteLine($"Elapsed Time for [String.Replace]: {K.Elapsed.TotalSeconds} sec");K.Reset(); K.Start();
for (var v = 1; v <= 1000000; v++)
{
int i = test.IndexOf("including versions of Lorem Ipsum");
}
K.Stop();
System.Console.WriteLine($"Elapsed Time for [String.IndexOf]: {K.Elapsed.TotalSeconds} sec");K.Reset(); K.Start();
for (var v = 1; v <= 600000000; v++)
{
var s = test.Substring(25, 50);
}
K.Stop();
System.Console.WriteLine($"Elapsed Time for [String.SubString]: {K.Elapsed.TotalSeconds} sec");K.Reset(); K.Start();
for (var v = 1; v <= 90000000; v++)
{
var s = test.Remove(45, 60);
}
K.Stop();
System.Console.WriteLine($"Elapsed Time for [String.Remove]: {K.Elapsed.TotalSeconds} sec");System.Console.WriteLine("Press a key to exit...");
System.Console.ReadKey();You got a really weird number at String.IndexOf. Here are my results:
Elapsed Time for [String.Replace]: 16,8695848 sec
Elapsed Time for [String.IndexOf]: 2,0153508 sec
Elapsed Time for [String.SubString]: 11,7445885 sec
Elapsed Time for [String.Remove]: 10,1579051 sec.NET 6 is 1.7x up to 2.9x faster. I think these numbers are pretty consistent.
-
Some tests to check some functions not tested in link below: Performance Improvements in .NET 6 - .NET Blog (arrays-strings-spans) .NET 6.0 is (comparing its performance to .NET Framework 4.8): • 3x faster on String.Replace operations • 16x slower on String.IndexOf operations, • 1.4x faster on String.Substring operations • The same on String.Remove operations Copy paste the code below and compile with .NET 6.0 and .NET Framework 4.8 and see by yourself.
//Simple Benchmark test for working with Strings in different versions of .NET Framework
string test = "Lorem Ipsum is simply dummy text" +
" of the printing and typesetting industry. " +
"Lorem Ipsum has been the industry's " +
"standard dummy text ever since the 1500s, " +
"when an unknown printer took a galley " +
"of type and scrambled it to make a type specimen book. " +
"It has survived not only " +
"five centuries, but also the leap into electronic typesetting," +
" remaining essentially unchanged." +
" It was popularised in the 1960s with the release" +
" of Letraset sheets containing Lorem Ipsum passages," +
" and more recently with desktop publishing software like " +
"Aldus PageMaker including versions of Lorem Ipsum.";System.Diagnostics.Stopwatch K = new System.Diagnostics.Stopwatch();
K.Reset(); K.Start();
for (var v = 1; v <= 10000000; v++)
{
test = test.Replace("a", "bla bla bla bla");
test = test.Replace("bla bla bla bla", "a");
}
K.Stop();
System.Console.WriteLine($"Elapsed Time for [String.Replace]: {K.Elapsed.TotalSeconds} sec");K.Reset(); K.Start();
for (var v = 1; v <= 1000000; v++)
{
int i = test.IndexOf("including versions of Lorem Ipsum");
}
K.Stop();
System.Console.WriteLine($"Elapsed Time for [String.IndexOf]: {K.Elapsed.TotalSeconds} sec");K.Reset(); K.Start();
for (var v = 1; v <= 600000000; v++)
{
var s = test.Substring(25, 50);
}
K.Stop();
System.Console.WriteLine($"Elapsed Time for [String.SubString]: {K.Elapsed.TotalSeconds} sec");K.Reset(); K.Start();
for (var v = 1; v <= 90000000; v++)
{
var s = test.Remove(45, 60);
}
K.Stop();
System.Console.WriteLine($"Elapsed Time for [String.Remove]: {K.Elapsed.TotalSeconds} sec");System.Console.WriteLine("Press a key to exit...");
System.Console.ReadKey();Prior to .NET 5, culture-specific comparisons used NLS[^] on Windows. Since .NET 5, they switched to using ICU[^] instead. Globalization and ICU | Microsoft Docs[^] There is a config switch to force .NET to use NLS instead, but it's not recommended: Breaking change with string.IndexOf(string) from .NET Core 3.0 -> .NET 5.0[^] If you change your
IndexOfcall to specifyStringComparison.OrdinalorStringComparison.OrdinalIgnoreCase, the .NET 6 code is roughly 1.5x faster than the .NET Framework 4.8 equivalent.
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
-
C because it close to assembly. Golang or rust aren't competing there. If it is a core function of what you do, then it'd make sense; eliminate the dependency.
Bastard Programmer from Hell :suss: "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
Machine code generated from Rust usually levels with C in terms of raw performance, often outperforming it
-
C because it close to assembly. Golang or rust aren't competing there. If it is a core function of what you do, then it'd make sense; eliminate the dependency.
Bastard Programmer from Hell :suss: "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
I'd be interested to learn why "rust isn't competing here".
-
Pete O'Hanlon wrote:
It's advisable to use a real benchmarking framework
Not in this case, StopWatch is sufficient.
georani wrote:
Not in this case, StopWatch is sufficient.
Benchmark.NET makes sure that code is warmed up properly, eliminating simple things like tiered compilation whose defaults may have changed between .NET versions, for example. It also makes it fairly trivial to compare different runtimes: it is, after all, the tool that Microsoft itself uses for comparison of runtimes (to consciously decide when performance regressions are acceptable).
-
Some tests to check some functions not tested in link below: Performance Improvements in .NET 6 - .NET Blog (arrays-strings-spans) .NET 6.0 is (comparing its performance to .NET Framework 4.8): • 3x faster on String.Replace operations • 16x slower on String.IndexOf operations, • 1.4x faster on String.Substring operations • The same on String.Remove operations Copy paste the code below and compile with .NET 6.0 and .NET Framework 4.8 and see by yourself.
//Simple Benchmark test for working with Strings in different versions of .NET Framework
string test = "Lorem Ipsum is simply dummy text" +
" of the printing and typesetting industry. " +
"Lorem Ipsum has been the industry's " +
"standard dummy text ever since the 1500s, " +
"when an unknown printer took a galley " +
"of type and scrambled it to make a type specimen book. " +
"It has survived not only " +
"five centuries, but also the leap into electronic typesetting," +
" remaining essentially unchanged." +
" It was popularised in the 1960s with the release" +
" of Letraset sheets containing Lorem Ipsum passages," +
" and more recently with desktop publishing software like " +
"Aldus PageMaker including versions of Lorem Ipsum.";System.Diagnostics.Stopwatch K = new System.Diagnostics.Stopwatch();
K.Reset(); K.Start();
for (var v = 1; v <= 10000000; v++)
{
test = test.Replace("a", "bla bla bla bla");
test = test.Replace("bla bla bla bla", "a");
}
K.Stop();
System.Console.WriteLine($"Elapsed Time for [String.Replace]: {K.Elapsed.TotalSeconds} sec");K.Reset(); K.Start();
for (var v = 1; v <= 1000000; v++)
{
int i = test.IndexOf("including versions of Lorem Ipsum");
}
K.Stop();
System.Console.WriteLine($"Elapsed Time for [String.IndexOf]: {K.Elapsed.TotalSeconds} sec");K.Reset(); K.Start();
for (var v = 1; v <= 600000000; v++)
{
var s = test.Substring(25, 50);
}
K.Stop();
System.Console.WriteLine($"Elapsed Time for [String.SubString]: {K.Elapsed.TotalSeconds} sec");K.Reset(); K.Start();
for (var v = 1; v <= 90000000; v++)
{
var s = test.Remove(45, 60);
}
K.Stop();
System.Console.WriteLine($"Elapsed Time for [String.Remove]: {K.Elapsed.TotalSeconds} sec");System.Console.WriteLine("Press a key to exit...");
System.Console.ReadKey();I can reproduce the nature (if not the exact numbers) of these results using BenchmarkDotNet and the same code being benchmarked (i.e. the loop bodies). I added a fifth benchmark for
IndexOfwithStringComparison.Ordinal.// * Summary *
BenchmarkDotNet=v0.13.1, OS=Windows 10.0.18363.2274 (1909/November2019Update/19H2)
Intel Core i9-9980HK CPU 2.40GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK=6.0.202
[Host] : .NET 6.0.4 (6.0.422.16404), X64 RyuJIT
.NET 6.0 : .NET 6.0.4 (6.0.422.16404), X64 RyuJIT
.NET Framework 4.8 : .NET Framework 4.8 (4.8.4510.0), X64 RyuJITMethod Job Runtime Mean Error StdDev Ratio RatioSD Replace .NET 6.0 .NET 6.0 1,350.99 ns 7.502 ns 7.017 ns 0.33 0.01 Replace .NET Framework 4.8 .NET Framework 4.8 4,150.69 ns 80.004 ns 82.158 ns 1.00 0.00 IndexOf .NET 6.0 .NET 6.0 22,561.81 ns 435.204 ns 446.923 ns 10.44 0.34 IndexOf .NET Framework 4.8 .NET Framework 4.8 2,189.63 ns 43.739 ns 88.354 ns 1.00 0.00 IndexOfOrdinal .NET 6.0 .NET 6.0 261.22 ns 2.760 ns 2.446 ns 0.65 0.02 IndexOfOrdinal .NET Framework 4.8 .NET Framework 4.8 403.50 ns 7.982 ns 9.502 ns 1.00 0.00 Substring .NET 6.0 .NET 6.0 12.12 ns 0.310 ns 0.413 ns 0.93 0.05 Substring .NET Framework 4.8 .NET Framework 4.8 13.07 ns 0.326 ns 0.563 ns 1.00 0.00 Remove .NET 6.0 .NET 6.0 64.00 ns 1.208 ns 1.770 ns 1.09 0.04 Remove .NET Framework 4.8 .NET Framework 4.8 59.23 ns 1.020 -
Prior to .NET 5, culture-specific comparisons used NLS[^] on Windows. Since .NET 5, they switched to using ICU[^] instead. Globalization and ICU | Microsoft Docs[^] There is a config switch to force .NET to use NLS instead, but it's not recommended: Breaking change with string.IndexOf(string) from .NET Core 3.0 -> .NET 5.0[^] If you change your
IndexOfcall to specifyStringComparison.OrdinalorStringComparison.OrdinalIgnoreCase, the .NET 6 code is roughly 1.5x faster than the .NET Framework 4.8 equivalent.
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
likly this. core 3.1 default basic just fine, and then 5.1 is playing a different sport. This sounds like same issue with defaults switching from Newtonsoft JSON to System.Text that Text uses case sensitive name matching :mad:
\[Replace\]: \[IndexOf\]: \[SubString\]: \[Remove\]:core 3.1 run 1 21.7278133 sec 1.7657007 sec 10.897253 sec 7.2913159 sec
core 3.1 run 2 17.5640173 sec 1.4541324 sec 8.3179752 sec 6.0721578 sec
core 5.0 run 1 14.7799258 sec 20.315649 sec 7.6012973 sec 5.9379139 sec
core 5.0 run 2 15.2000948 sec 26.041709 sec 10.142171 sec 7.3596202 sec
core 6.0 run 1 10.7217398 sec 18.049464 sec 7.5585288 sec 7.5387282 sec
core 6.0 run 2 10.6481228 sec 17.215285 sec 7.5210471 sec 6.4885804 sec
fw 4.8 run 1 33.0995089 sec 1.7132278 sec 9.5126196 sec 6.9524378 sec
fw 4.8 run 2 28.0019003 sec 1.4240000 sec 7.9734736 sec 5.6638693 sec -
Some tests to check some functions not tested in link below: Performance Improvements in .NET 6 - .NET Blog (arrays-strings-spans) .NET 6.0 is (comparing its performance to .NET Framework 4.8): • 3x faster on String.Replace operations • 16x slower on String.IndexOf operations, • 1.4x faster on String.Substring operations • The same on String.Remove operations Copy paste the code below and compile with .NET 6.0 and .NET Framework 4.8 and see by yourself.
//Simple Benchmark test for working with Strings in different versions of .NET Framework
string test = "Lorem Ipsum is simply dummy text" +
" of the printing and typesetting industry. " +
"Lorem Ipsum has been the industry's " +
"standard dummy text ever since the 1500s, " +
"when an unknown printer took a galley " +
"of type and scrambled it to make a type specimen book. " +
"It has survived not only " +
"five centuries, but also the leap into electronic typesetting," +
" remaining essentially unchanged." +
" It was popularised in the 1960s with the release" +
" of Letraset sheets containing Lorem Ipsum passages," +
" and more recently with desktop publishing software like " +
"Aldus PageMaker including versions of Lorem Ipsum.";System.Diagnostics.Stopwatch K = new System.Diagnostics.Stopwatch();
K.Reset(); K.Start();
for (var v = 1; v <= 10000000; v++)
{
test = test.Replace("a", "bla bla bla bla");
test = test.Replace("bla bla bla bla", "a");
}
K.Stop();
System.Console.WriteLine($"Elapsed Time for [String.Replace]: {K.Elapsed.TotalSeconds} sec");K.Reset(); K.Start();
for (var v = 1; v <= 1000000; v++)
{
int i = test.IndexOf("including versions of Lorem Ipsum");
}
K.Stop();
System.Console.WriteLine($"Elapsed Time for [String.IndexOf]: {K.Elapsed.TotalSeconds} sec");K.Reset(); K.Start();
for (var v = 1; v <= 600000000; v++)
{
var s = test.Substring(25, 50);
}
K.Stop();
System.Console.WriteLine($"Elapsed Time for [String.SubString]: {K.Elapsed.TotalSeconds} sec");K.Reset(); K.Start();
for (var v = 1; v <= 90000000; v++)
{
var s = test.Remove(45, 60);
}
K.Stop();
System.Console.WriteLine($"Elapsed Time for [String.Remove]: {K.Elapsed.TotalSeconds} sec");System.Console.WriteLine("Press a key to exit...");
System.Console.ReadKey();Here's the relevant discussion thread on the dotnet GitHub repo: List of performance regressions caused by switching to ICU · Issue #40942 · dotnet/runtime · GitHub[^]
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
-
I'd be interested to learn why "rust isn't competing here".
-
Machine code generated from Rust usually levels with C in terms of raw performance, often outperforming it
-
It's a LLVM. Running in a fictional machine. On top of a real machine. :)
Bastard Programmer from Hell :suss: "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
LLVM is not a VM, but a (native) language compiler. Rust compiles to native code just as C and C++ do, and it uses LLVM as the compiler back-end. Rust will frequently be faster that C or C++ for implementing the same algorithm, and sometimes a bit slower.
-
Hi, Can you run the
String.IndexOfbenchmark again usingStringComparison.Ordinaland post the result?Randor wrote:
Can you run the String.IndexOf benchmark again using StringComparison.Ordinal and post the result?
Thanks, it is the solution: Now .NET 6.0 is 2X times FASTER than .NET Framework 4.8
Starting tests for .NET Framework 3.5, wait...
Elapsed Time for [String.Replace]: 51,0830947 sec
Elapsed Time for [String.IndexOf]: 0,5451713 sec
Elapsed Time for [String.Contains]: 0,5424978 sec
Elapsed Time for [String.SubString]: 17,7345287 sec
Elapsed Time for [String.Remove]: 15,021131 secElapsed Time for [.NET 3.5]: 84,9272387 sec
Starting tests for .NET Framework 4.8, wait...
Elapsed Time for [String.Replace]: 53,2363039 sec
Elapsed Time for [String.IndexOf]: 0,5760411 sec
Elapsed Time for [String.Contains]: 0,595356 sec
Elapsed Time for [String.SubString]: 14,8117435 sec
Elapsed Time for [String.Remove]: 11,2053769 secElapsed Time for [.NET 4.8]: 80,4258634 sec
Starting tests for .NET 6.0, wait...
Elapsed Time for [String.Replace]: 18,5094685 sec
Elapsed Time for [String.IndexOf]: 0,3495122 sec
Elapsed Time for [String.Contains]: 0,32621 sec
Elapsed Time for [String.SubString]: 12,1062252 sec
Elapsed Time for [String.Remove]: 10,1427749 secElapsed Time for [.NET 6.0]: 41,4367067 sec
-
Randor wrote:
Can you run the String.IndexOf benchmark again using StringComparison.Ordinal and post the result?
Thanks, it is the solution: Now .NET 6.0 is 2X times FASTER than .NET Framework 4.8
Starting tests for .NET Framework 3.5, wait...
Elapsed Time for [String.Replace]: 51,0830947 sec
Elapsed Time for [String.IndexOf]: 0,5451713 sec
Elapsed Time for [String.Contains]: 0,5424978 sec
Elapsed Time for [String.SubString]: 17,7345287 sec
Elapsed Time for [String.Remove]: 15,021131 secElapsed Time for [.NET 3.5]: 84,9272387 sec
Starting tests for .NET Framework 4.8, wait...
Elapsed Time for [String.Replace]: 53,2363039 sec
Elapsed Time for [String.IndexOf]: 0,5760411 sec
Elapsed Time for [String.Contains]: 0,595356 sec
Elapsed Time for [String.SubString]: 14,8117435 sec
Elapsed Time for [String.Remove]: 11,2053769 secElapsed Time for [.NET 4.8]: 80,4258634 sec
Starting tests for .NET 6.0, wait...
Elapsed Time for [String.Replace]: 18,5094685 sec
Elapsed Time for [String.IndexOf]: 0,3495122 sec
Elapsed Time for [String.Contains]: 0,32621 sec
Elapsed Time for [String.SubString]: 12,1062252 sec
Elapsed Time for [String.Remove]: 10,1427749 secElapsed Time for [.NET 6.0]: 41,4367067 sec
