In .NET enumeration is slow
-
Quote:
Linq makes it not a fringe case since it uses enumerations all over the place.
You disliked Linq? Normal code doesn't require Linq? Did you make this microoptimization to prove you're better than Linq?
Quote:
It's also a known documented issue with .NET. I'm far from the first person to make this observation. Folks from Microsoft have said as much.
Is that the basis of your post? Maybe you missed the point; 20 ms is nothing in a background thread, and I would remove your code from any codebase for being unreadable and obfuscating. I gladly pay a few ms for readability, maintenance and fewer bugs. That is implying that there's only .NET devs and that there's no person available who can write in a language that compiles to native. You call those things "libraries".
Quote:
And yes, you're looking for a fight. I'm an adult, which is why I was looking for a discussion. You're clearly not capable of something like that so we're done.
So, that is your defense? You little twat copied MS and you can't handle me saying it ain't so? Thank you for the limitless list of arguments and examples. Wait, there's none? "Folks from MS have said".. and you copied and try to impress. Sorry, but you didn't with your 20 ms. As if all code is a regex-lookup, which you present as if your micro optimization invalidates every foreach. If you want to regex, use a native language. I will
BREAKyour nonsense. I do not care about you copying MS and presenting it as "your" observation. I just puke over the generalization :suss:Bastard Programmer from Hell :suss: "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
*sigh* You're a child. And you can't even code in C or C++. Stop wagging your tiny little peen around like it's something. :laugh:
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
-
Quote:
Linq makes it not a fringe case since it uses enumerations all over the place.
You disliked Linq? Normal code doesn't require Linq? Did you make this microoptimization to prove you're better than Linq?
Quote:
It's also a known documented issue with .NET. I'm far from the first person to make this observation. Folks from Microsoft have said as much.
Is that the basis of your post? Maybe you missed the point; 20 ms is nothing in a background thread, and I would remove your code from any codebase for being unreadable and obfuscating. I gladly pay a few ms for readability, maintenance and fewer bugs. That is implying that there's only .NET devs and that there's no person available who can write in a language that compiles to native. You call those things "libraries".
Quote:
And yes, you're looking for a fight. I'm an adult, which is why I was looking for a discussion. You're clearly not capable of something like that so we're done.
So, that is your defense? You little twat copied MS and you can't handle me saying it ain't so? Thank you for the limitless list of arguments and examples. Wait, there's none? "Folks from MS have said".. and you copied and try to impress. Sorry, but you didn't with your 20 ms. As if all code is a regex-lookup, which you present as if your micro optimization invalidates every foreach. If you want to regex, use a native language. I will
BREAKyour nonsense. I do not care about you copying MS and presenting it as "your" observation. I just puke over the generalization :suss:Bastard Programmer from Hell :suss: "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
You seriously called me a "twat"? What the hell is wrong with you? You are a child. You can't even code in C or C++, and you talk about Delphi like that's a flex. It's a joke. You're a joke, and you're a belligerent clown. I've reported your account because this isn't the first time you've been abusive and hostile to other people.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
-
You seriously called me a "twat"? What the hell is wrong with you? You are a child. You can't even code in C or C++, and you talk about Delphi like that's a flex. It's a joke. You're a joke, and you're a belligerent clown. I've reported your account because this isn't the first time you've been abusive and hostile to other people.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
Quote:
I've reported your account because this isn't the first time you've been abusive and hostile to other people.
I cannot argue that, as it is true. I do not have much patience.
Quote:
You are a child
You meant "childish". :suss:
Bastard Programmer from Hell :suss: "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
-
Quote:
I've reported your account because this isn't the first time you've been abusive and hostile to other people.
I cannot argue that, as it is true. I do not have much patience.
Quote:
You are a child
You meant "childish". :suss:
Bastard Programmer from Hell :suss: "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
I would have been happy to debate this with you. That changed when it became clear to me that weren't interested in actually debating anyone. You came here for a fight. You came here because you wanted to abuse other people. You have some issues, and you are making them the problem of other people here. That's not cool.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
-
> 30% in overall execution time? Does your code consist mostly of enumerations and while loops? In this case yes, in fact I posted the code, which is a regex matching algorithm. > And yes, mucking about a for loop vs enumerable in an VB6 runtime (which .NET IS) is not even a microoptimization, it is purely whining. Write in a goddamn real language if it is that important an link to it from .NET where you need it. I wasn't whining. I was making an observation. You're looking for a fight. I have better things to do with my time. Grow up. Also VB.NET and VB6 runtimes have nothing to do with each other. You don't even know what you're talking about. Funny how arrogance and ignorance go hand in hand so often.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
Regex, now there's your real problem. :laugh: [xkcd: Perl Problems](https://xkcd.com/1171/)
-
I just switched IEnumerable to IList and removed foreach (preferring for) and cut my execution time in my test from 65ms to about 45ms. I've put a stripped down version of the code here. The first argument of each emphasized routine was IEnumerable, is now IList with no foreach. This, ladies and gents, is why I don't like LINQ.
// here this._fa is the target machine we will be parsing.
// parse this or otherwise build it and use it here.
IList initial = FA.FillEpsilonClosure(this._fa);
IList next = new List();
IList states = new List(initial);
// start out with an empty capture buffer
this.capture.Clear();
// first move:
if (this.current == -2)
{
this.Advance();
}
// store the current position
long cursor_pos = this.position;
int line = this.line;
int column = this.column;
while(true) {
// try to transition from states on
// the current codepoint under the
// cursor
next.Clear();
FA.FillMove(states, this.current, next);
if (next.Count > 0)
{
// found at least one transition
// capture the current
// char, advance the input
// position:
this.Advance();
// move to the next states
states.Clear();
FA.FillEpsilonClosure(next, states);
} else {
// no matching transition
// is any current state accepting?
int acc = FA.GetFirstAcceptSymbol(states);
if(acc>-1) {
// accept
return FAMatch.Create(
acc,
this.capture.ToString(),
cursor_pos,
line,
column);
}
// not accepting - error
// keep capturing input until we find a
// valid move or there's no more input
while (this.current != -1 &&
FA.FillMove(initial, this.current).Count == 0)
{
this.Advance();
}
if (capture.Length == 0)
{
// end of input
return FAMatch.Create(-2, null, 0, 0, 0);
}
// error
return FAMatch.Create(-1,
capture.ToString(),
cursor_pos,
line,
column);}}
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
Having written .Net enumerators, yes, it can be slow. It can also be a very powerful abstraction when used properly. Linq on the other hand, is a nightmare to write in VB. C# supports the Lambda syntax while VB doesn't. Personally, most Linq uses can easily be rewritten as a for loop which is easier to read and understand. Some require a function call, but then so do all Linq statements.
-
Quote:
I've reported your account because this isn't the first time you've been abusive and hostile to other people.
I cannot argue that, as it is true. I do not have much patience.
Quote:
You are a child
You meant "childish". :suss:
Bastard Programmer from Hell :suss: "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
Not taking sides, but your posts are so much like the arguments I used to hear from my fellow Navy nucs. I kind of miss it, but we Navy nucs re an odd lot. :)
-
A fringe case, as most of us do not write regex-libs, and anyone who would doesn't use an interpreter but a compiler. No, not a byte code compiler, that's just a fancy marketing shit for an interpreter that doesn't compile to native. And yes, I'm looking for a fight; you are implying that some loop in an interpreter is interesting. It isn't. VB.NET and VB6 are ridiculously the same. I've done that discussion a thousand times, where a manager imagined C# to be superior to VB.NET. It is a different syntax for the same VBRUN300.DLL, a fakkin interpreter that does bytecode like VB6 did with the same memory manager. C# is marketing, but under the hood it is just VB7 with a different style of writing. Which is brilliant from an MS perspective btw, which proves MS is still the best. Now, get off my lawn.
Bastard Programmer from Hell :suss: "If you just follow the bacon Eddy, wherever it leads you, then you won't have to think about politics." -- Some Bell.
"and anyone who would doesn't use an interpreter but a compiler. No, not a byte code compiler, that's just a fancy marketing sh*t for an interpreter that doesn't compile to native." Far as my limited experience goes C#, Java and JavaScript use an interpreter for their regexes. And I know Perl does. I suspect JavaScript like Perl cannot be anything but an interpreter (in most usages.) At least for Java it might at some point go Native. Same could be true of C#.
-
Regex, now there's your real problem. :laugh: [xkcd: Perl Problems](https://xkcd.com/1171/)
-
I just switched IEnumerable to IList and removed foreach (preferring for) and cut my execution time in my test from 65ms to about 45ms. I've put a stripped down version of the code here. The first argument of each emphasized routine was IEnumerable, is now IList with no foreach. This, ladies and gents, is why I don't like LINQ.
// here this._fa is the target machine we will be parsing.
// parse this or otherwise build it and use it here.
IList initial = FA.FillEpsilonClosure(this._fa);
IList next = new List();
IList states = new List(initial);
// start out with an empty capture buffer
this.capture.Clear();
// first move:
if (this.current == -2)
{
this.Advance();
}
// store the current position
long cursor_pos = this.position;
int line = this.line;
int column = this.column;
while(true) {
// try to transition from states on
// the current codepoint under the
// cursor
next.Clear();
FA.FillMove(states, this.current, next);
if (next.Count > 0)
{
// found at least one transition
// capture the current
// char, advance the input
// position:
this.Advance();
// move to the next states
states.Clear();
FA.FillEpsilonClosure(next, states);
} else {
// no matching transition
// is any current state accepting?
int acc = FA.GetFirstAcceptSymbol(states);
if(acc>-1) {
// accept
return FAMatch.Create(
acc,
this.capture.ToString(),
cursor_pos,
line,
column);
}
// not accepting - error
// keep capturing input until we find a
// valid move or there's no more input
while (this.current != -1 &&
FA.FillMove(initial, this.current).Count == 0)
{
this.Advance();
}
if (capture.Length == 0)
{
// end of input
return FAMatch.Create(-2, null, 0, 0, 0);
}
// error
return FAMatch.Create(-1,
capture.ToString(),
cursor_pos,
line,
column);}}
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
Yeah, and it's not like there aren't developers that that have sweated over how to make that particular segment of code as fast as they can. There is a crap ton of flexibility built into IEnumerable that would be difficult to optimize away. I worked on a project years back where that level of optimization made a big difference, and I still used the convenience of LINQ group by, distinct, orderby, etc. to build up hashtables that I used in the optimized sections of code I wrote. Using LINQ helped in the part it was good at.
Curvature of the Mind now with 3D
-
It's not 20ms. it's a 30% improvement in overall execution time. If I increase the test size to run for 90 seconds, it would run for about 60 after the optimization. If you get even 20% off the execution in critical code paths it's generally worth the optimization. I mean, of course it depends on the circumstances, and is less true of business development, or development with large teams or teams with with a lot of turnover, where you can't afford the additional maintenance overhead, limited knowledge transferability and cognitive load of optimized code. That is not a microoptimization. 30% off total execution time is a significant savings. Adding, I used to use garbage collection in my ISAPI applications because it prevented nasty heap fragmentation due to all the string processing required of web servers. It made things faster. GC isn't always a losing performance proposition. When the situation calls for it, it can increase overall performance.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
-
I love GC for jobs that allocate 14 Gigs of memory and then finish. Poof! 14 gigs of memory returned to the OS very quickly. No need to clean the heap.
I mean, if you have it, why not, with a modern damned paged vmem system and gobs of RAM on a modern machine? Spare your program having to garbage collect as often.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
-
I mean, if you have it, why not, with a modern damned paged vmem system and gobs of RAM on a modern machine? Spare your program having to garbage collect as often.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
honey the codewitch wrote:
Spare your program having to garbage collect as often.
So you prefer to collect tons of garbage before doing anything about it? Then you may have some job to do when space runs out ... Nothing wrong to be said about having 'enough' RAM, but if there is anything risk at all of having to garbage collect (and if you use heap allocation at all, there is :-)), I would much prefer to do it in small steps! If you have got plenty of RAM, I'd much rather close my ears to all the whining about the internal fragmentation of buddy allocation (the only serious argument against buddy that I have encountered), to have a very fast allocation / deallocation mechanism, that also lends itself to incremental garbage collection.
Religious freedom is the freedom to say that two plus two make five.
-
honey the codewitch wrote:
Spare your program having to garbage collect as often.
So you prefer to collect tons of garbage before doing anything about it? Then you may have some job to do when space runs out ... Nothing wrong to be said about having 'enough' RAM, but if there is anything risk at all of having to garbage collect (and if you use heap allocation at all, there is :-)), I would much prefer to do it in small steps! If you have got plenty of RAM, I'd much rather close my ears to all the whining about the internal fragmentation of buddy allocation (the only serious argument against buddy that I have encountered), to have a very fast allocation / deallocation mechanism, that also lends itself to incremental garbage collection.
Religious freedom is the freedom to say that two plus two make five.
It depends on what I'm doing. In some scenarios, such as when you don't need to garbage collect but for at the end, such as CLI tools often do then yes, absolutely, because you've finished doing useful work and you don't need to make the user wait for the collection (even if the process is still running at that point you can have written out all of your output and everything.) I write a lot of command line tools that do complicated things, like Deslang: From Code to CodeDOM and Back[^] that absolutely benefit from doing things this way. I should add, that modern GCs collect in the background, and that should perhaps influence one's decision as it's probably less expensive overall to do one large collection than a bunch of little ones, particularly when asynchronicity is involved.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
-
It depends on what I'm doing. In some scenarios, such as when you don't need to garbage collect but for at the end, such as CLI tools often do then yes, absolutely, because you've finished doing useful work and you don't need to make the user wait for the collection (even if the process is still running at that point you can have written out all of your output and everything.) I write a lot of command line tools that do complicated things, like Deslang: From Code to CodeDOM and Back[^] that absolutely benefit from doing things this way. I should add, that modern GCs collect in the background, and that should perhaps influence one's decision as it's probably less expensive overall to do one large collection than a bunch of little ones, particularly when asynchronicity is involved.
Check out my IoT graphics library here: https://honeythecodewitch.com/gfx And my IoT UI/User Experience library here: https://honeythecodewitch.com/uix
If you do buddy with a set of freelist heads, one for each size, and your buddy combiner orders the freelist, you have an extra benefit of locality: most accesses would go to the lower end of the heap, making better use of virtual memory (less paging). A background GC could unhook a freelist (maybe leaving a couple entries in the list for use while the GC was working), returning with one shorter list for the original freelist and one list of combined buddies to be put into the next higher size freelist. The head end of the freelist may be rather unordered - this is where all the allocation and freeing is taking place. If the list is long - it hasn't been emptied for quite some time - the tail end may be perfectly sorted after the previous GC/combination round. If you do sorting e.g. by Smoothsort, handling the already sorted part has complexity O(n), so most likely, the long freelist will not required much effort. You find buddies by traversing a sorted list, so the list of buddy pairs will also be sorted. If the next higher freelist is also mostly sorted, all buddy pairs is inserted into this is list in a single traversal. I would do real timing tests with a synthetic heap load (modeled after a relevant usage scenario) to see if it really is worthwhile the resource cost of an asynchronous GC thread - strongly suspecting that a finely tuned incremental but synchronous buddy manager can do it both at a lower total resource cost and with so small delays that it would be a much better solution. Final remark: "you've finished doing useful work and you don't need to make the user wait for the collection". In most systems, each process has its own heap. Multiple processes allocating from one common global heap requires a lot of resource consuming synchronization. Most CLI programs are run in their own processes. So when they complete, noone cares about what their heap looks like at that time. There is no reason to do any garbage collection at that time. The entire data segment holding the heap is released en bloc. In an embedded system, you often have a single systemwide heap. But few embedded system have CLI interfaces for running arbitrary programs that start up and terminate as a function of user operations. Even if the embedded system has some sort of UI, user actions are usually limited to activating specific built-in operations in the embedded code, not separate CLI oriented programs. But of course, there may be exceptions :-)
Religious freedom is the freedom to say th