Unpopular opinions: LINQ
-
> Object instantiation is cheap, or so I've been told. It can be (depends on the object of course), but the problem starts when the garbage collector comes calling
Right? I didn't want to get into it and potentially start an argument over GC arcana, but basically the concept behind a GC isn't so much that you save on allocations, but you pay for them after-the-fact. I recently had a project that needed fast pointer increment allocation like a GC has but I didn't want to pay for the object sweeping so I simply made it so my little heaps could be defined to a fixed size (of which allocations would come out of) and you couldn't delete objects at all. You could clear the entire heap in one sweep, invalidating all the data at once though. Practically free, and the use case was such that it handled everything I needed. GCs aren't all that and a bag of chips. :) But then i'm not telling you anything you don't already know. *hides from @SanderRossel*
Real programmers use butterflies
-
Assuming PLINQ's implementation is not terrible, you're probably incurring locking overhead. It doesn't make sense to try to use any kind of parallelization in the following scenarios a) your problem has interdependent components such that you can't decouple the work done by B from the result of A and C depends on the result of both, so you're elephanted. b) it doesn't do you a heck of a lot of good to query the same source in parallel with itself. It's hard to give you a good example in PLINQ but you want parallel op A to use a different datasource than B. In an RDBMS this principle is easier to understand. If I run a join across two tables, i don't have a lot i can do to make it parallel *unless* each table is on a separate drive ("spindle" in DB parlance) meaning the read operations of table A aren't dependent on waiting for read operations from B since they are different drive controllers working in parallel. The same basic idea would apply to PLINQ If a or b are an issue, you'll probably end up incurring more overhead than you gain in throughput
Real programmers use butterflies
I tried with some code I have from my long past Physics PhD that integrate some equation over time... I have a multidimensional field and each dimension was in its own thread... Mmm... come to think of it now, there is coupling between some variable I think, I wonder if it was the cause of the slow down... no matter.. not sure where this code even is now ^^
A new .NET Serializer All in one Menu-Ribbon Bar Taking over the world since 1371!
-
I tried with some code I have from my long past Physics PhD that integrate some equation over time... I have a multidimensional field and each dimension was in its own thread... Mmm... come to think of it now, there is coupling between some variable I think, I wonder if it was the cause of the slow down... no matter.. not sure where this code even is now ^^
A new .NET Serializer All in one Menu-Ribbon Bar Taking over the world since 1371!
It's very likely. It can be really easy to miss interdependencies in formulas.
Real programmers use butterflies
-
That's like putting it on steak! C++ i have mixed feelings about. I love the flexibility. Almost every feature has an actual purpose. You'd think that wouldn't be an exceptional thing, but consider C#8 and parameter "deconstruction" and all that other nonsense we don't need. But C++ has gotten so complicated that it's about as bad C# has gotten if not worse now. So even if it is purposeful, it's hell to navigate.
Real programmers use butterflies
I like how it was phrased in one of the Star Trek movies. I think it was #3. Anyway, the president of the Federation said, "Let us define progress to mean just because we can do a thing does not mean we must do that thing." I take that approach to a large amount of stuff when it comes to programming. Especially new language features. So far, I have seen exactly one thing in C++17 that I really, really like and use often : inline static variables with initialization.
"They have a consciousness, they have a life, they have a soul! Damn you! Let the rabbits wear glasses! Save our brothers! Can I get an amen?"
-
I like how it was phrased in one of the Star Trek movies. I think it was #3. Anyway, the president of the Federation said, "Let us define progress to mean just because we can do a thing does not mean we must do that thing." I take that approach to a large amount of stuff when it comes to programming. Especially new language features. So far, I have seen exactly one thing in C++17 that I really, really like and use often : inline static variables with initialization.
"They have a consciousness, they have a life, they have a soul! Damn you! Let the rabbits wear glasses! Save our brothers! Can I get an amen?"
I hear you. The one "must have" feature in C++ that isn't actually in C++ but should be is real forward type declarations. Remove the restriction where I can only declare things like a pointer to a type before the type itself is fully declared. It might require using a GLR parser or something equally complicated to parse your code (like C# does) but given the complexity of a standards compliant C++ compiler already, what's changing the parsing engine used by most C++ compilers to something more advanced in the big scheme of things? Basically to bottom line it, I want to be able to declare my types anywhere in my files and use them anywhere in my files.
Real programmers use butterflies
-
I'm not a fan of LINQ. I love functional programming but .NET's enumerator paradigm is not up to the task. It creates too many objects too quickly to be a real grown up functional language, whose iteration is highly optimized because it's a first class operation. I've benched LINQ against hand written pseudo-functional operations that do the same thing. It was not encouraging. For things that make heavy use of functional computation like parser generators, where your LINQ query might be half a page, it's a Bad Idea(TM) Worse, I think its use has been over encouraged by Microsoft. It makes green developers write even worse code, and makes it harder for a seasoned developer to understand the performance implications of the code they are writing (and I'm not talking about bit twiddling here, I'm talking about figuring out your Big O expression) I tend to avoid its use, preferring - at least in C# - to make my iteration operations explicit and long hand. If .NET had truly optimized iteration paradigm - one that didn't create new objects for every single iteration operation** - i might consider using it. ** yes i understand that LINQ combines multiple operations into a single iteration *sometimes* - in practice it's not often enough to make up for the overhead of enumerators. Now, there's a case where all of the above doesn't matter, and that's PLINQ. Theoretically, for a large enough operation, that can be highly parallelized, the overhead of enumerators suddenly isn't the biggest part of the performance equation. What I mean is it essentially pays for itself. Also, given the issues with synchronization and other cross task communication (is your operation clustered over a network?) enumerators are actually not a bad idea since you can lock behind them or RPC behind them. Contrast that with C++ iterators that are usually lightly wrapped pointer ops and you realize their limitations fast: In order to enable all of the stuff you need to make iteration operations work with each other in parallel you have to wrap every iterator operator anyway, making it as "heavy" as an enumerator in .NET, not counting the general overhead of running managed code. So basically, PLINQ is where LINQ finally covers its costs - where it reaches the point where its advantages outweigh its disadvantages. All of this of course, is one developer's opinion. And some of this doesn't necessarily apply to business software, where performance almost doesn't matter for most scenarios.
Real programmers
Is LINQ bad'ish or orders of magnitude slower, than those hand-written operations? I ask because I wonder about performance implications myself, while also regarding code read-/maintainability (after all, if performance was all I cared about, I'd hand-optimize everything in assembly anyway, engineering is all about trade-offs).
-
Sander Rossel wrote:
For my customers a few extra objects, iterations and even MB's are no issue at all (they don't even know it exists), but my hourly rate is :-D
Yeah I can understand that, but also I'm glad that's not my situation. I like having to cram as much functionality I can into modest hardware.
Real programmers use butterflies
My customers like having me cram as much functionality as I can into modest invoices :laugh:
Best, Sander Azure DevOps Succinctly (free eBook) Azure Serverless Succinctly (free eBook) Migrating Apps to the Cloud with Azure arrgh.js - Bringing LINQ to JavaScript
-
I'm not a fan of LINQ. I love functional programming but .NET's enumerator paradigm is not up to the task. It creates too many objects too quickly to be a real grown up functional language, whose iteration is highly optimized because it's a first class operation. I've benched LINQ against hand written pseudo-functional operations that do the same thing. It was not encouraging. For things that make heavy use of functional computation like parser generators, where your LINQ query might be half a page, it's a Bad Idea(TM) Worse, I think its use has been over encouraged by Microsoft. It makes green developers write even worse code, and makes it harder for a seasoned developer to understand the performance implications of the code they are writing (and I'm not talking about bit twiddling here, I'm talking about figuring out your Big O expression) I tend to avoid its use, preferring - at least in C# - to make my iteration operations explicit and long hand. If .NET had truly optimized iteration paradigm - one that didn't create new objects for every single iteration operation** - i might consider using it. ** yes i understand that LINQ combines multiple operations into a single iteration *sometimes* - in practice it's not often enough to make up for the overhead of enumerators. Now, there's a case where all of the above doesn't matter, and that's PLINQ. Theoretically, for a large enough operation, that can be highly parallelized, the overhead of enumerators suddenly isn't the biggest part of the performance equation. What I mean is it essentially pays for itself. Also, given the issues with synchronization and other cross task communication (is your operation clustered over a network?) enumerators are actually not a bad idea since you can lock behind them or RPC behind them. Contrast that with C++ iterators that are usually lightly wrapped pointer ops and you realize their limitations fast: In order to enable all of the stuff you need to make iteration operations work with each other in parallel you have to wrap every iterator operator anyway, making it as "heavy" as an enumerator in .NET, not counting the general overhead of running managed code. So basically, PLINQ is where LINQ finally covers its costs - where it reaches the point where its advantages outweigh its disadvantages. All of this of course, is one developer's opinion. And some of this doesn't necessarily apply to business software, where performance almost doesn't matter for most scenarios.
Real programmers
So true.
-
That's like putting it on steak! C++ i have mixed feelings about. I love the flexibility. Almost every feature has an actual purpose. You'd think that wouldn't be an exceptional thing, but consider C#8 and parameter "deconstruction" and all that other nonsense we don't need. But C++ has gotten so complicated that it's about as bad C# has gotten if not worse now. So even if it is purposeful, it's hell to navigate.
Real programmers use butterflies
I wouldn't say C++ has gotten more complicated. Au contraire - I would say it's never been as easy to learn and use as it is now, I almost never have to rely on raw pointers, manual memory management, and all the stuff that C++ haters love. The problem of C++ is that it has become so feature rich, that it's virtually impossible to know all of it, and there really are a lot of features, which, although I'm sure somebody out there uses, but I don't see them as belonging in the standard. It's just so freaking huge. But, of course, you don't really need to know and use all of it. Unless you're a C++ compiler developer.
-
I'm not a fan of LINQ. I love functional programming but .NET's enumerator paradigm is not up to the task. It creates too many objects too quickly to be a real grown up functional language, whose iteration is highly optimized because it's a first class operation. I've benched LINQ against hand written pseudo-functional operations that do the same thing. It was not encouraging. For things that make heavy use of functional computation like parser generators, where your LINQ query might be half a page, it's a Bad Idea(TM) Worse, I think its use has been over encouraged by Microsoft. It makes green developers write even worse code, and makes it harder for a seasoned developer to understand the performance implications of the code they are writing (and I'm not talking about bit twiddling here, I'm talking about figuring out your Big O expression) I tend to avoid its use, preferring - at least in C# - to make my iteration operations explicit and long hand. If .NET had truly optimized iteration paradigm - one that didn't create new objects for every single iteration operation** - i might consider using it. ** yes i understand that LINQ combines multiple operations into a single iteration *sometimes* - in practice it's not often enough to make up for the overhead of enumerators. Now, there's a case where all of the above doesn't matter, and that's PLINQ. Theoretically, for a large enough operation, that can be highly parallelized, the overhead of enumerators suddenly isn't the biggest part of the performance equation. What I mean is it essentially pays for itself. Also, given the issues with synchronization and other cross task communication (is your operation clustered over a network?) enumerators are actually not a bad idea since you can lock behind them or RPC behind them. Contrast that with C++ iterators that are usually lightly wrapped pointer ops and you realize their limitations fast: In order to enable all of the stuff you need to make iteration operations work with each other in parallel you have to wrap every iterator operator anyway, making it as "heavy" as an enumerator in .NET, not counting the general overhead of running managed code. So basically, PLINQ is where LINQ finally covers its costs - where it reaches the point where its advantages outweigh its disadvantages. All of this of course, is one developer's opinion. And some of this doesn't necessarily apply to business software, where performance almost doesn't matter for most scenarios.
Real programmers
There is only one language integrated query, and it is scatter memvar / gather memvar.
-
I wouldn't say C++ has gotten more complicated. Au contraire - I would say it's never been as easy to learn and use as it is now, I almost never have to rely on raw pointers, manual memory management, and all the stuff that C++ haters love. The problem of C++ is that it has become so feature rich, that it's virtually impossible to know all of it, and there really are a lot of features, which, although I'm sure somebody out there uses, but I don't see them as belonging in the standard. It's just so freaking huge. But, of course, you don't really need to know and use all of it. Unless you're a C++ compiler developer.
afigegoznaet wrote:
But, of course, you don't really need to know and use all of it. Unless you're a C++ compiler developer.
Or need to read someone else's code (or your own, later). Oops.
-
afigegoznaet wrote:
But, of course, you don't really need to know and use all of it. Unless you're a C++ compiler developer.
Or need to read someone else's code (or your own, later). Oops.
Well, that's not really language specific. I find myself reading online documentation even when reading bash scripts.
-
My customers like having me cram as much functionality as I can into modest invoices :laugh:
Best, Sander Azure DevOps Succinctly (free eBook) Azure Serverless Succinctly (free eBook) Migrating Apps to the Cloud with Azure arrgh.js - Bringing LINQ to JavaScript
-
Well, that's not really language specific. I find myself reading online documentation even when reading bash scripts.
Hey now. Bash scripts get arcane. I just got done writing a "bash course" for a client of mine (technically a client of an outfit i work for from time to time) who is trying to educate their own customers on how to use it. Talk to me after you've passed what bash calls an "array" to a function. :laugh:
Real programmers use butterflies
-
Is LINQ bad'ish or orders of magnitude slower, than those hand-written operations? I ask because I wonder about performance implications myself, while also regarding code read-/maintainability (after all, if performance was all I cared about, I'd hand-optimize everything in assembly anyway, engineering is all about trade-offs).
Usually it's not terrible. Not orders of magnitude slower for what most people seem to use it for - queries in business software. However, don't use it for what I'd call "real" functional programming. If you're going to write a parser generator or scanner generator for example, you don't want to compute your tables using linq. In that case it *will* be orders of magnitude slower than most anything you could write by hand. And I guess now you can tell what kind of software I write. :laugh:
Real programmers use butterflies
-
I wouldn't say C++ has gotten more complicated. Au contraire - I would say it's never been as easy to learn and use as it is now, I almost never have to rely on raw pointers, manual memory management, and all the stuff that C++ haters love. The problem of C++ is that it has become so feature rich, that it's virtually impossible to know all of it, and there really are a lot of features, which, although I'm sure somebody out there uses, but I don't see them as belonging in the standard. It's just so freaking huge. But, of course, you don't really need to know and use all of it. Unless you're a C++ compiler developer.
I'd argue that this *is* added complexity. Every layer you use to abstract away pointers and other C++ arcana removes you that much further from what your code is doing. And you know, std::auto_ptr works fantastic, until it doesn't. And when it doesn't you better know exactly how it's spec'd to function. Things can be complicated and easy to use at the same time. Hell, that describes the average IoT device these days.
Real programmers use butterflies
-
I'd argue that this *is* added complexity. Every layer you use to abstract away pointers and other C++ arcana removes you that much further from what your code is doing. And you know, std::auto_ptr works fantastic, until it doesn't. And when it doesn't you better know exactly how it's spec'd to function. Things can be complicated and easy to use at the same time. Hell, that describes the average IoT device these days.
Real programmers use butterflies
Well, no. It is added complexity if you don't know how to use raw pointers. If you do - you really appreciate the simplicity added by RAII, smart_ptrs and optionals, RAII makes sense in any language, actually. P. S. std::auto_ptr, really?
-
I hear you. The one "must have" feature in C++ that isn't actually in C++ but should be is real forward type declarations. Remove the restriction where I can only declare things like a pointer to a type before the type itself is fully declared. It might require using a GLR parser or something equally complicated to parse your code (like C# does) but given the complexity of a standards compliant C++ compiler already, what's changing the parsing engine used by most C++ compilers to something more advanced in the big scheme of things? Basically to bottom line it, I want to be able to declare my types anywhere in my files and use them anywhere in my files.
Real programmers use butterflies
You mean like this?: class FWD; struct STR { FWD *fwdPtr; }; int main() { STR tst; return 0; }
-
You mean like this?: class FWD; struct STR { FWD *fwdPtr; }; int main() { STR tst; return 0; }
No. I mean like this:
struct myForward;
std::unordered_map m;
...Except I'd even go as far as to eliminate the forward declaration entirely. The above won't work, BTW on most implementations of the STL, although there's nothing in the spec that says it won't work but nor does it guarantee it does. It works in Boost's unordered_map, but because they went out of their way to make it work in how they implemented unordered_map. Strangely enough, the above will usually work if you replace it with std::map
Real programmers use butterflies
-
Well, no. It is added complexity if you don't know how to use raw pointers. If you do - you really appreciate the simplicity added by RAII, smart_ptrs and optionals, RAII makes sense in any language, actually. P. S. std::auto_ptr, really?
afigegoznaet wrote:
P. S. std::auto_ptr, really?
What? You just said you liked RAII. ;P
Real programmers use butterflies
