Unpopular opinions: LINQ
-
Well, that's not really language specific. I find myself reading online documentation even when reading bash scripts.
Hey now. Bash scripts get arcane. I just got done writing a "bash course" for a client of mine (technically a client of an outfit i work for from time to time) who is trying to educate their own customers on how to use it. Talk to me after you've passed what bash calls an "array" to a function. :laugh:
Real programmers use butterflies
-
Is LINQ bad'ish or orders of magnitude slower, than those hand-written operations? I ask because I wonder about performance implications myself, while also regarding code read-/maintainability (after all, if performance was all I cared about, I'd hand-optimize everything in assembly anyway, engineering is all about trade-offs).
Usually it's not terrible. Not orders of magnitude slower for what most people seem to use it for - queries in business software. However, don't use it for what I'd call "real" functional programming. If you're going to write a parser generator or scanner generator for example, you don't want to compute your tables using linq. In that case it *will* be orders of magnitude slower than most anything you could write by hand. And I guess now you can tell what kind of software I write. :laugh:
Real programmers use butterflies
-
I wouldn't say C++ has gotten more complicated. Au contraire - I would say it's never been as easy to learn and use as it is now, I almost never have to rely on raw pointers, manual memory management, and all the stuff that C++ haters love. The problem of C++ is that it has become so feature rich, that it's virtually impossible to know all of it, and there really are a lot of features, which, although I'm sure somebody out there uses, but I don't see them as belonging in the standard. It's just so freaking huge. But, of course, you don't really need to know and use all of it. Unless you're a C++ compiler developer.
I'd argue that this *is* added complexity. Every layer you use to abstract away pointers and other C++ arcana removes you that much further from what your code is doing. And you know, std::auto_ptr works fantastic, until it doesn't. And when it doesn't you better know exactly how it's spec'd to function. Things can be complicated and easy to use at the same time. Hell, that describes the average IoT device these days.
Real programmers use butterflies
-
I'd argue that this *is* added complexity. Every layer you use to abstract away pointers and other C++ arcana removes you that much further from what your code is doing. And you know, std::auto_ptr works fantastic, until it doesn't. And when it doesn't you better know exactly how it's spec'd to function. Things can be complicated and easy to use at the same time. Hell, that describes the average IoT device these days.
Real programmers use butterflies
Well, no. It is added complexity if you don't know how to use raw pointers. If you do - you really appreciate the simplicity added by RAII, smart_ptrs and optionals, RAII makes sense in any language, actually. P. S. std::auto_ptr, really?
-
I hear you. The one "must have" feature in C++ that isn't actually in C++ but should be is real forward type declarations. Remove the restriction where I can only declare things like a pointer to a type before the type itself is fully declared. It might require using a GLR parser or something equally complicated to parse your code (like C# does) but given the complexity of a standards compliant C++ compiler already, what's changing the parsing engine used by most C++ compilers to something more advanced in the big scheme of things? Basically to bottom line it, I want to be able to declare my types anywhere in my files and use them anywhere in my files.
Real programmers use butterflies
You mean like this?: class FWD; struct STR { FWD *fwdPtr; }; int main() { STR tst; return 0; }
-
You mean like this?: class FWD; struct STR { FWD *fwdPtr; }; int main() { STR tst; return 0; }
No. I mean like this:
struct myForward;
std::unordered_map m;
...Except I'd even go as far as to eliminate the forward declaration entirely. The above won't work, BTW on most implementations of the STL, although there's nothing in the spec that says it won't work but nor does it guarantee it does. It works in Boost's unordered_map, but because they went out of their way to make it work in how they implemented unordered_map. Strangely enough, the above will usually work if you replace it with std::map
Real programmers use butterflies
-
Well, no. It is added complexity if you don't know how to use raw pointers. If you do - you really appreciate the simplicity added by RAII, smart_ptrs and optionals, RAII makes sense in any language, actually. P. S. std::auto_ptr, really?
afigegoznaet wrote:
P. S. std::auto_ptr, really?
What? You just said you liked RAII. ;P
Real programmers use butterflies
-
afigegoznaet wrote:
P. S. std::auto_ptr, really?
What? You just said you liked RAII. ;P
Real programmers use butterflies
std::auto_ptr was deprecated even in C++11 it's not valid C++ anymore. And RAII is not smart pointers.
-
No. I mean like this:
struct myForward;
std::unordered_map m;
...Except I'd even go as far as to eliminate the forward declaration entirely. The above won't work, BTW on most implementations of the STL, although there's nothing in the spec that says it won't work but nor does it guarantee it does. It works in Boost's unordered_map, but because they went out of their way to make it work in how they implemented unordered_map. Strangely enough, the above will usually work if you replace it with std::map
Real programmers use butterflies
This is supported by all compilers I use, clang, gcc and msvc. Just curious, what did you use that didn't support this? BTW, if you don't like forward declaration, perhaps you need an untyped language. Here, the type of "m" depends on the type "myForward", there's no way around this, and it's not just in C++.
-
My customers are willing to pay for me because the alternative is worse. :laugh:
Real programmers use butterflies
My customers are willing to pay for me, but they'll never admit it ;p Although I've had one saying "Sander, I trust you. If you send me an invoice I can be sure you're charging an honest price and that it's not too expensive." Which of course raised his invoices by 10% ;p :laugh: (no really, I'm joking!) I recently had a talk with two customers (from the same company) and one of them said "Sander, it's very nice what you've made for us, but that invoice was quite high." To which the other person replied "Think of it like this, [name], if it wasn't for Sander we wouldn't have it at all and we really need it." Despite me costing a lot of money, I'm don't think I'm expensive. I add direct value with my custom software and most of my competition is slower and/or more expensive :D All in all it's a fun job and haggling is just a part of it.
Best, Sander Azure DevOps Succinctly (free eBook) Azure Serverless Succinctly (free eBook) Migrating Apps to the Cloud with Azure arrgh.js - Bringing LINQ to JavaScript
-
Hey now. Bash scripts get arcane. I just got done writing a "bash course" for a client of mine (technically a client of an outfit i work for from time to time) who is trying to educate their own customers on how to use it. Talk to me after you've passed what bash calls an "array" to a function. :laugh:
Real programmers use butterflies
Of course, I had to do some lookup. Nevertheless: my_array=(foo bar) print_array(){ for i in $* do echo $i done } print_array ${my_array[*]}
-
My customers are willing to pay for me, but they'll never admit it ;p Although I've had one saying "Sander, I trust you. If you send me an invoice I can be sure you're charging an honest price and that it's not too expensive." Which of course raised his invoices by 10% ;p :laugh: (no really, I'm joking!) I recently had a talk with two customers (from the same company) and one of them said "Sander, it's very nice what you've made for us, but that invoice was quite high." To which the other person replied "Think of it like this, [name], if it wasn't for Sander we wouldn't have it at all and we really need it." Despite me costing a lot of money, I'm don't think I'm expensive. I add direct value with my custom software and most of my competition is slower and/or more expensive :D All in all it's a fun job and haggling is just a part of it.
Best, Sander Azure DevOps Succinctly (free eBook) Azure Serverless Succinctly (free eBook) Migrating Apps to the Cloud with Azure arrgh.js - Bringing LINQ to JavaScript
I'm a jerk i guess. I don't haggle. I charge based on how much I like the project, and I don't take jobs I don't like. I am fair about my invoices, and I itemize my time, but again, I don't haggle. Pay me or find someone else. I cost what I cost, and I'm always told I'm worth it when I'm told anything. One of my current clients actually told me I "walk on water" so I had to reduce his expectations for fear of drowning. :laugh:
Real programmers use butterflies
-
I'm not a fan of LINQ. I love functional programming but .NET's enumerator paradigm is not up to the task. It creates too many objects too quickly to be a real grown up functional language, whose iteration is highly optimized because it's a first class operation. I've benched LINQ against hand written pseudo-functional operations that do the same thing. It was not encouraging. For things that make heavy use of functional computation like parser generators, where your LINQ query might be half a page, it's a Bad Idea(TM) Worse, I think its use has been over encouraged by Microsoft. It makes green developers write even worse code, and makes it harder for a seasoned developer to understand the performance implications of the code they are writing (and I'm not talking about bit twiddling here, I'm talking about figuring out your Big O expression) I tend to avoid its use, preferring - at least in C# - to make my iteration operations explicit and long hand. If .NET had truly optimized iteration paradigm - one that didn't create new objects for every single iteration operation** - i might consider using it. ** yes i understand that LINQ combines multiple operations into a single iteration *sometimes* - in practice it's not often enough to make up for the overhead of enumerators. Now, there's a case where all of the above doesn't matter, and that's PLINQ. Theoretically, for a large enough operation, that can be highly parallelized, the overhead of enumerators suddenly isn't the biggest part of the performance equation. What I mean is it essentially pays for itself. Also, given the issues with synchronization and other cross task communication (is your operation clustered over a network?) enumerators are actually not a bad idea since you can lock behind them or RPC behind them. Contrast that with C++ iterators that are usually lightly wrapped pointer ops and you realize their limitations fast: In order to enable all of the stuff you need to make iteration operations work with each other in parallel you have to wrap every iterator operator anyway, making it as "heavy" as an enumerator in .NET, not counting the general overhead of running managed code. So basically, PLINQ is where LINQ finally covers its costs - where it reaches the point where its advantages outweigh its disadvantages. All of this of course, is one developer's opinion. And some of this doesn't necessarily apply to business software, where performance almost doesn't matter for most scenarios.
Real programmers
I actually love Linq. I find just the opposite. After you learn it well, it is very powerful and very fast. I once wrote an in-house version of the FBI’s CODIS search engine that requires very complicated operations and millions of them because you are searching huge genetic identity (DNA) databases. Linq handles them very well. Plus, because you can split operations between cores\processors, it is very fast. Linq is my go to solution for lots of scientific software. I especially like it’s Join and GroupBy. I do a lot of Sql. The way I do it now is that I developed a very fast transfer From Sql to a list of classes and then do Linq operations on the list. Extremely fast and more powerful than using Sql operations. Certainly a lot easier than Sql. So, I have to respectfully disagree with the common bashing in these posts and recommend investigating how Linq works to its fullest and suggest that you will change your minds.
MeziLu
-
I actually love Linq. I find just the opposite. After you learn it well, it is very powerful and very fast. I once wrote an in-house version of the FBI’s CODIS search engine that requires very complicated operations and millions of them because you are searching huge genetic identity (DNA) databases. Linq handles them very well. Plus, because you can split operations between cores\processors, it is very fast. Linq is my go to solution for lots of scientific software. I especially like it’s Join and GroupBy. I do a lot of Sql. The way I do it now is that I developed a very fast transfer From Sql to a list of classes and then do Linq operations on the list. Extremely fast and more powerful than using Sql operations. Certainly a lot easier than Sql. So, I have to respectfully disagree with the common bashing in these posts and recommend investigating how Linq works to its fullest and suggest that you will change your minds.
MeziLu
MeziLu wrote:
Plus, because you can split operations between cores\processors, it is very fast.
That's PLINQ. Read my comment again.
Real programmers use butterflies
-
I'm not a fan of LINQ. I love functional programming but .NET's enumerator paradigm is not up to the task. It creates too many objects too quickly to be a real grown up functional language, whose iteration is highly optimized because it's a first class operation. I've benched LINQ against hand written pseudo-functional operations that do the same thing. It was not encouraging. For things that make heavy use of functional computation like parser generators, where your LINQ query might be half a page, it's a Bad Idea(TM) Worse, I think its use has been over encouraged by Microsoft. It makes green developers write even worse code, and makes it harder for a seasoned developer to understand the performance implications of the code they are writing (and I'm not talking about bit twiddling here, I'm talking about figuring out your Big O expression) I tend to avoid its use, preferring - at least in C# - to make my iteration operations explicit and long hand. If .NET had truly optimized iteration paradigm - one that didn't create new objects for every single iteration operation** - i might consider using it. ** yes i understand that LINQ combines multiple operations into a single iteration *sometimes* - in practice it's not often enough to make up for the overhead of enumerators. Now, there's a case where all of the above doesn't matter, and that's PLINQ. Theoretically, for a large enough operation, that can be highly parallelized, the overhead of enumerators suddenly isn't the biggest part of the performance equation. What I mean is it essentially pays for itself. Also, given the issues with synchronization and other cross task communication (is your operation clustered over a network?) enumerators are actually not a bad idea since you can lock behind them or RPC behind them. Contrast that with C++ iterators that are usually lightly wrapped pointer ops and you realize their limitations fast: In order to enable all of the stuff you need to make iteration operations work with each other in parallel you have to wrap every iterator operator anyway, making it as "heavy" as an enumerator in .NET, not counting the general overhead of running managed code. So basically, PLINQ is where LINQ finally covers its costs - where it reaches the point where its advantages outweigh its disadvantages. All of this of course, is one developer's opinion. And some of this doesn't necessarily apply to business software, where performance almost doesn't matter for most scenarios.
Real programmers
Performance should never be the only consideration, and in most cases not even the most important one. One big advantage of LINQ is that you have the same set of functions, no matter what data source lies behind the data. It might be a local System.Collections.Generic.List generated on-the-fly, but it might as well be an SQL or SOAP connection providing the data collection. Either way, you should always consider the cost when calling LINQ functions. So calling Enumerable.Count() without important reason is usually not a good idea, as this will iterate over all items. Another advantage is readability. Of course only when you know how to read LINQ. In performance-critical scenarios though, the cost of generating an enumerator just for the sake of being able to use LINQ (e.g. for an array, which doesn't come with its own enumerator) might be relevant. But you should decide on a case-by-case basis instead of generalizing the decision for or against LINQ. "Only a Sith deals in absolutes." ;-)
-
MeziLu wrote:
Plus, because you can split operations between cores\processors, it is very fast.
That's PLINQ. Read my comment again.
Real programmers use butterflies
-
Performance should never be the only consideration, and in most cases not even the most important one. One big advantage of LINQ is that you have the same set of functions, no matter what data source lies behind the data. It might be a local System.Collections.Generic.List generated on-the-fly, but it might as well be an SQL or SOAP connection providing the data collection. Either way, you should always consider the cost when calling LINQ functions. So calling Enumerable.Count() without important reason is usually not a good idea, as this will iterate over all items. Another advantage is readability. Of course only when you know how to read LINQ. In performance-critical scenarios though, the cost of generating an enumerator just for the sake of being able to use LINQ (e.g. for an array, which doesn't come with its own enumerator) might be relevant. But you should decide on a case-by-case basis instead of generalizing the decision for or against LINQ. "Only a Sith deals in absolutes." ;-)
I feel justified in making the general observations I made about linq. And my comment wasn't just about performance. It was also about cognitive load in terms of understanding what your code is doing as well. And performance considerations are indeed important *if* they influence architecture, at which point potential perf problems are best identified at design time rather than after you've already architected something that will not perform to requirements. Using LINQ to implement all of your functional-programming style operations is a "Bad Idea(TM)" when you're doing loads of heavy iteration, like building parser or scanner tables. Most pure functional languages like Haskell handle iteration a lot better than LINQ if nothing else than for the fact that it's a first class operation. Enumerators in .NET were designed not as first class operations but built on top of the existing operations in .NET, and practically, that comes with performance considerations, like all the object creation it does. If you don't believe me, write a LALR(1) parser generator using LINQ and then one without. As long as you know what you're doing latter will be at least twice as fast.
Real programmers use butterflies
-
Thank you for your comment. I meant it as part of the entire package, not as a singled out feature to make the point that you need to consider Linq in its entirety, not this feature or that feature.
MeziLu
I feel like I did, which is why I made a whole exception for PLINQ. I even agree with you, if you'll read my comment that PLINQ is where LINQ finally pays for itself. Where we disagree is on basic LINQ. Not PLINQ. I covered readability. I don't think LINQ helps that, because I operate under the idea that if you can't understand exactly what your code is doing it doesn't matter if the code is concise. LINQ is concise, but not easy to read: It's harder to translate a LINQ call of any real world complexity mentally into the series of iteration operations its going to perform than it is to do the same with nested foreach and if statements. LINQ is shorter, but it doesn't lend itself to readability, it just means the code is concise, which isn't the same thing, IMO.
Real programmers use butterflies
-
Thank you for your comment. I meant it as part of the entire package, not as a singled out feature to make the point that you need to consider Linq in its entirety, not this feature or that feature.
MeziLu
One other point about parallel processing: When I do that, I always actually check the speed by setting a stopwatch to make sure I am getting a faster speed. Some times it is actually slower, but for the kinds of things I do, it is most often a lot faster. Just a word of advice to not assume speed boosts. Overall, without parallel processing, I find that Linq is blisteringly fast.
MeziLu
-
I feel like I did, which is why I made a whole exception for PLINQ. I even agree with you, if you'll read my comment that PLINQ is where LINQ finally pays for itself. Where we disagree is on basic LINQ. Not PLINQ. I covered readability. I don't think LINQ helps that, because I operate under the idea that if you can't understand exactly what your code is doing it doesn't matter if the code is concise. LINQ is concise, but not easy to read: It's harder to translate a LINQ call of any real world complexity mentally into the series of iteration operations its going to perform than it is to do the same with nested foreach and if statements. LINQ is shorter, but it doesn't lend itself to readability, it just means the code is concise, which isn't the same thing, IMO.
Real programmers use butterflies
Wow, I must be wired differently. Readability is one of the things I like most about Linq. I place each operation on its own line and it’s like this “outline” or flow diagram that I and my colleagues can instantly understand. Anyway, good chatting with you but I need to take care of some urgent things now
MeziLu
