Does anybody really use a Profiler?
-
Would like to mention here that depending on what you are doing to time the code, that a good profiler will show you automatically. it will point out graphically where the slow code is. No need to time. It will show you how much time is spent in exact portions of the code. The SharpDev profiler is amazine because it creates these very cool graphics which are concentric circles of the method calls, then inside those calls the calls the user calls that are made then finally the method may hit some system APIs and spend time in those DLLs and it shows you how much time is spent as a part of the entire method call. It is amazing and easy to use and it provides a very cool view of the code.
I did recently try out the Jet Brains profiler on that block of code that did the dictionary tuple lookup just for fun to see if profilers have improved since I last used them. It showed that my bottleneck was in GetHashCode(). That was identified as my #1 application hot-spot. The method that actually did the dictionary lookup was ranked almost dead last. That's a major profiler failure in my book. It didn't point me anywhere close to the right place. I messed around with the profiler for about half an hour trying various options to try to get it to show me the call tree that was at fault, but it didn't seem possible. Maybe it is and I just didn't set it up correctly. Like I said, I only spent about half an hour on it. However, to be fair, I guess GetHashCode() *was* the ultimate method at fault, but none of the views in Jet Brains indicated that it was from the dictionary lookup and because I wasn't seeing call trees, if I didn't know what was going on, I probably would have never connected the two. Probably the top 10 methods identified were internal .NET methods like GetHashCode() that really had nothing to do with figuring out the performance issue. Since it isn't really GetHashCode() that's the problem, but using a tuple in a dictionary is. The profiler also ranked TryGetValue as pretty low, so there was no indication that that method was calling GetHashCode() so much.
-
I recently posted a question: "I am trying to solve this problem. I solved it by using Solution1. I'm not happy with Solution1 because it is going to be a performance issue and I want to use Solution2 which I think will be a lot better. How do people typically implement Solution2?" So, of course, somebody chimes in and responds: "What makes you think Solution1 is a performance issue? Don't guess. Use a profiler!" Are you serious? LOL... Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow. If you rely on a profiler as a crutch, you should really work on your analytical skills as I think that's a requirement to be a good software ENGINEER and not just a coder. I dunno, maybe it's just a talent I have, but I've never used a profiler more then maybe once or twice and I've never had an issue QUICKLY optimizing the hell out of code. For example, just the other day, a junior co-worker complained to me that he was given a simple task and he has it working, but it's taking 2.5 minutes to run. I agreed that was waaay to slow and asked him how he did it. He said it was very basic, he just did this 1 simple step. So I said, it would be a lot faster if you used this 2 step solution instead. At first, he argued with me (cuz he's really cocky) that my 2 step solution couldn't possibly be faster then his 1 step solution. I told him that my 2 step solution would blow his 1 step solution out of the water guaranteed and that it would finish "instantly". He still didn't believe me (cuz he's really, REALLY cocky). Finally, after trying more stuff that didn't work, he implemented my 2 step solution and what do you know... it came back in "0ms" vs his 1 step solution that took 2.5 minutes. To me, optimizing code & identifying bottlenecks isn't rocket science. It's always the same: 1) move any repetitive work that's static outside the loop 2) cache objects that are expensive to create and setup 3) cache results that are expensive to calculate 4) don't use reflection in highly trafficked code 5) don't use linq in highly trafficked code 6) don't inline SQL code, use stored procs 7) don't suck down an entire database, only grab the data as you need it Those 7 basics will generally get you at least 50% of the way towards optimized code. There are some more "advanced" concepts that'll get you the rest of the way (and there are of course some other basic things I didn't list). I think if you use C# every day
Yes a profiler can't always get you the stats you need. Like others said, they are useful for finding hotspots in large distributed applications. For example I worked on a large web based CRM system that had randomly had extremely slow page render times. The whole system used DataSets as core data objects that were associated with classes defined in C# code that was stored in the database and dynamically compiled. Now, I could have gone in and manually instrumented several hundreds of methods and objects, but using a profiler I was able to find the slow spots in several of the subsystems including the process of compiling the data objects. It took a little experimentation to find out which of several approaches was fast enough. The worst offender ended up being really inefficient code in the site navigation that involved a bad caching implementation. I'm skipping around to your last example. A memory profiler will find that issue pretty quickly, in addition you will have a significant amount of time in garbage collection when you look at performance counters. I also like to instrument my code with custom performance counters so I can monitor it in production easily. Ok, the highest performance C# app I worked on was a click scoring web service that took web server log lines and had to return a score in 100ms or less. It peaked out at about 6,000 scores per second averaging less than 1ms average response time. The entire first stage involved mapping strings to ids using Dictionary objects. Then there were a number of shared statistics that were maintained based on those integer lookups. Several hundred rules were run on all that computed data. That was combined with a naive bayes classifer, and the statistics were updated with the results of the scoring, and the score was returned to the client. The first versions loaded data as needed, but eventually everything was hugely sped up by loading the entire database in memory as the first step and then generating all the ids in the client. Also writing data out to the database was done with sql batch update objects which out perform any kind of stored procedures by wide margins. But even then, stored procedures haven't been faster than inline sql for years. Micro-optimization like you are talking about can get you a long ways, but at some point secondary effects like garbage collection, memory paging, working set management, cache lines, etc. are going to become issues especially when you are working with something like a 16 Gigabyte process s
-
I recently posted a question: "I am trying to solve this problem. I solved it by using Solution1. I'm not happy with Solution1 because it is going to be a performance issue and I want to use Solution2 which I think will be a lot better. How do people typically implement Solution2?" So, of course, somebody chimes in and responds: "What makes you think Solution1 is a performance issue? Don't guess. Use a profiler!" Are you serious? LOL... Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow. If you rely on a profiler as a crutch, you should really work on your analytical skills as I think that's a requirement to be a good software ENGINEER and not just a coder. I dunno, maybe it's just a talent I have, but I've never used a profiler more then maybe once or twice and I've never had an issue QUICKLY optimizing the hell out of code. For example, just the other day, a junior co-worker complained to me that he was given a simple task and he has it working, but it's taking 2.5 minutes to run. I agreed that was waaay to slow and asked him how he did it. He said it was very basic, he just did this 1 simple step. So I said, it would be a lot faster if you used this 2 step solution instead. At first, he argued with me (cuz he's really cocky) that my 2 step solution couldn't possibly be faster then his 1 step solution. I told him that my 2 step solution would blow his 1 step solution out of the water guaranteed and that it would finish "instantly". He still didn't believe me (cuz he's really, REALLY cocky). Finally, after trying more stuff that didn't work, he implemented my 2 step solution and what do you know... it came back in "0ms" vs his 1 step solution that took 2.5 minutes. To me, optimizing code & identifying bottlenecks isn't rocket science. It's always the same: 1) move any repetitive work that's static outside the loop 2) cache objects that are expensive to create and setup 3) cache results that are expensive to calculate 4) don't use reflection in highly trafficked code 5) don't use linq in highly trafficked code 6) don't inline SQL code, use stored procs 7) don't suck down an entire database, only grab the data as you need it Those 7 basics will generally get you at least 50% of the way towards optimized code. There are some more "advanced" concepts that'll get you the rest of the way (and there are of course some other basic things I didn't list). I think if you use C# every day
SledgeHammer01 wrote:
Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow.
Unfortunately, while there are millions of programmers, only a few have the analytic skills that Jon Skeet ... and, presumably, you ? ... have: [^]. To the rarer programmer who has "got down" in the plumbing, and studied the internal behavior of every operator in their language, and the structure of its framework, carried out timing experiments, etc. : I salute you ! cheers, Bill
“I have diligently numbered the days of pure and genuine happiness which have fallen to my lot: They amount to 14.” Abd-Ar Rahman III, Caliph of Cordoba, circa 950CE.
-
I did recently try out the Jet Brains profiler on that block of code that did the dictionary tuple lookup just for fun to see if profilers have improved since I last used them. It showed that my bottleneck was in GetHashCode(). That was identified as my #1 application hot-spot. The method that actually did the dictionary lookup was ranked almost dead last. That's a major profiler failure in my book. It didn't point me anywhere close to the right place. I messed around with the profiler for about half an hour trying various options to try to get it to show me the call tree that was at fault, but it didn't seem possible. Maybe it is and I just didn't set it up correctly. Like I said, I only spent about half an hour on it. However, to be fair, I guess GetHashCode() *was* the ultimate method at fault, but none of the views in Jet Brains indicated that it was from the dictionary lookup and because I wasn't seeing call trees, if I didn't know what was going on, I probably would have never connected the two. Probably the top 10 methods identified were internal .NET methods like GetHashCode() that really had nothing to do with figuring out the performance issue. Since it isn't really GetHashCode() that's the problem, but using a tuple in a dictionary is. The profiler also ranked TryGetValue as pretty low, so there was no indication that that method was calling GetHashCode() so much.
Your point is much clearer, and more powerfully expressed ... to me ... with this excellently written personal example. thanks, Bill
“I have diligently numbered the days of pure and genuine happiness which have fallen to my lot: They amount to 14.” Abd-Ar Rahman III, Caliph of Cordoba, circa 950CE.
-
I recently posted a question: "I am trying to solve this problem. I solved it by using Solution1. I'm not happy with Solution1 because it is going to be a performance issue and I want to use Solution2 which I think will be a lot better. How do people typically implement Solution2?" So, of course, somebody chimes in and responds: "What makes you think Solution1 is a performance issue? Don't guess. Use a profiler!" Are you serious? LOL... Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow. If you rely on a profiler as a crutch, you should really work on your analytical skills as I think that's a requirement to be a good software ENGINEER and not just a coder. I dunno, maybe it's just a talent I have, but I've never used a profiler more then maybe once or twice and I've never had an issue QUICKLY optimizing the hell out of code. For example, just the other day, a junior co-worker complained to me that he was given a simple task and he has it working, but it's taking 2.5 minutes to run. I agreed that was waaay to slow and asked him how he did it. He said it was very basic, he just did this 1 simple step. So I said, it would be a lot faster if you used this 2 step solution instead. At first, he argued with me (cuz he's really cocky) that my 2 step solution couldn't possibly be faster then his 1 step solution. I told him that my 2 step solution would blow his 1 step solution out of the water guaranteed and that it would finish "instantly". He still didn't believe me (cuz he's really, REALLY cocky). Finally, after trying more stuff that didn't work, he implemented my 2 step solution and what do you know... it came back in "0ms" vs his 1 step solution that took 2.5 minutes. To me, optimizing code & identifying bottlenecks isn't rocket science. It's always the same: 1) move any repetitive work that's static outside the loop 2) cache objects that are expensive to create and setup 3) cache results that are expensive to calculate 4) don't use reflection in highly trafficked code 5) don't use linq in highly trafficked code 6) don't inline SQL code, use stored procs 7) don't suck down an entire database, only grab the data as you need it Those 7 basics will generally get you at least 50% of the way towards optimized code. There are some more "advanced" concepts that'll get you the rest of the way (and there are of course some other basic things I didn't list). I think if you use C# every day
SledgeHammer01 wrote:
look at a block of code for a few minutes and instantly identify why it's slow
I hate to destroy your self confidence, but some code is work just perfectly with input A and perform terribly with input B (see compression algorithms), so looking at the code and even understanding it means nothing without the real-time environment...
SledgeHammer01 wrote:
maybe it's just a talent
The word you looking for is neglection me think...
SledgeHammer01 wrote:
from my limited experience with profilers
Better learn profilesr (how to use and interpret its output) and after that we will hear you again...
I'm not questioning your powers of observation; I'm merely remarking upon the paradox of asking a masked man who he is. (V)
-
SledgeHammer01 wrote:
look at a block of code for a few minutes and instantly identify why it's slow
I hate to destroy your self confidence, but some code is work just perfectly with input A and perform terribly with input B (see compression algorithms), so looking at the code and even understanding it means nothing without the real-time environment...
SledgeHammer01 wrote:
maybe it's just a talent
The word you looking for is neglection me think...
SledgeHammer01 wrote:
from my limited experience with profilers
Better learn profilesr (how to use and interpret its output) and after that we will hear you again...
I'm not questioning your powers of observation; I'm merely remarking upon the paradox of asking a masked man who he is. (V)
Kornfeld Eliyahu Peter wrote:
I hate to destroy your self confidence, but some code is work just perfectly with input A and perform terribly with input B (see compression algorithms), so looking at the code and even understanding it means nothing without the real-time environment...
I think you are very confused. I never said I can compile or disassemble code in my head. Nor did I ever say I can optimize code for every single condition that will ever come up. OF COURSE you need the input that causes the issue LOL. You couldn't use your profiler without the proper input either :confused: :confused: :confused:. Given the problem input, yes, I can solve the issue. Duh. That was the whole point of the discussion. EDIT: Actually, I take that back. If the code is in a highly specialized area like math or science where you need a PHD in the specialized area to understand the algorithm, that's a different story. In that case, a profiler wouldn't help you either. So your point is moot. Let's say you are an expert on compression and have a few PHDs in math. I'll give you all the time in the world and any and all profilers you could ever want. I'm willing to bet you $$$ that you would be unable to resolve a performance issue in a DIFFERENT highly specialized area like say nuclear physics.
-
Kornfeld Eliyahu Peter wrote:
I hate to destroy your self confidence, but some code is work just perfectly with input A and perform terribly with input B (see compression algorithms), so looking at the code and even understanding it means nothing without the real-time environment...
I think you are very confused. I never said I can compile or disassemble code in my head. Nor did I ever say I can optimize code for every single condition that will ever come up. OF COURSE you need the input that causes the issue LOL. You couldn't use your profiler without the proper input either :confused: :confused: :confused:. Given the problem input, yes, I can solve the issue. Duh. That was the whole point of the discussion. EDIT: Actually, I take that back. If the code is in a highly specialized area like math or science where you need a PHD in the specialized area to understand the algorithm, that's a different story. In that case, a profiler wouldn't help you either. So your point is moot. Let's say you are an expert on compression and have a few PHDs in math. I'll give you all the time in the world and any and all profilers you could ever want. I'm willing to bet you $$$ that you would be unable to resolve a performance issue in a DIFFERENT highly specialized area like say nuclear physics.
SledgeHammer01 wrote:
You couldn't use your profiler without the proper input either
That's profiler IS. Check your code in real-time environment. And if you believe that you can solve the issue by seeing the input, you never saw real input and never understood the issue!
I'm not questioning your powers of observation; I'm merely remarking upon the paradox of asking a masked man who he is. (V)
-
I recently posted a question: "I am trying to solve this problem. I solved it by using Solution1. I'm not happy with Solution1 because it is going to be a performance issue and I want to use Solution2 which I think will be a lot better. How do people typically implement Solution2?" So, of course, somebody chimes in and responds: "What makes you think Solution1 is a performance issue? Don't guess. Use a profiler!" Are you serious? LOL... Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow. If you rely on a profiler as a crutch, you should really work on your analytical skills as I think that's a requirement to be a good software ENGINEER and not just a coder. I dunno, maybe it's just a talent I have, but I've never used a profiler more then maybe once or twice and I've never had an issue QUICKLY optimizing the hell out of code. For example, just the other day, a junior co-worker complained to me that he was given a simple task and he has it working, but it's taking 2.5 minutes to run. I agreed that was waaay to slow and asked him how he did it. He said it was very basic, he just did this 1 simple step. So I said, it would be a lot faster if you used this 2 step solution instead. At first, he argued with me (cuz he's really cocky) that my 2 step solution couldn't possibly be faster then his 1 step solution. I told him that my 2 step solution would blow his 1 step solution out of the water guaranteed and that it would finish "instantly". He still didn't believe me (cuz he's really, REALLY cocky). Finally, after trying more stuff that didn't work, he implemented my 2 step solution and what do you know... it came back in "0ms" vs his 1 step solution that took 2.5 minutes. To me, optimizing code & identifying bottlenecks isn't rocket science. It's always the same: 1) move any repetitive work that's static outside the loop 2) cache objects that are expensive to create and setup 3) cache results that are expensive to calculate 4) don't use reflection in highly trafficked code 5) don't use linq in highly trafficked code 6) don't inline SQL code, use stored procs 7) don't suck down an entire database, only grab the data as you need it Those 7 basics will generally get you at least 50% of the way towards optimized code. There are some more "advanced" concepts that'll get you the rest of the way (and there are of course some other basic things I didn't list). I think if you use C# every day
Many, many times. Both static and dynamic profiling is an important tool in enterprise development. Yes, your basic steps are useful in a single application, but when you move into a large distributed, multi user system simple code viewing doesn't cut the mustard any more. The time it takes to look through hundreds of methods and calculate which ones will be a problem under which conditions is waisted if you have a decent set of diagnostic tools at your hand. Real example. I had a junior put together a piece of work. Simple service for validating the entry from a form. Now applying your logic above the code would check out fine. Performance was good on the test system and there were no obvious conflicts, data was being cached, look ups avoided, etc, etc, etc. The memory footprint was a pig, absolutely horrendous. And for each user the problem increased. We needed to pool resources, reduce cached data and in some places slowed down response times.
-
I recently posted a question: "I am trying to solve this problem. I solved it by using Solution1. I'm not happy with Solution1 because it is going to be a performance issue and I want to use Solution2 which I think will be a lot better. How do people typically implement Solution2?" So, of course, somebody chimes in and responds: "What makes you think Solution1 is a performance issue? Don't guess. Use a profiler!" Are you serious? LOL... Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow. If you rely on a profiler as a crutch, you should really work on your analytical skills as I think that's a requirement to be a good software ENGINEER and not just a coder. I dunno, maybe it's just a talent I have, but I've never used a profiler more then maybe once or twice and I've never had an issue QUICKLY optimizing the hell out of code. For example, just the other day, a junior co-worker complained to me that he was given a simple task and he has it working, but it's taking 2.5 minutes to run. I agreed that was waaay to slow and asked him how he did it. He said it was very basic, he just did this 1 simple step. So I said, it would be a lot faster if you used this 2 step solution instead. At first, he argued with me (cuz he's really cocky) that my 2 step solution couldn't possibly be faster then his 1 step solution. I told him that my 2 step solution would blow his 1 step solution out of the water guaranteed and that it would finish "instantly". He still didn't believe me (cuz he's really, REALLY cocky). Finally, after trying more stuff that didn't work, he implemented my 2 step solution and what do you know... it came back in "0ms" vs his 1 step solution that took 2.5 minutes. To me, optimizing code & identifying bottlenecks isn't rocket science. It's always the same: 1) move any repetitive work that's static outside the loop 2) cache objects that are expensive to create and setup 3) cache results that are expensive to calculate 4) don't use reflection in highly trafficked code 5) don't use linq in highly trafficked code 6) don't inline SQL code, use stored procs 7) don't suck down an entire database, only grab the data as you need it Those 7 basics will generally get you at least 50% of the way towards optimized code. There are some more "advanced" concepts that'll get you the rest of the way (and there are of course some other basic things I didn't list). I think if you use C# every day
I also find this curious. "You didn't use a profile, did you" is a pretty common piece of bullshit that people sling. It's almost always perfectly obviously what the problem is, or that something will be a problem, without even running the code let along profiling it. Basically what that comes down to is, if you do something stupid, it's going to suck. And preemptively not doing stupid shit is not "premature optimization". Of course it's not always obvious. For example,
xor ecx, ecx
_benchloop:
lzcnt eax, edx
add ecx, 1
jnz _benchloopWhy does this measure the latency of
lzcnt, instead of its throughput? It doesn't look like it should do that, so in the original code that contained the "problemlzcnt", that problem gave me quite the chase. Initially I didn't even notice something was wrong. No actual profiler was involved though. -
I recently posted a question: "I am trying to solve this problem. I solved it by using Solution1. I'm not happy with Solution1 because it is going to be a performance issue and I want to use Solution2 which I think will be a lot better. How do people typically implement Solution2?" So, of course, somebody chimes in and responds: "What makes you think Solution1 is a performance issue? Don't guess. Use a profiler!" Are you serious? LOL... Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow. If you rely on a profiler as a crutch, you should really work on your analytical skills as I think that's a requirement to be a good software ENGINEER and not just a coder. I dunno, maybe it's just a talent I have, but I've never used a profiler more then maybe once or twice and I've never had an issue QUICKLY optimizing the hell out of code. For example, just the other day, a junior co-worker complained to me that he was given a simple task and he has it working, but it's taking 2.5 minutes to run. I agreed that was waaay to slow and asked him how he did it. He said it was very basic, he just did this 1 simple step. So I said, it would be a lot faster if you used this 2 step solution instead. At first, he argued with me (cuz he's really cocky) that my 2 step solution couldn't possibly be faster then his 1 step solution. I told him that my 2 step solution would blow his 1 step solution out of the water guaranteed and that it would finish "instantly". He still didn't believe me (cuz he's really, REALLY cocky). Finally, after trying more stuff that didn't work, he implemented my 2 step solution and what do you know... it came back in "0ms" vs his 1 step solution that took 2.5 minutes. To me, optimizing code & identifying bottlenecks isn't rocket science. It's always the same: 1) move any repetitive work that's static outside the loop 2) cache objects that are expensive to create and setup 3) cache results that are expensive to calculate 4) don't use reflection in highly trafficked code 5) don't use linq in highly trafficked code 6) don't inline SQL code, use stored procs 7) don't suck down an entire database, only grab the data as you need it Those 7 basics will generally get you at least 50% of the way towards optimized code. There are some more "advanced" concepts that'll get you the rest of the way (and there are of course some other basic things I didn't list). I think if you use C# every day
They always just say "male, aged 18-35, who used to kick cats", so it's a waste of time even consulting them.
I wanna be a eunuchs developer! Pass me a bread knife!
-
Are you talking about thread deadlocking and/or resource contention? I wouldn't consider deadlocking a performance issue, but I guess you could take it as one. I'd consider it more a threading issue. Big performance killer in multi-threading is if you use a lot of thread synchronization when you don't absolutely have to. Microsoft has a nice solution for that with the TPL library. I've not done games or graphics programming, so I couldn't comment on that portion.
Much worse is when two or more threads keep each other waiting, but the synchronization is needed. For example a thread that reads input from devices and inserts the data as messages into a queue for the UI. Obviously, the UI threads frequently are going to check the queue for new messages and this access must be synchronized. If the threads lock each other out too frequently, you will have to come up with something more clever than a simple queue to eliminate the need for synchronization.
The language is JavaScript. that of Mordor, which I will not utter here
I hold an A-7 computer expert classification, Commodore. I'm well acquainted with Dr. Daystrom's theories and discoveries. The basic design of all our ship's computers are JavaScript. -
I recently posted a question: "I am trying to solve this problem. I solved it by using Solution1. I'm not happy with Solution1 because it is going to be a performance issue and I want to use Solution2 which I think will be a lot better. How do people typically implement Solution2?" So, of course, somebody chimes in and responds: "What makes you think Solution1 is a performance issue? Don't guess. Use a profiler!" Are you serious? LOL... Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow. If you rely on a profiler as a crutch, you should really work on your analytical skills as I think that's a requirement to be a good software ENGINEER and not just a coder. I dunno, maybe it's just a talent I have, but I've never used a profiler more then maybe once or twice and I've never had an issue QUICKLY optimizing the hell out of code. For example, just the other day, a junior co-worker complained to me that he was given a simple task and he has it working, but it's taking 2.5 minutes to run. I agreed that was waaay to slow and asked him how he did it. He said it was very basic, he just did this 1 simple step. So I said, it would be a lot faster if you used this 2 step solution instead. At first, he argued with me (cuz he's really cocky) that my 2 step solution couldn't possibly be faster then his 1 step solution. I told him that my 2 step solution would blow his 1 step solution out of the water guaranteed and that it would finish "instantly". He still didn't believe me (cuz he's really, REALLY cocky). Finally, after trying more stuff that didn't work, he implemented my 2 step solution and what do you know... it came back in "0ms" vs his 1 step solution that took 2.5 minutes. To me, optimizing code & identifying bottlenecks isn't rocket science. It's always the same: 1) move any repetitive work that's static outside the loop 2) cache objects that are expensive to create and setup 3) cache results that are expensive to calculate 4) don't use reflection in highly trafficked code 5) don't use linq in highly trafficked code 6) don't inline SQL code, use stored procs 7) don't suck down an entire database, only grab the data as you need it Those 7 basics will generally get you at least 50% of the way towards optimized code. There are some more "advanced" concepts that'll get you the rest of the way (and there are of course some other basic things I didn't list). I think if you use C# every day
Good luck finding the most important bottle necks in applications of several millions lines of code without a profiler! :laugh: Hell, even in smaller applications I wouldn't even bother finding bottlenecks without a profiler. Why would you deny yourself a helpful tool?
Wout
-
Good luck finding the most important bottle necks in applications of several millions lines of code without a profiler! :laugh: Hell, even in smaller applications I wouldn't even bother finding bottlenecks without a profiler. Why would you deny yourself a helpful tool?
Wout
wout de zeeuw wrote:
Why would you deny yourself a helpful tool?
Because he thinks the tool sucks
M.D.V. ;) If something has a solution... Why do we have to worry about?. If it has no solution... For what reason do we have to worry about? Help me to understand what I'm saying, and I'll explain it better to you Rating helpful answers is nice, but saying thanks can be even nicer.
-
I recently posted a question: "I am trying to solve this problem. I solved it by using Solution1. I'm not happy with Solution1 because it is going to be a performance issue and I want to use Solution2 which I think will be a lot better. How do people typically implement Solution2?" So, of course, somebody chimes in and responds: "What makes you think Solution1 is a performance issue? Don't guess. Use a profiler!" Are you serious? LOL... Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow. If you rely on a profiler as a crutch, you should really work on your analytical skills as I think that's a requirement to be a good software ENGINEER and not just a coder. I dunno, maybe it's just a talent I have, but I've never used a profiler more then maybe once or twice and I've never had an issue QUICKLY optimizing the hell out of code. For example, just the other day, a junior co-worker complained to me that he was given a simple task and he has it working, but it's taking 2.5 minutes to run. I agreed that was waaay to slow and asked him how he did it. He said it was very basic, he just did this 1 simple step. So I said, it would be a lot faster if you used this 2 step solution instead. At first, he argued with me (cuz he's really cocky) that my 2 step solution couldn't possibly be faster then his 1 step solution. I told him that my 2 step solution would blow his 1 step solution out of the water guaranteed and that it would finish "instantly". He still didn't believe me (cuz he's really, REALLY cocky). Finally, after trying more stuff that didn't work, he implemented my 2 step solution and what do you know... it came back in "0ms" vs his 1 step solution that took 2.5 minutes. To me, optimizing code & identifying bottlenecks isn't rocket science. It's always the same: 1) move any repetitive work that's static outside the loop 2) cache objects that are expensive to create and setup 3) cache results that are expensive to calculate 4) don't use reflection in highly trafficked code 5) don't use linq in highly trafficked code 6) don't inline SQL code, use stored procs 7) don't suck down an entire database, only grab the data as you need it Those 7 basics will generally get you at least 50% of the way towards optimized code. There are some more "advanced" concepts that'll get you the rest of the way (and there are of course some other basic things I didn't list). I think if you use C# every day
-
I recently posted a question: "I am trying to solve this problem. I solved it by using Solution1. I'm not happy with Solution1 because it is going to be a performance issue and I want to use Solution2 which I think will be a lot better. How do people typically implement Solution2?" So, of course, somebody chimes in and responds: "What makes you think Solution1 is a performance issue? Don't guess. Use a profiler!" Are you serious? LOL... Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow. If you rely on a profiler as a crutch, you should really work on your analytical skills as I think that's a requirement to be a good software ENGINEER and not just a coder. I dunno, maybe it's just a talent I have, but I've never used a profiler more then maybe once or twice and I've never had an issue QUICKLY optimizing the hell out of code. For example, just the other day, a junior co-worker complained to me that he was given a simple task and he has it working, but it's taking 2.5 minutes to run. I agreed that was waaay to slow and asked him how he did it. He said it was very basic, he just did this 1 simple step. So I said, it would be a lot faster if you used this 2 step solution instead. At first, he argued with me (cuz he's really cocky) that my 2 step solution couldn't possibly be faster then his 1 step solution. I told him that my 2 step solution would blow his 1 step solution out of the water guaranteed and that it would finish "instantly". He still didn't believe me (cuz he's really, REALLY cocky). Finally, after trying more stuff that didn't work, he implemented my 2 step solution and what do you know... it came back in "0ms" vs his 1 step solution that took 2.5 minutes. To me, optimizing code & identifying bottlenecks isn't rocket science. It's always the same: 1) move any repetitive work that's static outside the loop 2) cache objects that are expensive to create and setup 3) cache results that are expensive to calculate 4) don't use reflection in highly trafficked code 5) don't use linq in highly trafficked code 6) don't inline SQL code, use stored procs 7) don't suck down an entire database, only grab the data as you need it Those 7 basics will generally get you at least 50% of the way towards optimized code. There are some more "advanced" concepts that'll get you the rest of the way (and there are of course some other basic things I didn't list). I think if you use C# every day
I find the idea of not being able to use every tool available to you is incredibly naive. Granted that you should be able to solve simple performance problems just by looking at them, but some problems are just too big and interwoven to solve like this. Also, it's not always just your code that you're profiling - don't forget that you can often identify problem areas in code that you're using that there might be alternatives to. So yes, I use profilers regularly. I use them because I'm aware enough to know I don't know everything and that there's always something new that I can learn.
-
I recently posted a question: "I am trying to solve this problem. I solved it by using Solution1. I'm not happy with Solution1 because it is going to be a performance issue and I want to use Solution2 which I think will be a lot better. How do people typically implement Solution2?" So, of course, somebody chimes in and responds: "What makes you think Solution1 is a performance issue? Don't guess. Use a profiler!" Are you serious? LOL... Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow. If you rely on a profiler as a crutch, you should really work on your analytical skills as I think that's a requirement to be a good software ENGINEER and not just a coder. I dunno, maybe it's just a talent I have, but I've never used a profiler more then maybe once or twice and I've never had an issue QUICKLY optimizing the hell out of code. For example, just the other day, a junior co-worker complained to me that he was given a simple task and he has it working, but it's taking 2.5 minutes to run. I agreed that was waaay to slow and asked him how he did it. He said it was very basic, he just did this 1 simple step. So I said, it would be a lot faster if you used this 2 step solution instead. At first, he argued with me (cuz he's really cocky) that my 2 step solution couldn't possibly be faster then his 1 step solution. I told him that my 2 step solution would blow his 1 step solution out of the water guaranteed and that it would finish "instantly". He still didn't believe me (cuz he's really, REALLY cocky). Finally, after trying more stuff that didn't work, he implemented my 2 step solution and what do you know... it came back in "0ms" vs his 1 step solution that took 2.5 minutes. To me, optimizing code & identifying bottlenecks isn't rocket science. It's always the same: 1) move any repetitive work that's static outside the loop 2) cache objects that are expensive to create and setup 3) cache results that are expensive to calculate 4) don't use reflection in highly trafficked code 5) don't use linq in highly trafficked code 6) don't inline SQL code, use stored procs 7) don't suck down an entire database, only grab the data as you need it Those 7 basics will generally get you at least 50% of the way towards optimized code. There are some more "advanced" concepts that'll get you the rest of the way (and there are of course some other basic things I didn't list). I think if you use C# every day
>>> Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow. Are you serious!? Maybe for a single method or simple application, but for a complex application there's no way you can "just see it". Here's a simple real world example, recently I used a profiler to see where the bottleneck was in a web service which sits on top of 3 or 4 other layers (and I won't elaborate cause I'm keeping this simple). It turns out the bottleneck was in a piece of Microsoft code used within the ORM we use, so that was refactored to use a different method to do the same thing. No way you would have just seen that by looking, not least cause the code wasn't there to see. I'd happily admit I also wouldn't be able to just see an issue in my own code some of the time. In the above example we're talking 1000s of lines spread over multiple assemblies, why would I bother trawling through all that when I can just use a profiler? I was lucky enough to be able to test this for a single thread, good luck if you come across an issue that only occurs when you've got multiple threads involved.... :) The profiler is your friend. PS. does this mean you never use SHOWPLAN/EXPLAIN either?
-
I recently posted a question: "I am trying to solve this problem. I solved it by using Solution1. I'm not happy with Solution1 because it is going to be a performance issue and I want to use Solution2 which I think will be a lot better. How do people typically implement Solution2?" So, of course, somebody chimes in and responds: "What makes you think Solution1 is a performance issue? Don't guess. Use a profiler!" Are you serious? LOL... Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow. If you rely on a profiler as a crutch, you should really work on your analytical skills as I think that's a requirement to be a good software ENGINEER and not just a coder. I dunno, maybe it's just a talent I have, but I've never used a profiler more then maybe once or twice and I've never had an issue QUICKLY optimizing the hell out of code. For example, just the other day, a junior co-worker complained to me that he was given a simple task and he has it working, but it's taking 2.5 minutes to run. I agreed that was waaay to slow and asked him how he did it. He said it was very basic, he just did this 1 simple step. So I said, it would be a lot faster if you used this 2 step solution instead. At first, he argued with me (cuz he's really cocky) that my 2 step solution couldn't possibly be faster then his 1 step solution. I told him that my 2 step solution would blow his 1 step solution out of the water guaranteed and that it would finish "instantly". He still didn't believe me (cuz he's really, REALLY cocky). Finally, after trying more stuff that didn't work, he implemented my 2 step solution and what do you know... it came back in "0ms" vs his 1 step solution that took 2.5 minutes. To me, optimizing code & identifying bottlenecks isn't rocket science. It's always the same: 1) move any repetitive work that's static outside the loop 2) cache objects that are expensive to create and setup 3) cache results that are expensive to calculate 4) don't use reflection in highly trafficked code 5) don't use linq in highly trafficked code 6) don't inline SQL code, use stored procs 7) don't suck down an entire database, only grab the data as you need it Those 7 basics will generally get you at least 50% of the way towards optimized code. There are some more "advanced" concepts that'll get you the rest of the way (and there are of course some other basic things I didn't list). I think if you use C# every day
Absolutely I use a profiler. You're right that looking at code can often provide the answer, but when you're code base is several thousand lines of code split across multiple components using a profiler can at the very least help identify where the problem is. It's also useful to spot things which might not be a problem with your code. Maybe a library you're using is doing something stupid, perhaps it's a little known feature of the .NET framework which is causing you problems. We recently had an issue with an older piece of software which was running into that, looking at the code would not have helped in the slightest but a profiler quickly sorted out where the problem was. A profiler should be a part of any developers toolbox, fair enough that it shouldn't be the only solution to every problem, but it should be a tool that you are comfortable using and comfortable to know when to use.
Eagles may soar, but weasels don't get sucked into jet engines
-
I recently posted a question: "I am trying to solve this problem. I solved it by using Solution1. I'm not happy with Solution1 because it is going to be a performance issue and I want to use Solution2 which I think will be a lot better. How do people typically implement Solution2?" So, of course, somebody chimes in and responds: "What makes you think Solution1 is a performance issue? Don't guess. Use a profiler!" Are you serious? LOL... Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow. If you rely on a profiler as a crutch, you should really work on your analytical skills as I think that's a requirement to be a good software ENGINEER and not just a coder. I dunno, maybe it's just a talent I have, but I've never used a profiler more then maybe once or twice and I've never had an issue QUICKLY optimizing the hell out of code. For example, just the other day, a junior co-worker complained to me that he was given a simple task and he has it working, but it's taking 2.5 minutes to run. I agreed that was waaay to slow and asked him how he did it. He said it was very basic, he just did this 1 simple step. So I said, it would be a lot faster if you used this 2 step solution instead. At first, he argued with me (cuz he's really cocky) that my 2 step solution couldn't possibly be faster then his 1 step solution. I told him that my 2 step solution would blow his 1 step solution out of the water guaranteed and that it would finish "instantly". He still didn't believe me (cuz he's really, REALLY cocky). Finally, after trying more stuff that didn't work, he implemented my 2 step solution and what do you know... it came back in "0ms" vs his 1 step solution that took 2.5 minutes. To me, optimizing code & identifying bottlenecks isn't rocket science. It's always the same: 1) move any repetitive work that's static outside the loop 2) cache objects that are expensive to create and setup 3) cache results that are expensive to calculate 4) don't use reflection in highly trafficked code 5) don't use linq in highly trafficked code 6) don't inline SQL code, use stored procs 7) don't suck down an entire database, only grab the data as you need it Those 7 basics will generally get you at least 50% of the way towards optimized code. There are some more "advanced" concepts that'll get you the rest of the way (and there are of course some other basic things I didn't list). I think if you use C# every day
-
I recently posted a question: "I am trying to solve this problem. I solved it by using Solution1. I'm not happy with Solution1 because it is going to be a performance issue and I want to use Solution2 which I think will be a lot better. How do people typically implement Solution2?" So, of course, somebody chimes in and responds: "What makes you think Solution1 is a performance issue? Don't guess. Use a profiler!" Are you serious? LOL... Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow. If you rely on a profiler as a crutch, you should really work on your analytical skills as I think that's a requirement to be a good software ENGINEER and not just a coder. I dunno, maybe it's just a talent I have, but I've never used a profiler more then maybe once or twice and I've never had an issue QUICKLY optimizing the hell out of code. For example, just the other day, a junior co-worker complained to me that he was given a simple task and he has it working, but it's taking 2.5 minutes to run. I agreed that was waaay to slow and asked him how he did it. He said it was very basic, he just did this 1 simple step. So I said, it would be a lot faster if you used this 2 step solution instead. At first, he argued with me (cuz he's really cocky) that my 2 step solution couldn't possibly be faster then his 1 step solution. I told him that my 2 step solution would blow his 1 step solution out of the water guaranteed and that it would finish "instantly". He still didn't believe me (cuz he's really, REALLY cocky). Finally, after trying more stuff that didn't work, he implemented my 2 step solution and what do you know... it came back in "0ms" vs his 1 step solution that took 2.5 minutes. To me, optimizing code & identifying bottlenecks isn't rocket science. It's always the same: 1) move any repetitive work that's static outside the loop 2) cache objects that are expensive to create and setup 3) cache results that are expensive to calculate 4) don't use reflection in highly trafficked code 5) don't use linq in highly trafficked code 6) don't inline SQL code, use stored procs 7) don't suck down an entire database, only grab the data as you need it Those 7 basics will generally get you at least 50% of the way towards optimized code. There are some more "advanced" concepts that'll get you the rest of the way (and there are of course some other basic things I didn't list). I think if you use C# every day
Any reason why parametrized SQL code would be slower than a stored procedure? I mean, unless your statement is so big that the network overhead of sending it would be significant, a parametrized statement would have its execution plan cached by the server, same as the stored procedure. Of course it's a lot more interesting to get all the results you need in one statement instead of looping over every row and issuing a SQL statement for every row. Same goes for LDAP.
-
I recently posted a question: "I am trying to solve this problem. I solved it by using Solution1. I'm not happy with Solution1 because it is going to be a performance issue and I want to use Solution2 which I think will be a lot better. How do people typically implement Solution2?" So, of course, somebody chimes in and responds: "What makes you think Solution1 is a performance issue? Don't guess. Use a profiler!" Are you serious? LOL... Personally, I think anybody who has been programming a while should be able to look at a block of code for a few minutes and instantly identify why it's slow. If you rely on a profiler as a crutch, you should really work on your analytical skills as I think that's a requirement to be a good software ENGINEER and not just a coder. I dunno, maybe it's just a talent I have, but I've never used a profiler more then maybe once or twice and I've never had an issue QUICKLY optimizing the hell out of code. For example, just the other day, a junior co-worker complained to me that he was given a simple task and he has it working, but it's taking 2.5 minutes to run. I agreed that was waaay to slow and asked him how he did it. He said it was very basic, he just did this 1 simple step. So I said, it would be a lot faster if you used this 2 step solution instead. At first, he argued with me (cuz he's really cocky) that my 2 step solution couldn't possibly be faster then his 1 step solution. I told him that my 2 step solution would blow his 1 step solution out of the water guaranteed and that it would finish "instantly". He still didn't believe me (cuz he's really, REALLY cocky). Finally, after trying more stuff that didn't work, he implemented my 2 step solution and what do you know... it came back in "0ms" vs his 1 step solution that took 2.5 minutes. To me, optimizing code & identifying bottlenecks isn't rocket science. It's always the same: 1) move any repetitive work that's static outside the loop 2) cache objects that are expensive to create and setup 3) cache results that are expensive to calculate 4) don't use reflection in highly trafficked code 5) don't use linq in highly trafficked code 6) don't inline SQL code, use stored procs 7) don't suck down an entire database, only grab the data as you need it Those 7 basics will generally get you at least 50% of the way towards optimized code. There are some more "advanced" concepts that'll get you the rest of the way (and there are of course some other basic things I didn't list). I think if you use C# every day
I rarely use profilers, but there are a few cases when even good programmers may have trouble seeing where code is slow. Sometimes there may be a library call which you think will be fast, but turns out to be slow. This could be due to system call(s) that take longer than the programmer realizes or due to the library function's algorithm working very differently than the programmer realized. We all make assumptions about things that sometimes aren't true. One very big issue is using something like using strlen(some_string) as a constant in a C program. Many programmers use that as though it is a constant in the compiler's mind, when it may not be (GCC has an __attribute__ that lets the compiler see that it is, but it's not standard). That's the kind of thing that could end up in a loop, possibly as part of the loop check, without many programmers realizing that it's a problem. While you might see that code, there are similar circumstances which are less obvious. These are often the results of invisible code introduced by compilers for languages higher than C. Overloaded operators, copy constructors, and destructors can all have much more impact on code than programmers realize. Knowing what kinds of things your language may hide is a good way to avoid these, but some things still slip through. Other things that are easy to miss is how contentious locks, semaphores, mutexes, and other blocking operations are. These are difficult to figure out sometimes. There are also times when you need to demonstrate to someone else where the bottle necks in code are. If your cocky junior programmer weren't your subordinate but rather your boss then the output of a profiler would be something that you could use to combat the cockiness. This is particularly useful when you get "Your code is slow! Fix it!" and you can come back with "This external code/hardware/whatever, which isn't under my control, is what is slow." Another time when using a profiler would be very useful would be if there were disagreement or uncertainty over which of multiple parts of a program were responsible for the different amounts of the slowness. Maybe you need to prioritize the order in which these are addressed. For you or I it may be obvious that a section of code is slower than it has to be, but when comparing two suboptimal sections of code it is very often that eyeballs alone are not enough. There are also times when a profiler can be used to find bugs that would take a lot of stepping in a debugger to locate.
