Getting the algorithm right
-
PaltryProgrammer wrote:
Isn't every routine/function/method/class/program an "algorithm" of sorts?
Sure, but not an algorithm that takes a great amount of mental effort to develop. Just like every little shed is a "building", you do not to into a great "building construction task" that really takes you expertise as an construction engineer to calculate right. I see myself more as a carpenter putting boards together, laying the tiles on the roof. Some effort goes on deciding the "floor plan", which pieces to put together and in which way, but for most things I do, that is really a minor part, both in hours and mental effort. The carpentry is the essential thing: Coding, debugging, testing, documenting. Not the architectural work.
As for mental effort, programmers are merely engineers. In my view there are three levels, i.e. to wit [0] scientists discover new knowledge, [1] engineers utilize these new knowledge to solve new problems, [2] technicians utilize the tools engineers fabricate to repeatedly solve the same problems again and again. - Best
-
Are you saying that 80% of your time goes to delvelop the algorithm? That only 20% goes to typing it in, building, debugging, documenting, communicating with users/customers and other developers?
-
Still, if you spend 80% of your mental effort on algorithmic decisions, it sounds to me as if you consider e.g. choosing an if - elseif - else sequence over a switch statement part of 'algorithm design'. Or while(){} versus do{}while(). I see those as trivial coding details. I'd say that if you by a short glimpse on the alternatives can say that they have the same complexity, in the big O sense, then there is no significant algorithmic development from one to the other. I am probably too ambitious. I really wish that University professors required any hand-in code taking variable size input to be followed by documentation stating the O() complexity of the function. Actually, I never ever saw any lecturer, professor or lower level, make such requirements. I wish it not only for college homework, but to be the norm for any published library or source code. It really should be part of our professional code of conduct to always include complexity as a basic part of the documentation. You might develop an algorithm of the same complexity as an existing one for the same task. It may have other traits, e.g. execution complexity and data space complexity are not necessarily parallell. But trivial code changes / decisions affect neither.
-
I had your experience with business development. With IoT it's all about stuffing everything you can into as small a footprint as possible, and I mean that, because once you get it to actually run on the hardware, you still have power concerns. ;P Unless you want the user going to the charging station once every couple of hours. So algorithms are king. It's why I enjoy it so much, I think. After all this time it feels like *real programming* again.
To err is human. Fortune favors the monsters.
I only do line-of-business stuff, and the OP's experience pretty much mirrors my own. Despite my work falling under the umbrella of "computer science," I do not like to think of myself as a computer scientist (or even an engineer, really), I tend to reserve that term for the people who are devising new technologies and novel algorithms. I take the excellent, innovative ideas of other people and apply them more or less intelligently to the business problem at hand. I'm comfortable with being good at that, even though I'm certain I'm not changing the world :)
-
Spurred by this week's survey (Do you get up in the middle of the night to code because you can't sleep?[^]): How much of your work hours and/or mental energy goes to find an "algorithmic solution" to your problem? I have been coding for a few decades, but my experience is that at least 90% of my time and energy goes to collecting background information, putting pieces together (rather mechanically), typing the code, writing tests, managing the build scripts, reading compiler listings and logs, writing documentation, presenting stuff to coworkers, ... I cannot recite all sorts of algorithms by heart, so sometimes I dig up a text (or open-source code) describing how to solve the problem. Very rarely am I stuck with a problem where I cannot quite easily either devise a method (usually composed from a set of partial solutions), or where I can find a workable solution in literature or on internet. Those 'eureka moments' are for the most part limited to when I understand the logic in a textbook presentation of an algorithm. I can't imagine not falling asleep because I am unable to devise a new, great, hitherto unknown algorithm. Of course: If you are an advanced research scientist in a field such as eg. numerical methods, then you job is to develop new algorithms for the algorithm's sake. Few of us are. Maybe I am different. Do you really spend any significant fraction of your working hours or mental capacity on developing new methods/algorithms?
For me it's 80% algorithm / 20% code. Once the algorithm is clear in my mind (and maybe on paper) the coding part is easy. If I can explain my approach to my colleagues - and they understand it, then I know I'm on the right track. If I can't explain it adequately or their eyes glaze over then I need to do more thinking.
-
Spurred by this week's survey (Do you get up in the middle of the night to code because you can't sleep?[^]): How much of your work hours and/or mental energy goes to find an "algorithmic solution" to your problem? I have been coding for a few decades, but my experience is that at least 90% of my time and energy goes to collecting background information, putting pieces together (rather mechanically), typing the code, writing tests, managing the build scripts, reading compiler listings and logs, writing documentation, presenting stuff to coworkers, ... I cannot recite all sorts of algorithms by heart, so sometimes I dig up a text (or open-source code) describing how to solve the problem. Very rarely am I stuck with a problem where I cannot quite easily either devise a method (usually composed from a set of partial solutions), or where I can find a workable solution in literature or on internet. Those 'eureka moments' are for the most part limited to when I understand the logic in a textbook presentation of an algorithm. I can't imagine not falling asleep because I am unable to devise a new, great, hitherto unknown algorithm. Of course: If you are an advanced research scientist in a field such as eg. numerical methods, then you job is to develop new algorithms for the algorithm's sake. Few of us are. Maybe I am different. Do you really spend any significant fraction of your working hours or mental capacity on developing new methods/algorithms?
Yes, I spent most of a year developing two completely novel, efficient algorithms, resulting in half-a-dozen U.S. patents. I'm "just a developer" but I happened to work at a company where a much more efficient algorithm became necessary because Moore's Law was making our product way too slow on modern circuits. To develop these algorithms, I also had to characterize the performance in big-O terms, of a dozen existing algorithms, including three algorithms we had previously developed ourselves. In another instance, I had to reverse-engineer the inefficient sort algorithm a customer was using from the English description of a non-technical person, and then implement an efficient algorithm to sort his data in an acceptable time. I have the same experience as trønderen, spending much of my life integrating known puzzle-pieces off the web into an effective, novel solution. But I have to be aware of algorithms to separate the efficient answers from the inefficient ones. There are recurring patterns underlying efficient algorithms that you can use every day if you are aware of them, and only thinking about and studying algorithms leads you to these patterns. You can tweak your code forever to make it faster, but the only way to achieve an order-of-magnitude performance improvement is to find a faster algorithm. It's just something I do.
-
Spurred by this week's survey (Do you get up in the middle of the night to code because you can't sleep?[^]): How much of your work hours and/or mental energy goes to find an "algorithmic solution" to your problem? I have been coding for a few decades, but my experience is that at least 90% of my time and energy goes to collecting background information, putting pieces together (rather mechanically), typing the code, writing tests, managing the build scripts, reading compiler listings and logs, writing documentation, presenting stuff to coworkers, ... I cannot recite all sorts of algorithms by heart, so sometimes I dig up a text (or open-source code) describing how to solve the problem. Very rarely am I stuck with a problem where I cannot quite easily either devise a method (usually composed from a set of partial solutions), or where I can find a workable solution in literature or on internet. Those 'eureka moments' are for the most part limited to when I understand the logic in a textbook presentation of an algorithm. I can't imagine not falling asleep because I am unable to devise a new, great, hitherto unknown algorithm. Of course: If you are an advanced research scientist in a field such as eg. numerical methods, then you job is to develop new algorithms for the algorithm's sake. Few of us are. Maybe I am different. Do you really spend any significant fraction of your working hours or mental capacity on developing new methods/algorithms?
The amount of think time is usually proportional to how much you "own" what you're working on. As technical lead, you might spend all your time "thinking"; assuming someone else is doing the coding. A system / app is just a big algorithm, IMO; but if you don't get the overall right, the little ones don't matter.
"Before entering on an understanding, I have meditated for a long time, and have foreseen what might happen. It is not genius which reveals to me suddenly, secretly, what I have to say or to do in a circumstance unexpected by other people; it is reflection, it is meditation." - Napoleon I
-
I only do line-of-business stuff, and the OP's experience pretty much mirrors my own. Despite my work falling under the umbrella of "computer science," I do not like to think of myself as a computer scientist (or even an engineer, really), I tend to reserve that term for the people who are devising new technologies and novel algorithms. I take the excellent, innovative ideas of other people and apply them more or less intelligently to the business problem at hand. I'm comfortable with being good at that, even though I'm certain I'm not changing the world :)
-
Spurred by this week's survey (Do you get up in the middle of the night to code because you can't sleep?[^]): How much of your work hours and/or mental energy goes to find an "algorithmic solution" to your problem? I have been coding for a few decades, but my experience is that at least 90% of my time and energy goes to collecting background information, putting pieces together (rather mechanically), typing the code, writing tests, managing the build scripts, reading compiler listings and logs, writing documentation, presenting stuff to coworkers, ... I cannot recite all sorts of algorithms by heart, so sometimes I dig up a text (or open-source code) describing how to solve the problem. Very rarely am I stuck with a problem where I cannot quite easily either devise a method (usually composed from a set of partial solutions), or where I can find a workable solution in literature or on internet. Those 'eureka moments' are for the most part limited to when I understand the logic in a textbook presentation of an algorithm. I can't imagine not falling asleep because I am unable to devise a new, great, hitherto unknown algorithm. Of course: If you are an advanced research scientist in a field such as eg. numerical methods, then you job is to develop new algorithms for the algorithm's sake. Few of us are. Maybe I am different. Do you really spend any significant fraction of your working hours or mental capacity on developing new methods/algorithms?
At present, most of my mental capacity is spent on refining and adding to the implementation of my novel native-variable-length hash table, part of which is covered by U.S. Patent No.: 11,254,590. But I spent many restless nights trying to devise the basic algorithm that allows the storage of multiple record types directly in a hash table; after all, that has been considered impossible since the invention of the hash table in the 1950's!
-
Isn't every routine/function/method/class/program an "algorithm" of sorts? As for my own time much of it is spent watching the twirly blue circle in Visual Studio and occasionally the white screen of death. Then there are the many many many 15m builds and the occasional language error leaving me scratching my head and attempting to discern the cryptic not very informative compiler error message. - Cheerio "I once put instant coffee into the microwave and went back in time." - Steven Wright "Shut up and calculate" - apparently N. David Mermin possibly Richard Feynman My sympathies to the SPAM moderator “I want to sing, I want to cry, I want to laugh. Everything together. And jump and dance. The day has arrived — yippee!” - Desmond Tutu “When the green flag drops the bullshit stops!” "It is cheaper to save the world than it is to ruin it." "I must have had lessons" - Reverend Jim Ignatowski / Christopher Lloyd
-
How big is your cade base? I have a fairly big code base (50k lines maybe) and it takes about 15s, not 15m to compile on my 32-hyperthread CPU.
Approximately 15K lines mostly templates plus 1.5K lines test code. When this project is done pretty soon now I intend to return to a project finished some years ago 50K lines in C and convert to C++ w/ modern UI/GUI and many improvements. No way I will tolerate the build times for that size. I will have to get a proper development machine. My current horse and buggy is https://www.hp.com/us-en/shop/pdp/hp-slimline-desktop-290-a0035z[^]
-
Approximately 15K lines mostly templates plus 1.5K lines test code. When this project is done pretty soon now I intend to return to a project finished some years ago 50K lines in C and convert to C++ w/ modern UI/GUI and many improvements. No way I will tolerate the build times for that size. I will have to get a proper development machine. My current horse and buggy is https://www.hp.com/us-en/shop/pdp/hp-slimline-desktop-290-a0035z[^]