Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Code Project
  1. Home
  2. The Lounge
  3. performance or accuracy?

performance or accuracy?

Scheduled Pinned Locked Moved The Lounge
databaseperformancealgorithmstutorialquestion
31 Posts 17 Posters 0 Views 1 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • R ravikhoda

    Today i came across a scene where we had a discussion on this topic. we have some work on the database side where batch files data is insert (row count is several thousand per operations )/ some text files generated by windows services / flag status maintain based on the data insert in the database per operation and many other small operations happen during this process. obviously correctness of the operation is must but at the same time performance of the operation came in to the picture. some teammates says that as there are lot of process happens during per operations it will take time to complete things while some believe that there can be something done to reduce the time. Client is okay even if that takes time as for him accuracy is more important rather than time taken. How to manage such scenarios ? after some level we can not think of optimization on query as all the things which is written is necessary. does anyone face such things ? how do you manage things to improve the performance?

    Ravi Khoda

    D Offline
    D Offline
    Daniel Lieberwirth BrainInBlack
    wrote on last edited by
    #5

    Basically you can answer this question by your self, by asking youself a bunch of questions: Is the process a critical part of the application? Depending on how critical the process is, you can do several things to optimize performance. For example, if the process isn't that important to the basic operation, then you can try to push the process in the background, i.e. give it a lower priority. On the other hand, if it is a critical operation, you have to find other means to optimize the process. And that can be very hard, especially if the code has to keep it's accuracy.In that case you have to decide if the work, required to optimize the code, is worth your client's money. Because that is what it comes down to. Are there opportunities to do optimizations/micro-optimizations on several parts of the code? This will probably cause a heated discussion, but micro-optimizations do work, if there are plenty of opportunities. I.e. benefit > time. The basic rule here is ~+5% performance, or more, should be worth the time. In reality those figures will come down to 2-3% performance plus, but depending on how much time your process takes, the total benefit can be huge. Can a technology change improve performance? This is a big one, a very big one. Before you even consider to switch to a different technology, you have to do "the math". Is it even possible to migrate the "old" storage, to the "new" one? How much time, and in the end money, does this cost? Is the benefit high enough? And so on, and so forth. This is a step i usually don't recommend, but it can pay out in the long run. Is that what the client wants? In the end, the client has the last word. If he is satisfied with the performance of the application and doesn't want any optimizations, then the situation is very clear. Even if other people, colleagues, think otherwise. Greetings Daniel

    1 Reply Last reply
    0
    • R ravikhoda

      Today i came across a scene where we had a discussion on this topic. we have some work on the database side where batch files data is insert (row count is several thousand per operations )/ some text files generated by windows services / flag status maintain based on the data insert in the database per operation and many other small operations happen during this process. obviously correctness of the operation is must but at the same time performance of the operation came in to the picture. some teammates says that as there are lot of process happens during per operations it will take time to complete things while some believe that there can be something done to reduce the time. Client is okay even if that takes time as for him accuracy is more important rather than time taken. How to manage such scenarios ? after some level we can not think of optimization on query as all the things which is written is necessary. does anyone face such things ? how do you manage things to improve the performance?

      Ravi Khoda

      J Offline
      J Offline
      Jorgen Andersson
      wrote on last edited by
      #6

      You lost me already at the subject line. If it's not accurate, it's buggy, and therefore wrong.

      Wrong is evil and must be defeated. - Jeff Ello[^]

      S 1 Reply Last reply
      0
      • R ravikhoda

        Today i came across a scene where we had a discussion on this topic. we have some work on the database side where batch files data is insert (row count is several thousand per operations )/ some text files generated by windows services / flag status maintain based on the data insert in the database per operation and many other small operations happen during this process. obviously correctness of the operation is must but at the same time performance of the operation came in to the picture. some teammates says that as there are lot of process happens during per operations it will take time to complete things while some believe that there can be something done to reduce the time. Client is okay even if that takes time as for him accuracy is more important rather than time taken. How to manage such scenarios ? after some level we can not think of optimization on query as all the things which is written is necessary. does anyone face such things ? how do you manage things to improve the performance?

        Ravi Khoda

        V Offline
        V Offline
        V 0
        wrote on last edited by
        #7

        Increasing performance is never a lost exercise even if you shouldn't implement it right away. When the product matures and new functionality is added you can optimize (or often by then, it is needed). Although the first thought should always be to optimize algorithms or indexes, other tricks do exist which make applications perform better. eg. We used to have an application that showed a form with many comboboxes loaded with thousands and thousands of records from the database. They ALL had to be in there. To increase the performance of that winforms application we did the following: * first used suspendlayout-resumelayout functions to avoid reloading of the comboboxes. * replaced the comboboxes with a textbox and a button. the button opened a small dialog with the records (which were loaded only then). Since nothing had to be loaded on form load the user had the feeling the application behaved better. others: * When using grids you can also use paging. In listboxes there's a way to load only the visible part (+ a small extra part above and below). When scrolling the rest is loaded. * You can use seperate threads * you can perform calculations on the server and notify the application when done... * ... Point is, there are many different ways to improve performance. You don't need to implement it right away, but it's a good exercise to start thinking about options that might be acceptable for the client. hope this helps.

        V.
        (MQOTD rules and previous solutions)

        1 Reply Last reply
        0
        • R ravikhoda

          Today i came across a scene where we had a discussion on this topic. we have some work on the database side where batch files data is insert (row count is several thousand per operations )/ some text files generated by windows services / flag status maintain based on the data insert in the database per operation and many other small operations happen during this process. obviously correctness of the operation is must but at the same time performance of the operation came in to the picture. some teammates says that as there are lot of process happens during per operations it will take time to complete things while some believe that there can be something done to reduce the time. Client is okay even if that takes time as for him accuracy is more important rather than time taken. How to manage such scenarios ? after some level we can not think of optimization on query as all the things which is written is necessary. does anyone face such things ? how do you manage things to improve the performance?

          Ravi Khoda

          J Offline
          J Offline
          JimmyRopes
          wrote on last edited by
          #8

          ravikhoda wrote:

          Client is okay even if that takes time as for him accuracy is more important rather than time taken.

          The client is always right. :cool:

          The report of my death was an exaggeration - Mark Twain
          Simply Elegant Designs JimmyRopes Designs
          I'm on-line therefore I am. JimmyRopes

          D M 2 Replies Last reply
          0
          • J JimmyRopes

            ravikhoda wrote:

            Client is okay even if that takes time as for him accuracy is more important rather than time taken.

            The client is always right. :cool:

            The report of my death was an exaggeration - Mark Twain
            Simply Elegant Designs JimmyRopes Designs
            I'm on-line therefore I am. JimmyRopes

            D Offline
            D Offline
            Daniel Lieberwirth BrainInBlack
            wrote on last edited by
            #9

            Not always, sometimes you have to convince the client to do the right thing. For example, i had a client that had a existing web application and my task was it to add features for which the original code never was designed for. I.e. it would have take me longer to implement the new code, than rewrite the whole application from scratch. So i had to convince my client to do "the right thing", and let me rework the whole thing for the same price.

            J J 2 Replies Last reply
            0
            • J JimmyRopes

              ravikhoda wrote:

              Client is okay even if that takes time as for him accuracy is more important rather than time taken.

              The client is always right. :cool:

              The report of my death was an exaggeration - Mark Twain
              Simply Elegant Designs JimmyRopes Designs
              I'm on-line therefore I am. JimmyRopes

              M Offline
              M Offline
              Mycroft Holmes
              wrote on last edited by
              #10

              I call bullshit, I don't know how his job is done, why should I expect him to know mine. The client has the privilege of pay the bill and requesting the solution, the detail are not his concern.

              Never underestimate the power of human stupidity RAH

              1 Reply Last reply
              0
              • D Daniel Lieberwirth BrainInBlack

                Not always, sometimes you have to convince the client to do the right thing. For example, i had a client that had a existing web application and my task was it to add features for which the original code never was designed for. I.e. it would have take me longer to implement the new code, than rewrite the whole application from scratch. So i had to convince my client to do "the right thing", and let me rework the whole thing for the same price.

                J Offline
                J Offline
                Jorgen Andersson
                wrote on last edited by
                #11

                Hi there, I have an application in the need of a new feature... :)

                Wrong is evil and must be defeated. - Jeff Ello[^]

                D 1 Reply Last reply
                0
                • J Jorgen Andersson

                  Hi there, I have an application in the need of a new feature... :)

                  Wrong is evil and must be defeated. - Jeff Ello[^]

                  D Offline
                  D Offline
                  Daniel Lieberwirth BrainInBlack
                  wrote on last edited by
                  #12

                  How about, NO? :laugh: Done it once, and will never again. ;P

                  J 1 Reply Last reply
                  0
                  • D Daniel Lieberwirth BrainInBlack

                    How about, NO? :laugh: Done it once, and will never again. ;P

                    J Offline
                    J Offline
                    Jorgen Andersson
                    wrote on last edited by
                    #13

                    Fair enough. :)

                    Wrong is evil and must be defeated. - Jeff Ello[^]

                    1 Reply Last reply
                    0
                    • D Daniel Lieberwirth BrainInBlack

                      Not always, sometimes you have to convince the client to do the right thing. For example, i had a client that had a existing web application and my task was it to add features for which the original code never was designed for. I.e. it would have take me longer to implement the new code, than rewrite the whole application from scratch. So i had to convince my client to do "the right thing", and let me rework the whole thing for the same price.

                      J Offline
                      J Offline
                      JimmyRopes
                      wrote on last edited by
                      #14

                      Daniel Lieberwirth (BrainInBlack) wrote:

                      Not always, sometimes you have to convince the client to do the right thing.

                      In this case the client is doing the right thing. They wart accuracy over performance. How can you argue with that? :~

                      The report of my death was an exaggeration - Mark Twain
                      Simply Elegant Designs JimmyRopes Designs
                      I'm on-line therefore I am. JimmyRopes

                      D S 2 Replies Last reply
                      0
                      • J JimmyRopes

                        Daniel Lieberwirth (BrainInBlack) wrote:

                        Not always, sometimes you have to convince the client to do the right thing.

                        In this case the client is doing the right thing. They wart accuracy over performance. How can you argue with that? :~

                        The report of my death was an exaggeration - Mark Twain
                        Simply Elegant Designs JimmyRopes Designs
                        I'm on-line therefore I am. JimmyRopes

                        D Offline
                        D Offline
                        Daniel Lieberwirth BrainInBlack
                        wrote on last edited by
                        #15

                        I didn't argue, i just challenged your somewhat general statement that the client is alway right. He isn't and that's the point i wanted to make clear with my statement.

                        J 2 Replies Last reply
                        0
                        • D Daniel Lieberwirth BrainInBlack

                          I didn't argue, i just challenged your somewhat general statement that the client is alway right. He isn't and that's the point i wanted to make clear with my statement.

                          J Offline
                          J Offline
                          JimmyRopes
                          wrote on last edited by
                          #16

                          Whenever in doubt the customer is always right. :doh:

                          The report of my death was an exaggeration - Mark Twain
                          Simply Elegant Designs JimmyRopes Designs
                          I'm on-line therefore I am. JimmyRopes

                          1 Reply Last reply
                          0
                          • D Daniel Lieberwirth BrainInBlack

                            I didn't argue, i just challenged your somewhat general statement that the client is alway right. He isn't and that's the point i wanted to make clear with my statement.

                            J Offline
                            J Offline
                            JimmyRopes
                            wrote on last edited by
                            #17

                            Whenever in doubt the customer is always right. The only exception to that is if you do not want to be paid for your efforts. :doh:

                            The report of my death was an exaggeration - Mark Twain
                            Simply Elegant Designs JimmyRopes Designs
                            I'm on-line therefore I am. JimmyRopes

                            J 1 Reply Last reply
                            0
                            • R ravikhoda

                              Today i came across a scene where we had a discussion on this topic. we have some work on the database side where batch files data is insert (row count is several thousand per operations )/ some text files generated by windows services / flag status maintain based on the data insert in the database per operation and many other small operations happen during this process. obviously correctness of the operation is must but at the same time performance of the operation came in to the picture. some teammates says that as there are lot of process happens during per operations it will take time to complete things while some believe that there can be something done to reduce the time. Client is okay even if that takes time as for him accuracy is more important rather than time taken. How to manage such scenarios ? after some level we can not think of optimization on query as all the things which is written is necessary. does anyone face such things ? how do you manage things to improve the performance?

                              Ravi Khoda

                              Sander RosselS Offline
                              Sander RosselS Offline
                              Sander Rossel
                              wrote on last edited by
                              #18

                              I run into performance issues all the time. These are not easy to fix and sometimes it means looking at your design critically and throw it overboard. That's not something you can sell to customers. When you're lucky it's a matter of adding some indexes to your database, but sometimes you just have to face the fact that further performance optimization is not possible unless you spend days, or even weeks, working on a single issue. For our customers the costs that come with such tasks far outweigh the benefits that come with it (a form might load a few seconds faster). Accuracy should always be your most important objective. If your results are inaccurate they are useless and if the data is useless performance is not an issue either. Inaccurate data is still inaccurate at the speed of light.

                              It's an OO world.

                              public class SanderRossel : Lazy<Person>
                              {
                              public void DoWork()
                              {
                              throw new NotSupportedException();
                              }
                              }

                              1 Reply Last reply
                              0
                              • R ravikhoda

                                Today i came across a scene where we had a discussion on this topic. we have some work on the database side where batch files data is insert (row count is several thousand per operations )/ some text files generated by windows services / flag status maintain based on the data insert in the database per operation and many other small operations happen during this process. obviously correctness of the operation is must but at the same time performance of the operation came in to the picture. some teammates says that as there are lot of process happens during per operations it will take time to complete things while some believe that there can be something done to reduce the time. Client is okay even if that takes time as for him accuracy is more important rather than time taken. How to manage such scenarios ? after some level we can not think of optimization on query as all the things which is written is necessary. does anyone face such things ? how do you manage things to improve the performance?

                                Ravi Khoda

                                K Offline
                                K Offline
                                kalberts
                                wrote on last edited by
                                #19

                                1st law of optimization: Don't do it! 2nd law of optimization: If you HAVE TO do, it don't do it yet! (To me, it sounds as if the 2nd law is not appliccable in your case)

                                1 Reply Last reply
                                0
                                • J Jorgen Andersson

                                  You lost me already at the subject line. If it's not accurate, it's buggy, and therefore wrong.

                                  Wrong is evil and must be defeated. - Jeff Ello[^]

                                  S Offline
                                  S Offline
                                  SortaCore
                                  wrote on last edited by
                                  #20

                                  But all doubles can be rounded up into ints... We'll save 4 bytes! That's FIFTY PERCENT LESS MEMORY USAGE!!! :omg:

                                  1 Reply Last reply
                                  0
                                  • R ravikhoda

                                    Today i came across a scene where we had a discussion on this topic. we have some work on the database side where batch files data is insert (row count is several thousand per operations )/ some text files generated by windows services / flag status maintain based on the data insert in the database per operation and many other small operations happen during this process. obviously correctness of the operation is must but at the same time performance of the operation came in to the picture. some teammates says that as there are lot of process happens during per operations it will take time to complete things while some believe that there can be something done to reduce the time. Client is okay even if that takes time as for him accuracy is more important rather than time taken. How to manage such scenarios ? after some level we can not think of optimization on query as all the things which is written is necessary. does anyone face such things ? how do you manage things to improve the performance?

                                    Ravi Khoda

                                    B Offline
                                    B Offline
                                    BobJanova
                                    wrote on last edited by
                                    #21

                                    Accuracy is the only thing that matters until the client starts worrying about performance. In your case that seems to be clearly not the case so why are you worrying about it? For any algorithm there is a minimum time that it will take, and while you never know what that time is or manage to get particularly close to it, optimisation attempts do often hit a wall where any further increase in performance is not worth the effort. If you've done the quick wins (batching data at an appropriate size, indexing your tables on the right columns, using fast text writing) then you are almost certainly against that wall.

                                    1 Reply Last reply
                                    0
                                    • R ravikhoda

                                      Today i came across a scene where we had a discussion on this topic. we have some work on the database side where batch files data is insert (row count is several thousand per operations )/ some text files generated by windows services / flag status maintain based on the data insert in the database per operation and many other small operations happen during this process. obviously correctness of the operation is must but at the same time performance of the operation came in to the picture. some teammates says that as there are lot of process happens during per operations it will take time to complete things while some believe that there can be something done to reduce the time. Client is okay even if that takes time as for him accuracy is more important rather than time taken. How to manage such scenarios ? after some level we can not think of optimization on query as all the things which is written is necessary. does anyone face such things ? how do you manage things to improve the performance?

                                      Ravi Khoda

                                      C Offline
                                      C Offline
                                      Christophe Van Olmen
                                      wrote on last edited by
                                      #22

                                      Probably something you already considered but which can have an enormous impact in loading your data: transactions. Grouping as much data together before committing can increase loading performance to a factor 100 (as usual, experiment with size of transaction for best result in your situation). Timing of the process is also extremely important: you might experience slow performance due to locking issues. To me, accuracy seems the most important criterium (inaccurate = wrong).

                                      1 Reply Last reply
                                      0
                                      • R ravikhoda

                                        Today i came across a scene where we had a discussion on this topic. we have some work on the database side where batch files data is insert (row count is several thousand per operations )/ some text files generated by windows services / flag status maintain based on the data insert in the database per operation and many other small operations happen during this process. obviously correctness of the operation is must but at the same time performance of the operation came in to the picture. some teammates says that as there are lot of process happens during per operations it will take time to complete things while some believe that there can be something done to reduce the time. Client is okay even if that takes time as for him accuracy is more important rather than time taken. How to manage such scenarios ? after some level we can not think of optimization on query as all the things which is written is necessary. does anyone face such things ? how do you manage things to improve the performance?

                                        Ravi Khoda

                                        R Offline
                                        R Offline
                                        rnbergren
                                        wrote on last edited by
                                        #23

                                        You can have it - Correctly Done - Quickly Done - Cheaply Done Pick no more than 2 I think your client understands this.

                                        To err is human to really mess up you need a computer

                                        1 Reply Last reply
                                        0
                                        • J JimmyRopes

                                          Whenever in doubt the customer is always right. The only exception to that is if you do not want to be paid for your efforts. :doh:

                                          The report of my death was an exaggeration - Mark Twain
                                          Simply Elegant Designs JimmyRopes Designs
                                          I'm on-line therefore I am. JimmyRopes

                                          J Offline
                                          J Offline
                                          jeffreystacks
                                          wrote on last edited by
                                          #24

                                          1- The client is right about WHAT they want. Client is usually never right about HOW to do it or how much it will cost. 2- Accuracy over performance. Accuracy is usually a black or white determination (not always, but usually). Performance can be tweaked and dialed up in degrees over time and iterations. To paraphrase...first make it work, then make it work fast, then make it elegant.

                                          J 1 Reply Last reply
                                          0
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes


                                          • Login

                                          • Don't have an account? Register

                                          • Login or register to search.
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • World
                                          • Users
                                          • Groups