How NOT to optimize a database!
-
That isn't as silly as it sounds - for a couple of reasons: 1) If the table is reasonably dynamic and the Key is defined as an AutoNumber, its value might get out of the int range in finite time, even if there were never more than a few hundred records in the table simultaneously. 2) If you maintain the conventions that your Primary Key is always a bigint, you always know how to declare linking fields in other tables without going back to check.
Sure, having a bigint as key is not the end of the world (certainly better than char(3) for storing numerics or (tiny)int where more is needed), but if none of your points apply it certainly deserves no beauty award ;) What I am talking about is a few hundred records ever (simultaneously or not) and every table having defined another key anyway (char, varchar, byte, tinyint, int, bigint, multiple columns etc.). Which might not deserve a beauty award at all ;p
It's an OO world.
public class Naerling : Lazy<Person>{}
-
Certainly, but there are usually a lot of other optimizations that can be made that will make much more difference. You're almost always better off trying NOT to design something that can be easily broken through usage patterns. The PITA you save may be your own.
Other optimizations exist, but making things as small as possible is often overlooked. People believe what you said - that other optimizations are better. I think this is backwards thinking. In my 25 years of doing this stuff, optimizing the size of your objects has fallen out of favor, but it still has the same performance impact that it has always had, and we see it in the results. I'm willing to go out on a limb and say, probably, most of the performance issues we face with modern software are rooted in this problem. Sure, you can take something that's too big and make the performance even worse, but if you don't start with an efficient object, you've shot yourself in the foot before you even started the race.
-
The spec is " to hold the reservation token (001 to 999) on a per-train basis", reservations per train, not tickets per country if read correctly.
aha! its 999 e-reservations per train considering that not all people book tickets online which would mean in the worst case scenario if you have 999 e-bookings and lets say even 1 non-electronic booking the system would still fail. Coz as i understand electronic reservation system is the same as the one used by staff over the counters. Weird. Also one ticket means >= 1 person (not sure what relevance that has to my statement :) ).
-
The spec is " to hold the reservation token (001 to 999) on a per-train basis", reservations per train, not tickets per country if read correctly.
In retrospect though, this is the first time I have come across the Indian railway reservation system crashing which probably could have been avoided by a software check which could have just rejected 1000th reservation. Short-sightedness on the developers' part or over-confidence? who knows....
-
Other optimizations exist, but making things as small as possible is often overlooked. People believe what you said - that other optimizations are better. I think this is backwards thinking. In my 25 years of doing this stuff, optimizing the size of your objects has fallen out of favor, but it still has the same performance impact that it has always had, and we see it in the results. I'm willing to go out on a limb and say, probably, most of the performance issues we face with modern software are rooted in this problem. Sure, you can take something that's too big and make the performance even worse, but if you don't start with an efficient object, you've shot yourself in the foot before you even started the race.
Perhaps, but I think there is a distinction to be made in WHY they are too big. One can minimize the size in ways that are more or less flexible. Obviously they chose one that was less rather then more.
-
http://articles.economictimes.indiatimes.com/2011-10-23/news/30313183_1_diwali-rush-western-railway-berths[^] TLDR : The IT company that designed the online reservation system for India's western railways used a 3-character field storing numbers to hold the reservation token (001 to 999) on a per-train basis. This limit was never ever hit until last week when (due to Diwali, an Indian festival) enough people made online bookings that the database crashed and with it the reservation system. Funny side-bit : 3 characters (even non-Unicode) is still larger than a short int (memory wise). So whoever did this pseudo-optimization was terribly ignorant.
Regards, Nish
My technology blog: voidnish.wordpress.com Part 2 in my WinRT/C++ series : Visual C++ and WinRT/Metro - Databinding Basics
-
The strange thing is that the 3 char field uses 3 bytes and only allows up to 999. Depending on the platform, a 2 byte int would allow at least 32767 entries.
rjemp wrote:
The strange thing is that the 3 char field uses 3 bytes and only allows up to 999. Depending on the platform, a 2 byte int would allow at least 32767 entries.
You probably missed this but the original post mentions this very thing. :rolleyes:
Regards, Nish
My technology blog: voidnish.wordpress.com Part 2 in my WinRT/C++ series : Visual C++ and WinRT/Metro - Databinding Basics
-
rjemp wrote:
The strange thing is that the 3 char field uses 3 bytes and only allows up to 999. Depending on the platform, a 2 byte int would allow at least 32767 entries.
You probably missed this but the original post mentions this very thing. :rolleyes:
Regards, Nish
My technology blog: voidnish.wordpress.com Part 2 in my WinRT/C++ series : Visual C++ and WinRT/Metro - Databinding Basics
-
If every bit helps, they could have saved 8 for every record recorded by using smallint and this would have allowed more than 32 times the 1000 limit. They weren't saving bits, they were showing ignorance. (Per another poster, the ignorance of the manager who can't code.)
-
Every bit counted when your memory was limited to 4K and disk sizes were 5 MB. That gave us the Y2K problem. With 4GB of main memory and terabyte disks aplenty, there is NO reason at all today for this kind of design.
-
rjemp wrote:
The strange thing is that the 3 char field uses 3 bytes and only allows up to 999. Depending on the platform, a 2 byte int would allow at least 32767 entries.
You probably missed this but the original post mentions this very thing. :rolleyes:
Regards, Nish
My technology blog: voidnish.wordpress.com Part 2 in my WinRT/C++ series : Visual C++ and WinRT/Metro - Databinding Basics
-
Heh :-)
Regards, Nish
My technology blog: voidnish.wordpress.com Part 2 in my WinRT/C++ series : Visual C++ and WinRT/Metro - Databinding Basics
-
You have to understand the work culture in India. You as a programmer go into a meeting to discuss the Reservation System. The manager tells you not just what the requirements are but also how exactly it must be designed. If you don't follow his instructions to the letter (this assumes you actually have the ability to think for yourself, which is a rare commodity in Indian IT graduates) but deviate even slightly from what you are instructed to do, you will be forced to re-do it the manager's way. Any suggestions from you to improve usability will be turned down. Just go to the online reservation system for Indian Railways. To get from Chennai to Thiruvananthapuram (get Nishant to say it aloud so you know how to pronounce it :laugh: ), you have to know the station codes for both stations, and the train numbers of the various trains between the two stations. You have to backtrack from the reservations screen to a different screen to get the station code, to yet another screen to get all the train numbers which you must write down on paper so that you can input it when needed. The woes go on. We may use computers in India but we will make them as difficult as a face-to-face encounter at the ticket counter! Yes, at the ticket counter, they want you write the train numbers on the ticket rquest form too and you will have no computers to look them up!
Err.. I think you went to some other website, because I have been using irctc website for quite sometime and I needn't do any of such things you mentioned. Infact, its a simple wizard kind of interface where you type in source & destination city, select from a list of available trains and book it - simple. Anyway, coming to topic, I think the fault lies with lack of foresightedness. One rule of thumb which I use personally is to assume Moore's Law on expected data usage -- because software generally end up using more memory than initially planned and secondly, it happen to last more than designed for.
-
Chris Meech wrote:
Today, the moment 999 was breached, the program developed a bug
No, the bug was already there, guys.
cheers, Chris Maunder The Code Project | Co-founder Microsoft C++ MVP
Yes, it just got visible. Funny old cartoons on bugs: Software Bug's Life[^]
Regards, Jwalant Natvarlal Soneji
-
The Y2K problem came from mainframes which completely pre-date micro-computers and had nothing to do with memory or disk size. Actually, they were storing 2 digit numbers in 16 bits of storage. Don't tell me it had anything to do with memory or storage.
It had every thing to do with disk or main memory sizes. Unless of course you think that programmers were stupid in the 1960s through the 1990s and Intelligent Life dawned on Earth only after the PC arrived. The largest IBM 370/168 shipped usually with 4 MB of main memory. Anything more was tremendously expensive, though it could go up to 16MB, if memory serves me right. Disk sizes were 350MB each on the IBM 3350 disk drives and a big bank could afford 20 of those drives. So programmers counted every bit and byte when they created records for storing on the disk drives.
-
If every bit helps, they could have saved 8 for every record recorded by using smallint and this would have allowed more than 32 times the 1000 limit. They weren't saving bits, they were showing ignorance. (Per another poster, the ignorance of the manager who can't code.)
The world didn't use C to write business programs but COBOL, which you would probably say is evidence of their stupidity. Programmers used Binary Coded Decimal (BCD) to represent numbers. They could have used COMPUTATIONAL-3 which would be pure binary representation but conversion from binary to BCD and back was expensive in terms of instruction execution. Computers existed before you were born as did beer and sex but I am willing to grant PC programmers the right to claim they invented all three. ;P
-
The Y2K problem came from mainframes which completely pre-date micro-computers and had nothing to do with memory or disk size. Actually, they were storing 2 digit numbers in 16 bits of storage. Don't tell me it had anything to do with memory or storage.
You might want to look up the Y2038 problem created by the oh-so-clever Unix programmers. http://en.wikipedia.org/wiki/Year_2038_problem[^]
-
It had every thing to do with disk or main memory sizes. Unless of course you think that programmers were stupid in the 1960s through the 1990s and Intelligent Life dawned on Earth only after the PC arrived. The largest IBM 370/168 shipped usually with 4 MB of main memory. Anything more was tremendously expensive, though it could go up to 16MB, if memory serves me right. Disk sizes were 350MB each on the IBM 3350 disk drives and a big bank could afford 20 of those drives. So programmers counted every bit and byte when they created records for storing on the disk drives.
Y2K came straight from COBOL formatted input. I've forgotten my COBOL formatting, but it was something like 99V99V99 for a date format. That became an industry standard when people didn't expect code to last 5 years let alone 25. That was a handy format because when you looked at a keypunch card, read 122773, you could tell that was December 27, 1973. The cards used 9 holes per character but only used a byte per character. After COBOL read it in, it retained it in the 6 byte format for a total range of valid values around 36,530. You are totally wrong about memory having anything whatsoever to do with the Y2K problem. Why did Y2K have ANYTHING to do with the year 2000, if the above wasn't the reason for it happening. The Unix Y2K problem is interesting. Didn't even know about it until now. Perfectly fits into the binary world. There is an easy fix if they are swapping into negative numbers in 2038. Store it in uint format. Same number of bits, all former dates exactly translate into the new format and you add another 138+ years before you run into the problem again. Sure, it'll rear it's ugly head again, but do you think you'll care about it? Who knows, maybe they'll even plan ahead by then. Of course you lose predating 1900 dates.
-
Another case of Famous. Last Words. When I was programming databases (actually flat files) back in '76, I encountered a date field that needed to be stored. I asked my manager whether I should store a 2 digit or 4 digit year. I was concerned about the upcoming millennium. He said, "Don't worry, by that time, the software will be replaced." Famous. Last. Words. I have had a few other times where managers would tell me limits that would not be exceeded and in the end, I learned to ignore them and build something that had excess capacity.
Psychosis at 10 Film at 11 Those who do not remember the past, are doomed to repeat it. Those who do not remember the past, cannot build upon it.
This is classic, absolutely classic example of the narrow minded, short-sightedness that so many in the corporate world engage in. It never ceases to amaze me how stupid these so called leaders in bussiness can be at times. I bet that behind %90 of all incidents like this, where poor decision making that could have been avoided causes woes for many, had at least 1 low to entry level employee who tried to point out [to management] the potential downfall from the bad decission about to be made only to be met with the classic response "That'll never happen". It took 10 years (many years back) for me to see the results of a bad decission made by management that I tried to warn them about only to b met with the it will never happen line. ANyone else.
-
Chris Meech wrote:
Today, the moment 999 was breached, the program developed a bug
No, the bug was already there, guys.
cheers, Chris Maunder The Code Project | Co-founder Microsoft C++ MVP
