Best Database Configures with c# win. form
-
yes, i mean store and search. As my work is little tricky it has to search through 100 million records every hour.
You have to manipulate ALL of those 100 million records every hour? Or is it just some, and then what percentage? And what kind of reporting do you need to do? On how many records? Fact is that it's not so important which DB you're having as long as you're just doing CRUD. The size of the database is also not very important for the performance (Were talking about a factor of a couple of hundred times larger for every new level in an index, assuming B-Tree index). It's the number of transactions! And how many records that are affected by the queries. So the limiting factors will be the hardware (mostly the harddrives), the configuration of the hardware (It doesn't matter if the drives are fast if they're in a RAID5), and the indexes. If you have no indexes at all, all inserts will be ridiculously fast, but querying or updating will be just as ridiculously slow. Adding every index you can think about will make both inserts and updates ridiculously slow instead, while the queries will be faster. Note that updates needs to both read and write. Finding the right indexes will give you a good balance.
Wrong is evil and must be defeated. - Jeff Ello[^]
-
yes, i mean store and search. As my work is little tricky it has to search through 100 million records every hour.
-
sir, in this context i used "manipulate" for "searching"
-
sir, in this context i used "manipulate" for "searching"
Then you are using the wrong terms. "Manipulate" means updating, inserting or deleting records. Reading doesn't manipulate the data, it fetches it. Selecting a few million should be doable, depending on the hardware and software-combination and the skillset of the dba.
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^]
-
Then you are using the wrong terms. "Manipulate" means updating, inserting or deleting records. Reading doesn't manipulate the data, it fetches it. Selecting a few million should be doable, depending on the hardware and software-combination and the skillset of the dba.
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^]
sorry, for all this.i have corrected my reply.
-
sorry, for all this.i have corrected my reply.
Databases are optimized to work with records; there will be a HUGE difference between searching records (select using a where) and searching a specific substring within a NTEXT field. If you are going to search within the contents of the fiels, you'd be wanting a full-text search catalog. Again, supported by most major databases, but their speed may vary wildly. Can you post a schema of the data that you'll be storing? Should I be thinking about simple data like measurements or prices? (lots o' fields with numbers) Or more toward text? (lots of short readable text-fields, like profiles), more toward memo's (single large textfield), or even documents (Word, PDF?) In the case of documents I'd suggest to dump the files in the filesystem - and to use something like Google Desktop Search to search for specific terms.
Agent_Spock wrote:
sorry, for all this
Don't be; for someone who doesn't code all those things might sound roughly the same. Sorry for my short and blunt answers.
Live long and prosper :)
-
Databases are optimized to work with records; there will be a HUGE difference between searching records (select using a where) and searching a specific substring within a NTEXT field. If you are going to search within the contents of the fiels, you'd be wanting a full-text search catalog. Again, supported by most major databases, but their speed may vary wildly. Can you post a schema of the data that you'll be storing? Should I be thinking about simple data like measurements or prices? (lots o' fields with numbers) Or more toward text? (lots of short readable text-fields, like profiles), more toward memo's (single large textfield), or even documents (Word, PDF?) In the case of documents I'd suggest to dump the files in the filesystem - and to use something like Google Desktop Search to search for specific terms.
Agent_Spock wrote:
sorry, for all this
Don't be; for someone who doesn't code all those things might sound roughly the same. Sorry for my short and blunt answers.
Live long and prosper :)
no sir, it consists of data which is converted into bytes.
-
no sir, it consists of data which is converted into bytes.
Everything that a computer stores is encoded in bytes; images, text, applications - they're all stored as bytes. Hence, the remark that it's going to store bytes is not very helpfull. That way I'd assume a large binary blob, and "searching" to be a series of bytes. Those bytes represent something; data, in whatever form. What "kind" of data you're going to store determines the best approach.
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^]
-
Everything that a computer stores is encoded in bytes; images, text, applications - they're all stored as bytes. Hence, the remark that it's going to store bytes is not very helpfull. That way I'd assume a large binary blob, and "searching" to be a series of bytes. Those bytes represent something; data, in whatever form. What "kind" of data you're going to store determines the best approach.
Bastard Programmer from Hell :suss: If you can't read my code, try converting it here[^]
no i am using it in such way that each record has max. 25 letters
-
no i am using it in such way that each record has max. 25 letters
-
actually i am already working on sql but i am not sure that it can manage 100 million record. somebody suggested me oracle is oracle better than SQL.
Agent_Spock wrote:
sql but i am not sure that it can manage 100 million record.
I've worked with a SQL database that had a single table with a few billion rows (Inventory type stuff). So, yea SQL can scale as far as you need it too.
Common sense is admitting there is cause and effect and that you can exert some control over what you understand.
-
Agent_Spock wrote:
sql but i am not sure that it can manage 100 million record.
I've worked with a SQL database that had a single table with a few billion rows (Inventory type stuff). So, yea SQL can scale as far as you need it too.
Common sense is admitting there is cause and effect and that you can exert some control over what you understand.
and sir, is the searching good (fast).
-
and sir, is the searching good (fast).
-
Yup, never a problem. The table was indexed for the usage.
Common sense is admitting there is cause and effect and that you can exert some control over what you understand.
Does "Indexed" mean using primary key as filter or something else?