Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Code Project
CODE PROJECT For Those Who Code
  • Home
  • Articles
  • FAQ
Community
  1. Home
  2. The Lounge
  3. How Do I Write A Data Management System? PLEASE HELP!

How Do I Write A Data Management System? PLEASE HELP!

Scheduled Pinned Locked Moved The Lounge
questioncomhelpannouncement
40 Posts 16 Posters 13 Views 1 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • T TheGreatAndPowerfulOz

    Paul Watson wrote: Imagine the detail in the Playboy shots eeeewww! gross!

    C Offline
    C Offline
    Christian Graus
    wrote on last edited by
    #4

    Add one more to the list of female CPians. Christian Graus - Microsoft MVP - C++

    E T 2 Replies Last reply
    0
    • M Marc Clifton

      I hope you can all help me! I need to write a data management system to manage 30 terabytes of data that is being generated nightly! Yes, there's quite a night life where I live, I guess! Here's[^] a link describing this project! (My professor wants it done by the end of the semister, BTW) But hey, I get to play with a cool 3000M pixel camera!!! :-D Marc My website Traceract

      R Offline
      R Offline
      Rolando Cruz
      wrote on last edited by
      #5

      Hi, OK, 30 terabytes. Is this data formated in any way? What is represented wihtin the 30 TBytes? Images? Dates? Times? Stellar Coordinates? As you can see your initial question is too general. :doh: Please be more specific. :) What are the results you want to get out of the DMS? What information do you want to store in this DMS? I'm willing to point you in the right direction. If you want you can directly send me an email to discuss this project of yours in more detail. Rolando :suss:

      M 1 Reply Last reply
      0
      • M Marc Clifton

        I hope you can all help me! I need to write a data management system to manage 30 terabytes of data that is being generated nightly! Yes, there's quite a night life where I live, I guess! Here's[^] a link describing this project! (My professor wants it done by the end of the semister, BTW) But hey, I get to play with a cool 3000M pixel camera!!! :-D Marc My website Traceract

        E Offline
        E Offline
        El Corazon
        wrote on last edited by
        #6

        Sounds like it is already done... your problem is solved. Congradulations! This unique system for surveying the heavens **is made possible by advances in several technologies including**: * Large optics fabrication to create the telescope's distinctive 3-mirror design which includes a convex 4-meter secondary mirror, the size of many primary mirrors on today's large research telescopes. *** Data management systems to process and catalog the 30 terabytes of data generated nightly, the equivalent of 7,000 DVDs.** * New detectors needed to build the LSST's 3 billion pixel digital camera, the largest ever created. _________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb) -- modified at 20:05 Monday 5th September, 2005

        1 Reply Last reply
        0
        • C Christian Graus

          Add one more to the list of female CPians. Christian Graus - Microsoft MVP - C++

          E Offline
          E Offline
          El Corazon
          wrote on last edited by
          #7

          Christian Graus wrote: Add one more to the list of female CPians. sorry, I agree with ahz. Looking at pores just doesn't seem appealing. Plus you can easily use area-of-interest level-of-detail since you can pretty much guarentee zoom of only a few areas of interest.... chances are you could go months before someone discovered you didn't store it all. :rolleyes: _________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)

          C 1 Reply Last reply
          0
          • P Paul Watson

            Go back to school kid and then when you get out get some experience and then maybe you can come back here and be privileged enough to ask a question. Now get! :P (Cool camera though. Imagine the detail in the Playboy shots you could get from it.) regards, Paul Watson South Africa Colib and WebTwoZero. K(arl) wrote: oh, and BTW, CHRISTIAN ISN'T A PARADOX, HE IS A TASMANIAN!

            J Offline
            J Offline
            Jim Crafton
            wrote on last edited by
            #8

            Paul Watson wrote: (Cool camera though. Imagine the detail in the Playboy shots you could get from it.) Well as they say, there's levels of detail, and then there's levels of detail. ¡El diablo está en mis pantalones! ¡Mire, mire! Real Mentats use only 100% pure, unfooled around with Sapho Juice(tm)! SELECT * FROM User WHERE Clue > 0 0 rows returned Save an Orange - Use the VCF!

            1 Reply Last reply
            0
            • E El Corazon

              Christian Graus wrote: Add one more to the list of female CPians. sorry, I agree with ahz. Looking at pores just doesn't seem appealing. Plus you can easily use area-of-interest level-of-detail since you can pretty much guarentee zoom of only a few areas of interest.... chances are you could go months before someone discovered you didn't store it all. :rolleyes: _________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)

              C Offline
              C Offline
              Christian Graus
              wrote on last edited by
              #9

              I more meant that I couldn't imagine a guy saying 'eeeww gross'. Christian Graus - Microsoft MVP - C++

              E 1 Reply Last reply
              0
              • C Christian Graus

                I more meant that I couldn't imagine a guy saying 'eeeww gross'. Christian Graus - Microsoft MVP - C++

                E Offline
                E Offline
                El Corazon
                wrote on last edited by
                #10

                Christian Graus wrote: I couldn't imagine a guy saying Ahhhh.... true.... I think my phrasing would have been... "uhh... I'll pass on THAT much detail." although compression using self-similarity and area related Level of detail could compress such images... after all, though one pore may not look exactly the same as another, it will look similar, you can use difference compression with area-offsets as you get higher and higher compression on similar looking details. :) Interesting idea.... I think I could do it... but I would need volunteer for the gigapixel imaging.... somehow I doubt playboy magazine would provide that detail, I'd be looking at paper fibers. ;P _________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)

                1 Reply Last reply
                0
                • R Rolando Cruz

                  Hi, OK, 30 terabytes. Is this data formated in any way? What is represented wihtin the 30 TBytes? Images? Dates? Times? Stellar Coordinates? As you can see your initial question is too general. :doh: Please be more specific. :) What are the results you want to get out of the DMS? What information do you want to store in this DMS? I'm willing to point you in the right direction. If you want you can directly send me an email to discuss this project of yours in more detail. Rolando :suss:

                  M Offline
                  M Offline
                  Marc Clifton
                  wrote on last edited by
                  #11

                  Rolando E. Cruz-Marshall wrote: I'm willing to point you in the right direction. Well, I was hoping to hear from Bill S., as I understand he has some lossless compression algorithms that will reduce 30 TB down to 1 bit! Marc My website Traceract

                  E M 2 Replies Last reply
                  0
                  • M Marc Clifton

                    Rolando E. Cruz-Marshall wrote: I'm willing to point you in the right direction. Well, I was hoping to hear from Bill S., as I understand he has some lossless compression algorithms that will reduce 30 TB down to 1 bit! Marc My website Traceract

                    E Offline
                    E Offline
                    El Corazon
                    wrote on last edited by
                    #12

                    Marc Clifton wrote: Well, I was hoping to hear from Bill S., as I understand he has some lossless compression algorithms that will reduce 30 TB down to 1 bit! ahhhh, a sense of humor too. :) okay I'll be serious then... several ways to do this... We do it for the earth all the time, though we worry about textures and elevation, you need location and time, so multiple images for a given location (bernard's star moving as well as catching asteroids and comets in reference to the slower moving stars). http://www.vterrain.org/LOD/spherical.html[^] and http://www.vterrain.org/LOD/Papers/index.html[^] they are all terrain based, but many include image referencing with gridded terrain, or spherical mapping and referencing (which the stars in reference to eart are mapped in Ra Dec for spherical coordinates). _________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)

                    M S 2 Replies Last reply
                    0
                    • E El Corazon

                      Marc Clifton wrote: Well, I was hoping to hear from Bill S., as I understand he has some lossless compression algorithms that will reduce 30 TB down to 1 bit! ahhhh, a sense of humor too. :) okay I'll be serious then... several ways to do this... We do it for the earth all the time, though we worry about textures and elevation, you need location and time, so multiple images for a given location (bernard's star moving as well as catching asteroids and comets in reference to the slower moving stars). http://www.vterrain.org/LOD/spherical.html[^] and http://www.vterrain.org/LOD/Papers/index.html[^] they are all terrain based, but many include image referencing with gridded terrain, or spherical mapping and referencing (which the stars in reference to eart are mapped in Ra Dec for spherical coordinates). _________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)

                      M Offline
                      M Offline
                      Marc Clifton
                      wrote on last edited by
                      #13

                      Jeffry J. Brickley wrote: many include image referencing with gridded terrain, or spherical mapping and referencing (which the stars in reference to eart are mapped in Ra Dec for spherical coordinates). Very cool stuff! Wow! :-D Marc My website Traceract

                      E 1 Reply Last reply
                      0
                      • M Marc Clifton

                        Jeffry J. Brickley wrote: many include image referencing with gridded terrain, or spherical mapping and referencing (which the stars in reference to eart are mapped in Ra Dec for spherical coordinates). Very cool stuff! Wow! :-D Marc My website Traceract

                        E Offline
                        E Offline
                        El Corazon
                        wrote on last edited by
                        #14

                        Marc Clifton wrote: Very cool stuff! Wow! A friend's place since I can't show my work. Though I will be talking in November locally... probably boring stuff though, as well as hard to get into. :-D _________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)

                        1 Reply Last reply
                        0
                        • C Christian Graus

                          Add one more to the list of female CPians. Christian Graus - Microsoft MVP - C++

                          T Offline
                          T Offline
                          TheGreatAndPowerfulOz
                          wrote on last edited by
                          #15

                          actually, not female. sorry to burst your bubble that a guy would say "ewwww gross!". But I don't think that looking that close at a *picture* would be that interesting. Looking that close at the *subject* of the picture would be only slightly less appealing. I'd rather be *in* the subject!

                          C 1 Reply Last reply
                          0
                          • T TheGreatAndPowerfulOz

                            actually, not female. sorry to burst your bubble that a guy would say "ewwww gross!". But I don't think that looking that close at a *picture* would be that interesting. Looking that close at the *subject* of the picture would be only slightly less appealing. I'd rather be *in* the subject!

                            C Offline
                            C Offline
                            Christian Graus
                            wrote on last edited by
                            #16

                            ahz wrote: sorry to burst your bubble that a guy would say "ewwww gross!". OK, I guess guys are just more manly around where I live :-) ahz wrote: But I don't think that looking that close at a *picture* would be that interesting. Looking that close at the *subject* of the picture would be only slightly less appealing. I'd rather be *in* the subject! So you want to be 'in' it, but you don't want to 'see' it ? I'm not arguing that ultra close ups of femaleness don't lose their appeal after you hit the age of 12 ( they do ), but you're saying that you prefer the lights off all the time ? That's taking it a bit far, IMO. Christian Graus - Microsoft MVP - C++

                            E 1 Reply Last reply
                            0
                            • C Christian Graus

                              ahz wrote: sorry to burst your bubble that a guy would say "ewwww gross!". OK, I guess guys are just more manly around where I live :-) ahz wrote: But I don't think that looking that close at a *picture* would be that interesting. Looking that close at the *subject* of the picture would be only slightly less appealing. I'd rather be *in* the subject! So you want to be 'in' it, but you don't want to 'see' it ? I'm not arguing that ultra close ups of femaleness don't lose their appeal after you hit the age of 12 ( they do ), but you're saying that you prefer the lights off all the time ? That's taking it a bit far, IMO. Christian Graus - Microsoft MVP - C++

                              E Offline
                              E Offline
                              El Corazon
                              wrote on last edited by
                              #17

                              wondering how far from or over the kid sister rule we are... I never had a kid sister.... Well, I could say he could try marriage, then it's don't look, don't touch. ;) but I do recognize mine was the exception, not the rule. _________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)

                              C 1 Reply Last reply
                              0
                              • E El Corazon

                                wondering how far from or over the kid sister rule we are... I never had a kid sister.... Well, I could say he could try marriage, then it's don't look, don't touch. ;) but I do recognize mine was the exception, not the rule. _________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)

                                C Offline
                                C Offline
                                Christian Graus
                                wrote on last edited by
                                #18

                                Jeffry J. Brickley wrote: wondering how far from or over the kid sister rule we are... I never had a kid sister.... I do, but she's a bit of a tart, so... Jeffry J. Brickley wrote: but I do recognize mine was the exception, not the rule. Depends on how long you were married. My life was like that between the conception of the first child and the birth of the second ( 4 years ). Christian Graus - Microsoft MVP - C++

                                D E PJ ArendsP 3 Replies Last reply
                                0
                                • C Christian Graus

                                  Jeffry J. Brickley wrote: wondering how far from or over the kid sister rule we are... I never had a kid sister.... I do, but she's a bit of a tart, so... Jeffry J. Brickley wrote: but I do recognize mine was the exception, not the rule. Depends on how long you were married. My life was like that between the conception of the first child and the birth of the second ( 4 years ). Christian Graus - Microsoft MVP - C++

                                  D Offline
                                  D Offline
                                  David Stone
                                  wrote on last edited by
                                  #19

                                  How are things going with the missus now, CG?


                                  [Cheshire] I can't afford those plastic things to cover the electric sockets so I just draw bunny faces on the electric outlets to scare the kids away from them... [RLtim] Newsflash! Kids aren't afraid of bunnies. [Cheshire] Oh they will be... -Bash.org

                                  C 1 Reply Last reply
                                  0
                                  • D David Stone

                                    How are things going with the missus now, CG?


                                    [Cheshire] I can't afford those plastic things to cover the electric sockets so I just draw bunny faces on the electric outlets to scare the kids away from them... [RLtim] Newsflash! Kids aren't afraid of bunnies. [Cheshire] Oh they will be... -Bash.org

                                    C Offline
                                    C Offline
                                    Christian Graus
                                    wrote on last edited by
                                    #20

                                    Swimmingly, thanks. I'm not sure why, but it's feast or famine at my place, there doesn't appear to be an average position. Christian Graus - Microsoft MVP - C++

                                    1 Reply Last reply
                                    0
                                    • M Marc Clifton

                                      I hope you can all help me! I need to write a data management system to manage 30 terabytes of data that is being generated nightly! Yes, there's quite a night life where I live, I guess! Here's[^] a link describing this project! (My professor wants it done by the end of the semister, BTW) But hey, I get to play with a cool 3000M pixel camera!!! :-D Marc My website Traceract

                                      PJ ArendsP Offline
                                      PJ ArendsP Offline
                                      PJ Arends
                                      wrote on last edited by
                                      #21

                                      Marc Clifton wrote: 30 terabytes of data that is being generated nightly! Sounds like a job for him who's name can not be mentioned.


                                      "You're obviously a superstar." - Christian Graus about me - 12 Feb '03 "Obviously ???  You're definitely a superstar!!!" - mYkel - 21 Jun '04 "There's not enough blatant self-congratulatory backslapping in the world today..." - HumblePie - 21 Jun '05 Within you lies the power for good - Use it!

                                      Within you lies the power for good; Use it!

                                      1 Reply Last reply
                                      0
                                      • C Christian Graus

                                        Jeffry J. Brickley wrote: wondering how far from or over the kid sister rule we are... I never had a kid sister.... I do, but she's a bit of a tart, so... Jeffry J. Brickley wrote: but I do recognize mine was the exception, not the rule. Depends on how long you were married. My life was like that between the conception of the first child and the birth of the second ( 4 years ). Christian Graus - Microsoft MVP - C++

                                        E Offline
                                        E Offline
                                        El Corazon
                                        wrote on last edited by
                                        #22

                                        Christian Graus wrote: Depends on how long you were married. She already had kids... life was just like that after the first year... _________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)

                                        C 1 Reply Last reply
                                        0
                                        • E El Corazon

                                          Christian Graus wrote: Depends on how long you were married. She already had kids... life was just like that after the first year... _________________________ Asu no koto o ieba, tenjo de nezumi ga warau. Talk about things of tomorrow and the mice in the ceiling laugh. (Japanese Proverb)

                                          C Offline
                                          C Offline
                                          Christian Graus
                                          wrote on last edited by
                                          #23

                                          OK, well if there's no biological excuse, I'd say run for your life. Christian Graus - Microsoft MVP - C++

                                          E 1 Reply Last reply
                                          0
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes


                                          • Login

                                          • Don't have an account? Register

                                          • Login or register to search.
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • World
                                          • Users
                                          • Groups