Weird memory problem

Skippy II

Hi all, I have 2 forms (Form1 and Form2) in my app. Form1 has 2 buttons (Button1 and Button2). The Load event of Form2 has this code: Private Sub Form2_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load Dim obj(,,) As Object ReDim obj(50, 24000, 100) End Sub When you click Button1 on Form1 this code is executed: Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click Dim obj(,,) As Object ReDim obj(50, 25000, 100) End Sub The ReDim statement gives an OutOfMemoryException. When I change the statement to ReDim obj(50, 24000, 100) the exception isn't thrown anymore. When Button2 is clicked, this code is executed: Private Sub Button2_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles Button2.Click For tel As Integer = 1 To 10 Dim f As New Form2 f.Show() Next End Sub This works just fine. After searching the web for answers, I think it may have to do with the heap size, but I'm not sure at all. My questions: Why is the Button1 code giving an exception and why is this not happening when clicking Button2? What should I do to make the Button1 code work without throwing an exception? Skippy II

Dave Kreskowiak

I've got a better question. Why are you creating such MASSIVE arrays of objects?? 120,000,000+?? Just creating that many Objects, that don't do anything, in a single one of your arrays takes over a 1GB of RAM?? And you're creating at least 10 of these?? What on earth for?? BTW: I can't tell how much memory your code will use, because it ran my machine out of RAM. But I can make an educated guess that it's probably going to be around a 40 gigabytes if all of those arrays are filled with objects.

A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007

Skippy II

Thanks for your reaction. I know it doesn't make much sense to create such big arrays. At least not 10 of them. But I would like to use the idea in a grid control. The array would contain columnnumbers, rownumbers and cell properties. Now the code provided is just an example and is not the whole project of course. It is only a simple way to explain the problem. In fact, what I wanted to show, is that when allocating memory for 125,000,000 objects an error is thrown. But when allocating memory for 10 times 120,000,000 objects, no error is thrown at all. I hope your "better" question is answered. :) Any idea about my questions? Skippy II

Dave Kreskowiak

Skippy II wrote:

In fact, what I wanted to show, is that when allocating memory for 125,000,000 objects an error is thrown. But when allocating memory for 10 times 120,000,000 objects, no error is thrown at all.

All you did was declare an array that MIGHT hold that many objects. In this case, it'll allocate pointers to Nothing, which, on a 32-bit machine, is a 4 byte value containing all 0's. So, 120,000,000 * 4 = 480,000,000 bytes will be allocated to hold 120,000,000 address that point to nowhere. So, when you allocate one of those empty arrays, you're actually reserving 480MB of memory to hold pointers to nowhere, and when you create 10 of them, 4.8GB of RAM. Since the standard user mode address space can only handle 2GB per process and 2GB for the kernel space, BOOM - OutOfMemory. You can change the boot config of Windows so the split is 3GB for user space and 1GB for kernel, but you still won't get enough to do what you want. AND you still haven't created the objects that these arrays will point to! Try it:

Dim arr(50,25000,100) As Object
For a As Integer = 0 to 49
    For b As Integer = 0 to 24999
        For c As Integer = 0 to 99
            arr(a, b, c) = New Object()
        Next
    Next
Next

You'll reach 1GB of RAM before you get half way through the outside loop.

A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007

Skippy II

I cannot deny what you write. You're absolutely right. The idea may not be the best. But you still do not answer the questions. Why is an error thrown when declaring an array that might hold 125,000,000 objects, but not when declaring 10 times an array of 120,000,000 items? Skippy II

Luc Pattyn

Hi, the array you allocate in Form2_Load() is no longer alive when Form2_Load terminates, hence the garbage collector can reclaim it when it feels a need, such as when your next form instance runs its Form2_Load(). Therefore your app probably never holds more than one of those huge arrays. :)

Luc Pattyn [Forum Guidelines] [My Articles]

this months tips: - use PRE tags to preserve formatting when showing multi-line code snippets - before you ask a question here, search CodeProject, then Google

Skippy II

Thanks for replying, Luc. That is indeed very true. I completely overlooked this . Do you know if there is a way to increase the available memory allocation? Skippy II

Luc Pattyn

Hi,

Skippy II wrote:

a way to increase the available memory allocation?

Don't know, I never needed anything above normal size; to me a multi- dimensional array, or a very big one/two-dimensional array, indicate a wrong aproach; if it takes that much memory, how long will it take to initialize, compute, and draw conclusions from it? Maybe this could help.[^] :)

Luc Pattyn [Forum Guidelines] [My Articles]

this months tips: - use PRE tags to preserve formatting when showing multi-line code snippets - before you ask a question here, search CodeProject, then Google

Skippy II

I understand your reaction. Perhaps it is indeed a wrong approach. But the problem is much bigger than the provided code. Actually the whole project is about creating a very fast virtual grid (a bit comparable to a virtual listview, but even a lot faster). Accessing an object array (with property values at a specific location) is way faster than accessing the regular properties. The first coding steps led to a very fast, but extremely complex way of programming (a combination of virtual and real cells in the grid). One solution is to extend the object array so every cell can be virtual. But then this OutOfMemoryException came up... Anyway, thanks for your contribution. Skippy II

Dave Kreskowiak

You can only get your hands on a total of 3GB of memory, IF you put the /3GB[^] switch in the boot.ini for Windows. There is no way to get any more, and even this technique is NOT recommened.

A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007