Code problem or exceeded array limits?
-
Hi all, I have a little problem with a pair of arrays, when them are involved in a comparation work: In particular, I have an Array A As String and another array B As String. Normally the program works with a single array A, but sometimes I need a second array B. Array A is filled with numerical string (sometime with up 15000 elements). Array B is filled with numerical string (sometime with up 15000 elements). The code works in this way: Each element of array B is compared with Each element of array A. If a value of array B is = to some value in array A then the value in the array B is removed and the array is resized (progressively size-decreased) but preserving remaining different values (Redim Preserve). In this way, at the end of comparation loop, the array B will contain only values that are not already contained in array A. The code seems to be perfectly working until the number of elements in array A and B is down about 2200 + 2200 elements, but if numbers of arrays element go up, the program seems to work (in an infinite loop) but in reality it is quited (closed) from O.S. In the fact the program name disappears from list of running program. The following is the schematic code:
Do Until IndexB = ArrayB.GetLength(0)
For IndexA = 1 to ArrayA.GetLength(0)
If Val(ArrayB(IndexB)) = Val(ArrayA(IndexA)) Then
ArrayB(IndexB) = ArrayB(ArrayB.GetLength(0) - 1)
ReDim Preserve ArrayB((ArrayB.GetLength(0) - 1) - 1)
IndexB = IndexB - 1
Exit For
End If
Next
IndexB = IndexB + 1
LoopApplication is running in a Windows Mobile 6.1 Device. Apparentely seems to be a memory problem, but I don't think so, because size of arrays is not exagerate and the device have enough available memory. Thanks for help.
-
Hi all, I have a little problem with a pair of arrays, when them are involved in a comparation work: In particular, I have an Array A As String and another array B As String. Normally the program works with a single array A, but sometimes I need a second array B. Array A is filled with numerical string (sometime with up 15000 elements). Array B is filled with numerical string (sometime with up 15000 elements). The code works in this way: Each element of array B is compared with Each element of array A. If a value of array B is = to some value in array A then the value in the array B is removed and the array is resized (progressively size-decreased) but preserving remaining different values (Redim Preserve). In this way, at the end of comparation loop, the array B will contain only values that are not already contained in array A. The code seems to be perfectly working until the number of elements in array A and B is down about 2200 + 2200 elements, but if numbers of arrays element go up, the program seems to work (in an infinite loop) but in reality it is quited (closed) from O.S. In the fact the program name disappears from list of running program. The following is the schematic code:
Do Until IndexB = ArrayB.GetLength(0)
For IndexA = 1 to ArrayA.GetLength(0)
If Val(ArrayB(IndexB)) = Val(ArrayA(IndexA)) Then
ArrayB(IndexB) = ArrayB(ArrayB.GetLength(0) - 1)
ReDim Preserve ArrayB((ArrayB.GetLength(0) - 1) - 1)
IndexB = IndexB - 1
Exit For
End If
Next
IndexB = IndexB + 1
LoopApplication is running in a Windows Mobile 6.1 Device. Apparentely seems to be a memory problem, but I don't think so, because size of arrays is not exagerate and the device have enough available memory. Thanks for help.
Hi, this is horribly inefficient for many reasons: - you are converting strings to numbers all the time; it they really are numbers, store them in a numeric data type, not string. - ReDim Preserve is expensive as it has to allocate a new array, then copy all the data. When the number of items in a collection varies, you should consider using a real collection instead of an array; use a collection type if those are available, or create your own linked list structures. - For two arrays of size S1 and S2, you are performing S1*S2 comparisons. It would be cheaper to sort the first array, sort the second array, and then perform the equivalent of a merge-sort on both. This will improve speed dramatically and reduce the memory load drastically. So throw it all out; start using lists of numbers; then sort and merge. It would handle millions of numbers in a matter of seconds. :)
Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles]
I only read code that is properly formatted, adding PRE tags is the easiest way to obtain that.
All Toronto weekends should be extremely wet until we get it automated in regular forums, not just QA.
modified on Friday, February 26, 2010 1:50 PM
-
Hi all, I have a little problem with a pair of arrays, when them are involved in a comparation work: In particular, I have an Array A As String and another array B As String. Normally the program works with a single array A, but sometimes I need a second array B. Array A is filled with numerical string (sometime with up 15000 elements). Array B is filled with numerical string (sometime with up 15000 elements). The code works in this way: Each element of array B is compared with Each element of array A. If a value of array B is = to some value in array A then the value in the array B is removed and the array is resized (progressively size-decreased) but preserving remaining different values (Redim Preserve). In this way, at the end of comparation loop, the array B will contain only values that are not already contained in array A. The code seems to be perfectly working until the number of elements in array A and B is down about 2200 + 2200 elements, but if numbers of arrays element go up, the program seems to work (in an infinite loop) but in reality it is quited (closed) from O.S. In the fact the program name disappears from list of running program. The following is the schematic code:
Do Until IndexB = ArrayB.GetLength(0)
For IndexA = 1 to ArrayA.GetLength(0)
If Val(ArrayB(IndexB)) = Val(ArrayA(IndexA)) Then
ArrayB(IndexB) = ArrayB(ArrayB.GetLength(0) - 1)
ReDim Preserve ArrayB((ArrayB.GetLength(0) - 1) - 1)
IndexB = IndexB - 1
Exit For
End If
Next
IndexB = IndexB + 1
LoopApplication is running in a Windows Mobile 6.1 Device. Apparentely seems to be a memory problem, but I don't think so, because size of arrays is not exagerate and the device have enough available memory. Thanks for help.
As an alternative to the sort and merge approach, you may also try using a Dictionary/HashSet. If you have .NET 3.5 available, you could use the Except extension method (which I understand uses a HashSet internally) like so:
Dim temp = ArrayB.Select(AddressOf Double.Parse) _
.Except(ArrayA.Select(AddressOf Double.Parse)) _
.ToArray()If you don't have the LINQ methods, you could do this yourself:
Dim parsedToRemove As New Dictionary(Of Double, Double)
Dim noDuplicates As New List(Of Double)For Each parsedA In Array.ConvertAll(ArrayA, AddressOf Double.Parse)
If Not parsedToRemove.ContainsKey(parsedA) Then
parsedToRemove.Add(parsedA, parsedA)
End If
Next
For Each parsedB In Array.ConvertAll(ArrayB, AddressOf Double.Parse)
If Not parsedToRemove.ContainsKey(parsedB) Then
noDuplicates.Add(parsedB)
End If
Next
'noDuplicates now has the values in B not in A -
As an alternative to the sort and merge approach, you may also try using a Dictionary/HashSet. If you have .NET 3.5 available, you could use the Except extension method (which I understand uses a HashSet internally) like so:
Dim temp = ArrayB.Select(AddressOf Double.Parse) _
.Except(ArrayA.Select(AddressOf Double.Parse)) _
.ToArray()If you don't have the LINQ methods, you could do this yourself:
Dim parsedToRemove As New Dictionary(Of Double, Double)
Dim noDuplicates As New List(Of Double)For Each parsedA In Array.ConvertAll(ArrayA, AddressOf Double.Parse)
If Not parsedToRemove.ContainsKey(parsedA) Then
parsedToRemove.Add(parsedA, parsedA)
End If
Next
For Each parsedB In Array.ConvertAll(ArrayB, AddressOf Double.Parse)
If Not parsedToRemove.ContainsKey(parsedB) Then
noDuplicates.Add(parsedB)
End If
Next
'noDuplicates now has the values in B not in AThanks for replies. Both contains good ideas to resolving the problem.