Problem with reading big amount of numbers from file
-
Hi! I have to do a project for my studies. It is a insertion algorithm. I'm reading data from file with using vector. then I'm copying vector to a normal array (the reason is simple: I don't know how to send vector to function, but this is not a main problem). After copying, I'm sending (with using pointers) array to sorting function. Everything is good, algorithm is working but! (yep, there is always but... ;) When I save a lot of data to file, program just after start crashes. The numbers in file are written in that way:
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10
So there are 40 numbers in one line. I have 4.000 lines, so after simple math we have 160.000 numbers. The main problem: when I add some new numbers for example 40.000 program crashes I must have more numbers because the task is to compare few sorting algorithms with minimum 15 minutes of sorting. (160.000 digits is sorting in ~1 minute) Here's the code. Maybe it is poor written, but I did my best.
//written with Qt
#include <QtCore/QCoreApplication>
#include <iostream>
#include <time.h>
#include <iomanip>
#include <string>
#include <fstream>
#include <vector>using namespace std;
//sorting function
void insertion_sort(int *wsk, unsigned long size);int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);//vector to hold data from file vector <int> array\_vector; //stream to file ifstream plik ("numbers.txt",ios::in); //single number that is being readed from file unsigned long single\_number; while(plik >> single\_number) { array\_vector.push\_back(single\_number); } int array\_nonsorted\[array\_vector.size()\]; //copying from vector to array for (unsigned long a = 0; a < array\_vector.size(); a++) { array\_nonsorted\[a\] = array\_vector\[a\]; } time\_t start,end; double long dif; cout << "START\\n"; time(&start); insertion\_sort(array\_nonsorted,array\_vector.size()); time(&end); cout << "STOP\\n"; dif = difftime(end,start); cout << "Array sorted in " << setprecision(5) << dif << "sec"; dif = dif/60; cout << " = " << dif << "minutes"; //cout << "End."; return a.exec();}
void insertion_sort(int *wsk, unsigned long size)
{
int tab_sorted[size];//In final version -
Hi! I have to do a project for my studies. It is a insertion algorithm. I'm reading data from file with using vector. then I'm copying vector to a normal array (the reason is simple: I don't know how to send vector to function, but this is not a main problem). After copying, I'm sending (with using pointers) array to sorting function. Everything is good, algorithm is working but! (yep, there is always but... ;) When I save a lot of data to file, program just after start crashes. The numbers in file are written in that way:
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10
So there are 40 numbers in one line. I have 4.000 lines, so after simple math we have 160.000 numbers. The main problem: when I add some new numbers for example 40.000 program crashes I must have more numbers because the task is to compare few sorting algorithms with minimum 15 minutes of sorting. (160.000 digits is sorting in ~1 minute) Here's the code. Maybe it is poor written, but I did my best.
//written with Qt
#include <QtCore/QCoreApplication>
#include <iostream>
#include <time.h>
#include <iomanip>
#include <string>
#include <fstream>
#include <vector>using namespace std;
//sorting function
void insertion_sort(int *wsk, unsigned long size);int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);//vector to hold data from file vector <int> array\_vector; //stream to file ifstream plik ("numbers.txt",ios::in); //single number that is being readed from file unsigned long single\_number; while(plik >> single\_number) { array\_vector.push\_back(single\_number); } int array\_nonsorted\[array\_vector.size()\]; //copying from vector to array for (unsigned long a = 0; a < array\_vector.size(); a++) { array\_nonsorted\[a\] = array\_vector\[a\]; } time\_t start,end; double long dif; cout << "START\\n"; time(&start); insertion\_sort(array\_nonsorted,array\_vector.size()); time(&end); cout << "STOP\\n"; dif = difftime(end,start); cout << "Array sorted in " << setprecision(5) << dif << "sec"; dif = dif/60; cout << " = " << dif << "minutes"; //cout << "End."; return a.exec();}
void insertion_sort(int *wsk, unsigned long size)
{
int tab_sorted[size];//In final version -
Have you built a debug version and used the debugger to see where it crashes? That's what I'd to as a starting point.
You measure democracy by the freedom it gives its dissidents, not the freedom it gives its assimilated conformists.
hmm, I didn't thought about this, because algorithm works fine, but I did it as you said, and just after debugger started error message came out
Process stoped. Process was stopped because it received signal from operating system. Name of signal :SIGSEGV Meaning of singal : Segmentation faultNow it is coming to be more interesting..... ;) I can only click . After click nothing happened. Even debugging is still on, but do nothing. I read about SIGSEGV on wiki, but how to avoid it? -
hmm, I didn't thought about this, because algorithm works fine, but I did it as you said, and just after debugger started error message came out
Process stoped. Process was stopped because it received signal from operating system. Name of signal :SIGSEGV Meaning of singal : Segmentation faultNow it is coming to be more interesting..... ;) I can only click . After click nothing happened. Even debugging is still on, but do nothing. I read about SIGSEGV on wiki, but how to avoid it? -
Sounds like you have a wild pointer somewhere. Or have managed to corrupt the heap.
You measure democracy by the freedom it gives its dissidents, not the freedom it gives its assimilated conformists.
Maybe someone in the future will be searching solution for similar problem so I wrote it here. So the problem is with stack array. Array is so large that it overflows the stack. Solution is to use dynamically sized array. It can be vector class. In this example, it is sufficient to send pointer to function and in the sorting function working on vector. That's all. // Thanks to helios for helping me out (from other cplusplus forum)
-
Hi! I have to do a project for my studies. It is a insertion algorithm. I'm reading data from file with using vector. then I'm copying vector to a normal array (the reason is simple: I don't know how to send vector to function, but this is not a main problem). After copying, I'm sending (with using pointers) array to sorting function. Everything is good, algorithm is working but! (yep, there is always but... ;) When I save a lot of data to file, program just after start crashes. The numbers in file are written in that way:
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10
So there are 40 numbers in one line. I have 4.000 lines, so after simple math we have 160.000 numbers. The main problem: when I add some new numbers for example 40.000 program crashes I must have more numbers because the task is to compare few sorting algorithms with minimum 15 minutes of sorting. (160.000 digits is sorting in ~1 minute) Here's the code. Maybe it is poor written, but I did my best.
//written with Qt
#include <QtCore/QCoreApplication>
#include <iostream>
#include <time.h>
#include <iomanip>
#include <string>
#include <fstream>
#include <vector>using namespace std;
//sorting function
void insertion_sort(int *wsk, unsigned long size);int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);//vector to hold data from file vector <int> array\_vector; //stream to file ifstream plik ("numbers.txt",ios::in); //single number that is being readed from file unsigned long single\_number; while(plik >> single\_number) { array\_vector.push\_back(single\_number); } int array\_nonsorted\[array\_vector.size()\]; //copying from vector to array for (unsigned long a = 0; a < array\_vector.size(); a++) { array\_nonsorted\[a\] = array\_vector\[a\]; } time\_t start,end; double long dif; cout << "START\\n"; time(&start); insertion\_sort(array\_nonsorted,array\_vector.size()); time(&end); cout << "STOP\\n"; dif = difftime(end,start); cout << "Array sorted in " << setprecision(5) << dif << "sec"; dif = dif/60; cout << " = " << dif << "minutes"; //cout << "End."; return a.exec();}
void insertion_sort(int *wsk, unsigned long size)
{
int tab_sorted[size];//In final versionvonpik wrote:
int array_nonsorted[array_vector.size()];
1.2MB in size, perhaps?
"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"Man who follows car will be exhausted." - Confucius
-
vonpik wrote:
int array_nonsorted[array_vector.size()];
1.2MB in size, perhaps?
"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"Man who follows car will be exhausted." - Confucius
-
You missed my subtlty. If you have 160,000 numbers and each is 4 bytes in size, that would require 1.28MB of stack space, obviously more than the default size of 1MB. Using the stack, I'd be surprised if you could get any more than 130,000 numbers.
"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"Man who follows car will be exhausted." - Confucius
-
You missed my subtlty. If you have 160,000 numbers and each is 4 bytes in size, that would require 1.28MB of stack space, obviously more than the default size of 1MB. Using the stack, I'd be surprised if you could get any more than 130,000 numbers.
"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"Man who follows car will be exhausted." - Confucius
-
So, you are saying that default and maximum size of stack size is 1 MB? I solved this problem with using vector, but still I want to know a C++ mechanisms :)
vonpik wrote:
So, you are saying that default...size of stack size is 1 MB?
Yes.
vonpik wrote:
I solved this problem with using vector,
Which gets its memory from the heap, not the stack.
"One man's wage rise is another man's price increase." - Harold Wilson
"Fireproof doesn't mean the fire will never come. It means when the fire comes that you will be able to withstand it." - Michael Simmons
"Man who follows car will be exhausted." - Confucius