Concept/example for virtual list
-
Hi, I need to display ~ 100000 entries in a windows list which are stored in an SQL database (queried by a SELECT query with a few JOINs). 1st question: In only need to display a simple list. I would really like to use a Listbox Control for the sake of simplicity. Do I really need to choose a (complex) List Control? 2nd question: I tried a virtual list with a simple SQL query:
OnInitDialog()
...
// SELECT COUNT(*) FROM...
m_ctrlWords.SetItemCountEx(count);
....OnLvnGetdispinfoWords(...)
...
sql.Format("SELECT keywordID, keyword FROM keywords ORDER BY keyword LIMIT %d, 1", index);
// query
const char *res = query.getStringField("keyword");
lstrcpy(pItem->pszText, res);but this is really too slow. I think I need OnLvnOdcachehintWords but it's somehow complicated. I could not find any concepts (which data structure to choose??) or examples concerning this :-( The problem is that I have to remove old entried in cache (or my app's memory consumption would explode). Does anyone have implemented such a virtual list (with data in SQL) already? Are there some examples or concepts available? Thank you very much, Niki
-
Hi, I need to display ~ 100000 entries in a windows list which are stored in an SQL database (queried by a SELECT query with a few JOINs). 1st question: In only need to display a simple list. I would really like to use a Listbox Control for the sake of simplicity. Do I really need to choose a (complex) List Control? 2nd question: I tried a virtual list with a simple SQL query:
OnInitDialog()
...
// SELECT COUNT(*) FROM...
m_ctrlWords.SetItemCountEx(count);
....OnLvnGetdispinfoWords(...)
...
sql.Format("SELECT keywordID, keyword FROM keywords ORDER BY keyword LIMIT %d, 1", index);
// query
const char *res = query.getStringField("keyword");
lstrcpy(pItem->pszText, res);but this is really too slow. I think I need OnLvnOdcachehintWords but it's somehow complicated. I could not find any concepts (which data structure to choose??) or examples concerning this :-( The problem is that I have to remove old entried in cache (or my app's memory consumption would explode). Does anyone have implemented such a virtual list (with data in SQL) already? Are there some examples or concepts available? Thank you very much, Niki
You'll probably be better off executing something like "SELECT keywordID, keyword FROM keywords ORDER BY keyword" within
OnInitDialogand caching the results in a resultset or whatever equivalent your database interface technology offers. Then you can interrogate the cache withinOnLvnGetdispinfoWordsand don't need to bother withOnLvnOdcachehintWords. -
Hi, I need to display ~ 100000 entries in a windows list which are stored in an SQL database (queried by a SELECT query with a few JOINs). 1st question: In only need to display a simple list. I would really like to use a Listbox Control for the sake of simplicity. Do I really need to choose a (complex) List Control? 2nd question: I tried a virtual list with a simple SQL query:
OnInitDialog()
...
// SELECT COUNT(*) FROM...
m_ctrlWords.SetItemCountEx(count);
....OnLvnGetdispinfoWords(...)
...
sql.Format("SELECT keywordID, keyword FROM keywords ORDER BY keyword LIMIT %d, 1", index);
// query
const char *res = query.getStringField("keyword");
lstrcpy(pItem->pszText, res);but this is really too slow. I think I need OnLvnOdcachehintWords but it's somehow complicated. I could not find any concepts (which data structure to choose??) or examples concerning this :-( The problem is that I have to remove old entried in cache (or my app's memory consumption would explode). Does anyone have implemented such a virtual list (with data in SQL) already? Are there some examples or concepts available? Thank you very much, Niki
nobaq wrote:
I need to display ~ 100000 entries in a windows list...
Why? Who would want to navigate through such a list?
"Love people and use things, not love things and use people." - Unknown
"The brick walls are there for a reason...to stop the people who don't want it badly enough." - Randy Pausch
-
You'll probably be better off executing something like "SELECT keywordID, keyword FROM keywords ORDER BY keyword" within
OnInitDialogand caching the results in a resultset or whatever equivalent your database interface technology offers. Then you can interrogate the cache withinOnLvnGetdispinfoWordsand don't need to bother withOnLvnOdcachehintWords. -
Hi, I need to display ~ 100000 entries in a windows list which are stored in an SQL database (queried by a SELECT query with a few JOINs). 1st question: In only need to display a simple list. I would really like to use a Listbox Control for the sake of simplicity. Do I really need to choose a (complex) List Control? 2nd question: I tried a virtual list with a simple SQL query:
OnInitDialog()
...
// SELECT COUNT(*) FROM...
m_ctrlWords.SetItemCountEx(count);
....OnLvnGetdispinfoWords(...)
...
sql.Format("SELECT keywordID, keyword FROM keywords ORDER BY keyword LIMIT %d, 1", index);
// query
const char *res = query.getStringField("keyword");
lstrcpy(pItem->pszText, res);but this is really too slow. I think I need OnLvnOdcachehintWords but it's somehow complicated. I could not find any concepts (which data structure to choose??) or examples concerning this :-( The problem is that I have to remove old entried in cache (or my app's memory consumption would explode). Does anyone have implemented such a virtual list (with data in SQL) already? Are there some examples or concepts available? Thank you very much, Niki
Use a CListBox,in virtual mode. If I remember correctly, You will need to manually draw and manage the data, but it will make the CListBox very fast; once you know how to do it, you will use it everywhere. You will need to override the
CListBox::DrawItem(and maybe theCListBox::MeasureItem) -
nobaq wrote:
I need to display ~ 100000 entries in a windows list...
Why? Who would want to navigate through such a list?
"Love people and use things, not love things and use people." - Unknown
"The brick walls are there for a reason...to stop the people who don't want it badly enough." - Randy Pausch
-
Hello, I have to correct: The entries are much more than 100000...it's a fulltext recherche of a huge database system. So it's definitifely too large to just read out all entries and cache them. Regards, Niki
I have tried it using caching and virtual list controls. Even if I read only the elements from LVN_ODCACHEHINT and cache them into a map, the list is really, really tooo slow. What could be the solution? I just want to have the same as WinHelp32's keyword (or topic) search. For some reason, this list is very very fast since over 10 years (with slow computers). Regards, Niki
-
Use a CListBox,in virtual mode. If I remember correctly, You will need to manually draw and manage the data, but it will make the CListBox very fast; once you know how to do it, you will use it everywhere. You will need to override the
CListBox::DrawItem(and maybe theCListBox::MeasureItem)Hi, thank you for this hint. I've been searching for a while now but I can not find any docs Do you have a reference, docs, howtos for subclassing CListBox in order to implement a virtual listbox? Any anyway, how do I query data from my database when "scrolling" very fast? I think i need to minimze my database queries but on the other hand I need to display something while scrolling... Regards, Niki
-
Hello, I have to correct: The entries are much more than 100000...it's a fulltext recherche of a huge database system. So it's definitifely too large to just read out all entries and cache them. Regards, Niki
OK - in that case, you want to cache maybe a couple of hundred rows starting at the row that's at the top of the list control. Then, as users move down the list, you can retrieve a few hundred rows at intervals, when required?
-
I have tried it using caching and virtual list controls. Even if I read only the elements from LVN_ODCACHEHINT and cache them into a map, the list is really, really tooo slow. What could be the solution? I just want to have the same as WinHelp32's keyword (or topic) search. For some reason, this list is very very fast since over 10 years (with slow computers). Regards, Niki
nobaq wrote:
I have tried it using caching and virtual list controls. Even if I read only the elements from LVN_ODCACHEHINT and cache them into a map, the list is really, really tooo slow.
I'm surprised - I've implemented virtual lists that provide views over 10s of thousands of rows (and 10s of columns) of data - they've been as quick as you could want. I feel sure you're using the right technology (virtual list controls) but there's something not quite right about your implementation?
-
I have tried it using caching and virtual list controls. Even if I read only the elements from LVN_ODCACHEHINT and cache them into a map, the list is really, really tooo slow. What could be the solution? I just want to have the same as WinHelp32's keyword (or topic) search. For some reason, this list is very very fast since over 10 years (with slow computers). Regards, Niki
One other thing - have you profiled the code to see what is slowing it down? I'd suggest implementing a virtual list control that just displays the numeric index of the list rows, to convince yourself that you've got the virtual list control set up correctly. Then work out how much of the SQL query you can do without taking too much time, and you can try to merge that into the list control implementation?