How to load 100,000 list view items without application freeze?
-
It is possible to run addition of items in background worker but it needs to add them in report progress handler, main thread only, which also hangs the application. Is there any other approaches to add them to list view without application freeze?
Чесноков
I use the telerik winforms suite for similar operations, because it can :laugh: Telerik gridview
-
It is possible to run addition of items in background worker but it needs to add them in report progress handler, main thread only, which also hangs the application. Is there any other approaches to add them to list view without application freeze?
Чесноков
I met the similar problem. Once I wrote a program for retrieving images from a website and I want that users can be notified about the downloading process, so I added a status bar on main GUI, and let the downloading process works on the other thread(let's say B), when it is retrieving the images , the B has to report its status to GUI which is on main thread(let's say A). But B can not access the status bar which belongs to A. So there is a method call BeginInvoke(xxxx); With this method, it enables B to access resourses on A. So I think it should be the answer that you are looking for. Give me reply whatever it works or not :) cheers
-
I met the similar problem. Once I wrote a program for retrieving images from a website and I want that users can be notified about the downloading process, so I added a status bar on main GUI, and let the downloading process works on the other thread(let's say B), when it is retrieving the images , the B has to report its status to GUI which is on main thread(let's say A). But B can not access the status bar which belongs to A. So there is a method call BeginInvoke(xxxx); With this method, it enables B to access resourses on A. So I think it should be the answer that you are looking for. Give me reply whatever it works or not :) cheers
You need to use background worker and report the progress with its ReportProgress() function.
Чесноков
-
Virtual mode is very easy to use and code... just look at the guidline i gave you from microsoft, it shows you the way. But u need to know that virtual mode has its limits, with what it can do, compared to normal mode. ;)
With great code, comes great complexity, so keep it simple stupid...:-\ :-\
Great, thanks, virtual mode is of value to handle large lists especially with large icons. I will use it from now for those purpouses. However I presume I will not be able to use virtual view for log event because the list with those events is not static. There are many threads which use global logger class to report events. They are stored in a queue and once queue limit exceeds some constant 100,000 previous one is dequeued. Log event addition is safe with lock. It is possible that in virtual list view between RetrieveVirtualItem() calls threads may report dozens of new events, queue will move and it will end up of showing the same or odd events several times.
Чесноков
-
this entire thread got way too much attention...
do or do not, there is no try
because the topic is a challenge to display 100,000 in as little time as possible, and to find a user who may process them all :)
Чесноков
-
can you help with dynamic queue for logs? http://www.codeproject.com/Messages/3754435/Re-How-to-load-100-000-list-view-items-without-app.aspx[^]
Чесноков
-
It is possible to run addition of items in background worker but it needs to add them in report progress handler, main thread only, which also hangs the application. Is there any other approaches to add them to list view without application freeze?
Чесноков
Is the code calling BeginUpdate and EndUpdate? It might make a difference, but it's still going to be slow...
-
Is the code calling BeginUpdate and EndUpdate? It might make a difference, but it's still going to be slow...
No, I set the list view Visible to false during events addition. Otherwise the process is very slow.
Чесноков
-
It is possible to run addition of items in background worker but it needs to add them in report progress handler, main thread only, which also hangs the application. Is there any other approaches to add them to list view without application freeze?
Чесноков
Thanks for this question. I have an application with the same problem. The answers by some smart folks here are very helpful in solving this problem.
-
Dave Kreskowiak wrote:
You have a serious design flaw in your app if you think you need to show 100,000 items in a single control.
Have you ever ran Windows Events on your machine? How many events are there in a list view for a couple of years e.g. windows applications :) It does not freeze as you run it either.
Чесноков
I think you have a complete misunderstanding of how event works. Events happen on demand, when you add 100,000 items all at once, you're simply blocking your application message pump from processing the messages sent by windows (also known as events) and therefore freezing. It's possible to add this 100,000 items in the control without freezing, as already mentioned, by adding in small iteractions and from time to time, allowing the messages to be processed (one simple, but not usually recommended way to do it is call Application.DoEvents()) and thus not freezing the application. In any case, I agree with the guys that 100,000 items in a listview is simply insane. It doesn't make sense, nobody want's to see 100,000 items all at once. One approach I like to take is to display a couple hundred at once at the most and have a textbox filtering the results as you type, very easy for the user.
-
can you help with dynamic queue for logs? http://www.codeproject.com/Messages/3754435/Re-How-to-load-100-000-list-view-items-without-app.aspx[^]
Чесноков
-
I think you have a complete misunderstanding of how event works. Events happen on demand, when you add 100,000 items all at once, you're simply blocking your application message pump from processing the messages sent by windows (also known as events) and therefore freezing. It's possible to add this 100,000 items in the control without freezing, as already mentioned, by adding in small iteractions and from time to time, allowing the messages to be processed (one simple, but not usually recommended way to do it is call Application.DoEvents()) and thus not freezing the application. In any case, I agree with the guys that 100,000 items in a listview is simply insane. It doesn't make sense, nobody want's to see 100,000 items all at once. One approach I like to take is to display a couple hundred at once at the most and have a textbox filtering the results as you type, very easy for the user.
Fabio Franco wrote:
think you have a complete misunderstanding of how event works
You do not know my design. Message pump is independent from the view representation, pump observer. There is no blocking of the pump no matter what the observer of the pump do. The application freeze only at one point, adding of the list view items to list view, either one by one all of them with AddRange.
Fabio Franco wrote:
100,000 items in a listview is simply insane
There are many customers and you can not develop one solution that suit them all. I'm one of those wierd ones who fond of scrolling 100,000 events ;)
Чесноков
-
Fabio Franco wrote:
think you have a complete misunderstanding of how event works
You do not know my design. Message pump is independent from the view representation, pump observer. There is no blocking of the pump no matter what the observer of the pump do. The application freeze only at one point, adding of the list view items to list view, either one by one all of them with AddRange.
Fabio Franco wrote:
100,000 items in a listview is simply insane
There are many customers and you can not develop one solution that suit them all. I'm one of those wierd ones who fond of scrolling 100,000 events ;)
Чесноков
Chesnokov Yuriy wrote:
You do not know my design. Message pump is independent from the view representation, pump observer. There is no blocking of the pump no matter what the observer of the pump do. The application freeze only at one point, adding of the list view items to list view, either one by one all of them with AddRange.
It's not your design, the view representation is totally dependant of the message pumb, read my article: FormEx[^] and perhaps you'll have some idea on how the view get's rendered, it's through windows messages. The application freezes because WM_PAINT message never gets processed by your application because there can be only one UI thread and while you're adding items to your listview, the thread get's dedicated to that alone and has no time to process paint, click and other messages because it's adding the items of the listview, UI operations do not run in parallel. Just do a while(true); on a click of a button and that will also block paint messages and will freeze the app. Add Range and adding one by one are doing the exact same things. AddRange is just to make some operations simpler to the control user. Again, I think you have a complete misunderstanding of how event works.
Chesnokov Yuriy wrote:
There are many customers and you can not develop one solution that suit them all. I'm one of those wierd ones who fond of scrolling 100,000 events Wink
Well, if you're developing this to yourself, I guess there's no harm then, but I really think that's not the best approach if you're developing to a client, I know we can't please them all, but I also know that there are best practices and consolidated designs that pleases the most and I really think your approach is not the most appropriate. But who to know better what your target audience wants but yourself? :-\
-
You have been given the same advice from several people now. Why are you continuing to argue the point? If you don't want to use the advice then don't ask for it.
I know the language. I've read a book. - _Madmatt
Mark Nischalke wrote:
You have been given the same advice from several people now. Why are you continuing to argue the point?
Déjà vu :laugh:
-
First, putting 100,000 items in a ListView control is ridiculous. Do you really think a user wants to scroll through all that just to find a particular record?? If you add all the items to the listView all at once, there's no way to avoid the "freeze". That's because the UI thread has to handle adding those items to the ListView. it cannot be done from another thread because you can only maniplute a control on the thread that created it. You can, however, add each item tot he ListView, one a few at time, from a background thread, by Invoking a method on the UI thread to add just a few items at a time. This will give the UI thread time to handle other requests, but it'll take considerably longer to add your 100,000 items. You have a serious design flaw in your app if you think you need to show 100,000 items in a single control.
A guide to posting questions on CodeProject[^]
Dave KreskowiakDave Kreskowiak wrote:
That's because the UI thread has to handle adding those items to the ListView. it cannot be done from another thread because you can only maniplute a control on the thread that created it. You can, however, add each item tot he ListView, one a few at time, from a background thread, by Invoking a method on the UI thread to add just a few items at a time.
What about using a delegate on the background worker thread? This worked for me.
' Because we run our routine that gets the selected class's properties on a background thread using the ' BackgroundWorker component, we can't update our listview directly using this thread, if we try, we get ' a cross-threading exception; our listview was created on the program's main thread and can't be changed ' directly by our background thread. ' To get round this little problem, we create delegate routines and then call the listview control's ' invoke method which uses the delegated routine to update the listview control on the main thread. Private Delegate Sub AddItem(ByVal lv As ListView, ByVal lvi As ListViewItem)Private Shared Sub AddListViewItem(ByVal listViewCtrl As ListView, ByVal lvi As ListViewItem)
listViewCtrl.Items.Add(lvi)
End SublistViewCtrl.Invoke(New AddItem(AddressOf AddListViewItem), New Object() {listViewCtrl, lvGroupItem})
:)
I'm too lazy to Google it for you.
-
Great, thanks, virtual mode is of value to handle large lists especially with large icons. I will use it from now for those purpouses. However I presume I will not be able to use virtual view for log event because the list with those events is not static. There are many threads which use global logger class to report events. They are stored in a queue and once queue limit exceeds some constant 100,000 previous one is dequeued. Log event addition is safe with lock. It is possible that in virtual list view between RetrieveVirtualItem() calls threads may report dozens of new events, queue will move and it will end up of showing the same or odd events several times.
Чесноков
Yes and No, Virtual mode is definitely for log events, Its the fastest way of getting data out on the GUI. Just force the listview to reload the single Item when it changes ;) But that might takes some Native calls ;)
With great code, comes great complexity, so keep it simple stupid...:-\ :-\
-
Yes and No, Virtual mode is definitely for log events, Its the fastest way of getting data out on the GUI. Just force the listview to reload the single Item when it changes ;) But that might takes some Native calls ;)
With great code, comes great complexity, so keep it simple stupid...:-\ :-\
Ohh and by the way its your background collection that has to change, and when it does, just reset the Virtual item count on the listview, then it will reload, all visible items (from index, to index) from the background collection. ;)
With great code, comes great complexity, so keep it simple stupid...:-\ :-\
-
Dave Kreskowiak wrote:
That's because the UI thread has to handle adding those items to the ListView. it cannot be done from another thread because you can only maniplute a control on the thread that created it. You can, however, add each item tot he ListView, one a few at time, from a background thread, by Invoking a method on the UI thread to add just a few items at a time.
What about using a delegate on the background worker thread? This worked for me.
' Because we run our routine that gets the selected class's properties on a background thread using the ' BackgroundWorker component, we can't update our listview directly using this thread, if we try, we get ' a cross-threading exception; our listview was created on the program's main thread and can't be changed ' directly by our background thread. ' To get round this little problem, we create delegate routines and then call the listview control's ' invoke method which uses the delegated routine to update the listview control on the main thread. Private Delegate Sub AddItem(ByVal lv As ListView, ByVal lvi As ListViewItem)Private Shared Sub AddListViewItem(ByVal listViewCtrl As ListView, ByVal lvi As ListViewItem)
listViewCtrl.Items.Add(lvi)
End SublistViewCtrl.Invoke(New AddItem(AddressOf AddListViewItem), New Object() {listViewCtrl, lvGroupItem})
:)
I'm too lazy to Google it for you.
A delegate doesn't get around the thread/control problem. It just sets up a parameter for the Invoke call so the AddItem call gets made on the UI thread and not the background thread.
A guide to posting questions on CodeProject[^]
Dave Kreskowiak -
It is possible to run addition of items in background worker but it needs to add them in report progress handler, main thread only, which also hangs the application. Is there any other approaches to add them to list view without application freeze?
Чесноков