Live video processing / DirectX

[merlinos]

Hi, I am pretty new to DirectX and I need to accomplish the following thing: Preview a live video stream from a webcam and be able to select (draggable&resizable rectangle) an area of the live video and apply some real-time image processing only on that area. This involves video overlay (for the selection rectangle) and video processing of a portion of the video stream. I understand DirectShow will let me preview a live video stream "quite easily". But will it give me the flexibility I need for overlaying the video and processing a portion of it? If so how? And if not, what could I use? Anything that could point me in the right direction would be greatly appreciated. Thanks!

[Taha Elsayed]

go to see my article http://www.codeproject.com/useritems/Motion\_Detection.asp :-)

[merlinos]

Thanks, but I am looking for something in C++, not C#. :-) Also I would like to know what the best way is to achieve what I am trying to do. DirectShow + DirectDraw, or DirectDraw only? Since I am starting with DirectX, I would like to get a few pointers in the right direction... For example, once I manage to get a preview of the live video feed using DirectShow, how do I implement localized video processing? What's the best way to do that? Thanks!

[Andrew Walker]

I've done something similar to this for the project I'm working on at the moment. I've used a DirectShow filter graph to capture the images, inside that graph is a sample grabber filter which pushes images into my application. They are then connverted into textures which are drawn with Direct3D. The key to this is that is that there are two distinct parts, both of which can be implemented in a number of ways. Frame capturing can be done with DirectShow, VFW or Video4Linux. Once you've got the frames in your image you can present them any way you like (GDI, Direct3D, DirectDraw, OpenGL, ...) Adding the image processing is fairly simple. You can either add the image processing into your application after you have retrieved the data or you can modularise it by making it a directshow filter. My personal opinion is to do as little manipulation of frames as possible in DirectShow to ensure you have the most upto date frames flowing through the filter graph. Either way you end up dealing with an array of bytes which represent the image (depending on the camera format typically red, green blue triplets) Drawing rectangles around regions is a minor problem compared to getting everything else correct, but all the drawing API's support this to some degree. Simple Image Processing Architecture with DirectShow:

Source Filter
|
CSampleGrabber Filter -- Application -- Texture -- Direct3D Surface
|
Null Renderer

---

If you can keep you head when all about you Are losing theirs and blaming it on you; If you can dream - and not make dreams your master; If you can think - and not make thoughts you aim; Yours is the Earth and everything that's in it. Rudyard Kipling

[merlinos]

Thanks Andrew! I think I need to spend some time learning DirectShow. I know DirectDraw and I understand how to apply image processing and overlay to a video buffer (primary surface, back buffer...). However I am missing the piece to get the video from the webcam to a buffer (DirectShow). Can you tell me briefly how it works? Merlinos

[Andrew Walker]

Have a look at the playcap sample (\DXSDK\Samples\C++\DirectShow\Capture\PlayCap) in the DirectX SDK. It runs through the steps to create a filter graph, and add a renderer. You should be able to build on the example to add an instance of the CSampleGrabber filter (\DXSDK\Samples\C++\DirectShow\Filters\Grabber) into the graph. Once the sample grabber is in the graph you pass it a callback, which is triggered when a frame arrives. In that callback, copy the image data onto the DirectDraw surface. Hopefully that will get you started :) - good luck!

---

If you can keep you head when all about you Are losing theirs and blaming it on you; If you can dream - and not make dreams your master; If you can think - and not make thoughts you aim; Yours is the Earth and everything that's in it. Rudyard Kipling

[merlinos]

I am trying to add the CSamlpeGrabber into the playcap sample but I am running into some issues. I get the following linking errors: playcap.obj : error LNK2001: unresolved external symbol "public: __thiscall CSampleGrabber::CSampleGrabber(struct IUnknown *,long *,int)" (??0CSampleGrabber@@QAE@PAUIUnknown@@PAJH@Z) strmbasd.lib(wxdebug.obj) : error LNK2001: unresolved external symbol __imp__timeGetTime@0 Can you tell me what I am missing here? Thanks! Also, I tried a different way to insert the grabber sample using simply its CLSID and when I do that I get the following error: playcap.obj : error LNK2001: unresolved external symbol _CLSID_GrabberSample I am sorry to ask all these newbie's questions but I am a DirectShow newbie. :)

[Andrew Walker]

to get timeGetTime working you need to link to winmm.lib including <initguid.h> should eliminate the problem with the clsid

---

If you can keep you head when all about you Are losing theirs and blaming it on you; If you can dream - and not make dreams your master; If you can think - and not make thoughts you aim; Yours is the Earth and everything that's in it. Rudyard Kipling