|
|
The 3D tracker uses the 2D positions of objects as viewed by multiple cameras to determine the 3D position of the objects. The 3D Tracker filter is a DirectShow filter that runs under Microsoft Windows. The filter receives input from two or more cameras, passes it to 2D trackers to process the video frames, then calls the OpenCV 3D tracker to combine the 2D results to compute a 3D position.
The 2D trackers are COM objects that
implement an interface called ITracker.
The available trackers are listed in registry in the
Video Trackers category.
Before using the 3D Tracker filter, you must register the tracker(s) you
want to use with regsvr32.exe.
There is an example COM object tracker that uses the CamShift tracker
already in OpenCV. There is also an example tracker called
BlobTracker.
You can run the 3D Tracker filter using the DirectShow filter graph editor,
graphedt.exe;
using the sample application provided, Tracker3dDemo.exe;
or by writing your own application.
(You can use Tracker3dDemo.exe as a starting point.)
To use Tracker3dDemo.exe, you must first use graphedt.exe to create a filter graph
for the sample app to load.
Before using the 3D Tracker filter, run the Calib filter with each of your cameras to determine the camera intrinsics. This needs to be done only once for each camera. Even if the cameras are moved, the intrinsics don't change. Save the intrinsics for each camera in a separate file. The 3D Tracker filter will ask for these filenames when you begin camera calibration in the 3D Tracker.
When you first insert the 3D Tracker filter into a graph, it will have no pins. Use the property page to set the number of cameras, as described below, and then in GraphEdit choose Refresh on the View menu to cause GraphEdit to redisplay the filter with the selected number of input and output pins. Before you connect the inputs and outputs, set the tracker type and input frame size also. If you save the graph, the number of cameras, the tracker type, and the input size are saved with the graph.
After you have set up the filter and connected it up, the graph will look something like this. You may have additional decompression or color conversion filters, depending on your cameras and your computer's display.
Right click on the 3D Tracker and choose Filter Properties... to bring up the filter's property page. A tab for the trackers' property page will be displayed as well if a tracker has been selected.
The controls in the box can be changed only when none of the pins of the filter are connected. You must click Apply after changing any of the controls in the box for them to take effect.
The checkerboard that is used for camera intrinsics calibration should have a large number of squares and be held close to the camera. A different checkerboard may be used to calibrate the 3D Tracker filter. It can have fewer, larger squares so it can be held farther from the camera. It must be visible to all the cameras at once. The position of the checkerboard when the 3D Tracker camera calibration is done determines the 3D coordinate system that is used to report the locations of the tracked objects. The plane of the checkerboard is the x-y plane, with the origin being the lower-left intersection on the checkerboard.
When you begin camera calibration, you will see red circles on the images of the checkerboard as it finds corners. When it finds all the corners in an image, the circles change to rainbow colors. Once it finds all the corners in all the images at the same time, the locations of the cameras is saved and calibration stops. If this happens instantly, you may only see a flash of color in the images. To force it to keep running the calibration step even after all the corners are found, hold down the control key when you click on Calibrate cameras. Click Calibrate cameras again (without the control key) to cause it to finish calibrating.
If you are using the BlobTracker to track infrared LEDs and you have infrared filters on the cameras, you must evenly illuminate the checkerboard with a fairly intense infrared source. A 250 watt infrared flood lamp works well. A regular (non-infrared) flood lamp probably would work as well.