CAMSHIFT DEMO
Face and Color Tracking


Description

This application demonstrates a fast, simple color tracking algorithm that can be used to track faces, hands etc described in camshift.pdf. The CAMSHIFT algorithm is a modification of the Meanshift algorithm which is a robust statistical method of finding the mode (top) of a probability distribution. Both CAMSHIFT and Meanshift algorithms exist in the library. While it is a very fast and simple method of tracking, because CAMSHIFT tracks the center and size of the probability distribution of an object, it is only as good as the probability distribution that you produce for the object. Typically the probability distribution is derived from color via a histogram, although it could be produced from correlation, recognition scores or bolstered by frame differencing or motion detection schemes, or joint probabilities of different colors/motions etc.

In this application, we use only the most simplistic approach: A 1-D Hue histogram is sampled from the object in an HSV color space version of the image. To produce the probability image to track, histogram "back projection" (we replace image pixels by their histogram hue value) is used. The raw video image and backprojection is shown in the following image:

Figure 1

Since we use such a simple method in this application, you must tune the parameters for your camera and the lighting in the image as described below.

Note that this filter requires that you have DirectShow 6.0 or greater installed. DirectShow 6.0 for NT4.0, DirectShow 7.0 for Win 98 or Win2000). You will also need a video camera source filter which should be supplied by your USB camera vendor. Microsoft* Windows* 2000 may be the most suitable operating system for development since many vendors do not supply filters for Windows NT 4.0. However, Windows 98 will be adequate for just running the filter graph. 

Controls

Figure 2
[1] Select input from USB video camera.
[2] Select input from "bouncing ball" simulation.
[3] Select input from an AVI movie file.
[4] Stop and start AVI movie file if [3] is selected.
[5] Threshold the histogram bins to clean noise.
[6] Select the number of histogram bins to use in the 1-D Hue histogram.
[7] Minimum color saturation to accept threshold on Hue pixels.
[8] Minimum brightness to accept threshold on Hue pixels.
[9] Maximum brightness to accept threshold on Hue pixels.
[10] View (top) Raw video; (middle) Histogram backprojection; (bottom) Histogram bins.
[11] Select display window size -- on slower machines, a smaller display window will run faster.

How to

Track a colored object (face, ball etc):

(a) Reset thresholds: Set Threshold [5] to zero; Saturation [7] far left; Minimum brightness [8] far left; Maximum brightness [9] far right.
(b) Click video [1], bouncing ball [2] or AVI [3] to select your input source.
(c) Check the raw video display [10] button.
(d) Set the number of bins [6] you want in your histogram (32-100 is a good number).
(e) Use your mouse to draw a sample box over the colored object that you want to track.

...Tuning...

(f) Check the histogram "Back Project" [10] button. You want the object to be well segmented as in figure 1 above. To get things well segmented, adjust things in the order described below:
(g) Increase (move the slider to the right) the mimimum acceptable saturation threshold [7]. Make sure you allow a little time to see how your adjustment is working. Often moving it about 25% of the way to the right is sufficient.
(h) If (g) doesn't do the whole job, then increase the minimum acceptable brightness threshold [8] to the right allowing enough time to see the effect at each position. Usually 20-50% of the way to the right works.

...not working...

(i) If (g) & (h) do not give you a well segmented object, you probably need to change your camera settings and/or lighting. Histogram threshold [5] can have a marginal effect on edge regions. Decreasing (move to the left) high brightness threshold [9] can diminish edges and glaring white regions in the video image, usually moving it no more than 10% of the way to the left. Setting more histogram bins [6] sometimes helps.