Adds frame to accumulator
void cvAcc( const CvArr* I, CvArr* S, const CvArr* mask=0 );
The function cvAcc adds the whole image I or its selected region to accumulator S:
S(x,y)=S(x,y)+I(x,y) if mask(x,y)!=0
Adds the square of source image to accumulator
void cvSquareAcc( const CvArr* img, CvArr* sqSum, const CvArr* mask=0 );
The function cvSquareAcc adds the square of input image I or its selected region to accumulator Sq:
Sq(x,y)=Sq(x,y)+I(x,y)2 if mask(x,y)!=0
Adds product of two input images to accumulator
void cvMultiplyAcc( const CvArr* I, const CvArr* J, CvArr* Sp, const CvArr* mask=0 );
The function cvMultiplyAcc adds product of the whole images I and J or their selected regions to accumulator Sp:
Sp(x,y)=Sp(x,y)+I(x,y)•J(x,y) if mask(x,y)!=0
Updates running average
void cvRunningAvg( const CvArr* I, CvArr* R, double alpha, const CvArr* mask=0 );
The function cvRunningAvg calculates weighted sum of input image I and accumulator R so that R becomes a running average of frame sequence:
R(x,y)=(1-α)•R(x,y) + α•I(x,y) if mask(x,y)!=0
where α (alpha) regulates update speed (how fast accumulator forgets about previous frames).
Updates motion history image by moving silhouette
void cvUpdateMotionHistory( const CvArr* S, CvArr* MHI,
double timestamp, double duration );
The function cvUpdateMotionHistory updates the motion history image as following:
MHI(x,y)=timestamp if S(x,y)!=0
0 if S(x,y)=0 and MHI(x,y)<timestamp-duration
MHI(x,y) otherwise
That is, MHI pixels where motion occurs are set to the current timestamp, while the pixels where motion happened far ago are cleared.
Calculates gradient orientation of motion history image
void cvCalcMotionGradient( const CvArr* MHI, CvArr* mask, CvArr* orientation,
double delta1, double delta2, int apertureSize=3 );
min(delta1,delta2) <= M(x,y)-m(x,y) <= max(delta1,delta2).
The function cvCalcMotionGradient calculates the derivatives Dx and Dy of MHI and then calculates gradient orientation as:
orientation(x,y)=arctan(Dy(x,y)/Dx(x,y))
where both Dx(x,y)' and Dy(x,y)' signs are taken into account (as in cvCartToPolar function). After that mask is filled to indicate where the orientation is valid (see delta1 and delta2 description).
Calculates global motion orientation of some selected region
double cvCalcGlobalOrientation( const CvArr* orientation, const CvArr* mask, const CvArr* MHI,
double currTimestamp, double mhiDuration );
The function cvCalcGlobalOrientation calculates the general motion direction in the selected region and returns the angle between 0° and 360°. At first the function builds the orientation histogram and finds the basic orientation as a coordinate of the histogram maximum. After that the function calculates the shift relative to the basic orientation as a weighted sum of all orientation vectors: the more recent is the motion, the greater is the weight. The resultant angle is a circular sum of the basic orientation and the shift.
Segments whole motion into separate moving parts
CvSeq* cvSegmentMotion( const CvArr* MHI, CvArr* segMask, CvMemStorage* storage,
double timestamp, double segthresh );
The function cvSegmentMotion finds all the motion segments and marks them in segMask with individual values each (1,2,...). It also returns a sequence of CvConnectedComp structures, one per each motion components. After than the motion direction for every component can be calculated with cvCalcGlobalOrientation using extracted mask of the particular component (using cvCmp)
Finds object center on back projection
int cvMeanShift( const CvArr* imgProb, CvRect windowIn,
CvTermCriteria criteria, CvConnectedComp* comp );
The function cvMeanShift iterates to find the object center given its back projection and initial position of search window. The iterations are made until the search window center moves by less than the given value and/or until the function has done the maximum number of iterations. The function returns the number of iterations made.
Finds object center, size, and orientation
int cvCamShift( const CvArr* imgProb, CvRect windowIn, CvTermCriteria criteria,
CvConnectedComp* comp, CvBox2D* box=0 );
The function cvCamShift implements CAMSHIFT object tracking algrorithm ([Bradski98]). First, it finds an object center using cvMeanShift and, after that, calculates the object size and orientation. The function returns number of iterations made within cvMeanShift.
CvCamShiftTracker class declared in cv.hpp implements color object tracker that uses the function.
[Bradski98] G.R. Bradski. Computer vision face tracking as a component of a perceptual
user interface. In Workshop on Applications of Computer Vision, pages 214–219,
Princeton, NJ, Oct. 1998.
Updated version can be viewed online at
http://www.intel.com/technology/itj/q21998/articles/art_2.htm.
Also, it is included into OpenCV distribution (camshift.pdf)
Changes contour position to minimize its energy
void cvSnakeImage( const IplImage* image, CvPoint* points, int length,
float* alpha, float* beta, float* gamma, int coeffUsage,
CvSize win, CvTermCriteria criteria, int calcGradient=1 );
The function cvSnakeImage updates snake in order to minimize its total energy that is a sum of internal energy that depends on contour shape (the smoother contour is, the smaller internal energy is) and external energy that depends on the energy field and reaches minimum at the local energy extremums that correspond to the image edges in case of image gradient.
The parameter criteria.epsilon is used to define the minimal number of points that must be moved during any iteration to keep the iteration process running.
If at some iteration the number of moved points is less than criteria.epsilon or the function performed criteria.maxIter iterations, the function terminates.
Calculates optical flow for two images
void cvCalcOpticalFlowHS( const CvArr* imgA, const CvArr* imgB, int usePrevious,
CvArr* velx, CvArr* vely, double lambda,
CvTermCriteria criteria );
The function cvCalcOpticalFlowHS computes flow for every pixel of the first input image using Horn & Schunck algorithm [Horn81].
[Horn81] Berthold K.P. Horn and Brian G. Schunck. Determining Optical Flow. Artificial Intelligence, 17, pp. 185-203, 1981.
Calculates optical flow for two images
void cvCalcOpticalFlowLK( const CvArr* imgA, const CvArr* imgB, CvSize winSize,
CvArr* velx, CvArr* vely );
The function cvCalcOpticalFlowLK computes flow for every pixel of the first input image using Lucas & Kanade algorithm [Lucas81].
[Lucas81] Lucas, B., and Kanade, T. An Iterative Image Registration Technique with an Application to Stereo Vision, Proc. of 7th International Joint Conference on Artificial Intelligence (IJCAI), pp. 674-679.
Calculates optical flow for two images by block matching method
void cvCalcOpticalFlowBM( const CvArr* imgA, const CvArr* imgB, CvSize blockSize,
CvSize shiftSize, CvSize maxRange, int usePrevious,
CvArr* velx, CvArr* vely );
The function cvCalcOpticalFlowBM calculates optical flow for overlapped blocks blockSize.width×blockSize.height pixels each, thus the velocity fields are smaller than the original images. For every block in imgA the functions tries to find a similar block in imgB in some neighborhood of the original block or shifted by (velx(x0,y0),vely(x0,y0)) block as has been calculated by previous function call (if usePrevious=1)
Calculates optical flow for a sparse feature set using iterative Lucas-Kanade method in pyramids
void cvCalcOpticalFlowPyrLK( const CvArr* imgA, const CvArr* imgB, CvArr* pyrA, CvArr* pyrB,
CvPoint2D32f* featuresA, CvPoint2D32f* featuresB,
int count, CvSize winSize, int level, char* status,
float* error, CvTermCriteria criteria , int flags );
The function cvCalcOpticalFlowPyrLK implements sparse iterative version of Lucas-Kanade optical flow in pyramids ([Bouguet00]). Calculates the optical flow between two images for the given set of points. The function finds the flow with sub-pixel accuracy.
Both parameters pyrA and pyrB comply with the following rules: if the image pointer is 0 , the function allocates the buffer internally, calculates the pyramid, and releases the buffer after processing. Otherwise, the function calculates the pyramid and stores it in the buffer unless the flag CV_LKFLOW_PYR_A[B]_READY is set. The image should be large enough to fit the Gaussian pyramid data. After the function call both pyramids are calculated and the ready flag for the corresponding image can be set in the next call.
[Bouguet00] Jean-Yves Bouguet. Pyramidal Implementation of the Lucas Kanade Feature Tracker.
The paper is included into OpenCV distribution (algo_tracking.pdf)
Kalman filter state
typedef struct CvKalman
{
int MP; /* number of measurement vector dimensions */
int DP; /* number of state vector dimensions */
int CP; /* number of control vector dimensions */
/* backward compatibility fields */
#if 1
float* PosterState; /* =state_pre->data.fl */
float* PriorState; /* =state_post->data.fl */
float* DynamMatr; /* =transition_matrix->data.fl */
float* MeasurementMatr; /* =measurement_matrix->data.fl */
float* MNCovariance; /* =measurement_noise_cov->data.fl */
float* PNCovariance; /* =process_noise_cov->data.fl */
float* KalmGainMatr; /* =gain->data.fl */
float* PriorErrorCovariance;/* =error_cov_pre->data.fl */
float* PosterErrorCovariance;/* =error_cov_post->data.fl */
float* Temp1; /* temp1->data.fl */
float* Temp2; /* temp2->data.fl */
#endif
CvMat* state_pre; /* predicted state (x'(k)):
x(k)=A*x(k-1)+B*u(k) */
CvMat* state_post; /* corrected state (x(k)):
x(k)=x'(k)+K(k)*(z(k)-H*x'(k)) */
CvMat* transition_matrix; /* state transition matrix (A) */
CvMat* control_matrix; /* control matrix (B)
(it is not used if there is no control)*/
CvMat* measurement_matrix; /* measurement matrix (H) */
CvMat* process_noise_cov; /* process noise covariance matrix (Q) */
CvMat* measurement_noise_cov; /* measurement noise covariance matrix (R) */
CvMat* error_cov_pre; /* priori error estimate covariance matrix (P'(k)):
P'(k)=A*P(k-1)*At + Q)*/
CvMat* gain; /* Kalman gain matrix (K(k)):
K(k)=P'(k)*Ht*inv(H*P'(k)*Ht+R)*/
CvMat* error_cov_post; /* posteriori error estimate covariance matrix (P(k)):
P(k)=(I-K(k)*H)*P'(k) */
CvMat* temp1; /* temporary matrices */
CvMat* temp2;
CvMat* temp3;
CvMat* temp4;
CvMat* temp5;
}
CvKalman;
The structure CvKalman is used to keep Kalman filter state. It is created by cvCreateKalman function, updated by cvKalmanPredict and cvKalmanCorrect functions and released by cvReleaseKalman functions. Normally, the structure is used for standard Kalman filter (notation and formulae are borrowed from excellent Kalman tutorial [Welch95]):
xk=A•xk-1+B•uk+wk zk=H•xk+vk,
where:
xk (xk-1) - state of the system at the moment k (k-1) zk - measurement of the system state at the moment k uk - external control applied at the moment k wk and vk are normally-distributed process and measurement noise, respectively: p(w) ~ N(0,Q) p(v) ~ N(0,R), that is, Q - process noise covariance matrix, constant or variable, R - measurement noise covariance matrix, constant or variable
In case of standard Kalman filter, all the matrices: A, B, H, Q and R are initialized once after CvKalman structure is allocated via cvCreateKalman. However, the same structure and the same functions may be used to simulate extended Kalman filter by linearizing extended Kalman filter equation in the current system state neighborhood, in this case A, B, H (and, probably, Q and R) should be updated on every step.
[Welch95] Greg Welch, Gary Bishop. An Introduction To the Kalman Filter. Technical Report TR95-041, University of North Carolina at Chapel Hill, 1995. Online version is available at http://www.cs.unc.edu/~welch/kalman/kalman_filter/kalman.html
Allocates Kalman filter structure
CvKalman* cvCreateKalman( int dynamParams, int measureParams, int controParams=0 );
The function cvCreateKalman allocates CvKalman and all its matrices and initializes them somehow.
Deallocates Kalman filter structure
void cvReleaseKalman(CvKalman** kalman );
The function cvReleaseKalman releases the structure CvKalman and all underlying matrices.
Estimates subsequent model state
const CvMat* cvKalmanPredict( CvKalman* kalman, const CvMat* control=NULL ); #define cvKalmanUpdateByTime cvKalmanPredict
The function cvKalmanPredict estimates the subsequent stochastic model state by its current state and stores it at kalman->state_pre:
x'k=A•xk+B•uk
P'k=A•Pk-1*AT + Q,
where
x'k is predicted state (kalman->state_pre),
xk-1 is corrected state on the previous step (kalman->state_post)
(should be initialized somehow in the beginning, zero vector by default),
uk is external control (control parameter),
P'k is priori error covariance matrix (kalman->error_cov_pre)
Pk-1 is posteriori error covariance matrix on the previous step (kalman->error_cov_post)
(should be initialized somehow in the beginning, identity matrix by default),
The function returns the estimated state.
Adjusts model state
void cvKalmanCorrect( CvKalman* kalman, const CvMat* measurement=NULL ); #define cvKalmanUpdateByMeasurement cvKalmanCorrect
The function cvKalmanCorrect adjusts stochastic model state on the basis of the given measurement of the model state:
Kk=P'k•HT•(H•P'k•HT+R)-1 xk=x'k+Kk•(zk-H•x'k) Pk=(I-Kk•H)•P'k where zk - given measurement (mesurement parameter) Kk - Kalman "gain" matrix.
The function stores adjusted state at kalman->state_post and returns it on output.
#include "cv.h"
#include "highgui.h"
#include <math.h>
int main(int argc, char** argv)
{
/* A matrix data */
const float A[] = { 1, 1, 0, 1 };
IplImage* img = cvCreateImage( cvSize(500,500), 8, 3 );
CvKalman* kalman = cvCreateKalman( 2, 1, 0 );
/* state is (phi, delta_phi) - angle and angle increment */
CvMat* state = cvCreateMat( 2, 1, CV_32FC1 );
CvMat* process_noise = cvCreateMat( 2, 1, CV_32FC1 );
/* only phi (angle) is measured */
CvMat* measurement = cvCreateMat( 1, 1, CV_32FC1 );
CvRandState rng;
int code = -1;
cvRandInit( &rng, 0, 1, -1, CV_RAND_UNI );
cvZero( measurement );
cvNamedWindow( "Kalman", 1 );
for(;;)
{
cvRandSetRange( &rng, 0, 0.1, 0 );
rng.disttype = CV_RAND_NORMAL;
cvRand( &rng, state );
memcpy( kalman->transition_matrix->data.fl, A, sizeof(A));
cvSetIdentity( kalman->measurement_matrix, cvRealScalar(1) );
cvSetIdentity( kalman->process_noise_cov, cvRealScalar(1e-5) );
cvSetIdentity( kalman->measurement_noise_cov, cvRealScalar(1e-1) );
cvSetIdentity( kalman->error_cov_post, cvRealScalar(1));
/* choose random initial state */
cvRand( &rng, kalman->state_post );
rng.disttype = CV_RAND_NORMAL;
for(;;)
{
#define calc_point(angle) \
cvPoint( cvRound(img->width/2 + img->width/3*cos(angle)), \
cvRound(img->height/2 - img->width/3*sin(angle)))
float state_angle = state->data.fl[0];
CvPoint state_pt = calc_point(state_angle);
/* predict point position */
const CvMat* prediction = cvKalmanPredict( kalman, 0 );
float predict_angle = prediction->data.fl[0];
CvPoint predict_pt = calc_point(predict_angle);
float measurement_angle;
CvPoint measurement_pt;
cvRandSetRange( &rng, 0, sqrt(kalman->measurement_noise_cov->data.fl[0]), 0 );
cvRand( &rng, measurement );
/* generate measurement */
cvMatMulAdd( kalman->measurement_matrix, state, measurement, measurement );
measurement_angle = measurement->data.fl[0];
measurement_pt = calc_point(measurement_angle);
/* plot points */
#define draw_cross( center, color, d ) \
cvLine( img, cvPoint( center.x - d, center.y - d ), \
cvPoint( center.x + d, center.y + d ), color, 1, 0 ); \
cvLine( img, cvPoint( center.x + d, center.y - d ), \
cvPoint( center.x - d, center.y + d ), color, 1, 0 )
cvZero( img );
draw_cross( state_pt, CV_RGB(255,255,255), 3 );
draw_cross( measurement_pt, CV_RGB(255,0,0), 3 );
draw_cross( predict_pt, CV_RGB(0,255,0), 3 );
cvLine( img, state_pt, predict_pt, CV_RGB(255,255,0), 3, 0 );
/* adjust Kalman filter state */
cvKalmanCorrect( kalman, measurement );
cvRandSetRange( &rng, 0, sqrt(kalman->process_noise_cov->data.fl[0]), 0 );
cvRand( &rng, process_noise );
cvMatMulAdd( kalman->transition_matrix, state, process_noise, state );
cvShowImage( "Kalman", img );
code = cvWaitKey( 100 );
if( code > 0 ) /* break current simulation by pressing a key */
break;
}
if( code == 27 ) /* exit by ESCAPE */
break;
}
return 0;
}
ConDenstation state
typedef struct CvConDensation
{
int MP; //Dimension of measurement vector
int DP; // Dimension of state vector
float* DynamMatr; // Matrix of the linear Dynamics system
float* State; // Vector of State
int SamplesNum; // Number of the Samples
float** flSamples; // array of the Sample Vectors
float** flNewSamples; // temporary array of the Sample Vectors
float* flConfidence; // Confidence for each Sample
float* flCumulative; // Cumulative confidence
float* Temp; // Temporary vector
float* RandomSample; // RandomVector to update sample set
CvRandState* RandS; // Array of structures to generate random vectors
} CvConDensation;
The structure CvConDensation stores CONditional DENSity propagATION tracker state. The information about the algorithm can be found at http://www.dai.ed.ac.uk/CVonline/LOCAL_COPIES/ISARD1/condensation.html
Allocates ConDensation filter structure
CvConDensation* cvCreateConDensation( int DynamParams, int MeasureParams, int SamplesNum );
The function cvCreateConDensation creates CvConDensation structure and returns pointer to the structure.
Deallocates ConDensation filter structure
void cvReleaseConDensation( CvConDensation** ConDens );
The function cvReleaseConDensation releases the structure CvConDensation (see cvConDensation) and frees all memory previously allocated for the structure.
Initializes sample set for condensation algorithm
void cvConDensInitSampleSet( CvConDensation* ConDens, CvMat* lowerBound, CvMat* upperBound );
The function cvConDensInitSampleSet fills the samples arrays in the structure CvConDensation with values within specified ranges.
Estimates subsequent model state
void cvConDensUpdateByTime( CvConDensation* ConDens );
The function cvConDensUpdateByTime estimates the subsequent stochastic model state from its current state.