Veranstalter	Giorgio Panin, Ph.D.
Modul	IN3150
Typ	Vorlesung
Sprache	Englisch
Semester	WS 2009/2010
ECTS	3.0
SWS	2V
Hörerkreis	Wahlfach für Studenten der Informatik (Master, Diploma)
Zeit & Ort	Di 10:00 - 12:00 MI 03.07.023
Schein	Nach erfolgreicher mündlichen Prüfung

News

Next lecture (n. 5: Bayesian tracking with Kalman and Particle filters) will take place regularly, on Tuesday 24.11.

Open and currently running Theses

Thesis proposals can be found at the Vision section of our student projects webpage.

For information about our research group, see also the ITrackU webpage, and the OpenTL library.

Course description

The course aims to provide a structured overview of model-based object tracking, with the purpose of estimating and following in real-time the spatial pose (rotation, translation etc.) of one or more objects, by using digital cameras and fast computer vision techniques.

The first part of the course will introduce the general tools for object tracking:

1. Pose and deformation models, and camera projection

2. Methods for pose estimation from geometric feature correspondences

3. Bayesian tracking concepts (state dynamics, measurement likelihood)

4. Bayesian filters for linear and nonlinear models, with single or multi-hypothesis state distributions

Afterwards, we will concentrate on the visual part: among the many modalities available, we will focus in particular on the following ones:

1. Color-based: Matching color statistics, from the visible object surface to the underlying image area.

2. Keypoint- and Motion-based: Detection and tracking of single point features, possibly making use of image motion information (optical flow).

3. Contour-based: Matching the object boundary line, as it deforms with the object roto-translation (also called Active Contours).

4. Template-based: Registration of a fully textured surface (Template) to the image gray-level intensities.

Finally, the last lecture will introduce advanced topics, concerning: multiple cameras, multiple simultaneous objects, and data fusion with multiple modalities (colors, edges, ...).

Pre-requisites

The course will also provide the following pre-requisites in a self-contained fashion (a basic knowledge would be in any case recommended):

• Basic math and algebra (nonlinear functions and derivatives, matrix computation)

• Basic geometry: 3D transformations, projective geometry, camera imaging

• Probability theory and statistics

• Basic image processing (representation, filtering etc.)

• System theory: state-space representation, dynamics, observation

Material

Lecture slides for WS09/10 are currently in preparation:

Part I - General tools for object tracking

• Lecture1.pdf: Introduction to model-based visual tracking
• Lecture2.pdf: Object and camera geometry
• Lecture3.pdf: Pose estimation from feature correspondences (with Matlab exercises)
• Lecture4.pdf: Bayesian tracking: dynamics and observation models (with Matlab exercises)
• Lecture5.pdf: Bayesian tracking: Kalman and Particle filters (with Matlab exercises)

Bibliographical references

Lecture 1:

• Survey book [1] (Introduction)
• Survey paper [11]

Lecture 2:

• General transformations: [3], Chapter 2
• Rigid body motion, exponential representation: [1], Sec. 2.2; [7]
• Camera model: [1], Sec 2.1 (and references), [3], Chapter 6
• Camera calibration: [3], Chapter 7

Lecture 3:

• Pose estimation from corresponding features: [3], Chapter 4
• P3P problem: [1], Sec. 2.3.3
• Similarity estimation (in N-dimensions): [10]
• Linear and Nonlinear LSE: [1], Sec. 2.4 (and references)
• Robust LSE: [9], and [1], Sec.2.5

Lecture 4:

• General tracking concepts (not only vision): [6], Introduction
• Dynamical models: [5], Chapter (...), [6], Chapter (...)
• The three levels of visual measurements: taken from the data fusion literature [8] (data-, feature-, decision-level)
• General Bayesian tracking equations: [1], Sec. 2.6

Reference textbooks:

• [1] V. Lepetit, P. Fua, Monocular Model-Based 3D Tracking of Rigid Objects: A Survey, Foundations and Trends in Computer Graphics and Vision, 2005

• [2] G. Bradski, A. Kaehler, Learning OpenCV - Computer Vision with the OpenCV Library, First Edition, O' Reilly, 2008

• [3] R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision, 2nd Edition, Cambridge University Press, 2004

• [4] O. Faugeras, Three-Dimensional Computer Vision, MIT Press, 1993

• [5] A. Blake, M. Isard, Active Contours, Springer-Verlag, 1998.

• [6] Y. Bar-Shalom, X.-R. Li, T. Kirubarajan, Estimation with Applications to Tracking and Navigation, J. Wiley & Sons, 2001

• [7] R. Murray, Z. Li, S. Sastry, A Mathematical Introduction to Robotic Manipulation, CRC Press, 2002

• [8] D. Hall, J. Llinas, Handbook of multisensor data fusion, CRC Press, 2nd Edition, 2008

Reference papers:

• [9] M. A. Fischler, R. C. Bolles, Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography Comm. of the ACM, Vol 24, pp 381-395, 1981

• [10] S. Umeyama, Least-Squares Estimation of Transformation Parameters Between Two Point Patterns IEEE Trans. Pattern Anal. Mach. Intell. 13(4): 376-380 (1991)

• [11] Yilmaz, A., Javed, O., and Shah, M. 2006. Object tracking: A survey. ACM Comput. Surv. 38, 4 (Dec. 2006), 13