Challenges:

- Mathematical representation of model
- learning models parameters from data

Overview:

- generative face model
- space of faces: goals
- each face represented as high dimensional vector
- each vector in high dimensional space represents a face

- Each face consists of
- shape vector Si
- Texture vector Ti

- Shape and texture vectors:
- Assumption: known point to point corrsp between faces
- construction: for eaxample 2D parameterization using few manually selected feature points
- shape, texture vecs: sampled at m common locations in parameterization
- Typically m = few thousand
- shape texture vecs 3m dimensional (x, y, z & r, g, b)
- *feature points may not be sampled.

- linear face model
- linear combinations of faces in database
- s = sum(ai * Si)
- t = sum(bi * Ti)
- Basis vectors Si, Ti
- Avg face Savg = 1/n sum(Si)
- Avg face Tavg = 1/n sum(Ti)

- linear combinations of faces in database
- Probabilistic modeling
- PDF over space of faces gives probability to encounter certain face in a population
- Sample the PDF generates random new faces.
- Ovservation
- Shape & tex vecs are not suitable for probabilistic modeling
- Too much redundancy
- many vecs do not resemble real faces
- real faces occupy

- AssumptionL faces occupy linear subspace of high dimensional space.
- Faces lie on hyperplane
- Illustration in 3D
- faces lie on 2D plane in 3D

- How to determine hyperplane?
- PCA:
- find orthogonal sets of vecs that best represent input data points
- first basis vec: largest variance along its direction
- Following basis vecs: orthogonal, ordered according to variance along their directions

- Dimensionality reduction:
- discard basis vectors with low variance
- represent each data point using remaining k basis vecs (k-dim hyperplane)
- can show: hyperplanes obtained via PCA minimize L2 distance to plane

- First basis vec maximized variance along its direction
- w{1} = argmanx{sum((t1)i^2)} – argmax sum(xi * w)^2 = argmax{||Xw||^2} = argmax{w’X’Xw}
- data points as row vecs xi , zero mean
- matrix X consists of row vecs xi

- w{1} is the eigenvec corespp to the max eigenvalue of X’X

- PCA:
- Properties;
- Matrix X’X is proportional to so-called sample covariance matrix of dataset X
- if dataset has multi-variance normal distribution, maximum likelihood estimation of distribution is
- f(x) = (2* pi) ….

- Node
- X is very large X is n x m matrix
- n is # data vec
- m is length of data vec

- m>> n in general
- X is m x m matrix, very large

- X is very large X is n x m matrix
- SVD of X
- X = U sigma V’
- X’X = V sigma’ UU’ sigma V’ = V sigma^2 V’
- right singular vecs V of X are eigenvec of X’X
- singular values sigma(k) of X are square roots of eigenvalues lambda(k) of X’X
- change of bassis into orthogonal PCA basis by projection onto eigenvectors

- T = XV = Usigma V’V = uSigma
- left singular vectors U multiplied by singular velues in sigma

- Dimensionality reduction
- only consider l eigenvectors/ singular vectors correp to l largest singular values
- Tl = XVl = U l * sigma l
- Matrix Tl in R (mxl) contains coord of m samples in reduced number l of dim
- computation of only l components directly via truncated SVD
- multivariate normal distribution in reduced space has covariance matrix
- Diagonal matrix, l largest eigenvalues of X’X
- PCA diagonalizes covariance matrix decorrelates data

- attribute based modeling
- mnually define attributes, label each face i with a weight mui for each attribute mu
- attribute vecs
- add/ subtract multiple of attribute vecs

- model fitting, tracking
- Assume a parametric shape model
- Given parameters,can generate shape

- model fitting, tracking problem
- given some observation, find shape
- parameters that most likely to produce the observation

- bayes theorem

- Assume a parametric shape model
- Matching to images
- Model parameters to generate an image
- Shape vec: alpha
- tex vec: beta
- rendering parameters (projection, lighting) rou

- Given image, what are the most likely rendering parameters that generate that image
- MAP
- BAyes
- compute max p

- Model parameters to generate an image

- space of faces: goals
- Discussion
- adv:
- Disadv:
- linear mdoel may not be accurate
- linear model not suitable for large geometric deformations (rotations)