1128

Challenges:

1. Mathematical representation of model
2. learning models parameters from data

Overview:

• generative face model
  • space of faces: goals
    • each face represented as high dimensional vector
    • each vector in high dimensional space represents a face
  • Each face consists of
    • shape vector Si
    • Texture vector Ti
  • Shape and texture vectors:
    • Assumption: known point to point corrsp between faces
    • construction: for eaxample 2D parameterization using few manually selected feature points
    • shape, texture vecs: sampled at m common locations in parameterization
      • Typically m = few thousand
      • shape texture vecs 3m dimensional (x, y, z & r, g, b)
      • *feature points may not be sampled.
  • linear face model
    • linear combinations of faces in database
      • s = sum(ai * Si)
      • t = sum(bi * Ti)
      • Basis vectors Si, Ti
      • Avg face Savg = 1/n sum(Si)
      • Avg face Tavg = 1/n sum(Ti)
  • Probabilistic modeling
    • PDF over space of faces gives probability to encounter certain face in a population
    • Sample the PDF generates random new faces.
    • Ovservation
      • Shape & tex vecs are not suitable for probabilistic modeling
      • Too much redundancy
      • many vecs do not resemble real faces
      • real faces occupy
    • AssumptionL faces occupy linear subspace of high dimensional space.
      • Faces lie on hyperplane
      • Illustration in 3D
        • faces lie on 2D plane in 3D
      • How to determine hyperplane?
        • PCA:
          • find orthogonal sets of vecs that best represent input data points
          • first basis vec: largest variance along its direction
          • Following basis vecs: orthogonal, ordered according to variance along their directions
        • Dimensionality reduction:
          • discard basis vectors with low variance
          • represent each data point using remaining k basis vecs (k-dim hyperplane)
          • can show: hyperplanes obtained via PCA minimize L2 distance to plane
        • First basis vec maximized variance along its direction
          • w{1} = argmanx{sum((t1)i^2)} – argmax sum(xi * w)^2 = argmax{||Xw||^2} = argmax{w’X’Xw}
          • data points as row vecs xi , zero mean
          • matrix X consists of row vecs xi
        • w{1} is the eigenvec corespp to the max eigenvalue of X’X
      • Properties;
        • Matrix X’X is proportional to so-called sample covariance matrix of dataset X
        • if dataset has multi-variance normal distribution, maximum likelihood estimation of distribution is
          • f(x) = (2* pi) ….
      • Node
        • X is very large X is n x m matrix
          • n is # data vec
          • m is length of data vec
        • m>> n in general
        • X is m x m matrix, very large
      • SVD of X
        • X = U sigma V’
        • X’X  = V sigma’ UU’ sigma V’ = V sigma^2 V’
        • right singular vecs V of X are eigenvec of X’X
        • singular values sigma(k) of X are square roots of eigenvalues lambda(k) of X’X
        • change of bassis into orthogonal PCA basis by projection onto eigenvectors
      • T = XV = Usigma V’V = uSigma
      • left singular vectors U multiplied by singular velues in sigma
    • Dimensionality reduction
      • only consider l eigenvectors/ singular vectors correp to l largest singular values
      • Tl = XVl = U l * sigma l
      • Matrix Tl in R (mxl)  contains coord of m samples in reduced number l of dim
      • computation of only l components directly via truncated SVD
      • multivariate normal distribution in reduced space has covariance matrix
      • Diagonal matrix, l largest eigenvalues of X’X
      • PCA diagonalizes covariance matrix decorrelates data
  • attribute based modeling
    • mnually define attributes, label each face i with a weight mui for each attribute mu
    • attribute vecs
    • add/ subtract multiple of attribute vecs
  • model fitting, tracking
    • Assume a parametric shape model
      • Given parameters,can generate shape
    • model fitting, tracking problem
      • given some observation, find shape
      • parameters that most likely to produce the observation
    • bayes theorem
  • Matching to images
    • Model parameters to generate an image
      • Shape vec: alpha
      • tex vec: beta
      • rendering parameters (projection, lighting) rou
    • Given image, what are the most likely rendering parameters that generate that image
      • MAP
      • BAyes
      • compute max p
• Discussion
  • adv:
  • Disadv:
    • linear mdoel may not be accurate
    • linear model not suitable for large geometric deformations (rotations)