Title:
- Commercial: Frontal Face Reconstruction using Multiple Low Resolution Videos of a Scene (for Denso)
- Application in Brain Science: Neuron/Cell Reconstruction ...
- Generalization: 3D Object Reconstruction using Multiple Low Res Videos of a Scene using a Priori Model
Scenario:
- Several low resolution cameras monitoring a scene from various angles
- Subject is in the intersection zone of cameras
- Typical cameras:Shops, Parking Lots, Traffic Control Cameras, Cellphone,...
- Limited maximum velocity for subject
- Chances of partial and temporal occlusion of the subject's face
Scope:
- Security project: Not real-time
- Ideal system (100% accuracy) to face detection level → Bounding box of face is available
- Relatively static illumination and background clutter
Applications:
- Survellience: Suspect recognition (watchlist)
- Fornesic: Suspect face reconstruction, Recorded video automatic review
- Authentication: Biometric identification, HCI
- Scientific: Neuron reconstruction, Tumor reconst., Incorperating priori knowledge to fMRI
- Search Engine: Relevancy of faces, Facial expression
Motivations:
- Technological advancements: Storage devices, processing speed
- Low Resolution Video acquiring: cheap, popular, daily life
- Need to automation / intelligent apps.
Challenges:
- Spatial resolution
- Sensors: Shot Noise + Cost vs. High density
- Optics: Physical Limitation + Cost vs. Fine Instrument
- Low Resolution Challenges
- Missing information
- Insufficient high-freq features
- Noise & blur
- Insufficient sample (ill-posed problems)
- Anti-aliasing post processing
- Face Processing Challenges:
- View point: Camera-subject relative position, Face Pose
- Illumination: Shadows, ...
- Occlusion: Other objects, other faces, other part of same face(self-occ.)
- Scale: Distance to camera, Geometry of Face
- Deformation: Facial Expression, Structural Differences, Plastic Surgery
- Background clutter
- Intra-class variation: Age, Race, Gender, ...
Solutions Possible:
- Modular solutions are discussed in slides.
-
After face detection in low-res domain, my proposed system will try to reconstruct the face incorporating all the factors in one fitness function and minimize it under meaningful constraints. So it needs a robust framework to balance between speed and accurancy (something like Evoltion Strategy) and the fitness function can have elements from every domain: Illumination, Low Resolution, Template Matching, Multple Camera, etc.
Literature:
- Super Resolution
- 3D Reconstruction
- Active Appearance Model
- Structure from Motion
- Shape from Silhouette
- 3D Morphable Model
Road Map:
- Automation (Pose, Illumination, Scale, Single-Image Fitting, Optimization Balance, Accuracy vs. Speed Framework, Fitness Function Design)
- Occlusion (Outlier, Occlusion, Accessories and Make-up)
- Low Resolution (Low Res, Missing Data, Noise)
- Multiple Camera/Observation (Registration, Blend)
- Video/Temporal Properties (Video Synchronization, Model Correction using SfM, Sequential Input)
- Building 3DMM (Hardware, Acquisition, Processing)
- Commercial Implementation (Low Level language Software Design, Library Design, Integrating Modules)
