3D Reconstruction and Semantic Modeling of Eyelashes

Abstract

High-fidelity digital human modeling has become crucial in various applications, including gaming, visual effects and virtual reality. Despite the significant impact of eyelashes on facial aesthetics, their reconstruction and modeling have been largely unexplored. In this paper, we introduce the first data-driven generative model of eyelashes based on semantic features. This model is derived from real data by introducing a new 3D eyelash reconstruction method from multi-view images. The reconstructed data is made available which constitutes the first dataset of 3D eyelashes ever published. Through an innovative extraction process, we determine the features of any set of eyelashes, and present detailed descriptive statistics of human eyelashes shape. The proposed eyelashes model, which exclusively relies on semantic parameters, effectively captures the appearance of a set of eyelashes. Results show that the proposed model enables interactive, intuitive and realistic eyelashes modeling for non-experts, enriching avatar creation and synthetic data generation pipelines.&

Eyelash Reconstruction

Video

Visual Comparisons

Beeler et al. - “Coupled 3D Reconstruction of Sparse Facial Hair and Skin” - TOG 2012

Nam et al. - “Strand-accurate Multi-view Hair Capture” - CVPR 2019

Semantic Editing

To validate the semantic editing capability of our model, we developed an intuitive and interactive application with a dedicated user interface.

Style Transfer

Eyelashes swapping with our model. The eyelashes of the subject of the i-th row are transferred to the subject of the j-th column. In the diagonal, the reference image is shown.

Challenging Reconstruction

Challenging reconstruction results with (a) very short eyelashes, (b) light colored eyelashes, (c) low contrast due to a dark skin, (d) recessed eyelashes occluded in most view angles.

Dataset

A publicly available dataset of 3D eyelashes containing a total of 57 subjects. Each subject is composed by:

the left-side face skin mesh (untextured)
the eyeball (untextured)
the reconstructed eyelashes as points
the eyelash colors for each point
the eyelash radii for each point

(Textures have been removed for GDPR reasons)

Implantation area vertex mask and uv parameterization are provided once for all subjects.

Data is available on https://www.interdigital.com/data_sets

BibTeX

@article{kerbiriou2024,
  author    = {Kerbiriou, Glenn and Avril, Quentin and Marchal, Maud},
  title     = {3D Reconstruction and Semantic Modeling of Eyelashes},
  journal   = {Eurographics},
  year      = {2024},
}

Acknowledgments

The authors would like to thank Jules Saïdane for his precious work on manual annotation of our dataset, Derek Bradley (Disney Research) for providing a link to their dataset and Giljoo Nam (Meta) for running his LPMVS algorithm on our dataset.