PERSE: Personalized 3D Generative Avatars from A Single Portrait
In this article, I’ll walk you through an exciting AI tool called PERSE, which creates personalized 3D avatars from just a single portrait. I’ll break down how it works, its features, and the creative possibilities it offers.
What is PERSE?
PERSE is an AI tool that generates a 3D head model from just one photo of a person. Unlike other 3D head generators I’ve featured on my channel, such as Cap4D or FaceLift, PERSE stands out because of its flexibility. Not only does it create a 3D head that you can animate, but it also allows you to edit specific features like hair, beard, eyebrows, and more. This opens up a wide range of creative opportunities.
PERSE has an even more impressive feature: you can transfer attributes from one photo to another. For example, if you create a 3D avatar from one photo, you can upload another photo of a different person and transfer features like hairstyle or beard onto the avatar. You can even blend facial features from two different images using an interactive slider.
PERSE Overview
| Detail | Description |
|---|---|
| Name | PERSE |
| Purpose | Generates personalized 3D avatars from a single portrait. |
| GitHub Page | PERSE GitHub Page |
| Official Paper | PERSE Paper on arXiv |
| GitHub Repository | PERSE GitHub Repo |
This table provides a quick snapshot of PERSE, its components, and relevant links for further exploration. Let me know if you need additional details!
How PERSE Works?
PERSE operates in two main stages to create a 3D avatar from a single photo.
Stage 1: Generating a Synthetic Dataset
- Input Photo: The process starts with a single input photo of a person.
- Synthetic Data Creation: PERSE uses a video diffusion model called Portrait-CHAMP to generate multiple photos of the same person with different facial features and expressions.
- Result: This creates a synthetic dataset of 2D images that maintain the same identity as the input photo but vary in attributes like hairstyle, facial hair, and expressions.
Stage 2: Creating the 3D Avatar
- 3D Model Training: Using the synthetic dataset, PERSE trains a 3D Gaussian Splatting (3DGS) avatar model.
- Disentangled Control: The 3DGS avatar is conditioned on a latent space, allowing for independent control over different facial attributes. For example, you can change the hair color without affecting the rest of the face.
- Attribute Transfer: PERSE also enables attribute transfer from real-world 2D images. This means you can blend features from different photos, such as hairstyles or beards, into the 3D avatar.
Key Features of PERSE
Here are some of the standout features that make PERSE a powerful tool for creating 3D avatars:
- Editable 3D Avatars: You can animate the 3D head and edit specific features like hair, beard, and eyebrows.
- Attribute Transfer: Transfer features like hairstyles or beards from one photo to another.
- Interactive Blending: Use a slider to blend facial features from two different images.
- Disentangled Control: Change individual attributes (e.g., hair color) without affecting the rest of the face.
Creative Possibilities with PERSE
PERSE can be used in a variety of ways, making it a versatile tool for different applications:
- Personalized Avatars: Create custom avatars for video games or social media profiles.
- Hairstyle and Beard Experiments: Try out different hairstyles or beards before making real-life changes.
- Creative Projects: Use PERSE for animation, character design, or other creative endeavors.
How to Use PERSE Personalized 3D Generative Avatars?
While PERSE is still in its early stages, here’s what we know so far:
- GitHub Link: At the top of the project page, there’s a GitHub link. However, as of now, the repository appears to be empty, meaning the code hasn’t been published yet.
- Main Page: For now, you can visit the main project page to learn more about PERSE and its capabilities.
Additional Details from the Project Page
Here’s a summary of the additional details provided on the main project page:
- Components: PERSE consists of two main components and an application.
- Portrait-CHAMP: A video diffusion model used to generate a 2D synthetic dataset from a single image.
- 3DGS Avatar: A 3D avatar model trained on the synthetic dataset, allowing for disentangled control over attributes.
- Attribute Transfer: PERSE enables attribute transfer from real-world 2D images, making it possible to interpolate between the pretrained latent space and in-the-wild attributes.
Final Thoughts
PERSE is an impressive tool that pushes the boundaries of 3D avatar creation. Its ability to generate editable 3D heads from a single photo, combined with features like attribute transfer and disentangled control, makes it a valuable resource for creators, gamers, and anyone interested in personalized avatars.
While the GitHub repository is currently empty, I’m excited to see how PERSE evolves as more details and code are released. For now, I’ll keep an eye on the main project page and update you as soon as more information becomes available.