Digital human pipelines: from portrait driving to reviewable delivery

01 / Distinguish three common routes

“Digital human” is not one model. Portrait motion transfer starts from an identity image and a driving video, then transfers pose, expression and head motion; LivePortrait is an example. Audio-driven lip sync conditions mouth motion on speech; projects such as MuseTalk focus on audio-visual synchronization. A real-time 3D avatar instead combines a rig, facial blendshapes, speech or motion controls and a 3D engine, emphasizing latency, continuous control and scene rendering.

These methods solve different problems. A portrait-motion model does not get motion from speech alone if it expects a driving video; a driving video does not automatically acquire lip sync for a new voice. Before choosing a model, specify inputs, outputs, real-time needs, acceptable identity changes and permissions.

02 / The path from source media to an output

A typical portrait-animation pipeline detects and crops a face, encodes identity appearance, extracts pose and expression from the driver, maps motion, synthesizes or reenacts frames, blends boundaries and encodes video. A lip-sync route also aligns audio segments, phonemes or audio-visual features with video frames. Rendering a clip is an inference result; it does not by itself validate identity preservation, lip-sync quality, robustness across scenes or realism.

• Portrait motion transfer: identity image + driving video → motion extraction and mapping → synthesized video.
• Audio-driven lip sync: authorized portrait video + speech → lip-motion or audio-visual synthesis → synchronized video.
• Real-time 3D avatar: rigged character + motion or speech controls → real-time engine rendering → interactive scene.

03 / A repeatable portrait-motion exercise

The upstream LivePortrait quick-start is used here as a learning entry point. Select a release or commit and read its README and checkpoint terms first; match CUDA and PyTorch to the local GPU using the official installation guidance. The command uses the repository's sample assets; do not substitute another person's portrait or an unlicensed video. This procedure has not been executed locally for this note.

# Create an isolated environment, then follow the selected upstream release notes.
conda create -n portrait-lab python=3.10 -y
conda activate portrait-lab
git clone https://github.com/KwaiVGI/LivePortrait.git
cd LivePortrait
python -m pip install -r requirements.txt

# Download model checkpoints only from the official instructions.
# This quick-start uses the repository's own sample image and driving video.
python inference.py -s assets/examples/source/s9.jpg -d assets/examples/driving/d0.mp4

Environment versions, GPU support and command-line options can change upstream. If an option differs, follow the README for the pinned release. Never commit model weights, access tokens or private media to a public repository.

04 / What to inspect after generation

Sample frames from the start, middle, transitions and end of the clip. Inspect crop jumps, head drift, occlusion boundaries, distorted teeth or eyes and background flicker; for lip-sync, also check timing between audio and mouth opening. Compare versions with the same source media and settings, and retain failure cases. Human review is a useful first pass; research claims additionally need predefined splits, metrics, baselines and statistical uncertainty.

• Input quality: Is the face clear and unobstructed? Is the driver stable? Are audio and video aligned?
• Output quality: Does identity drift? Are expressions natural? Do boundaries flicker? Are speech and lip motion synchronized?
• Experiment record: model version, checkpoint source, input summary, parameters, random seed, runtime environment and failure notes.

Use your own media or obtain explicit permission for every portrait, voice and driving video, and disclose synthetic output. With real people, do not use cloned voices or realistic likenesses for undisclosed endorsements, impersonation or deception.