Auto Lip | Sync Blender

Assumption: you have shape keys for visemes (rest, MBP, FV, U, O, A, E, etc.)

Steps:

Rhubarb outputs phoneme intervals with timestamps.

Convert phoneme timings to Blender keyframes:

Tweak keyframe interpolation (Bezier/Ease) or use F-Curve modifiers (Noise, Limits) for natural motion.

Minimal example Python logic (conceptual): auto lip sync blender

- load JSON timings
- for each phoneme interval:
    frame_start = int(time_start * FPS)
    frame_end = int(time_end * FPS)
    sk = shape_keys[viseme_map[phoneme]]
    set keyframe sk.value = 1.0 at frame_start
    set keyframe sk.value = 0.0 at frame_end + small_offset

(Implement in Blender’s text editor, run to bake animation.) Assumption: you have shape keys for visemes (rest,

Blender has a built-in feature known as Shape Key Baker. This tool analyzes an audio file and converts the sound amplitude (loudness) into keyframes on a shape key. Rhubarb outputs phoneme intervals with timestamps

If you want, I can: