Face Tracking: Difference between revisions

From VRChat Wiki
mNo edit summary
mNo edit summary
Line 1: Line 1:
'''[[Face Tracking|Face tracking]] for VRChat''' refers to using hardware and software to capture a user's real-life facial expressions and map them onto their [[Avatars]] in real time. This makes avatars more expressive and lifelike, enhancing immersion and social interaction in VRChat.
'''[[Face Tracking|Face tracking]] for VRChat''' refers to using hardware and software to capture a user's real-life facial expressions and map them onto their [[Avatars]] in real time. This makes avatars more expressive and lifelike, enhancing immersion and social interaction in VRChat.
{{Noticebox/Stub}}
----{{Noticebox/Community}}
----{{Noticebox/Community}}



Revision as of 23:21, 16 June 2025

Face tracking for VRChat refers to using hardware and software to capture a user's real-life facial expressions and map them onto their Avatars in real time. This makes avatars more expressive and lifelike, enhancing immersion and social interaction in VRChat.

V · EThis page is a stub.
You can help the VRChat Wiki by improving it.
[Reason: You can contribute by expanding and proofreading this article, in accordance with the Manual of Style.]

V · ECommunity-written content
The following was created by the community. It may contain material not directly endorsed by the VRChat team. To learn more, consider reading Contributing to the VRChat Wiki.

How Face Tracking works

Face Tracking works with the combination of Eye Tracking and Mouth Tracking (some headsets aren't supporting both) :

  • Capture your Eyes and Mouth : Your facial movements—like eye blinks, eyebrow raises, and mouth movements—are captured using:
    • Webcams (e.g., using Selfie Expression since 2025.1.3. Previously, users would use iFacialMocap on iPhone, VSeeFace on PC)
    • Depth cameras (e.g., Intel RealSense or iPhone Face ID sensors data)
    • Dedicated trackers (e.g., Apple ARKit, HTC Vive Facial Tracker or Full Face Tracker for XR Elite, Quest Pro, Varjo XR4, Pimax Crystal Mod, BigScreen Beyond 2)
  • Send Data to VRChat

The facial data is converted into blendshape values (facial expression controls) and sent to VRChat using OSC or avatar-specific animations. Most commonly used OSC tool is VRCFT.

  • Drive your Avatar The avatar must be set up with:
    • Visemes (for lip syncing)
    • Expression blendshapes (for emotions like happy, sad, surprised)
    • Eye and head tracking support

High-End VR Headsets with Built-in or Add-on Face Tracking

Headset Face Tracking Support Notes
Meta Quest Pro ✅ Built-in Tracks facial expressions + eye movement with inward-facing sensors
Meta Quest 3 🚫 Not built-in

⚙️ Experimental (via phone/Webcam + OSC)

No native face tracking; requires external tools or mods
Apple Vision Pro ✅ Built-in (ARKit-based) Full face + eye tracking, but not officially compatible with VRChat
HTC Vive Pro 2 + Facial Tracker ✅ Via add-on Uses HTC Vive Facial Tracker (tracks lower face, jaw, lips)
HTC Vive XR Elite ⚙️ Add-on face tracking planned Eye tracking built-in; face tracking requires future accessory
Pimax Crystal ⚙️ Optional Eye tracking included, face tracking module sold separately
Varjo XR-4 ✅ Professional-grade Includes face + eye tracking; enterprise-focused (not ideal for consumer VRChat)
Valve Index 🚫 No built-in face tracking

⚙️ Third-party mods possible

Some DIY users use SlimeVR or OSC with webcams
Bigscreen Beyond 🚫 No face tracking No internal face/eye tracking hardware

VRChatFaceTracking (VRCFT) Tool

VRCFT stands for VRChat Face Tracking, a community-made tool that allows you to send real-time face tracking data into VRChat using OSC. It acts as a bridge between face tracking software (like iFacialMocap or VSeeFace) and your VRChat avatar, enabling expressive facial animations in VRChat.

What VRCFT Does

  • Receives facial tracking data (blendshapes) from supported modules
  • Maps those blendshapes to your avatar in VRChat using OSC
  • Works in desktop mode and VR

Requirements

Requirement Details
VRChat Must be OSC-enabled (Settings > OSC)
Avatar Must have blendshapes (shape keys) set up properly for face expressions

See VRCFT Standards Wiki

Tracking source e.g., iFacialMocap (iOS), VSeeFace (PC webcam), or Modules
VRCFT installed Steam (community tool, not VRChat official) or on their Website

Important Notes

  • Not officially made by VRChat, but widely used in the community
  • Your avatar must be properly set up with blendshapes and OSC parameters, see VRCFT Parameters
  • If you’re using Quest Pro, face tracking via VRCFT only works via PC + Link/Virtual Desktop

See Also