Face Tracking: Difference between revisions
mNo edit summary |
mNo edit summary |
||
| Line 1: | Line 1: | ||
'''[[Face Tracking|Face tracking]] for VRChat''' refers to using hardware and software to capture a user's real-life facial expressions and map them onto their [[Avatars]] in real time. This makes avatars more expressive and lifelike, enhancing immersion and social interaction in VRChat. | '''[[Face Tracking|Face tracking]] for VRChat''' refers to using hardware and software to capture a user's real-life facial expressions and map them onto their [[Avatars]] in real time. This makes avatars more expressive and lifelike, enhancing immersion and social interaction in VRChat. | ||
{{Noticebox/Stub}} | |||
----{{Noticebox/Community}} | ----{{Noticebox/Community}} | ||
Revision as of 23:21, 16 June 2025
Face tracking for VRChat refers to using hardware and software to capture a user's real-life facial expressions and map them onto their Avatars in real time. This makes avatars more expressive and lifelike, enhancing immersion and social interaction in VRChat.
[Reason: You can contribute by expanding and proofreading this article, in accordance with the Manual of Style.]
How Face Tracking works
Face Tracking works with the combination of Eye Tracking and Mouth Tracking (some headsets aren't supporting both) :
- Capture your Eyes and Mouth : Your facial movements—like eye blinks, eyebrow raises, and mouth movements—are captured using:
- Webcams (e.g., using Selfie Expression since 2025.1.3. Previously, users would use iFacialMocap on iPhone, VSeeFace on PC)
- Depth cameras (e.g., Intel RealSense or iPhone Face ID sensors data)
- Dedicated trackers (e.g., Apple ARKit, HTC Vive Facial Tracker or Full Face Tracker for XR Elite, Quest Pro, Varjo XR4, Pimax Crystal Mod, BigScreen Beyond 2)
- Send Data to VRChat
The facial data is converted into blendshape values (facial expression controls) and sent to VRChat using OSC or avatar-specific animations. Most commonly used OSC tool is VRCFT.
- Drive your Avatar The avatar must be set up with:
- Visemes (for lip syncing)
- Expression blendshapes (for emotions like happy, sad, surprised)
- Eye and head tracking support
High-End VR Headsets with Built-in or Add-on Face Tracking
| Headset | Face Tracking Support | Notes |
|---|---|---|
| Meta Quest Pro | ✅ Built-in | Tracks facial expressions + eye movement with inward-facing sensors |
| Meta Quest 3 | 🚫 Not built-in
⚙️ Experimental (via phone/Webcam + OSC) |
No native face tracking; requires external tools or mods |
| Apple Vision Pro | ✅ Built-in (ARKit-based) | Full face + eye tracking, but not officially compatible with VRChat |
| HTC Vive Pro 2 + Facial Tracker | ✅ Via add-on | Uses HTC Vive Facial Tracker (tracks lower face, jaw, lips) |
| HTC Vive XR Elite | ⚙️ Add-on face tracking planned | Eye tracking built-in; face tracking requires future accessory |
| Pimax Crystal | ⚙️ Optional | Eye tracking included, face tracking module sold separately |
| Varjo XR-4 | ✅ Professional-grade | Includes face + eye tracking; enterprise-focused (not ideal for consumer VRChat) |
| Valve Index | 🚫 No built-in face tracking
⚙️ Third-party mods possible |
Some DIY users use SlimeVR or OSC with webcams |
| Bigscreen Beyond | 🚫 No face tracking | No internal face/eye tracking hardware |
VRChatFaceTracking (VRCFT) Tool
VRCFT stands for VRChat Face Tracking, a community-made tool that allows you to send real-time face tracking data into VRChat using OSC. It acts as a bridge between face tracking software (like iFacialMocap or VSeeFace) and your VRChat avatar, enabling expressive facial animations in VRChat.
What VRCFT Does
- Receives facial tracking data (blendshapes) from supported modules
- Maps those blendshapes to your avatar in VRChat using OSC
- Works in desktop mode and VR
Requirements
| Requirement | Details |
|---|---|
| VRChat | Must be OSC-enabled (Settings > OSC) |
| Avatar | Must have blendshapes (shape keys) set up properly for face expressions |
| Tracking source | e.g., iFacialMocap (iOS), VSeeFace (PC webcam), or Modules |
| VRCFT installed | Steam (community tool, not VRChat official) or on their Website |
Important Notes
- Not officially made by VRChat, but widely used in the community
- Your avatar must be properly set up with blendshapes and OSC parameters, see VRCFT Parameters
- If you’re using Quest Pro, face tracking via VRCFT only works via PC + Link/Virtual Desktop