villa-X
- AI model
- Humanoids and robot learning
A vision-language-latent-action model for robots, from Microsoft Research, that learns latent actions from video and uses them to predict future frames. Released so far: the latent action model.
Made by Microsoft, United States (checked on github.com/microsoft (opens in a new tab)).
Source on GitHub: github.com/microsoft/villa-x (opens in a new tab). Facts read at commit 9b44899 (opens in a new tab).
Licenses
- Code
- MIT (opens in a new tab)
Read by Robotheca from LICENSE (opens in a new tab) (MIT) at commit 9b44899 (opens in a new tab), on 16 Oct 2026.
- Commercial use
- Yes
What it does
- Learns latent action tokens aligned with robot states and actions by adding an extra proprio forward dynamics model.
- Jointly trains a latent action expert and a robot action expert through joint diffusion in the policy model.
- Extracts latent actions from a video with the pre-trained latent action model and reconstructs future frames from them.
- Ships a pre-trained latent action model of 955M parameters on Hugging Face, with inference code and a Jupyter notebook demo.
- The actor model is not yet released; it is listed as pending in the release plan.
Written by Robotheca from the project's README (opens in a new tab) at commit 9b44899.
Getting started
- Difficulty
- Advanced
- How to start
- The project's README (opens in a new tab)
Maintenance
- Status
- Active: a commit in the last 90 days
- Newest commit
- 16 Sept 2026 on main
- Latest release
- None published on GitHub
Checked on GitHub on 16 Oct 2026.
Links
Robots that can run it
No robot on Robotheca is linked to it yet. Links come from a robot's robot.yaml, from its dependency files read as text, or from Robotheca's editors.
Robotheca lists AI models as a reference: nothing is hosted or run here, and every fact links to where it was read.