MuseVLA
- AI model
- Humanoids and robot learning
MuseVLA is a vision-language-action model for robotic manipulation that picks thermal, acoustic, radar, or RGB-only sensing per task. It is built on the VITRA model.
Made by Microsoft, United States (checked on github.com/microsoft (opens in a new tab)).
Source on GitHub: github.com/microsoft/MuseVLA (opens in a new tab). Facts read at commit a0b7d34 (opens in a new tab).
Licenses
- Code
- MIT (opens in a new tab)
Read by Robotheca from LICENSE.txt (opens in a new tab) (MIT) at commit a0b7d34 (opens in a new tab), on 16 Oct 2026.
- Commercial use
- Yes
What it does
- Chooses among thermal, acoustic, mmWave radar, or RGB-only sensing based on the task via sensor tokens.
- Overlays sensor heatmaps onto RGB images to create grounded sensor images.
- Provides a pipeline that synthesizes multisensory training data from existing RGB datasets.
- Ships a Flask inference API server that accepts RGB images, robot state, and optional sensor images.
- Inference runs in three stages: sensor selection by a VLM, SAM3 segmentation with overlay, and DiT action generation.
- Includes scripts for fine-tuning and training on a released dataset or on your own multi-sensor data.
Written by Robotheca from the project's README (opens in a new tab) at commit a0b7d34.
Getting started
- Difficulty
- Advanced
- How to start
- The project's README (opens in a new tab)
Maintenance
- Status
- Active: a commit in the last 90 days
- Newest commit
- 12 Aug 2026 on main
- Latest release
- None published on GitHub
Checked on GitHub on 16 Oct 2026.
Links
Robots that can run it
No robot on Robotheca is linked to it yet. Links come from a robot's robot.yaml, from its dependency files read as text, or from Robotheca's editors.
Robotheca lists AI models as a reference: nothing is hosted or run here, and every fact links to where it was read.