RoboVLMs

AI model
Humanoids and robot learning

RoboVLMs is a codebase for turning pretrained vision-language models into vision-language-action policies that control robots. It supports training and evaluation on CALVIN and SimplerEnv.

Made by Tsinghua University, ByteDance Research, CASIA MAIS-NLPR.

Source on GitHub: github.com/Robot-VLAs/RoboVLMs (opens in a new tab). Facts read at commit b18f25c (opens in a new tab).

Licenses

Code: Apache-2.0 (opens in a new tab)
Read by Robotheca from LICENCE.txt (opens in a new tab) (Apache-2.0) at commit b18f25c (opens in a new tab), on 16 Oct 2026.
Weights: Apache-2.0 (opens in a new tab)
Read on huggingface.co/robovlms/RoboVLMs (opens in a new tab).
Commercial use: Yes

What it does

Lets most vision-language models be integrated into the codebase with roughly 30 lines of code, following a documented tutorial.
Turns a vision-language model into an action policy by adding an action head such as LSTM, FC, GPT or discrete decoders.
Supports training on the CALVIN dataset, Open X-Embodiment datasets, and custom datasets in a stated format.
Provides evaluation scripts for CALVIN and for SimplerEnv's Google Robot and Bridge environments.
Reports benchmark results on CALVIN, where the KosMos-based policy is compared with other methods.
Lists supported backbones such as Flamingo, LLaVA, KosMos2 and PaliGemma, with a table marking which are fully tested.

Written by Robotheca from the project's README (opens in a new tab) at commit b18f25c.

Getting started

Difficulty: Advanced
Hardware it needs, as the project states it: It mentions installing CUDA toolkit packages (cudatoolkit) and needing a GPU-oriented PyTorch setup, but gives no GPU, memory or operating system specification.
How to start: The project's README (opens in a new tab)

Maintenance

Status: Maintained: a commit in the last 12 months
Newest commit: 14 Apr 2026 on main
Latest release: v0.1.0 (29 Nov 2025)

Checked on GitHub on 16 Oct 2026.

Robots on Robotheca that use it

No robot on Robotheca is linked to it yet. Links come from a robot's robot.yaml, from its dependency files read as text, or from Robotheca's editors.

Robotheca lists AI models as a reference: nothing is hosted or run here, and every fact links to where it was read.