llama_ros
- Software
- Humanoids and robot learning
- Library
ROS 2 packages that embed llama.cpp so GGUF language, vision-language, audio and embedding models can run inside ROS 2 projects.
Source on GitHub: github.com/mgonzs13/llama_ros (opens in a new tab). Facts read at commit ee7fd13 (opens in a new tab).
Licenses
- Code
- MIT (opens in a new tab)
Read by Robotheca from LICENSE (opens in a new tab) (MIT) at commit ee7fd13 (opens in a new tab), on 16 Oct 2026.
- Commercial use
- Yes
What it does
- Runs GGUF-based LLMs, VLMs, audio-language, embedding and reranking models inside ROS 2 nodes.
- Offers an OpenAI-compatible chat completions interface with tool calling, reasoning and streaming.
- Supports speculative decoding with draft-model and ngram-based methods, which requires a single parallel context.
- Provides a llama_cli extension with ros2 llama launch and prompt commands to test models quickly.
- Configures models through ROS 2 parameters covering context, GPU, CPU, memory, multimodal and LoRA/control vector options.
- Can download models from HuggingFace by repository and filename, or load a local GGUF file.
Written by Robotheca from the project's README (opens in a new tab) at commit ee7fd13.
Getting started
- Difficulty
- Intermediate
- Hardware it needs, as the project states it
- Builds with CUDA when compiled with -DGGML_CUDA=ON, which requires the CUDA Toolkit; Docker images can be built with CUDA support and run with NVIDIA GPU access.
- How to start
- The project's website (opens in a new tab)
Maintenance
- Status
- Active: a commit in the last 90 days
- Newest commit
- 16 Oct 2026 on main
- Latest release
- 6.2.2 (16 Oct 2026)
Checked on GitHub on 16 Oct 2026.
Links
Robots on Robotheca that use it
No robot on Robotheca is linked to it yet. Links come from a robot's robot.yaml, from its dependency files read as text, or from Robotheca's editors.
Robotheca lists software as a reference: nothing is hosted or run here, and every fact links to where it was read.