Instructions to use xpeng-robotics/Capek0.5-35B-A3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use xpeng-robotics/Capek0.5-35B-A3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="xpeng-robotics/Capek0.5-35B-A3B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("xpeng-robotics/Capek0.5-35B-A3B")
model = AutoModelForMultimodalLM.from_pretrained("xpeng-robotics/Capek0.5-35B-A3B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use xpeng-robotics/Capek0.5-35B-A3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "xpeng-robotics/Capek0.5-35B-A3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "xpeng-robotics/Capek0.5-35B-A3B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/xpeng-robotics/Capek0.5-35B-A3B

SGLang

How to use xpeng-robotics/Capek0.5-35B-A3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "xpeng-robotics/Capek0.5-35B-A3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "xpeng-robotics/Capek0.5-35B-A3B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "xpeng-robotics/Capek0.5-35B-A3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "xpeng-robotics/Capek0.5-35B-A3B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use xpeng-robotics/Capek0.5-35B-A3B with Docker Model Runner:
```
docker model run hf.co/xpeng-robotics/Capek0.5-35B-A3B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Capek 0.5: An Execution-Centric Vision-Language Model for Embodied Intelligence

📄 Paper · 🌐 Project Page

Capek 0.5 is an execution-centric vision-language model. It brings spatial reasoning, temporal understanding, action guidance, and state verification into a single model for embodied reasoning. This repository provides Capek0.5-35B-A3B, built on Qwen3.6-35B-A3B, in Hugging Face Transformers format.

Spatial, temporal, guidance, and state capabilities recur throughout an embodied execution cycle

The four capabilities provide complementary evidence throughout the observation–reasoning–action cycle.

Capabilities and Training

Capek 0.5 organizes embodied reasoning around four recurring requirements of task execution:

Spatial Reasoning: geometry, relative positions, and viewpoint changes.
Temporal Understanding: event order, scene changes, and temporal localization.
Action Guidance: target points, affordance regions, and ordered image-space trajectories.
State Verification: physical predicates and task completion.

Candidate data inventory grouped into spatial, temporal, guidance, and state capabilities

Each capability is first learned by a dedicated specialist through reinforcement learning with verifiable rewards, starting from the same backbone. TIES weight merging combines the specialists to initialize a unified student, which is refined through routed multi-teacher on-policy distillation (MOPD). Specialist routing is used during training; inference runs on a single consolidated checkpoint.

Model Details

Architecture: Qwen3_5MoeForConditionalGeneration (qwen3_5_moe).
Scale: 35B total parameters, approximately 3B active per token, following the upstream architecture.
Inputs: text, images, and video through the supplied multimodal processor and chat template.
Outputs: generated text, including structured predictions when requested by the prompt.

Evaluation Results

Mean benchmark scores across four embodied capability domains and general capability

Under the paper's matched evaluation protocol, Capek0.5-35B-A3B improves 28 of 34 benchmark rows over its Qwen3.6-35B-A3B initialization. Selected results are shown as Qwen3.6 → Capek 0.5:

VSI-Bench: 60.83 → 70.69; Video-MME: 71.33 → 74.19.
PointBench: 76.66 → 81.22; StateBench-P: 65.80 → 76.80.
EmbodiedBench task success: EB-HAB 46.0% → 63.0%; EB-ALF 50.7% → 55.3%.

License and Attribution

Original Capek 0.5 contributions and the Qwen-derived components are distributed under Apache 2.0, as described in LICENSE.

LICENSE-Apache-2.0: the Apache License, Version 2.0.
LICENSE-Qwen3.6: the unchanged upstream license, including Copyright 2026 Alibaba Cloud.

This is a modified, post-trained derivative. Preserve the applicable license texts, attribution, and modification notices when redistributing. The upstream revision linked in LICENSE identifies the verified license text, not the exact training initialization revision.

Citation

@misc{chen2026capek05,
  title={Capek 0.5: An Execution-Centric Vision-Language Model for Embodied Intelligence},
  author={Ying Chen and Weizhen Li and Zhe Hu and Zhenjiang Li and Rui Jiang and Zhifeng Gu and Lihuang Fang and Jiangping Liu and Lei Yi and Jie Chen},
  year={2026},
  eprint={2608.06756},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2608.06756}
}