ZhiyingDu/HiMoE-VLA-Libero-Object

HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision–Language–Action Policies

This repository contains the model checkpoints for HiMoE-VLA, as presented in the paper HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies.

• Paper: arXiv:2512.05693
• GitHub Repository: ZhiyingDu/HiMoE-VLA

Model Description

HiMoE-VLA is a vision–language–action (VLA) framework designed to handle the heterogeneity of modern large-scale robotic datasets. It introduces a Hierarchical Mixture-of-Experts (HiMoE) action module that progressively abstracts away differences across layers (such as embodiments, action spaces, sensor configurations, and control frequencies), enabling unified learning of shared robot behaviors.

Sample Usage

For local inference, you can use the following pattern (make sure to set up the environment and submodules as specified in the GitHub Repository):

from moevla.policies import policy_config as _policy_config
from moevla.training import config as _config

# Specify these parameters
train_config = ""
dataset_config = ""
checkpoint_dir = ""

policy = _policy_config.create_trained_policy(
    _config.get_training_config(train_config),    
    _config.get_dataset_config(dataset_config), 
    checkpoint_dir, 
    default_prompt=None
)

# Run inference on a dummy example.
example = {
    "observation/exterior_image_1_left": ...,
    "observation/wrist_image_left": ...,
    "prompt": "fold clothes"
}

action_chunk = policy.infer(example)["actions"]

Citation

If you find our work useful in your research, please consider citing our paper:

@article{du2025himoe,
  title={HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies},
  author={Du, Zhiying and Liu, Bei and Liang, Yaobo and Shen, Yichao and Cao, Haidong and Zheng, Xiangyu and Feng, Zhiyuan and Wu, Zuxuan and Yang, Jiaolong and Jiang, Yu-Gang},
  journal={arXiv preprint arXiv:2512.05693},
  year={2025}
}