Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinese
Paper β’ 2110.06696 β’ Published β’ 2
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("Langboat/mengzi-oscar-base-caption")
model = AutoModelForMaskedLM.from_pretrained("Langboat/mengzi-oscar-base-caption", device_map="auto")Quick Links
Mengzi-oscar-base-caption (Chinese Multi-modal Image Caption model)
Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinese
Mengzi-oscar-base-caption is fine-tuned based on Chinese multi-modal pre-training model Mengzi-Oscar, on AIC-ICC Chinese image caption dataset.
Usage
Installation
Check INSTALL.md for installation instructions.
Pretrain & fine-tune
See the Mengzi-Oscar.md for details.
Citation
If you find the technical report or resource is useful, please cite the following technical report in your paper.
@misc{zhang2021mengzi,
title={Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinese},
author={Zhuosheng Zhang and Hanqing Zhang and Keming Chen and Yuhang Guo and Jingyun Hua and Yulong Wang and Ming Zhou},
year={2021},
eprint={2110.06696},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
- Downloads last month
- 53
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="Langboat/mengzi-oscar-base-caption")