ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
Paper • 2107.02137 • Published
AI & ML interests
Large Language Models
Recent Activity
View all activity
Papers
View all Papers
wangshuohuan
authored 2
papers about 11 hours ago
wangshuohuan
authored 18
papers about 12 hours ago
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora
Paper • 2012.15674 • Published • 1
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
Paper • 2112.12731 • Published • 1
On Training Data Influence of GPT Models
Paper • 2404.07840 • Published
Dual Modalities of Text: Visual and Textual Generative Pre-training
Paper • 2404.10710 • Published • 2
HFT: Half Fine-Tuning for Large Language Models
Paper • 2404.18466 • Published • 1
ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages
Paper • 2212.06742 • Published • 4
Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging
Paper • 2410.01610 • Published • 1
Curiosity-Driven Reinforcement Learning from Human Feedback
Paper • 2501.11463 • Published
Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking
Paper • 2502.13842 • Published
Mixture of Hidden-Dimensions Transformer
Paper • 2412.05644 • Published • 2
VideoAR: Autoregressive Video Generation via Next-Frame & Scale Prediction
Paper • 2601.05966 • Published • 23
DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion
Paper • 2406.06567 • Published • 1
NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time
Paper • 2408.03675 • Published • 1
ERNIE 5.0 Technical Report
Paper • 2602.04705 • Published • 269
Mixture of Universal Experts: Scaling Virtual Width via Depth-Width Transformation
Paper • 2603.04971 • Published • 4
CLEAR: Unlocking Generative Potential for Degraded Image Understanding in Unified Multimodal Models
Paper • 2604.04780 • Published • 11
Native Audio-Visual Alignment for Generation
Paper • 2605.30073 • Published • 33
ERNIE 2.0: A Continual Pre-training Framework for Language Understanding
Paper • 1907.12412 • Published • 1