SandLogicTechnologies/MiniCPM-V-4.7-35B-A3B-GGUF

MiniCPM-V 4.7 35B-A3B

MiniCPM-V 4.7 35B-A3B is a multimodal Mixture-of-Experts model developed by OpenBMB for visual and textual understanding. It is the first 35B-A3B MoE model in the MiniCPM-V family and extends the series with stronger multimodal capabilities while maintaining a focus on efficient inference and practical deployment.

This repository contains GGUF quantized variants of MiniCPM-V 4.7 35B-A3B for efficient local inference. The available quantization formats substantially reduce the model storage requirements while retaining the underlying multimodal architecture for image, video, and text-based workloads.

MiniCPM-V 4.7 35B-A3B is designed for single-image and multi-image understanding, video analysis, OCR, PDF parsing, visual grounding, and general multimodal interaction. The model also introduces techniques such as Lookahead Expert Scheduling (LES), fine-grained layer-wise quantization, and DSpark speculative sampling to improve the efficiency of MoE inference on resource-constrained devices.

---

Model Overview

• Model Name: MiniCPM-V 4.7 35B-A3B
• Base Model: openbmb/MiniCPM-V-4.7-35B-A3B
• Architecture: Multimodal Mixture-of-Experts Model
• Parameter Count: Approximately 35 Billion Parameters
• Active Parameters: Approximately 3 Billion Parameters per Token
• Modalities: Text, Image, Video
• Context Length: 262,144 tokens (256K)
• Vision Encoder: SigLIP2
• Language Backbone: Qwen3.5 / Qwen3.6-based MoE language backbone
• Experts: 256 total, 8 experts activated per token
• Language Model Layers: 40
• Developer: OpenBMB
• License: Apache 2.0

The model configuration specifies 256 total experts with 8 selected per token, 40 language-model layers, 16 attention heads, 2 key-value heads, and a 262,144-token maximum context length.

---

Quantization Formats

This repository provides GGUF quantized versions of MiniCPM-V 4.7 35B-A3B for reduced-storage local deployment. The three available formats offer different model-size and precision trade-offs.

IQ3_M

• Size reduction of approx 77.74% (14.38 GB) compared to 16-bit (64.61 GB)
• Aggressive 3-bit quantization providing the smallest representation among the supplied conversions
• Suitable for environments where reducing model storage and memory requirements is a primary concern
• Makes the large MoE multimodal model more practical to deploy on systems with tighter resource limits
• The lower precision may have a greater effect on detailed visual understanding, OCR, reasoning, and long-context workloads than higher-precision variants

IQ4_NL

• Size reduction of approx 71.24% (18.58 GB) compared to 16-bit (64.61 GB)
• Non-linear 4-bit quantization providing a higher-precision representation than IQ3_M
• Suitable for multimodal workloads involving image understanding, video analysis, OCR, PDF parsing, and visual reasoning
• Retains a substantially smaller stored representation than F16 while allocating more bits to model weights than the 3-bit variant
• Requires more storage than IQ3_M but provides an alternative for workloads where additional quantization precision is preferred

IQ4_XS

• Size reduction of approx 72.70% (17.64 GB) compared to 16-bit (64.61 GB)
• Compact 4-bit representation with a smaller file size than the IQ4_NL conversion
• Suitable for local multimodal applications involving visual question answering, document analysis, OCR, and video understanding
• Provides another 4-bit deployment option when storage requirements need to remain below the IQ4_NL conversion
• Its lower storage requirement comes with the same general 4-bit precision class while retaining the model's multimodal parameter structure

---

Training Background (Original Model)

MiniCPM-V 4.7 35B-A3B builds on the MiniCPM-V family's multimodal foundation and introduces a larger sparse MoE language backbone together with an efficient visual-processing architecture. The official model documentation identifies Qwen3.5 and Qwen3.6 as contributors to the language backbone and SigLIP2 as the vision-understanding component.

Multimodal Architecture

• Uses a 35B-A3B sparse Mixture-of-Experts language architecture
• Routes each token through 8 experts selected from a pool of 256 experts
• Uses a SigLIP2-based vision module for visual representation
• Supports image and video inputs alongside textual interaction
• Employs early visual-token compression to reduce computation in later vision-encoder layers
• Supports mixed 4x and 16x visual-token compression for different visual-detail and efficiency requirements

Efficient Inference

• Introduces Lookahead Expert Scheduling (LES) to predict and prefetch experts likely to be required by subsequent layers
• Uses fine-grained layer-wise quantization to reduce expert-weight storage and memory traffic
• Integrates DSpark speculative sampling to improve decoding efficiency
• The combined approach is intended to make sparse MoE inference more practical on devices with limited memory and bandwidth

Self-Replay Fine-Tuning

• The MiniCPM-V 4.7 release includes Self-Replay Fine-Tuning (SRFT) as a method for extending model capabilities
• SRFT is intended to support further customization while reducing forgetting of previously learned capabilities
• The model family provides fine-tuning support through frameworks including SWIFT and LLaMA-Factory

---

Key Capabilities

• Image Understanding Analyzes visual inputs and combines image information with textual instructions for multimodal responses.

• Multi-Image Understanding Supports workflows involving multiple images and comparisons or reasoning across visual inputs.

• PDF Parsing Supports understanding and processing of PDF-based document content.

• Visual Grounding Supports tasks that require connecting textual descriptions or questions with relevant regions within visual inputs.

• Multimodal Reasoning Combines visual and textual information when solving questions and interpreting complex visual content.

• Efficient MoE Inference Uses sparse expert activation together with scheduling and quantization techniques to reduce the practical cost of deploying the 35B-A3B architecture.

---

Usage Example

Using llama.cpp

For multimodal inference with the GGUF model, use the corresponding multimodal projector together with the selected GGUF model:

./llama-cli \
  -m SandlogicTechnologies/MiniCPM-V-4.7-35B-A3B-IQ4_NL.gguf \
  --mmproj mmproj-model-f16.gguf \
  -p "Describe the important visual information in this input."

---

Recommended Usecases

• Multimodal AI Assistants Build assistants capable of interpreting images alongside natural-language instructions.

• Visual Question Answering Answer questions that require understanding visual content and its relationship to accompanying text.

• Document Intelligence Process documents, PDFs, and visual pages containing structured textual information.

• OCR Applications Extract and interpret text from images and document-oriented visual inputs.

• Visual Grounding Develop applications that connect textual descriptions with relevant visual regions.

• Multimodal Research Experiment with vision-language reasoning, image understanding, video processing, and multimodal model evaluation.

• Resource-Constrained Inference Use GGUF quantized variants when the original F16 representation is too large for the available deployment environment.

---

Acknowledgments

These quantized models are based on the original work of the OpenBMB development team.

Special thanks to:

• The OpenBMB team for developing and releasing MiniCPM-V 4.7 35B-A3B and its multimodal inference ecosystem.

• The Qwen3.5 and Qwen3.6 teams for contributions to the language backbone, the SigLIP2 team for the vision-understanding component, and the Transformers project and broader open-source multimodal community for supporting the model's ecosystem.

• Georgi Gerganov and the llama.cpp community for enabling efficient GGUF-based local inference.

---

Contact

For questions, feedback, or support, please reach out at support@sandlogic.com or visit https://www.sandlogic.com/.