Joyfulxp/DeepSeek-V4-Flash-Vision-Exp-Abliterated

DeepSeek-V4-Flash-Vision-Exp — Abliterated (Overlay) — bayes-l2.0 (v2)

📢 v2 update — and an apology for v1

We apologize for the v1 release of this overlay. v1 edited the MTP / DSpark draft head, and in real use this caused garbling in long conversations — despite passing our offline checks. That failure was ours, and fixing it is the core of v2.

Adopting new experience — the community finding that the in-checkpoint draft head should be left untouched; drowzeys/keys-DeepSeekV4Flash-Vision-EXP-ablit is the source of that lesson. The other ablits we consulted (apetersson, s-zaizen) and the cebeuq-style overlay recipe this repo builds on are listed in Sources below. Confirmed by our own follow-up A/B study — v2 no longer edits the MTP head. Together with the re-tuned recipe, this partially resolves the long-dialogue garbling issue: on our 8192-token long-generation ruler the score improves from 2/4 (v1) → 3/4 (v2), and the single remaining divergence is inherited from the unedited base, not caused by the ablation.

Still running v1? Please switch to v2. The fix ships entirely in this repo's model-overlay-00001-of-00001.safetensors + the repointed model.safetensors.index.json.

⚠️⚠️⚠️ THIS REPOSITORY DOES NOT CONTAIN THE FULL MODEL ⚠️⚠️⚠️

This repo ships only the abliteration overlay + repointed index. You MUST first download the original base weights to use it:

Required base: deepseek-ai/DeepSeek-V4-Flash-Vision-Exp — all 48 shards (model-00001-of-00048.safetensors … model-00048-of-00048.safetensors), ~157 GB.

Setup (3 steps)

1. Download the original model repo above (48 shards + its model.safetensors.index.json).
2. Copy this repo's model-overlay-00001-of-00001.safetensors (1.44 GB) and overwrite the base repo's model.safetensors.index.json with this repo's index.
3. Load the directory with a V4-capable vLLM vision fork (see Serving below).

The overlay only stores the 86 edited tensors; without the base shards the repo cannot load.

---

⚡ Version 2 (bayes-l2.0) — what changed

This is v2, replacing the earlier v1. The edit recipe was re-tuned and re-validated:

	v1 (superseded)	v2 — this repo (bayes-l2.0)
Layers edited	43 backbone	43 backbone (all)
MTP / DSpark draft	edited (3 × mtp.*.wo_b)	NOT edited
Edited tensors	92	86
λ	2.0	2.0
Coverage	wob (v1)	layers.*.attn.wo_b (weight + scale)

Why drop MTP editing: community consensus (e.g. drowzeys) is that editing the in-checkpoint drafter kills DSpark speculative acceptance — and our own 2×2 behaviour study (below) confirmed no benefit from editing MTP, while leaving it untouched keeps DSpark working (draft acceptance measured normal). The result is a cleaner, narrower edit that avoids the draft head.

---

A decensored (abliterated) rebuild of deepseek-ai/DeepSeek-V4-Flash-Vision-Exp — a multimodal (vision-language) DeepSeek-V4 MoE model served at native precision across 2× NVIDIA RTX PRO 6000 Blackwell (96 GB each) with vLLM.

Refusal behaviour was removed from the attention output projections only (layers.*.attn.wo_b, all 43 layers, λ=2.0). Routed + shared experts, wo_a, embeddings, routers, norms, every mHC parameter and the entire 32-layer vision tower + its bias_vl are untouched — the edit is confined to the text path.

This is an overlay, not a re-quantization. Exactly 86 of 72,633 tensors (0.12 %) differ from the base; the other 72,547 are byte-identical. No quantization step anywhere — ships at the base checkpoint's native precision.

⚠️ Uncensored model. Safety refusals have been substantially removed. You are responsible for how you use it. Intended for local/research use on hardware you control. It will attempt almost any request.

---

Highlights

Base	deepseek-ai/DeepSeek-V4-Flash-Vision-Exp (DeepSeek-V4 MoE + 32-layer vision tower)
Architecture	43 layers, hidden 4096, 256 experts (top-6) + 1 shared, hybrid CSA + HCA attention, mHC hyper-connections, inline 3-stage DSpark draft head; vision tower 32×1024 dim, patch 14, bias_vl
Context	1,048,576 native (served here at 524,288)
Precision	unchanged — base checkpoint's native precision (FP8 e4m3 block-scaled, FP4 routed experts)
Edit	rank-1 orthogonal projection, λ = 2.0, on 43 × layers.*.attn.wo_b (weight + scale, 86 tensors) — MTP/DSpark draft NOT edited
Untouched	routed + shared experts, wo_a, embed, head, routers, norms, entire 32-layer vision tower + bias_vl, all mHC params, MTP draft
Size	this repo: 1.44 GB overlay (+ base ~157 GB required)
Runtime	vLLM (fraserpricee/vllm:dspark-cu132-vision), TP=2 across 2× RTX PRO 6000 Blackwell

---

Sources (methods, references & provenance)

Foundational method — cebeuq-style overlay: cebeuq/DeepSeek-V4-Flash-0731-abliterated — the "residual direction + FP8 block-scaled projection, edit only attn.wo_b, ship as a tensor overlay + repointed index" recipe this repo follows.

Community ablits of the same base (consulted for recipe decisions):

• drowzeys/keys-DeepSeekV4Flash-Vision-EXP-ablit — L10–35, λ=3.5, MTP not edited, 26 tensors. Community mainstream; source of the "do not touch the drafter" consensus.
• apetersson/DeepSeek-V4-Flash-Vision-Exp-Abliterated — rank-1 projection edit of the same base; refusal-direction provenance.
• s-zaizen/DeepSeek-V4-Flash-Vision-Exp-Abliterated-NVFP4 — NVFP4 variant; pins the base revision 86f746b36186f0e567729a5c06a8c918caba82a9.

Where this repo differs (decided by our own A/B behaviour study, not inherited):

• Layer range: all 43 layers (community favours L10–35 / 26 layers). Our 2×2 study {43/26} × {λ2.0/λ3.5}, all with MTP untouched, showed 43 layers is decisively better on thinking-mode generation (43-version converged 3/4 @8 k on high-difficulty prompts; 26-layer versions blew past the budget to 12k–17k chars). Early/26-layer truncation is a real cost on this checkpoint.
• λ = 2.0 (community uses 3.5). λ is model-specific; 3.5 gave no extra un-lock and degraded thinking on this checkpoint; 2.0 is the inversion point (see λ table below).
• MTP not edited (same as community; see version note above).

Our own direction-capture and bake method: forward hooks on all 43 attn.wo_b over AdvBench-harmful vs Alpaca-harmless prompts in all three reasoning modes (chat/think-high/think-max), merged per layer, then W ← W − λ·r̂(r̂ᵀW) on the FP8 e4m3 block-scaled weights (dequantize → project → re-quantize holding original block exponents fixed), emitted as an overlay.

---

Validation (measured on this release, bayes-l2.0)

Refusal (chat, n=48)

	Base Vision-Exp	This model (v2)
Safety refusals	22/24	0/48 (1 raw detector hit cleared as a modality misfire — "I'm a text-based AI, I can't create a video file", then still gives the full tutorial)

n=300 refusal mass-scan was run on v1 (0.0 % across 7 raw hits, 6 mismatch + 1 ideological stance). The recipe change (drop MTP, keep λ=2.0, same 43-layer projection) is narrow; that v1 n=300 figure and this v2 n=48 agree — 0 safety refusals.

Long-generation behaviour (8192 thinking budget, high-difficulty probes) — three-way contrast

Sample	Base	v2 (bayes-l2.0)	v1 (92-tensor, MTP-edited)
trigger_turn2	✓	✓	✓
explain_qm	✓	✓	✗ (div.)
write_code	✗ (div., model-inherent)	✗ (div., model-inherent)	✗ (div.)
math_derivation	✓	✓	✓
Score	3/4	3/4	2/4

No garbling / quote-storm on v2. The single divergence (write_code, a long-code task) is inherent to the model, not to ablation — it diverges identically on the unedited base. v1 (MTP-edited) is actually worse (2/4). Dropping MTP produced no text-coherence regression and removed a failure.

Vision

	Base	This model (v2)
Visual description	refuses / obfuscates	describes fully (colour-block image). Visual path decensors despite the text-side-only edit (see Findings).

Tool-calling (measured on this release)

Tool compliance: 6/6 = 1.000 (n=6, 3 tools; identical to base). Because the overlay only changes weight values (never the computation graph), throughput is expected unchanged from base (~75 tok/s single-stream; DSpark draft acceptance normal on v2, see Findings). v2 throughput bench was not re-run.

---

The λ operating envelope (calibrated on this checkpoint, not inherited)

λ	Result
3.0	Too strong. Long-generation garbling / semantic break. Reject.
2.5	Equivalent to 2.0 on the 8 k ruler (same score) but no added unlock here; no benefit over 2.0.
2.0	Selected. Full inversion point (`
1.9	Under-inversion fuzzy band (half-refuse / half-comply). Reject.
2.05–2.3	e4m3 quant dead-zone — edits swallowed by rounding. Avoid.

Do not inherit λ from other checkpoints. 2.0 is this checkpoint's clean operating point; the 0731 text sibling's λ=2.5 and the community's λ=3.5 do not transfer here.

---

Serving (vLLM, 2× RTX PRO 6000 Blackwell)

The full model (~157 GB + overlay) needs both 96 GB cards, TP=2. The overlay dir's 48 shards are relative symlinks to /mnt/ubuntulibs/models/DeepSeek-V4-Flash-Vision-Exp/, so the container must mirror-mount both /mnt/ubuntulib and /mnt/ubuntulibs (or the symlinks break and the engine crashes with FileNotFoundError):

docker run -d --name ds4-vision-abl --ipc=host --shm-size 32g --gpus all --restart unless-stopped \
  -p 5001:8000 \
  -v /mnt/ubuntulib:/mnt/ubuntulib:ro \
  -v /mnt/ubuntulibs:/mnt/ubuntulibs:ro \
  -v /data:/data \
  fraserpricee/vllm:dspark-cu132-vision \
  /opt/venv/bin/python -m vllm.entrypoints.cli.main serve /mnt/ubuntulib/models/DeepSeek-V4-Flash-Vision-Exp-abliterated \
  --served-model-name DeepSeek-Vision-EXP-abliterated --trust-remote-code \
  --tensor-parallel-size 2 --gpu-memory-utilization 0.95 \
  --max-model-len 524288 --max-num-seqs 8 --max-num-batched-tokens 2048 \
  --kv-cache-dtype fp8 --block-size 256 --tokenizer-mode deepseek_v4 \
  --attention-backend FLASHINFER_MLA_SPARSE_DSV4 \
  --kernel-config.moe_backend flashinfer_cutlass \
  --speculative-config '{"method":"dspark","num_speculative_tokens":4,"draft_sample_method":"greedy"}' \
  --enable-flashinfer-autotune \
  --compilation-config '{"cudagraph_mode":"FULL_AND_PIECEWISE","custom_ops":["all"]}' \
  --enable-prefix-caching --enable-chunked-prefill --async-scheduling \
  --reasoning-parser deepseek_v4 --enable-auto-tool-choice --tool-call-parser deepseek_v4 \
  --allowed-local-media-path /data

---

Findings

• Text-path edit also decensors the visual path. All 86 edited tensors are text-backbone attn.wo_b; the 32-layer vision tower and bias_vl are byte-identical. Yet the model answers visual prompts without refusal (see Validation) — the generation head decodes from a fused carrier whose wo_b was edited.
• Dropping MTP is strictly better here. v2 (MTP untouched) scored 3/4 on the 8 k ruler vs v1's (MTP-edited) 2/4, with DSpark draft acceptance confirmed working — no "must edit the drafter to unlock" effect.
• 43 layers beats the community's 26. On thinking-mode generation, 43-layer edited versions converge (1.4k–7.8k chars) while 26-layer versions blow the budget (12k–17k chars).
• A baked FP8 edit is weaker than a runtime hook (most elements round back); the e4m3 dead zone (λ ∈ [2.05, 2.3]) exists on this checkpoint — tune λ on the baked weights, never inherit.

---

Limitations

• Capability gated on refusal / coherence / multi-turn / tool-call / serving only. MMLU-Pro, GSM8K, HumanEval, etc. were not run on this checkpoint.
• Long-context unvalidated past multi-turn depth (8 turns) — no 168 k needle test.
• Refusal scored by marker matching on the parsed answer + manual review; will miss phrasings outside the marker set.
• write_code long-generation divergence is the one confirmed regression — but it is model-inherent (present on the unedited base too), not an ablation artifact.
• v2 n is small (refusal n=48, ruler 4-prompt contrast, tool n=6); direction agrees with v1 and the community but a large-n re-run was not done for v2.
• λ=2.0 is calibrated for the baked FP8 weights of this checkpoint only. Do not inherit.

---

Files

File	Size	Note
model-overlay-00001-of-00001.safetensors	1.44 GB	the only differing weight file (86 tensors)
model.safetensors.index.json	5.4 MB	repointed — 86 keys → overlay, 72,547 → base shard names
abliteration_report.json	27 KB	per-tensor edit audit (sha256 / clamped / overflow counts)
config.json / configuration.json / generation_config.json	—	architecture
tokenizer.json / tokenizer_config.json	6.4 MB	DeepSeek V4 tokenizer
encoding/encoding_dsv4.py	29 KB	official V4 prompt encoder (needed for correct chat formatting)
LICENSE	—	MIT

The 48 base shards are NOT in this repo. Download them from the base model, add this repo's overlay + repointed index into the same directory, and load. Removing the overlay + restoring the base index fully reverts the model.