kittn commited on
Commit
a46f068
·
verified ·
1 Parent(s): 1add252

Clarify exact parity under bf16 autocast

Browse files
Files changed (1) hide show
  1. README.md +2 -3
README.md CHANGED
@@ -60,7 +60,7 @@ If you want to minimize the discrepancy versus the original EUPE inference path,
60
 
61
  The stock Hugging Face DINOv3 implementation is internally correct and self-consistent, but it does not match the DINOv3 / EUPE reference implementations bitwise. The mismatch comes from the reference code persisting bf16-rounded RoPE `periods` in the checkpoint and computing angles as `coords / periods`, while Hugging Face reconstructs fp32 `inv_freq` from `rope_theta` and computes `coords * inv_freq`.
62
 
63
- If you want bitwise equivalence with the DINOv3 / EUPE references in pure fp32, run the following after loading `model` in the example above. It patches the already-loaded Hugging Face model to use the exact bf16-rounded `periods` and the reference RoPE forward:
64
 
65
  ```python
66
  import math
@@ -92,5 +92,4 @@ def forward(self, pixel_values):
92
 
93
  rope.forward = MethodType(forward, rope)
94
  ```
95
-
96
- Under `bf16` autocast, this still helps, but the remaining drift is no longer just RoPE.
 
60
 
61
  The stock Hugging Face DINOv3 implementation is internally correct and self-consistent, but it does not match the DINOv3 / EUPE reference implementations bitwise. The mismatch comes from the reference code persisting bf16-rounded RoPE `periods` in the checkpoint and computing angles as `coords / periods`, while Hugging Face reconstructs fp32 `inv_freq` from `rope_theta` and computes `coords * inv_freq`.
62
 
63
+ If you want bitwise equivalence with the DINOv3 / EUPE references, run the following after loading `model` in the example above. It patches the already-loaded Hugging Face model to use the exact bf16-rounded `periods` and the reference RoPE forward:
64
 
65
  ```python
66
  import math
 
92
 
93
  rope.forward = MethodType(forward, rope)
94
  ```
95
+ This restores bitwise equivalence in both pure fp32 and `bf16` autocast.