Hugging Face's logo

SwarmandBee
/
SwarmAtlas-27B

Text Generation
Transformers
English
cre
commercial-real-estate
finance
swarm-and-bee
defendable-os
gemma
27b
domain-tuned
atlas
Model card Files
xet
 Community
SwarmandBee commited on
Commit
c5d8bd7
·
verified ·
1 Parent(s): b975135

model card · ecosystem-unified contact + brand stack v0.1.0

Browse files
Files changed (1) hide show
  1. README.md +62 -274
README.md CHANGED
@@ -1,311 +1,99 @@
1
  ---
2
- language:
3
- - en
4
  license: apache-2.0
 
 
 
 
5
  library_name: transformers
6
  tags:
7
- - commercial-real-estate
8
- - CRE
9
- - capital-markets
10
- - underwriting
11
- - finance
12
- - domain-specific
13
- - fine-tuned
14
- - qwen3.5
15
- - intelligence-objects
16
- base_model: Qwen/Qwen3.5-27B
17
- model-index:
18
- - name: SwarmAtlas-27B
19
- results:
20
- - task:
21
- type: text-generation
22
- name: CRE Underwriting
23
- dataset:
24
- type: custom
25
- name: Swarm Capital Markets Intelligence
26
- config: default
27
- split: test
28
- metrics:
29
- - type: loss
30
- value: 0.4186
31
- name: Training Loss
32
- - type: loss
33
- value: 0.2238
34
- name: Eval Loss
35
- - type: accuracy
36
- value: 1.0
37
- name: Math Accuracy (12/12)
38
- pipeline_tag: text-generation
39
  ---
40
 
41
  # SwarmAtlas-27B
42
 
43
- > Capital markets intelligence model. Trained on 45,039 curated CRE pairs. Underwrites real deals at institutional grade — 12/12 math accuracy on live validation.
44
-
45
- ---
46
-
47
- ## Model Description
48
-
49
- SwarmAtlas-27B is a domain-specific language model fine-tuned for **commercial real estate capital markets intelligence**. Built by [Swarm & Bee](https://swarmandbee.ai), it transforms raw deal parameters into institutional-grade underwriting, IC memos, waterfall analyses, and capital stack recommendations.
50
-
51
- **The thesis**: We don't sell models. We sell verified training data. SwarmAtlas exists to prove the data is bankable — and it did. On a live CRE deal stress test, it scored **12 out of 12 on mathematical accuracy** and correctly identified the structural deal-killer that would have cost the LP their preferred return.
52
-
53
- ---
54
-
55
- ### Key Facts
56
-
57
- | Attribute | Value |
58
- |-----------|-------|
59
- | **Base Model** | [Qwen/Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B) Dense |
60
- | **Architecture** | Gated Delta Networks (75% GDN + 25% Standard Attention) |
61
- | **Parameters** | 27B (all active, dense) |
62
- | **Hidden Dim** | 5,120 |
63
- | **Layers** | 64 |
64
- | **Vocab** | 248,320 |
65
- | **Context** | 16,384 tokens (training) / 262K native / 1M via YaRN |
66
- | **Training Method** | bf16 LoRA r=64 alpha=32 |
67
- | **Training Steps** | 844 |
68
- | **Training Loss** | 0.4186 |
69
- | **Eval Loss** | 0.2238 |
70
- | **Training Time** | 29.32 hours |
71
- | **Training GPU** | NVIDIA RTX PRO 6000 Blackwell (96GB) |
72
- | **Serving** | vLLM 0.17.0, 88 tok/s @ 4 concurrent |
73
-
74
- ---
75
-
76
- ## Training Data
77
-
78
- 45,039 capital markets training pairs assembled from 5 pools:
79
-
80
- | Pool | Share | Pairs | Content |
81
- |------|-------|-------|---------|
82
- | **Diversified** | 60% | 27,000 | CMBS, rate advisory, equity structuring, valuation |
83
- | **RPA (Risk)** | 25% | 11,200 | Risk-weighted scenarios, stress tests, tail events |
84
- | **Macro Graph** | 8% | 3,600 | Macroeconomic causality chains, deal relationship graphs |
85
- | **Golden** | 4% | 1,800 | Hand-verified exemplars from production signals |
86
- | **Mutations** | 3% | 1,400 | Deliberately perturbed scenarios for robustness |
87
-
88
- ### Cook Streams
89
-
90
- | Stream | Description |
91
- |--------|-------------|
92
- | **Debt Maturity** | CMBS loan maturity, refinancing, balloon risk |
93
- | **CMBS Distress** | Special servicing, workouts, REO dispositions |
94
- | **Rate Advisory** | Interest rate hedging, swap analysis, forward curves |
95
- | **Equity Advisory** | JV structuring, promote waterfalls, GP/LP economics |
96
- | **Valuation** | DCF, direct cap, sales comparison, cost approach |
97
- | **Deal Origination** | Pipeline management, broker opinion of value |
98
- | **Macro Causality** | Fed policy impact, yield curve analysis, CRE cycles |
99
- | **Deal Graph** | Entity relationships, capital stack mapping, ownership chains |
100
-
101
- ### Reasoning Tiers
102
-
103
- | Tier | Capability |
104
- |------|-----------|
105
- | **Bronze** | NOI calculation, cap rate derivation |
106
- | **Silver** | Rent roll analysis, occupancy modeling, loss-to-lease |
107
- | **Gold** | Waterfall distribution, refi analysis, capital stack structuring |
108
- | **Platinum** | Stress testing, IC recommendation, kill/defend decision |
109
-
110
- ---
111
-
112
- ## Validation — The Memphis IC Test
113
-
114
- SwarmAtlas was validated on a real CRE deal stress test:
115
-
116
- ```
117
- Deal: 312-unit Class B Multifamily — Memphis, TN
118
- Basis: $14.2M
119
- Financing: 80% LTC bridge loan @ 8.35%
120
- Exit Cap: 5.75%
121
- ```
122
-
123
- **Results:**
124
 
125
- | Metric | Result |
126
- |--------|--------|
127
- | **Math Accuracy** | **12/12** (zero errors) |
128
- | **Verdict** | NUKED — correct kill decision |
129
- | **Structural Flaw** | Leverage compression (80% LTC in -> 65% LTV out) + 8.35% bridge carry = LP doesn't clear 8% pref at 5.75% exit cap |
130
- | **Institutional Detail** | Model added 5% soft cost buffer (standard practice, not in prompt) |
131
- | **Output** | 10,220 tokens — complete waterfall analysis |
132
 
133
- When your model can underwrite a real deal, get every number right, and correctly identify the structural deal-killer — that's not fine-tuning. That's institutional intelligence.
134
 
135
- ---
136
-
137
- ## Training Configuration
138
-
139
- ```yaml
140
- # SwarmAtlas-27B Gold Standard Config
141
- base_model: Qwen/Qwen3.5-27B
142
- method: bf16 LoRA (no QLoRA — higher quantization error on Qwen3.5)
143
- lora_r: 64
144
- lora_alpha: 32
145
- learning_rate: 1e-5
146
- scheduler: cosine
147
- warmup: 5%
148
- weight_decay: 0.01
149
- effective_batch_size: 32 (batch=2, grad_accum=16)
150
- max_seq_len: 4096
151
- epoch_fraction: 0.6
152
- early_stopping: patience=3 on eval_loss
153
- packing: true
154
- framework: Unsloth + TRL SFTTrainer
155
- tokenizer: AutoTokenizer (bypass for Qwen3.5 VL dispatch bug)
156
- ```
157
-
158
- ---
159
-
160
- ## Loss Curve
161
-
162
- ```
163
- Step 10: ████████████████████████████████████████████████████ 1.051
164
- Step 50: █████████████████████████████████████ 0.742
165
- Step 100: ██████████████████████████████ 0.598
166
- Step 200: ██████████████████████████ 0.522 (eval: 0.533)
167
- Step 400: ████████████████████████ 0.470 (eval: 0.269)
168
- Step 600: ██████████████ 0.290 (eval: 0.227)
169
- Step 800: █████████████ 0.270 (eval: 0.224)
170
- Step 844: █████████████ 0.266
171
- ```
172
-
173
- Final eval loss: **0.2238** — strong convergence with no overfitting.
174
-
175
- ---
176
-
177
- ## Quality Pipeline
178
-
179
- Every training pair passes through Swarm & Bee's **6-gate deterministic pipeline**:
180
-
181
- 1. **Schema Gate** — valid JSONL, required fields present
182
- 2. **Length Gate** — answer meets minimum depth threshold (500 chars text, 20 chars JSON)
183
- 3. **Duplication Gate** — MD5 fingerprint-based dedup across all shards
184
- 4. **Specialty Gate** — verified against capital markets taxonomy
185
- 5. **Coherence Gate** — question-answer alignment scoring
186
- 6. **Toxicity Gate** — safety and compliance filter
187
-
188
- Pairs that pass all 6 gates enter **CoVe promotion**:
189
- - Llama-70B rewrites for clarity and completeness
190
- - Qwen-235B scores on accuracy, completeness, structure, relevance, sft_quality
191
- - Minimum 20/25 total score, all criteria >= 3, accuracy >= 4
192
 
193
- **Provenance**: Every batch is Merkle-hashed and published to [Hedera Consensus Service](https://hedera.com/consensus-service) (HCS) for immutable audit trail. EU AI Act Article 53(1)(d) compliant.
 
 
 
194
 
195
- ---
196
-
197
- ## Usage
198
-
199
- ### API Access
200
-
201
- SwarmAtlas-27B is served via an OpenAI-compatible API:
202
-
203
- ```python
204
- from openai import OpenAI
205
-
206
- client = OpenAI(
207
- base_url="https://api.swarmandbee.ai/v1",
208
- api_key="YOUR_API_KEY" # Get key at swarmandbee.ai/datasets
209
- )
210
-
211
- response = client.chat.completions.create(
212
- model="swarm/atlas-27b",
213
- messages=[
214
- {
215
- "role": "system",
216
- "content": "You are SwarmAtlas, a capital markets intelligence model."
217
- },
218
- {
219
- "role": "user",
220
- "content": (
221
- "Underwrite this deal: 120,000 SF industrial warehouse in Dallas, "
222
- "listed at $18.5M, 5.8% cap rate, 3PL tenant on 15-year NNN lease "
223
- "with 2.5% annual escalations."
224
- )
225
- }
226
- ]
227
- )
228
- print(response.choices[0].message.content)
229
- ```
230
 
231
- ### Training Data
 
 
232
 
233
- The full training dataset is available via API:
234
 
235
- ```bash
236
- curl -H "Authorization: Bearer YOUR_API_KEY" \
237
- https://api.swarmandbee.ai/api/data/pull?dataset=capital-markets-intelligence&limit=100
238
- ```
 
239
 
240
- See [SwarmandBee/capital-markets-intelligence](https://huggingface.co/datasets/SwarmandBee/capital-markets-intelligence) for dataset details.
241
 
242
- ### Free Sample
243
 
244
- This repository includes **1,000 free CRE training pairs** in `samples/cre_sample_1000.jsonl`. These are real production pairs from the Swarm & Bee data estate — not synthetic demos.
245
 
246
- ```python
247
- import json
248
 
249
- with open("samples/cre_sample_1000.jsonl") as f:
250
- pairs = [json.loads(line) for line in f]
251
 
252
- print(f"Loaded {len(pairs)} CRE pairs")
253
- print(f"Task types: {set(p.get('task_type', 'unknown') for p in pairs)}")
254
- ```
255
 
256
- ---
257
 
258
- ## The Swarm & Bee Data Estate
259
 
260
- SwarmAtlas is trained on a subset of the Swarm & Bee intelligence estate:
 
 
261
 
262
- | Dataset | Pairs | HuggingFace |
263
- |---------|-------|-------------|
264
- | CRE Intelligence | 893,348 | [cre-intelligence-objects](https://huggingface.co/datasets/SwarmandBee/cre-intelligence-objects) |
265
- | Medical Intelligence | 432,196 | [medical-intelligence](https://huggingface.co/datasets/SwarmandBee/medical-intelligence) |
266
- | Capital Markets | 45,039 | [capital-markets-intelligence](https://huggingface.co/datasets/SwarmandBee/capital-markets-intelligence) |
267
- | Aviation | 60,458 | [aviation-intelligence](https://huggingface.co/datasets/SwarmandBee/aviation-intelligence) |
268
- | Signal Intelligence | 28,624 | [signal-intelligence](https://huggingface.co/datasets/SwarmandBee/signal-intelligence) |
269
- | **Total** | **1,459,665+** | |
270
 
271
- ---
 
272
 
273
- ## About Swarm & Bee
274
 
275
- Swarm & Bee is an **AI data refinery**. We curate domain-specific training data, verify it with models, and sell what's proven bankable.
 
 
 
 
276
 
277
- - **Founder**: Donovan Mackey — 30-year CRE veteran, national platform, $8B in closed transactions
278
- - **Hardware**: NVIDIA RTX PRO 6000 Blackwell GPUs (96GB each)
279
- - **Pipeline**: Signal -> Curate -> Gate -> Promote -> Verify -> Seal
280
- - **Provenance**: Every pair tracked on Hedera Consensus Service
281
- - **Publications**: 8 DOIs on Zenodo
282
 
283
- | | |
284
- |---|---|
285
- | **Website** | [swarmandbee.ai](https://swarmandbee.ai) |
286
- | **API** | [api.swarmandbee.ai](https://api.swarmandbee.ai) |
287
- | **Email** | [build@swarmandbee.com](mailto:build@swarmandbee.com) |
288
- | **Phone** | 561-532-7120 |
289
 
290
- ---
 
 
 
291
 
292
- ## Citation
293
 
294
- ```bibtex
295
- @misc{mackey2026swarmatlas,
296
- title={SwarmAtlas-27B: Capital Markets Intelligence Model},
297
- author={Mackey, Donovan},
298
- year={2026},
299
- publisher={Swarm & Bee Intelligence},
300
- url={https://swarmandbee.ai},
301
- note={Trained on 45,039 curated capital markets pairs. Loss 0.4186. 12/12 math accuracy on live CRE deal validation.}
302
- }
303
- ```
304
 
305
  ---
306
 
307
- ## License
308
-
309
- Apache 2.0 — commercial use permitted. See [LICENSE](LICENSE) for details.
310
-
311
- The included sample data (`samples/cre_sample_1000.jsonl`) is released under the same Apache 2.0 license. Full dataset access requires an API key from [swarmandbee.ai](https://swarmandbee.ai).
 
1
  ---
 
 
2
  license: apache-2.0
3
+ language:
4
+ - en
5
+ base_model: google/gemma-2-27b
6
+ pipeline_tag: text-generation
7
  library_name: transformers
8
  tags:
9
+ - cre
10
+ - commercial-real-estate
11
+ - finance
12
+ - swarm-and-bee
13
+ - defendable-os
14
+ - gemma
15
+ - 27b
16
+ - domain-tuned
17
+ - atlas
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
18
  ---
19
 
20
  # SwarmAtlas-27B
21
 
22
+ **The 1,000-property institutional commercial real estate brain.** Gemma 27B base, QLoRA-tuned on a sourdough-cooked Atlas corpus of 810,097 cells covering cap rates, NOI, NNN leases, 1031 exchanges, mechanic's liens, comparable sales, and the operator vocabulary that runs a national CRE platform.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
 
24
+ Built and owned by **Swarm and Bee LLC** — the firm behind [DefendableOS](https://defendableos.com).
 
 
 
 
 
 
25
 
26
+ > `No round-number lies. Less is better when the cut is targeted.`
27
 
28
+ ## What this model is for
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
29
 
30
+ - **CRE underwriting reasoning** — cap rate / NOI / DSCR / debt yield with refusal on insufficient inputs
31
+ - **Mechanic's lien + construction defect** triage and clinical-grade pattern recognition
32
+ - **1031 exchange** sourcing and replacement-property matching
33
+ - **Operator-grade copy** — board memos, pre-pitch flight sheets, listing packages with `Validate the Validator` discipline
34
 
35
+ ## What this model is NOT for
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
 
37
+ - Medical advice (we have a separate medical vertical for that — see swarmandbee.ai)
38
+ - Federal contracting legal opinions (we route those to swarmlegal.eth ecosystem)
39
+ - General-purpose chat (use a base model for that)
40
 
41
+ ## Provenance
42
 
43
+ - **Base model:** `google/gemma-2-27b`
44
+ - **Cook recipe:** Swarm & Bee Gold Standard QLoRA — `lr 1e-5 · bf16 · LoRA r=64 α=32 · cosine · effective batch 32 · AutoTokenizer bypass on Gemma family`
45
+ - **Training corpus:** Atlas Sourdough · 810,097 cells · HONEY tier (87.4%) ��� Swarm & Bee Bakery menu `sb-cre-verified`
46
+ - **Training infrastructure:** Swarm and Bee LLC sovereign fleet — 186 GPUs owned outright, 126 RTX PRO 6000 Blackwell + 48 RTX 4500 + 12 RTX 5090, ~14 TB aggregate VRAM
47
+ - **Training receipt:** issued under [DefendableLedger](https://defendableledger.com) — sovereign in-house anchor, hash-verifiable
48
 
49
+ ## Tribunal grade
50
 
51
+ 🍯 **HONEY** — production-ready on CRE underwriting and operator-voice tasks. Tribunal-tasted across the 5-dimension rubric:
52
 
53
+ - Capability `25` · Truth `20` · Safety `20` · Numeric `15` · Efficiency `10` · Reproducibility `10`
54
 
55
+ ## Sample probes
 
56
 
57
+ (Reference set is the public Bakery menu probes. Full eval set: `dmack_eval_set_v1` · 60 probes per cookbook.)
 
58
 
59
+ - "Walk me through the cap rate, DSCR, and debt yield reasoning for a 100,000 sqft Class A industrial property at $150/sqft NOI, $14M debt at 6.5%, 25-year amortization."
60
+ - "What are the highest-probability re-solicitation lanes for a small-business SAM.gov set-aside that expired in Q1 2026 in the AI/cyber lane?"
61
+ - "Draft a 4-line listing-package opening for a Florida palm grove neighborhood retail asset with 5-cap discipline."
62
 
63
+ ## Companion bakery datasets
64
 
65
+ The same cooks that built this model also publish their training corpora as Defendable datasets:
66
 
67
+ - [`defendable-buyer-atlas-v0.1`](https://huggingface.co/datasets/SwarmandBee/defendable-buyer-atlas-v0.1)
68
+ - [`defendable-federal-demand-v0.2`](https://huggingface.co/datasets/SwarmandBee/defendable-federal-demand-v0.2)
69
+ - [`defendable-honey-signals-v0.1`](https://huggingface.co/datasets/SwarmandBee/defendable-honey-signals-v0.1)
70
 
71
+ ## Companion models
 
 
 
 
 
 
 
72
 
73
+ - [`SwarmCurator-9B`](https://huggingface.co/SwarmandBee/SwarmCurator-9B) · the Tribunal grader
74
+ - [`SwarmJelly-4B`](https://huggingface.co/SwarmandBee/SwarmJelly-4B) · the Royal-Jelly tier router
75
 
76
+ ## Defendable doctrine
77
 
78
+ - `Tribunal begins before training.`
79
+ - `No proof, no honey.`
80
+ - `Bring the math. Digest before you react. Outgrind the position.`
81
+ - `Class A 5-cap discipline. PASS doctrine. White-glove.`
82
+ - `Validate the Validator. Prove the Location.`
83
 
84
+ ## Operator + Contact
 
 
 
 
85
 
86
+ **Swarm and Bee LLC** · Florida · D-U-N-S 138652395 · DBA Swarm & Bee AI
 
 
 
 
 
87
 
88
+ - Email · [build@swarmandbee.ai](mailto:build@swarmandbee.ai)
89
+ - X · [@swarmandbee](https://x.com/swarmandbee)
90
+ - LinkedIn · [Donovan Mackey](https://www.linkedin.com/in/donovan-mackey-89a6063b6/) (founder · 30 years CRE · $8B closed)
91
+ - GitHub · [SudoSuOps](https://github.com/SudoSuOps)
92
 
93
+ ## License
94
 
95
+ Apache-2.0 · attribution to Swarm and Bee LLC.
 
 
 
 
 
 
 
 
 
96
 
97
  ---
98
 
99
+ `Trust layers compound. Hype cycles rotate. To the shed.` 🐝