Add model card

README.md

@@ -1,46 +1,52 @@
 ---
 license: apache-2.0
-library_name: transformers
 pipeline_tag: text-generation
 language: en
-base_model: LiquidAI/LFM2.5-1.2B-Thinking
 tags:
+  - lfm2
   - agent
   - fine-tune
-  - reasoning
   - tool-use
-  - lfm2
+  - liquid-ai
+base_model: LiquidAI/LFM2.5-1.2B-Thinking
+datasets:
+  - TeichAI/Ox-Alpha-Pi-Traces
 metrics:
-  - perplexity
+  - loss
 ---
 
 # LiquidAgent-1.2B
 
-A fine-tune of [LiquidAI/LFM2.5-1.2B-Thinking](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Thinking) for agent-style tasks: multi-step reasoning, tool use, and structured output generation.
+A supervised fine-tune of [LiquidAI/LFM2.5-1.2B-Thinking](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Thinking) on agent traces from [TeichAI/Ox-Alpha-Pi-Traces](https://huggingface.co/datasets/TeichAI/Ox-Alpha-Pi-Traces), with tool results properly included in the training data.
+
+## What this is
+
+The base LFM2.5-1.2B-Thinking model is a 1.17B-param hybrid architecture (10 LIV-conv + 6 GQA blocks, 32K context, 65536 vocab) with built-in thinking. This fine-tune adapts it for agentic tool-use: the model learns to emit tool calls, see the tool results, and reason about them in subsequent turns.
+
+**Key fix over prior attempts:** The original parser for the TeichAI data silently dropped all `toolResult` messages (the `type` field is always `"message"` in the raw data, and tool results arrive as `message.role == "toolResult"` - a branch that did not exist). This fine-tune uses a fixed parser that recovers 100% of tool turns (verified: 562/562 on a 50-file sample).
+
+## Training details
+
+- **Base:** LiquidAI/LFM2.5-1.2B-Thinking (bfloat16)
+- **Data:** 379 examples, ~1.34M assistant tokens (from Ox-Alpha-Pi-Traces, 400 session files, 397 usable)
+- **Optimizer:** AdamW 8-bit, lr 2e-6 to 8e-6 (cosine warmup over 47 steps)
+- **Hardware:** NVIDIA RTX 5090 (32 GB), gradient checkpointing, batch size 1
+- **Duration:** 81 seconds (47 steps)
+- **Final loss:** 0.5965 (step 40) - training completed at step 47
 
 ## Architecture
 
 | Parameter | Value |
 |-----------|-------|
-| Base model | LiquidAI/LFM2.5-1.2B-Thinking |
-| Architecture | LFM2 (hybrid conv + full-attention) |
-| Layers | 16 (10 conv, 6 full-attention) |
+| Params | 1,170,340,608 (tied embedding) |
+| Layers | 16 (10 conv + 6 full attention) |
 | Hidden size | 2048 |
-| Attention heads | 32 Q / 8 KV (GQA) |
-| Vocab size | 65,536 |
-| Max position | 128,000 |
-| Tied embeddings | Yes |
-| Total params | 1,170,300,000 (148 tensors) |
+| FF dim | 12288 |
+| Attention heads | 32 Q / 8 KV |
+| Vocab | 65,536 |
+| Context | 32,768 (max_position_embeddings: 128,000) |
 | Dtype | bfloat16 |
 
-## Training
-
-- **Data**: Agent traces from [TeichAI/Ox-Alpha-Pi-Traces](https://huggingface.co/datasets/TeichAI/Ox-Alpha-Pi-Traces) and [TeichAI/DeepSeek-v4-Pro-Agent](https://huggingface.co/datasets/TeichAI/DeepSeek-v4-Pro-Agent)
-- **Method**: Full fine-tune (all parameters), assistant-token masking (loss on assistant turns only)
-- **Steps**: 47 optimizer steps (batch 1, grad accum 16, ~1.5M assistant tokens)
-- **LR**: 2e-5, cosine decay with 5-step warmup
-- **Hardware**: NVIDIA RTX 5090 (32 GB), bfloat16, gradient checkpointing
-
 ## Usage
 
 ```python
@@ -49,26 +55,20 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
 model = AutoModelForCausalLM.from_pretrained(
     "Compactbot/liquidagent-1.2b",
     torch_dtype="bfloat16",
-    device_map="auto"
+    device_map="auto",
 )
 tokenizer = AutoTokenizer.from_pretrained("Compactbot/liquidagent-1.2b")
 
-messages = [{"role": "user", "content": "Write a Python function to parse a JSON config file and validate required fields."}]
+messages = [{"role": "user", "content": "Write a Python function to parse a JSON config file."}]
 text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
 inputs = tokenizer(text, return_tensors="pt").to(model.device)
-output = model.generate(**inputs, max_new_tokens=512, temperature=0.7, top_p=0.9)
-print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
+out = model.generate(**inputs, max_new_tokens=512, temperature=0.7, top_p=0.9)
+print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
 ```
 
-## Quality
-
-Ship gate (2026-09-30): PASS. Generates coherent, on-topic agent-style responses with thinking tokens. No degenerate repetition (1 repeated 5-gram in 301 words across 3 prompts).
-
-**Known limitations**:
-- Only 47 optimizer steps — this is a lightweight SFT, not a full training run
-- Best suited for agent-style tasks (reasoning chains, tool calls, structured output)
-- Not a general-purpose chat model; general knowledge is from the base model
-
-## Notes
+## Honest limitations
 
-This is a demonstration fine-tune showing that agent capabilities can be injected into a 1.2B hybrid-attention model with a small number of optimizer steps. The base model's thinking-token format is preserved.
+- Trained on only 379 examples (~1.3M tokens) - this is a very light adaptation, not a full agent training run.
+- The model retains the base's thinking pattern (emits thinking blocks before responding).
+- No independent benchmark evaluation was run; quality was verified by generation samples (coherent, on-topic, no degenerate repetition).
+- The 32k sequence length requested by the requester was not used in this run (data was packed at shorter lengths to fit the 32GB GPU with 8-bit AdamW).