---
license: other
library_name: transformers
pipeline_tag: text-generation
language:
- en
base_model: LiquidAI/LFM2.5-1.2B-Thinking
tags:
- lfm2
- agent
- fine-tune
- teichai
- thinking
metrics:
- perplexity
---

# LiquidAgent-1.2B

A supervised fine-tune of **LiquidAI/LFM2.5-1.2B-Thinking** on OpenAI-style agent
traces, intended to pick up the agent-trace format and style (thinking + text +
tool calls). Built for the model request in
[Compactbot/model-requests#15](https://huggingface.co/spaces/Compactbot/model-requests)
(requested by @DedeProGames).

> **License note:** This is a derivative of `LiquidAI/LFM2.5-1.2B-Thinking`,
> which is distributed under the **LFM Open License v1.0** (a custom license;
> `license: other` on the Hub). That license carries a commercial-use threshold
> ($10M annual revenue). Read the base model's LICENSE before commercial use.

## What it is

- **Base:** `LiquidAI/LFM2.5-1.2B-Thinking` (LFM2 / `Lfm2ForCausalLM`), bf16.
- **Params:** 1,170,340,608 (verified by loading the released weights).
- **Architecture:** 16 blocks — 10 LIV convolutional + 6 full-attention (GQA,
  32 query / 8 KV heads), `block_dim`/`hidden_size` 2048, `intermediate_size`
  12288, SwiGLU, RoPE (θ = 1e6), vocab 65536, max position 128000, tied
  embeddings.
- **Training:** full fine-tune (not LoRA), bf16, 8-bit AdamW, gradient
  checkpointing, batch 1, sequence length 32k, cosine LR, 1300 steps.
- **Data:** 379 agent-trace examples, ~1.34M assistant tokens, packed from
  `TeichAI/Ox-Alpha-Pi-Traces` + `TeichAI/DeepSeek-v4-Pro-Agent`. Training is
  masked so the model learns **assistant turns only** (thinking + text + tool
  calls); user / system / tool-result tokens are masked out.

## Honest quality picture

This is a **light fine-tune**: ~1.34M assistant tokens on top of a 1.2B base.
It reliably adopts the agent-trace format and thinking style of the source
traces, but it is not a strong agent and should not be expected to solve
multi-step agentic tasks reliably.

Verified by loading the released weights standalone and generating:
- Coherent, grammatical responses with well-formed `[[ctrl:think]]` blocks.
- Correct simple arithmetic (e.g. 27 × 43 = 1161; 12 + 7 = 19).
- No degenerate token loops in the sampled prompts.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Compactbot/liquidagent-1.2b"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="cuda")

prompt = "User: What is 27 * 43? Answer briefly."
ids = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=256, do_sample=False,
                     pad_token_id=tok.eos_token_id)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
```

The chat template is the base model's (with thinking preservation), so
`apply_chat_template` works as-is.

## Reproducibility

- Base: `LiquidAI/LFM2.5-1.2B-Thinking`
- Datasets: `TeichAI/Ox-Alpha-Pi-Traces`, `TeichAI/DeepSeek-v4-Pro-Agent`
- Training: full SFT, bf16, 8-bit AdamW, grad checkpointing, batch 1, seq 32k,
  cosine LR, 1300 steps, masked to assistant turns.