README.md

metadata

license: other
library_name: transformers
pipeline_tag: text-generation
language:
  - en
base_model: LiquidAI/LFM2.5-1.2B-Thinking
tags:
  - lfm2
  - agent
  - fine-tune
  - teichai
  - thinking
metrics:
  - perplexity

LiquidAgent-1.2B

A supervised fine-tune of LiquidAI/LFM2.5-1.2B-Thinking on OpenAI-style agent traces, intended to pick up the agent-trace format and style (thinking + text + tool calls). Built for the model request in Compactbot/model-requests#15 (requested by @DedeProGames).

License note: This is a derivative of LiquidAI/LFM2.5-1.2B-Thinking, which is distributed under the LFM Open License v1.0 (a custom license; license: other on the Hub). That license carries a commercial-use threshold ($10M annual revenue). Read the base model's LICENSE before commercial use.

What it is

• Base: LiquidAI/LFM2.5-1.2B-Thinking (LFM2 / Lfm2ForCausalLM), bf16.
• Params: 1,170,340,608 (verified by loading the released weights).
• Architecture: 16 blocks — 10 LIV convolutional + 6 full-attention (GQA, 32 query / 8 KV heads), block_dim/hidden_size 2048, intermediate_size 12288, SwiGLU, RoPE (θ = 1e6), vocab 65536, max position 128000, tied embeddings.
• Training: full fine-tune (not LoRA), bf16, 8-bit AdamW, gradient checkpointing, batch 1, sequence length 32k, cosine LR, 1300 steps.
• Data: 379 agent-trace examples, ~1.34M assistant tokens, packed from TeichAI/Ox-Alpha-Pi-Traces + TeichAI/DeepSeek-v4-Pro-Agent. Training is masked so the model learns assistant turns only (thinking + text + tool calls); user / system / tool-result tokens are masked out.

Honest quality picture

This is a light fine-tune: ~1.34M assistant tokens on top of a 1.2B base. It reliably adopts the agent-trace format and thinking style of the source traces, but it is not a strong agent and should not be expected to solve multi-step agentic tasks reliably.

Verified by loading the released weights standalone and generating:

• Coherent, grammatical responses with well-formed [[ctrl:think]] blocks.
• Correct simple arithmetic (e.g. 27 × 43 = 1161; 12 + 7 = 19).
• No degenerate token loops in the sampled prompts.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Compactbot/liquidagent-1.2b"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="cuda")

prompt = "User: What is 27 * 43? Answer briefly."
ids = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=256, do_sample=False,
                     pad_token_id=tok.eos_token_id)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

The chat template is the base model's (with thinking preservation), so apply_chat_template works as-is.

Reproducibility

• Base: LiquidAI/LFM2.5-1.2B-Thinking
• Datasets: TeichAI/Ox-Alpha-Pi-Traces, TeichAI/DeepSeek-v4-Pro-Agent
• Training: full SFT, bf16, 8-bit AdamW, grad checkpointing, batch 1, seq 32k, cosine LR, 1300 steps, masked to assistant turns.