---
language:
- en
license: apache-2.0
library_name: transformers
tags:
- cybersecurity
- security
- infosec
- threat-modeling
- incident-response
- llm
- mistral
- dolphin
base_model: dphn/Dolphin3.0-R1-Mistral-24B
pipeline_tag: text-generation
model-index:
- name: nova-24b-cybersec
  results: []
---

# nova:24b - Cybersecurity Domain LLM

24B parameter LLM fine-tuned on 40,000+ cybersecurity examples. Built on Dolphin3.0-R1-Mistral-24B.

## Training Data (40,075 examples)

| Dataset | Examples | Source |
|---------|----------|--------|
| SecurityGPT | 16,000 | 407 security PDFs |
| PKI Context | 18,997 | Security Q&A pairs |
| Energy Sector Threats | 3,386 | ICS/SCADA scenarios |
| ISO 27001 Controls | 1,116 | 93 Annex A controls |
| ISO 27005 Threats | 576 | 48 threat categories |

## Domains Covered

- Threat modeling & risk assessment
- Incident response
- Cryptography
- Vulnerability management
- Compliance (ISO 27001/27005)
- Adversarial ML
- Secure coding
- ICS/SCADA security

## Usage

### Transformers

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("pki/nova-24b-cybersec")
tokenizer = AutoTokenizer.from_pretrained("pki/nova-24b-cybersec")

messages = [
    {"role": "system", "content": "You are a cybersecurity expert."},
    {"role": "user", "content": "Explain MITRE ATT&CK framework"}
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.1)
print(tokenizer.decode(outputs[0]))
```

### Ollama (GGUF)

```bash
# Download Q8 GGUF and create Modelfile
ollama create nova:24b -f Modelfile
ollama run nova:24b
```

## Model Details

| Aspect | Detail |
|--------|--------|
| Base Model | Dolphin3.0-R1-Mistral-24B |
| Parameters | 24 billion |
| Context Window | 32,768 tokens |
| Training Examples | 40,075 |
| Training Method | LoRA (r=32, alpha=64) |
| Hardware | RTX 4090, ~26 hours |

## Important: Temperature Setting

**Critical:** Use temperature 0.05-0.1. Mistral-24B requires very low temperature for coherent output.

```python
outputs = model.generate(inputs, temperature=0.08, top_p=0.9)
```

## Files

- `model-*.safetensors` - Model weights (transformers format)
- `nova-24b-q8.gguf` - Quantized GGUF for Ollama/llama.cpp

## Training Configuration

- LoRA rank: 32
- LoRA alpha: 64
- Learning rate: 5e-5
- Epochs: 5
- Batch size: 40 (effective)
- Optimizer: AdamW 8-bit

## Limitations

- Trained primarily on English text
- Best for security-focused tasks
- Requires low temperature (0.05-0.1)
- Large model - needs significant VRAM

## License

Apache 2.0

## Citation

```bibtex
@misc{nova24b-cybersec-2024,
  author = {PKI},
  title = {nova:24b - Cybersecurity Domain LLM},
  year = {2024},
  publisher = {HuggingFace},
}
```