README.md

---
title: ChatCraft
emoji: 🎮
colorFrom: red
colorTo: yellow
sdk: docker
app_port: 7860
pinned: false
license: mit
storage: small
short_description: 1v1 RTS game controlled by voice via Mistral
tags:
  - game
  - voice
  - mistral
  - rts
  - hackathon
  - svelte
  - fastapi
---

ChatCraft is a 1v1 real-time strategy game (inspired by StarCraft's Terran race) entirely controlled by voice. Speak your commands — build barracks, train marines, attack the enemy — and Mistral AI interprets them into live game actions. No mouse clicks to play, just your microphone.

<div align="center">

# 🎮 ChatCraft
### Voice-Controlled Real-Time Strategy Game

[![Mistral](https://img.shields.io/badge/Mistral-AI-FF6B35?style=for-the-badge&logo=data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMjQiIGhlaWdodD0iMjQiIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0id2hpdGUiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PHJlY3Qgd2lkdGg9IjI0IiBoZWlnaHQ9IjI0IiByeD0iNCIgZmlsbD0iI0ZGNkIzNSIvPjwvc3ZnPg==)](https://mistral.ai)
[![ElevenLabs](https://img.shields.io/badge/ElevenLabs-Voice-black?style=for-the-badge&logo=data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMjQiIGhlaWdodD0iMjQiIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0id2hpdGUiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PHJlY3Qgd2lkdGg9IjI0IiBoZWlnaHQ9IjI0IiByeD0iNCIgZmlsbD0iIzIyMjIyMiIvPjwvc3ZnPg==)](https://elevenlabs.io)
[![HuggingFace](https://img.shields.io/badge/Hugging%20Face-Spaces-FFD21E?style=for-the-badge&logo=huggingface&logoColor=black)](https://huggingface.co/spaces)
[![SvelteKit](https://img.shields.io/badge/SvelteKit-Frontend-FF3E00?style=for-the-badge&logo=svelte&logoColor=white)](https://kit.svelte.dev)
[![FastAPI](https://img.shields.io/badge/FastAPI-Backend-009688?style=for-the-badge&logo=fastapi&logoColor=white)](https://fastapi.tiangolo.com)

> **Mistral × ElevenLabs Hackathon** — Command your army. With your voice.

*A 1v1 real-time strategy game entirely controlled by voice. Inspired by StarCraft's Terran race — playable on mobile with just a microphone.*

</div>

---

## 🎯 Concept

> *"Build barracks, train 4 marines, attack the enemy base"*

Speak. Command. Conquer.

The AI (**Mistral**) interprets your natural language orders and translates them into real-time game actions. Voice feedback from the game (*"Construction complete", "Under attack!"*) is delivered instantly via the **Web Speech API**.

Touch / mouse is only used to navigate the map, inspect units or techs — **never to play**.

---

## ⚡ Tech Stack

| Layer | Technology |
|:------|:-----------|
| 🖥️ Frontend | SvelteKit + socket.io-client |
| ⚙️ Backend | FastAPI + python-socketio (asyncio) |
| 🧠 Command AI | Mistral API (`mistral-large-latest`) |
| 🎙️ Voice → Text | Voxtral (`voxtral-mini-latest`) |
| 🔊 Text → Voice | Web Speech API (browser, instant, free) |
| ☁️ Hosting | Hugging Face Spaces (Docker) |

---

## 🗣️ Voice Command Flow

```
🎤 [Player holds PTT]
        │
        ▼
  MediaRecorder (browser)
        │  audio blob
        ▼
  Socket.IO  ──voice_input──►  backend
                                  │
                                  ▼
                          🤖 Voxtral STT
                          "train 3 marines"
                                  │
                                  ▼
                      ✨ Mistral API  (system prompt = game state)
                      → { action: "train", unit: "marine", count: 3 }
                                  │
                                  ▼
                          engine.apply_command()
                                  │
                          ┌───────┴───────┐
                          ▼               ▼
                    game_state       feedback_text
                    broadcast        "Training 3 marines started"
                          │               │
                          ▼               ▼
                  ◄──game_update──  🔊 Web Speech API (browser)
```

---

## 🗺️ Game — Simplified Terran Race

### 💎 Resources
| Resource | Source | Usage |
|:---------|:-------|:------|
| 🔵 Minerals | Patches collected by SCV | Everything |
| 🟢 Vespene Gas | Geysers collected by SCV | Advanced units, upgrades |
| 👥 Supply | Supply Depots + Command Center | Population cap |

### 🏗️ Buildings
| Building | Prerequisite | Role |
|:---------|:-------------|:-----|
| 🏛️ Command Center | — | Starting building, produces SCV |
| 📦 Supply Depot | — | +8 supply |
| 🪖 Barracks | — | Marine, Medic |
| 🔧 Engineering Bay | Barracks | Infantry upgrades |
| 🏭 Factory | Barracks | Goliath, Tank |
| ⚔️ Armory | Factory | Vehicle upgrades |
| 🚀 Starport | Factory | Wraith |

### ⚔️ Units
| Unit | Cost | Prerequisite | Role |
|:-----|:-----|:-------------|:-----|
| 🔨 SCV | 50m | CC | Gathering, construction |
| 🪖 Marine | 50m | Barracks | Basic infantry, anti-air |
| 💊 Medic | 50m / 25g | Barracks + Academy* | Heals infantry |
| 🤖 Goliath | 100m / 50g | Factory | Anti-air + ground |
| 💣 Siege Tank | 150m / 100g | Factory + Machine Shop* | Ground DPS, siege mode |
| ✈️ Wraith | 150m / 100g | Starport | Air, cloaking |

*\*simplified: add-ons replaced by simple building prerequisites*

### 🎤 Voice Command Examples

```
"Build a supply depot near the command center"
"Train four marines"
"Send all marines to attack top-left"
"Put the tank in siege mode"
"Patrol the wraiths around my base"
"How much gas do I have?"
"Scout with an SCV toward the center"
```

---

## 🎮 Core Game Rules

- **🖥️ HUD layout**: resources top-left, production top-right, timer center; minimap bottom-left; control groups bottom-center and right; voice/text commands on the right.
- **🌫️ Fog of war**: visibility based on the player's units and buildings (vision radii per type); explored but not visible areas semi-transparent, never-seen areas in black.
- **🗺️ Minimap**: 40×40 map representation with fog, main map viewport, click to recenter.
- **🎯 Control groups**: 3 groups; add/remove units or buildings, or create empty; unified representation by icons (buildings, technologies, units).

---

## 🚀 Local Setup

```bash
# 1. Environment variables
cp .env.example .env
# Fill in MISTRAL_API_KEY

# 2. Backend
cd backend
pip install -r requirements.txt
uvicorn main:app --reload --port 8000

# 3. Frontend
cd frontend
npm install
npm run dev
```

### 🔑 Environment Variables

| Variable | Description |
|:---------|:------------|
| `MISTRAL_API_KEY` | Mistral API key (Voxtral STT + mistral-large chat) |
| `SECRET_KEY` | Key to sign lobby sessions |

---

## 🏗️ Project Architecture

```
hackaton/
├── Dockerfile                  # HF Spaces — exposes port 7860
├── docker-compose.yml          # Local dev
│
├── backend/
│   ├── main.py                 # FastAPI + Socket.IO entry point
│   ├── lobby/manager.py        # Room creation / joining, matchmaking
│   ├── game/
│   │   ├── engine.py           # Game loop (tick ~250ms)
│   │   ├── state.py            # GameState: resources, units, buildings
│   │   ├── units.py            # Marine / Medic / Goliath / Tank / Wraith / SCV
│   │   ├── buildings.py        # Command Center / Barracks / Factory…
│   │   ├── tech_tree.py        # Build prerequisites
│   │   └── map.py              # Grid + A* pathfinding
│   └── voice/
│       ├── stt.py              # Voxtral STT: audio → text
│       └── command_parser.py   # Mistral: text + game state → actions
│
└── frontend/
    └── src/
        ├── routes/
        │   ├── +page.svelte        # Lobby
        │   └── game/+page.svelte   # Main game view
        └── lib/
            ├── socket.ts           # socket.io-client singleton
            ├── voice.ts            # PTT mic capture → backend
            └── components/
                ├── Map.svelte          # Scrollable/zoomable SVG map
                ├── ResourceBar.svelte  # Minerals / Gas / Supply
                └── VoiceButton.svelte  # Push-to-talk button
```

---

## 🧠 Design Notes

- **python-socketio** with asyncio is chosen for room management — it natively handles rooms, namespaces and reconnection out of the box.
- The game loop runs **server-side** (tick every 250ms). Clients receive a full state snapshot each tick — no client-side reconciliation.
- Mistral receives a **summary of the player's current state** (resources, active buildings, supply) as system context for consistent command interpretation.
- **PTT (push-to-talk)** avoids false positives on mobile and reduces latency by limiting audio chunk size.