README.md

metadata

title: Pip - Emotional AI Companion
emoji: 🫧
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 6.0.1
app_file: app.py
pinned: true
license: mit
short_description: A blob friend who transforms your feelings into visual art
tags:
  - mcp-in-action-track-creative
  - mcp-in-action-track-consumer
  - agents
  - mcp

🎥 Demo Video: https://youtu.be/bWDj4gyngNI

📢 Social Post: https://x.com/07amit10/status/1995270517251801162

👥 Team: @Itsjustamit

🫧 Pip - Your Emotional AI Companion

Pip is a cute blob companion that understands your emotions and responds with conversation, context-specific imagery, and soothing voice.

Not a generic assistant - Pip is an emotional friend who knows when to reflect, celebrate, or gently intervene.

---

✨ What Makes Pip Special

Emotional Intelligence

Pip doesn't just respond - it understands. Using Claude's nuanced emotional analysis, Pip detects:

• Multiple co-existing emotions
• Emotional intensity
• Underlying needs (validation, comfort, celebration)
• When gentle intervention might help

Context-Specific Imagery

Every image Pip creates is unique to your conversation. Not generic stock photos - visual art that captures YOUR emotional moment:

• Mood Alchemist: Transform emotions into magical artifacts
• Day's Artist: Turn your day into impressionistic art
• Dream Weaver: Visualize thoughts in surreal imagery
• Night Companion: Calming visuals for 3am moments

Multi-Service Architecture

Pip uses multiple AI services intelligently:

Service	Role
Anthropic Claude	Deep emotional analysis, intervention logic
SambaNova	Fast acknowledgments, prompt enhancement
OpenAI	Image generation, speech-to-text (Whisper)
Google Gemini	Image generation (load balanced)
Flux/SDXL	Artistic image generation (via Modal/HuggingFace)
ElevenLabs	Expressive voice with emotional tone matching

Low-Latency Design

Pip is designed for responsiveness:

• Quick acknowledgment (< 500ms)
• Progressive state changes while processing
• Parallel task execution
• Streaming responses

---

🎮 How to Use

Chat Interface

1. Type how you're feeling or what's on your mind
2. Watch Pip's expression change as it processes
3. Receive a thoughtful response + custom image
4. Optionally enable voice to hear Pip speak

Voice Input

1. Click the microphone button
2. Speak your thoughts
3. Pip transcribes and responds with voice

Modes

• Auto: Pip decides the best visualization style
• Alchemist: Emotions become magical artifacts
• Artist: Your day becomes a painting
• Dream: Thoughts become surreal visions
• Night: Calming imagery for late hours

---

🤖 MCP Integration

Pip is available as an MCP (Model Context Protocol) server. Connect your AI agent!

For SSE-compatible clients (Cursor, Windsurf, Cline):

{
  "mcpServers": {
    "Pip": {
      "url": "https://YOUR-SPACE.hf.space/gradio_api/mcp/"
    }
  }
}

For stdio clients (Claude Desktop):

{
  "mcpServers": {
    "Pip": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://YOUR-SPACE.hf.space/gradio_api/mcp/sse",
        "--transport",
        "sse-only"
      ]
    }
  }
}

Available MCP Tools

• chat_with_pip(message, session_id) - Talk to Pip
• generate_mood_artifact(emotion, context) - Create emotional art
• get_pip_gallery(session_id) - View conversation history
• set_pip_mode(mode, session_id) - Change interaction mode

---

🧠 The Architecture

User Input
    ↓
┌─────────────────────────────────────┐
│  SambaNova: Quick Acknowledgment    │ ← Immediate response
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│  Claude: Emotion Analysis           │ ← Deep understanding
│  - Primary emotions                 │
│  - Intensity (1-10)                 │
│  - Intervention needed?             │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│  Claude: Action Decision            │ ← What should Pip do?
│  - reflect / celebrate / comfort    │
│  - calm / energize / intervene      │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│  SambaNova: Prompt Enhancement      │ ← Create vivid image prompt
│  (Context-specific, never generic)  │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│  Image Generation (Load Balanced)   │
│  ┌────────┐ ┌────────┐ ┌────────┐  │
│  │ OpenAI │ │ Gemini │ │  Flux  │  │
│  └────────┘ └────────┘ └────────┘  │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│  Claude/SambaNova: Response         │ ← Streaming text
│  (Load balanced for efficiency)     │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│  ElevenLabs: Voice (Optional)       │ ← Emotional tone matching
└─────────────────────────────────────┘

---

🎨 Pip's Expressions

Pip has 10 distinct emotional states with unique animations:

• Neutral (gentle wobble)
• Happy (bouncing)
• Sad (drooping)
• Thinking (looking up, swaying)
• Concerned (worried eyebrows, shaking)
• Excited (energetic bouncing with sparkles)
• Sleepy (half-closed eyes, breathing)
• Listening (wide eyes, pulsing)
• Attentive (leaning forward)
• Speaking (animated mouth)

---

💡 Key Features

Intervention Without Preaching

When Pip detects concerning emotional signals, it doesn't lecture. Instead:

• Brief acknowledgment
• Gentle redirect to curiosity/wonder
• Show something beautiful or intriguing
• Invite engagement, not advice

Not Generic

Every image prompt is crafted from YOUR specific words and context. Pip extracts:

• Specific details you mentioned
• Emotional undertones
• Time/context clues
• Your unique situation

---

🛠️ Tech Stack

• Frontend: Gradio
• Character: SVG + CSS animations
• LLMs: Anthropic Claude, SambaNova (Llama)
• Images: OpenAI DALL-E 3, Google Imagen, Flux
• Voice: ElevenLabs (Flash v2.5 for speed, v3 for expression)
• STT: OpenAI Whisper
• Compute: Modal (for Flux/SDXL)
• Hosting: HuggingFace Spaces

---

🔧 Environment Variables

ANTHROPIC_API_KEY=your_key
SAMBANOVA_API_KEY=your_key
OPENAI_API_KEY=your_key
GOOGLE_API_KEY=your_key
ELEVENLABS_API_KEY=your_key
HF_TOKEN=your_token (optional, for HuggingFace models)

---

📝 License

MIT License - Feel free to use, modify, and share!

---

Built with 💙 for MCP's 1st Birthday Hackathon 2025

Pip uses: Anthropic ($25K), OpenAI ($25), HuggingFace ($25), SambaNova ($25), ElevenLabs ($44), Modal ($250), Blaxel ($250)