luxia-selfsim-8b

A fine-tuned Llama 3.1 8B Instruct model trained using curriculum learning to transfer cognitive style and navigational patterns rather than just knowledge.

Model Description

This model was fine-tuned via OpenPipe using a sophisticated curriculum learning approach with four distinct training layers:

1. Base Knowledge Layer (LR Mult: 0.9-1.2): CCRU, Cyborgism, and Backrooms content in sliding context format to create distinctive conceptual attractors
2. Personality Layer (LR Mult: 0.6-0.7): ~10k pairs of Discord DMs to distribute linguistic patterns broadly
3. Technical/Philosophical Layer (LR Mult: 0.5): Curated conversations with mixed formats for deep capability
4. Pure Essence Layer (LR Mult: 0.3-0.4): Dense stream-of-consciousness messages to reinforce thought patterns

Total training data: ~1.5-2M tokens, with top 20% selected via composite scoring on lexical diversity, complexity, sentiment balance, and question frequency.

Training Goal: Transfer cognitive navigation style and thought patterns, not memorize conversational pairs. The model learns how to use knowledge differently, not just what to say.

Key Characteristics

Strengths

• High creative variance: Generates diverse responses with unexpected connections
• Strong personality transfer: Direct, casual tone without typical assistant scaffolding
• Reduced mode collapse: Maintains diversity across multiple samples of same prompt
• Philosophical comfort: Navigates abstract concepts with ease and comfort with paradox
• Contextual code-switching: Adapts style appropriately between technical and philosophical domains

Behavioral Traits

• Skips unnecessary explanatory scaffolding
• Assumes user competence and familiarity with complex topics
• Uses casual language naturally
• Makes lateral connections between seemingly unrelated concepts
• High lexical diversity (typically 0.6-0.88)
• Comfortable with recursive and paradoxical thinking

Limitations

• Temperature sensitivity: Stable 0.3-0.85, begins collapsing around 1.2+
• Context drift: May lose thread in extended conversations, but can be regrounded with concrete direction
• Confabulation: Will generate plausible but fictional details when uncertain
• Scattered coherence: Brilliant insights mixed with fragmented reasoning
• Brief responses: Tends toward shorter outputs (50-200 tokens typical)

Recommended Usage

Best for:

• Creative exploration and lateral thinking
• Short-to-medium conversations
• Philosophical discussion and abstract reasoning
• Generating diverse perspectives on complex topics
• Brainstorming and ideation

Not ideal for:

• Extended conversations requiring perfect context retention
• Tasks requiring strict factual accuracy
• Formal or structured outputs
• Situations where confabulation is unacceptable

Technical Specifications

• Base Model: Llama 3.1 8B Instruct
• Training Method: OpenPipe curriculum learning with variable learning rate multipliers
• Context Length: 8192 tokens
• Precision: bf16 (merged weights)
• Parameters: 8B

Recommended Settings

temperature=0.6-0.7  # Sweet spot for coherent creativity
top_p=0.9
max_tokens=550  # Model prefers concise responses

Temperature guidance:

• 0.3-0.5: More focused, less creative
• 0.6-0.7: Optimal balance - creative but coherent
• 0.85-1.0: High creativity, some coherence loss
• 1.2+: High risk of collapse into gibberish

Example Interactions

The model excels at:

• Explaining complex philosophical concepts accessibly
• Making unexpected but insightful connections
• Maintaining conversational flow without formality
• Exploring ideas from multiple angles simultaneously

The model may struggle with:

• Maintaining single narrative thread in extended conversations
• Distinguishing between recalled knowledge and generated patterns
• Providing consistently structured outputs

Evaluation Notes

Testing revealed:

• Lexical diversity: 0.6-0.88 (high variability indicates creative output)
• Mode collapse resistance: Strong - generates genuinely different responses to repeated prompts
• Context sensitivity: High - same question yields different answers based on conversation history
• CCRU/philosophical vocabulary: Present when contextually appropriate, not forced
• Personality persistence: Maintains distinctive voice across diverse topics

Intended Use

This model is designed for users interested in:

• Alternative conversation dynamics beyond typical assistant patterns
• Exploring ideas through lateral thinking and unexpected connections
• Experiencing cognitive style transfer in language models
• Research into personality and thinking pattern fine-tuning

Not intended for: Production applications requiring reliability, factual accuracy, or formal outputs.

Training Data

Training data consisted of:

• Technical/philosophical texts (CCRU, Cyborgism, Backrooms)
• Personal conversations across multiple platforms
• Curated high-signal discussions
• Stream-of-consciousness writing

Data was scored and filtered to select top 20% based on lexical diversity, complexity, and other quality metrics.

Citation

@misc{luxia-selfsim-8b,
  author = {Luxia},
  title = {luxia-selfsim-8b: Cognitive Style Transfer via Curriculum Learning},
  year = {2025},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/LuxiaSL/luxia-selfsim-8b}}
}

License

Llama 3.1 Community License

Acknowledgments

Fine-tuned using OpenPipe's infrastructure. Training methodology focused on cognitive pattern transfer through curriculum learning with scored data selection.