Duplicate from koyelog/MediMind-411M

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,129 @@
+---
+language:
+- en
+license: mit
+tags:
+- medical
+- llm
+- pytorch
+- text-generation
+- custom-model
+pipeline_tag: text-generation
+library_name: pytorch
+---
+
+# MediMind-411M
+
+MediMind-411M is a custom medical language model trained from scratch for biomedical and clinical text generation.
+
+This model was trained and uploaded by **Koyeliya Ghosh** under the Hugging Face account `koyelog`.
+
+## Overview
+
+MediMind-411M is a 411M-parameter transformer-based language model designed to generate medical-style text.  
+It was trained on a large medical text collection and uses a custom tokenizer.
+
+## Training Summary
+
+- Model name: `MediMind-411M`
+- Parameters: approximately 411.1M
+- Training device: Kaggle GPU T4 x2
+- Total texts loaded: 171,047
+- Training samples tokenized: 50,000
+- Total batches: 12,500
+- Final average loss: 4.9253
+- Total runtime: about 5536.5 seconds (~92 minutes)
+
+## Architecture
+
+This model uses a decoder-only transformer architecture with:
+
+- Embedding dimension: 1024
+- Layers: 24
+- Attention heads: 16
+- KV heads: 8
+- RoPE positional encoding
+- RMSNorm
+- SwiGLU-style feed-forward layers
+
+## Files in this Repository
+
+- `medimind_final.pt` — final trained model weights
+- `checkpoint_latest.pt` — latest training checkpoint
+- `vocab.json` — tokenizer vocabulary
+- `merges.txt` — tokenizer merges
+
+## Testing
+
+The model was tested locally in a Kaggle notebook by:
+
+1. Downloading the model files from this Hugging Face repository
+2. Loading the tokenizer using `vocab.json` and `merges.txt`
+3. Rebuilding the training architecture in PyTorch
+4. Loading `medimind_final.pt`
+5. Generating outputs from medical prompts
+
+### Example test prompts
+
+- `Patient presents with fever and cough. Diagnosis:`
+- `Symptoms of diabetes include`
+- `Treatment for hypertension includes`
+
+### Observed behavior
+
+The model successfully generates medical-style text and terminology.  
+Outputs show that the model has learned domain vocabulary and sentence patterns, but generations can still be noisy, mixed-topic, or clinically unreliable.
+
+## Limitations
+
+- This is an early-stage base language model, not an instruction-tuned chatbot.
+- It may produce incorrect, incomplete, or hallucinated medical statements.
+- It should **not** be used for real medical diagnosis, treatment, or decision-making.
+- Output quality can vary depending on prompt style and decoding settings.
+
+## Intended Use
+
+This model is intended for:
+
+- learning and experimentation
+- research practice
+- testing custom LLM training pipelines
+- educational exploration of medical text generation
+
+This model is **not intended** for direct clinical deployment or patient-facing use.
+
+## Example Usage
+
+```python
+from huggingface_hub import hf_hub_download
+from tokenizers import ByteLevelBPETokenizer
+import torch
+
+model_path = hf_hub_download(repo_id="koyelog/MediMind-411M", filename="medimind_final.pt")
+vocab_path = hf_hub_download(repo_id="koyelog/MediMind-411M", filename="vocab.json")
+merges_path = hf_hub_download(repo_id="koyelog/MediMind-411M", filename="merges.txt")
+
+tokenizer = ByteLevelBPETokenizer(vocab_path, merges_path)
+
+print("Load tokenizer and model architecture, then run generation.")
+```
+
+## Future Work
+
+Planned next improvements:
+
+- cleaner inference pipeline
+- better decoding and stopping rules
+- further training epochs
+- instruction tuning on medical QA data
+- model card improvements and benchmark evaluation
+
+## Author
+
+Created by **Koyeliya Ghosh**  
+Hugging Face: [koyelog](https://huggingface.co/koyelog)
+
+## Disclaimer
+
+This model is for research and educational purposes only.  
+It must not be used as a substitute for licensed medical advice or professional healthcare judgment.

checkpoint_latest.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c657060025a16ecedb618dd31423ae182725e6cd0b1001212771d10071bad17
+size 4958564991

medimind_final.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc9466f36b1bfc1a26e9b2a475a30ab3ef34f870397d5cebe6b50842dcf1d92b
+size 1669635739