Add PathoGen model: attention weights and configs

.gitattributes

@@ -0,0 +1,2 @@
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,129 @@
+---
+license: mit
+library_name: diffusers
+tags:
+  - diffusion
+  - inpainting
+  - histopathology
+  - medical-imaging
+  - pathology
+  - pytorch
+pipeline_tag: image-to-image
+---
+
+# PathoGen - Histopathology Image Inpainting
+
+PathoGen is a diffusion-based model for histopathology image inpainting. It enables realistic tissue pattern generation for filling masked regions in pathology whole slide images (WSI).
+
+## Model Description
+
+- **Model Type:** Diffusion model with custom attention processors
+- **Task:** Image inpainting for histopathology images
+- **Architecture:** UNet2DConditionModel with custom SkipAttnProcessor
+- **Input Size:** 512x512 pixels
+- **Framework:** PyTorch, Diffusers, PyTorch Lightning
+
+## Usage
+
+### Installation
+
+```bash
+git clone https://github.com/mkoohim/PathoGen.git
+cd PathoGen
+pip install -r requirements.txt
+```
+
+### Download Weights
+
+Download the attention weights and place them in your checkpoint directory:
+
+```python
+from huggingface_hub import hf_hub_download
+
+# Download attention weights
+hf_hub_download(
+    repo_id="mkoohim/PathoGen",
+    filename="attention.pt",
+    local_dir="./checkpoints"
+)
+```
+
+### Inference
+
+```python
+from src.models.pathogen import PathoGenModel
+from omegaconf import OmegaConf
+from PIL import Image
+
+# Load configuration
+config = OmegaConf.load("configs/config.yaml")
+
+# Initialize model
+model = PathoGenModel(config)
+model.load_attention_weights("./checkpoints/attention.pt")
+model.eval()
+
+# Load images
+image = Image.open("your_wsi_crop.jpg")
+mask = Image.open("your_mask.jpg")
+condition = Image.open("your_source_image.jpg")
+
+# Run inference
+result = model(image, mask, condition)
+```
+
+### Training
+
+```bash
+python train.py
+```
+
+See the [GitHub repository](https://github.com/mkoohim/PathoGen) for full training instructions.
+
+## Model Files
+
+| File | Description | Size |
+|------|-------------|------|
+| `attention.pt` | Trained attention module weights | ~190MB |
+
+## Training Details
+
+- **Base Model:** Stable Diffusion Inpainting UNet
+- **Training Data:** Histopathology whole slide image crops
+- **Optimizer:** AdamW
+- **Learning Rate:** 1e-5
+- **Precision:** Mixed precision (FP16)
+
+## Intended Use
+
+This model is designed for:
+- Histopathology image inpainting and augmentation
+- Research in computational pathology
+- Data augmentation for pathology AI training
+
+## Limitations
+
+- Optimized for 512x512 input images
+- Best results on H&E stained tissue images
+- Requires GPU for reasonable inference speed
+
+## Citation
+
+```bibtex
+@misc{pathogen2024,
+  title={PathoGen: Histopathology Image Inpainting with Diffusion Models},
+  author={mkoohim},
+  year={2024},
+  url={https://huggingface.co/mkoohim/PathoGen}
+}
+```
+
+## License
+
+This model is released under the MIT License.
+
+## Links
+
+- **GitHub:** [https://github.com/mkoohim/PathoGen](https://github.com/mkoohim/PathoGen)
+- **Hugging Face:** [https://huggingface.co/mkoohim/PathoGen](https://huggingface.co/mkoohim/PathoGen)
+

attention.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b722cdedabdfae97012dfb713f91d740b7945e6ccaae61745cfdf4ec5853fb0e
+size 198342750

config.json

@@ -0,0 +1,36 @@
+{
+    "_class_name": "UNet2DConditionModel",
+    "_diffusers_version": "0.6.0.dev0",
+    "act_fn": "silu",
+    "attention_head_dim": 8,
+    "block_out_channels": [
+        320,
+        640,
+        1280,
+        1280
+    ],
+    "center_input_sample": false,
+    "cross_attention_dim": 768,
+    "down_block_types": [
+        "CrossAttnDownBlock2D",
+        "CrossAttnDownBlock2D",
+        "CrossAttnDownBlock2D",
+        "DownBlock2D"
+    ],
+    "downsample_padding": 1,
+    "flip_sin_to_cos": true,
+    "freq_shift": 0,
+    "in_channels": 9,
+    "layers_per_block": 2,
+    "mid_block_scale_factor": 1,
+    "norm_eps": 1e-05,
+    "norm_num_groups": 32,
+    "out_channels": 4,
+    "sample_size": 64,
+    "up_block_types": [
+        "UpBlock2D",
+        "CrossAttnUpBlock2D",
+        "CrossAttnUpBlock2D",
+        "CrossAttnUpBlock2D"
+    ]
+}

scheduler/scheduler_config.json

@@ -0,0 +1,13 @@
+{
+    "_class_name": "DDIMScheduler",
+    "_diffusers_version": "0.6.0.dev0",
+    "beta_end": 0.012,
+    "beta_schedule": "scaled_linear",
+    "beta_start": 0.00085,
+    "clip_sample": false,
+    "num_train_timesteps": 1000,
+    "set_alpha_to_one": false,
+    "steps_offset": 1,
+    "trained_betas": null,
+    "skip_prk_steps": true
+}