---
tags:
  - language-model
  - gpt-2
  - fine-tuned
  - tiny-shakespeare
license: mit
datasets:
  - tiny_shakespeare
---

# GPT-2 Tiny Shakespeare Model

This is a small autoregressive language model based on the Transformer architecture trained on the Tiny Shakespeare dataset.

## Model Description
The model is a custom implementation of a TransformerDecoderModel, which uses a decoder-only architecture similar to GPT-2. It was trained on the Tiny Shakespeare dataset to generate text in the style of William Shakespeare.

## How to Use
To generate text with this model, you can load it and the tokenizer as follows:

```python
from transformers import GPT2LMHeadModel, GPT2Tokenizer

model = GPT2LMHeadModel.from_pretrained('NataliaH/gpt2-tiny-shakespeare')
tokenizer = GPT2Tokenizer.from_pretrained('NataliaH/gpt2-tiny-shakespeare')

input_text = 'To be or not to be'
inputs = tokenizer(input_text, return_tensors='pt')
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Tags
- Transformer
- GPT-2
- Tiny Shakespeare
- Language Model
- Text Generation
- Autoregressive

## Training Details

- **Epochs**: 3
- **Batch size**: 4
- **Learning Rate**: 5e-5
- **Loss Function**: Cross-Entropy Loss
- **Optimizer**: AdamW

## License
This model is licensed under the MIT license.