Instructions to use LargeWorldModel/LWM-Text-Chat-1M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LargeWorldModel/LWM-Text-Chat-1M with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LargeWorldModel/LWM-Text-Chat-1M")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("LargeWorldModel/LWM-Text-Chat-1M")
model = AutoModelForCausalLM.from_pretrained("LargeWorldModel/LWM-Text-Chat-1M")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use LargeWorldModel/LWM-Text-Chat-1M with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LargeWorldModel/LWM-Text-Chat-1M"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LargeWorldModel/LWM-Text-Chat-1M",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/LargeWorldModel/LWM-Text-Chat-1M

SGLang

How to use LargeWorldModel/LWM-Text-Chat-1M with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LargeWorldModel/LWM-Text-Chat-1M" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LargeWorldModel/LWM-Text-Chat-1M",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LargeWorldModel/LWM-Text-Chat-1M" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LargeWorldModel/LWM-Text-Chat-1M",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use LargeWorldModel/LWM-Text-Chat-1M with Docker Model Runner:
```
docker model run hf.co/LargeWorldModel/LWM-Text-Chat-1M
```

How to run this correctly

by blobpenguin - opened Feb 16, 2024

Discussion

blobpenguin

Feb 16, 2024

I've run it as I did with Lllama2 but the output seems random. Is the chat prompt identical to Llama2? Can please provide an example that works for you?

blobpenguin

Feb 16, 2024

Found the scripts folder and made it work to some extent. Please add some more documentation around running this. Thank you!

blobpenguin changed discussion status to closed Feb 16, 2024

Hoioi

Feb 16, 2024

Could you please share your experience? What's the prompt format for running this model?

mandeepbagga

Feb 17, 2024

You are a helpful assistant. USER: {context} {question} Don't give information outside the document or repeat your findings. Keep your response short and direct. ASSISTANT:

Found this in their GitHub Repo

HR1777

Feb 18, 2024

•

edited Feb 18, 2024

Thank you so much for sharing your experience.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment