Instructions to use Equall/Saul-7B-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Equall/Saul-7B-Base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Equall/Saul-7B-Base")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Equall/Saul-7B-Base")
model = AutoModelForCausalLM.from_pretrained("Equall/Saul-7B-Base")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Equall/Saul-7B-Base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Equall/Saul-7B-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Equall/Saul-7B-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Equall/Saul-7B-Base

SGLang

How to use Equall/Saul-7B-Base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Equall/Saul-7B-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Equall/Saul-7B-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Equall/Saul-7B-Base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Equall/Saul-7B-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Equall/Saul-7B-Base with Docker Model Runner:
```
docker model run hf.co/Equall/Saul-7B-Base
```

Dataset

by Rkoy - opened Apr 5, 2024

Discussion

Rkoy

Apr 5, 2024

Hi team , this seems to be a very great work in legal domain. Glad that you have open sourced the model. Is there any idea to open source the dataset used for continued pretraining and dataset used for IFT (General and Legal instructions)

PierreColombo

Equall org Apr 5, 2024

Thanks a lot for the kind words.
We are iterating internally and having some discussions! We will keep you posted!

Rkoy

Apr 15, 2024

@PierreColombo Could you provide a time frame approximately on when the dataset used for the base and Instruction fine tuned model will be available ?

Rkoy

Jun 28, 2024

Hi @PierreColombo , any update on this?

sirrosendo

Sep 21, 2024

I have an experimented with this one yet but why not just change the settings in the config file to adjust the model to perform to its needs because you can make a model more introspective on things or more philosophical was it answers at least it seems like it based on the config file you just got to find which corresponding numbers match which which internal architecture.

sirrosendo

Sep 21, 2024

What kind of update are you trying to do this llm I would like to know I would like to use this llm in experiment I'd like to extract its internal architecture for an experiment,

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment