Instructions to use malhajar/Platypus2-70B-instruct-Nietzsche with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use malhajar/Platypus2-70B-instruct-Nietzsche with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="malhajar/Platypus2-70B-instruct-Nietzsche")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("malhajar/Platypus2-70B-instruct-Nietzsche")
model = AutoModelForCausalLM.from_pretrained("malhajar/Platypus2-70B-instruct-Nietzsche")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use malhajar/Platypus2-70B-instruct-Nietzsche with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "malhajar/Platypus2-70B-instruct-Nietzsche"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "malhajar/Platypus2-70B-instruct-Nietzsche",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/malhajar/Platypus2-70B-instruct-Nietzsche

SGLang

How to use malhajar/Platypus2-70B-instruct-Nietzsche with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "malhajar/Platypus2-70B-instruct-Nietzsche" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "malhajar/Platypus2-70B-instruct-Nietzsche",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "malhajar/Platypus2-70B-instruct-Nietzsche" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "malhajar/Platypus2-70B-instruct-Nietzsche",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use malhajar/Platypus2-70B-instruct-Nietzsche with Docker Model Runner:
```
docker model run hf.co/malhajar/Platypus2-70B-instruct-Nietzsche
```

Model Card for Model ID

Platypus2-70B-instruct-Nietzsche is a finetuned version of garage-bAInd/Platypus2-70B-instruct using SFT Training. This model can answer information about different excplicit ideas mentioned in some of Fredrick Neitzche's books (all books will be added and are in progress).

Model Description

Developed by: Mohamad Alhajar
Language(s) (NLP): English
Finetuned from model: garage-bAInd/Platypus2-70B-instruct

Prompt Template

### Instruction:

<prompt> (without the <>)

### Response:

How to Get Started with the Model

Use the code sample provided in the original post to interact with the model.

from transformers import AutoTokenizer,AutoModelForCausalLM
 
model_id = "malhajar/Platypus2-70B-instruct-Nietzsche"
model = AutoModelForCausalLM.from_pretrained(model_name_or_path,
                                             device_map="auto",
                                             torch_dtype=torch.float16,
                                             revision="main")

tokenizer = AutoTokenizer.from_pretrained(model_id)

question: "what is the will to truth?"
# For generating a response
prompt = '''
### Instruction:
{question} 

### Response:'''
input_ids = tokenizer(prompt, return_tensors="pt").input_ids
output = model.generate(inputs=input_ids,max_new_tokens=512,pad_token_id=tokenizer.eos_token_id,top_k=50, do_sample=True,
        top_p=0.95)
response = tokenizer.decode(output[0])

print(response)

Example Generation

### Instruction
what is the will to truth?
### Response

1. An earnest desire for truth, coupled with a humble, discerning attitude towards sources of knowledge.
2.A robust, enduring curiosity that never fades, even in the face of extreme boredom.
3.An insatiable hunger for learning and intellectual stimulation, which always seeks and never satisfied.
4.A dedication to truth, even if it's uncomfortable or damaging. This requires a certain level of masochism and a willingness to sacrifice for the sake of knowledge.
5.An open, flexible mind that can adapt to new intellectual environments and concepts without rigidity or prejudice. This involves being receptive to change and admitting when one is wrong.

In essence, the key to truth lies in a dynamic, flexible perspective that allows for the continual acquisition of knowledge and the modification of existing beliefs. 
This requires a humble, discerning attitude towards sources of knowledge and an enduring intellectual curiosity. 
The desire for truth must be earnest and the dedication must be unwavering, even in the face of discomfort or damage. 
This involves a certain level of masochism and a willingness to sacrifice for the sake of knowledge. 
Additionally, one must be open to change and be able to admit when they are wrong. This involves having a non-dogmatic, flexible mindset that can adapt to new intellectual environments and concepts without rigidity or prejudice. 
Thus, the pursuit of truth demands a balance between curiosity and sternness, and between openness and discipline. This involves maintaining a delicate equilibrium between intellectual extremes, being neither too loose nor too rigid in one's intellectual attitudes.

Downloads last month: 8