Instructions to use Aculi/InternLM2-Chat-20B-ToxicRP-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Aculi/InternLM2-Chat-20B-ToxicRP-GGUF with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Aculi/InternLM2-Chat-20B-ToxicRP-GGUF", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Aculi/InternLM2-Chat-20B-ToxicRP-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Aculi/InternLM2-Chat-20B-ToxicRP-GGUF:Q2_K # Run inference directly in the terminal: llama cli -hf Aculi/InternLM2-Chat-20B-ToxicRP-GGUF:Q2_K
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Aculi/InternLM2-Chat-20B-ToxicRP-GGUF:Q2_K # Run inference directly in the terminal: llama cli -hf Aculi/InternLM2-Chat-20B-ToxicRP-GGUF:Q2_K
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Aculi/InternLM2-Chat-20B-ToxicRP-GGUF:Q2_K # Run inference directly in the terminal: ./build/bin/llama cli -hf Aculi/InternLM2-Chat-20B-ToxicRP-GGUF:Q2_K
Use Docker
docker model run hf.co/Aculi/InternLM2-Chat-20B-ToxicRP-GGUF:Q2_K
- Llama new
- LM Studio
- Jan
- Ollama
How to use Aculi/InternLM2-Chat-20B-ToxicRP-GGUF with Ollama:
ollama run hf.co/Aculi/InternLM2-Chat-20B-ToxicRP-GGUF:Q2_K
- Unsloth Desktop
- Docker Model Runner
How to use Aculi/InternLM2-Chat-20B-ToxicRP-GGUF with Docker Model Runner:
docker model run hf.co/Aculi/InternLM2-Chat-20B-ToxicRP-GGUF:Q2_K
- Lemonade
How to use Aculi/InternLM2-Chat-20B-ToxicRP-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Aculi/InternLM2-Chat-20B-ToxicRP-GGUF:Q2_K
Run and chat with the model
lemonade run user.InternLM2-Chat-20B-ToxicRP-GGUF-Q2_K
List all available models
lemonade list
- Atomic Chat
InternLM2-chat-20B-ToxicRP-QLORA-Merged
This Model was Finetuned by me, using the Machine Power of g4rg. Big Thanks to all people that helped me. Do whatever you want with this Model, just dont do anything illegal.
non Quantized here: Aculi/InternLM2-Chat-20B-ToxicRP
Have fun
This Model uses CHATML btw.
Merge Method
This model was merged using the passthrough merge method.
Models Merged
The following models were included in the merge:
- output/intervitens_internlm2-limarp-chat-20b-2 + Fischerboot/InternLM2-ToxicRP-QLORA-4Bit
Configuration
The following YAML configuration was used to produce this model:
dtype: bfloat16
merge_method: passthrough
slices:
- sources:
- layer_range: [0, 48]
model: output/intervitens_internlm2-limarp-chat-20b-2+Fischerboot/InternLM2-ToxicRP-QLORA-4Bit
- Downloads last month
- 279
2-bit
5-bit
6-bit
8-bit
Model tree for Aculi/InternLM2-Chat-20B-ToxicRP-GGUF
Base model
internlm/internlm2-base-20b