Instructions to use jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit with Transformers:

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit:Q4_K_M

Use Docker

docker model run hf.co/jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit:Q4_K_M

Llama new
LM Studio
Jan
Ollama
How to use jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit with Ollama:
```
ollama run hf.co/jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit with Docker Model Runner:
```
docker model run hf.co/jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit:Q4_K_M
```

Lemonade

How to use jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull jiya2/HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit:Q4_K_M

Run and chat with the model

lemonade run user.HRChatbot_unsloth_SmolLM2-360M-Instruct-bnb-4bit-Q4_K_M

List all available models

lemonade list

Atomic Chat

jiya2 commited on Apr 20, 2025

Commit

b728585

verified ·

1 Parent(s): 2ee9ed1

Trained with Unsloth

Browse files

Files changed (3) hide show

config.json +2 -3
generation_config.json +1 -1
pytorch_model.bin +1 -1

config.json CHANGED Viewed

@@ -1,5 +1,4 @@
 {
-  "_name_or_path": "unsloth/SmolLM2-360M-Instruct-bnb-4bit",
   "architectures": [
     "LlamaForCausalLM"
   ],
@@ -33,8 +32,8 @@
       "q4f16": "float16"
     }
   },
-  "transformers_version": "4.49.0",
-  "unsloth_version": "2025.3.17",
   "use_cache": true,
   "vocab_size": 49153
 }

 {
   "architectures": [
     "LlamaForCausalLM"
   ],
       "q4f16": "float16"
     }
   },
+  "transformers_version": "4.51.3",
+  "unsloth_version": "2025.3.19",
   "use_cache": true,
   "vocab_size": 49153
 }

generation_config.json CHANGED Viewed

@@ -4,5 +4,5 @@
   "eos_token_id": 2,
   "max_length": 8192,
   "pad_token_id": 49152,
-  "transformers_version": "4.49.0"
 }

   "eos_token_id": 2,
   "max_length": 8192,
   "pad_token_id": 49152,
+  "transformers_version": "4.51.3"
 }

pytorch_model.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e7102dd897edea293a28f43463ab8409b39d86678fb1bb394ff65c4817511a4e
 size 723742290

 version https://git-lfs.github.com/spec/v1
+oid sha256:ccc7de7f9e921c48a61b49e26f5caa2611f8a905a8f097bd606a61311a4332be
 size 723742290