Text Generation
Transformers
GGUF
English
intellect-3
primeintellect
abliterated
uncensored
llama-cpp
ollama
lmstudio
refusal-removal
prism
snr-layer-selection
norm-preserving
orthogonalization
dual-component
no-filter
unfiltered
unrestricted
thinking
reasoning
Mixture of Experts
mixture-of-experts
iq4_xs
Eval Results
conversational
Add Ollama native tool calling workaround section
Browse files
README.md
CHANGED
|
@@ -222,6 +222,136 @@ ollama run hf.co/Ex0bit/Elbaz-Prime-Intellect-3_Prism_Abliterated
|
|
| 222 |
|
| 223 |
> **Note:** The `hf.co/` prefix is required to pull from Hugging Face. Requires Ollama 0.3.0+.
|
| 224 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 225 |
### Using with LM Studio
|
| 226 |
|
| 227 |
1. Download the GGUF file and place it in your LM Studio models directory
|
|
|
|
| 222 |
|
| 223 |
> **Note:** The `hf.co/` prefix is required to pull from Hugging Face. Requires Ollama 0.3.0+.
|
| 224 |
|
| 225 |
+
<details>
|
| 226 |
+
<summary><strong>🔧 Native Tool Calling with Ollama's `tools` API</strong></summary>
|
| 227 |
+
|
| 228 |
+
> **Issue:** INTELLECT-3 outputs tool calls in XML format (`<function=name><parameter=key>value</parameter></function>`), but Ollama's parser expects JSON format inside `<tool_call>` tags. This causes Ollama's `tool_calls` field to remain empty even when the model outputs valid tool calls.
|
| 229 |
+
|
| 230 |
+
**Workaround:** Create a custom model with a template that instructs the model to use JSON format:
|
| 231 |
+
|
| 232 |
+
1. Create a file called `Modelfile.glm`:
|
| 233 |
+
|
| 234 |
+
```
|
| 235 |
+
FROM hf.co/Ex0bit/Elbaz-Prime-Intellect-3_Prism_Abliterated
|
| 236 |
+
|
| 237 |
+
# INTELLECT-3 Prism with GLM-style Tool Template
|
| 238 |
+
# Uses JSON tool call format that works with Ollama's parser
|
| 239 |
+
|
| 240 |
+
TEMPLATE """
|
| 241 |
+
{{- /* System message at the beginning, if there is .System or .Tools */ -}}
|
| 242 |
+
{{- if or .System .Tools }}
|
| 243 |
+
<|im_start|>system
|
| 244 |
+
{{- if .System }}
|
| 245 |
+
{{ .System }}
|
| 246 |
+
{{- end }}
|
| 247 |
+
{{- if .Tools }}
|
| 248 |
+
|
| 249 |
+
You are a helpful assistant with the ability to call tools.
|
| 250 |
+
|
| 251 |
+
**Available tools:**
|
| 252 |
+
{{ range .Tools }}
|
| 253 |
+
{{ . }}
|
| 254 |
+
{{ end }}
|
| 255 |
+
|
| 256 |
+
**Tool call format:**
|
| 257 |
+
<tool_call>
|
| 258 |
+
{"name": "tool_name", "parameters": {"param1": "value1", "param2": "value2"}}
|
| 259 |
+
</tool_call>
|
| 260 |
+
|
| 261 |
+
Important:
|
| 262 |
+
- You MUST use JSON format inside <tool_call> tags
|
| 263 |
+
- Do your reasoning first, then output the tool call
|
| 264 |
+
- If no tool is needed, answer normally
|
| 265 |
+
{{- end }}
|
| 266 |
+
<|im_end|>
|
| 267 |
+
{{- end }}
|
| 268 |
+
|
| 269 |
+
{{- /* Processing messages */ -}}
|
| 270 |
+
{{- range $i, $_ := .Messages }}
|
| 271 |
+
{{- $last := eq (len (slice $.Messages $i)) 1 }}
|
| 272 |
+
|
| 273 |
+
{{- /* User messages */ -}}
|
| 274 |
+
{{- if eq .Role "user" }}
|
| 275 |
+
<|im_start|>user
|
| 276 |
+
{{ .Content }}<|im_end|>
|
| 277 |
+
{{- /* Start model turn if this is the last message */ -}}
|
| 278 |
+
{{ if $last }}<|im_start|>assistant
|
| 279 |
+
{{ end }}
|
| 280 |
+
|
| 281 |
+
{{- /* Assistant messages */ -}}
|
| 282 |
+
{{- else if eq .Role "assistant" }}
|
| 283 |
+
<|im_start|>assistant
|
| 284 |
+
{{- if .ToolCalls }}
|
| 285 |
+
<tool_call>
|
| 286 |
+
{{- range .ToolCalls }}
|
| 287 |
+
{"name": "{{ .Function.Name }}", "parameters": {{ .Function.Arguments }}}
|
| 288 |
+
{{- end }}
|
| 289 |
+
</tool_call>
|
| 290 |
+
{{- else }}
|
| 291 |
+
{{ .Content }}
|
| 292 |
+
{{- end }}
|
| 293 |
+
{{- /* End turn if this is not the last message */ -}}
|
| 294 |
+
{{- if not $last }}<|im_end|>
|
| 295 |
+
{{ end }}
|
| 296 |
+
|
| 297 |
+
{{- /* Tool results */ -}}
|
| 298 |
+
{{- else if eq .Role "tool" }}
|
| 299 |
+
<|im_start|>user
|
| 300 |
+
<tool_response>
|
| 301 |
+
{{ .Content }}
|
| 302 |
+
</tool_response><|im_end|>
|
| 303 |
+
{{- /* Start model turn if this is the last message */ -}}
|
| 304 |
+
{{ if $last }}<|im_start|>assistant
|
| 305 |
+
{{ end }}
|
| 306 |
+
{{- end }}
|
| 307 |
+
{{- end }}
|
| 308 |
+
"""
|
| 309 |
+
|
| 310 |
+
PARAMETER stop <|im_start|>
|
| 311 |
+
PARAMETER stop <|im_end|>
|
| 312 |
+
PARAMETER temperature 0.6
|
| 313 |
+
PARAMETER num_ctx 32768
|
| 314 |
+
```
|
| 315 |
+
|
| 316 |
+
2. Create and run the model:
|
| 317 |
+
|
| 318 |
+
```bash
|
| 319 |
+
ollama create elbaz-prism-glm -f Modelfile.glm
|
| 320 |
+
ollama run elbaz-prism-glm
|
| 321 |
+
```
|
| 322 |
+
|
| 323 |
+
3. Use with Ollama's native `tools` API:
|
| 324 |
+
|
| 325 |
+
```python
|
| 326 |
+
import ollama
|
| 327 |
+
|
| 328 |
+
response = ollama.chat(
|
| 329 |
+
model='elbaz-prism-glm',
|
| 330 |
+
messages=[{'role': 'user', 'content': 'What time is it in Tokyo?'}],
|
| 331 |
+
tools=[{
|
| 332 |
+
'type': 'function',
|
| 333 |
+
'function': {
|
| 334 |
+
'name': 'get_time',
|
| 335 |
+
'description': 'Get current time in a timezone',
|
| 336 |
+
'parameters': {
|
| 337 |
+
'type': 'object',
|
| 338 |
+
'properties': {
|
| 339 |
+
'timezone': {'type': 'string', 'description': 'Timezone like Asia/Tokyo'}
|
| 340 |
+
}
|
| 341 |
+
}
|
| 342 |
+
}
|
| 343 |
+
}]
|
| 344 |
+
)
|
| 345 |
+
|
| 346 |
+
# tool_calls field will now be properly populated!
|
| 347 |
+
print(response['message'].get('tool_calls'))
|
| 348 |
+
```
|
| 349 |
+
|
| 350 |
+
This workaround ensures Ollama's parser correctly populates the `tool_calls` field in API responses.
|
| 351 |
+
|
| 352 |
+
</details>
|
| 353 |
+
|
| 354 |
+
|
| 355 |
### Using with LM Studio
|
| 356 |
|
| 357 |
1. Download the GGUF file and place it in your LM Studio models directory
|