Ex0bit commited on
Commit
f313a60
·
verified ·
1 Parent(s): f1f14f7

Add Ollama native tool calling workaround section

Browse files
Files changed (1) hide show
  1. README.md +130 -0
README.md CHANGED
@@ -222,6 +222,136 @@ ollama run hf.co/Ex0bit/Elbaz-Prime-Intellect-3_Prism_Abliterated
222
 
223
  > **Note:** The `hf.co/` prefix is required to pull from Hugging Face. Requires Ollama 0.3.0+.
224
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
225
  ### Using with LM Studio
226
 
227
  1. Download the GGUF file and place it in your LM Studio models directory
 
222
 
223
  > **Note:** The `hf.co/` prefix is required to pull from Hugging Face. Requires Ollama 0.3.0+.
224
 
225
+ <details>
226
+ <summary><strong>🔧 Native Tool Calling with Ollama's `tools` API</strong></summary>
227
+
228
+ > **Issue:** INTELLECT-3 outputs tool calls in XML format (`<function=name><parameter=key>value</parameter></function>`), but Ollama's parser expects JSON format inside `<tool_call>` tags. This causes Ollama's `tool_calls` field to remain empty even when the model outputs valid tool calls.
229
+
230
+ **Workaround:** Create a custom model with a template that instructs the model to use JSON format:
231
+
232
+ 1. Create a file called `Modelfile.glm`:
233
+
234
+ ```
235
+ FROM hf.co/Ex0bit/Elbaz-Prime-Intellect-3_Prism_Abliterated
236
+
237
+ # INTELLECT-3 Prism with GLM-style Tool Template
238
+ # Uses JSON tool call format that works with Ollama's parser
239
+
240
+ TEMPLATE """
241
+ {{- /* System message at the beginning, if there is .System or .Tools */ -}}
242
+ {{- if or .System .Tools }}
243
+ <|im_start|>system
244
+ {{- if .System }}
245
+ {{ .System }}
246
+ {{- end }}
247
+ {{- if .Tools }}
248
+
249
+ You are a helpful assistant with the ability to call tools.
250
+
251
+ **Available tools:**
252
+ {{ range .Tools }}
253
+ {{ . }}
254
+ {{ end }}
255
+
256
+ **Tool call format:**
257
+ <tool_call>
258
+ {"name": "tool_name", "parameters": {"param1": "value1", "param2": "value2"}}
259
+ </tool_call>
260
+
261
+ Important:
262
+ - You MUST use JSON format inside <tool_call> tags
263
+ - Do your reasoning first, then output the tool call
264
+ - If no tool is needed, answer normally
265
+ {{- end }}
266
+ <|im_end|>
267
+ {{- end }}
268
+
269
+ {{- /* Processing messages */ -}}
270
+ {{- range $i, $_ := .Messages }}
271
+ {{- $last := eq (len (slice $.Messages $i)) 1 }}
272
+
273
+ {{- /* User messages */ -}}
274
+ {{- if eq .Role "user" }}
275
+ <|im_start|>user
276
+ {{ .Content }}<|im_end|>
277
+ {{- /* Start model turn if this is the last message */ -}}
278
+ {{ if $last }}<|im_start|>assistant
279
+ {{ end }}
280
+
281
+ {{- /* Assistant messages */ -}}
282
+ {{- else if eq .Role "assistant" }}
283
+ <|im_start|>assistant
284
+ {{- if .ToolCalls }}
285
+ <tool_call>
286
+ {{- range .ToolCalls }}
287
+ {"name": "{{ .Function.Name }}", "parameters": {{ .Function.Arguments }}}
288
+ {{- end }}
289
+ </tool_call>
290
+ {{- else }}
291
+ {{ .Content }}
292
+ {{- end }}
293
+ {{- /* End turn if this is not the last message */ -}}
294
+ {{- if not $last }}<|im_end|>
295
+ {{ end }}
296
+
297
+ {{- /* Tool results */ -}}
298
+ {{- else if eq .Role "tool" }}
299
+ <|im_start|>user
300
+ <tool_response>
301
+ {{ .Content }}
302
+ </tool_response><|im_end|>
303
+ {{- /* Start model turn if this is the last message */ -}}
304
+ {{ if $last }}<|im_start|>assistant
305
+ {{ end }}
306
+ {{- end }}
307
+ {{- end }}
308
+ """
309
+
310
+ PARAMETER stop <|im_start|>
311
+ PARAMETER stop <|im_end|>
312
+ PARAMETER temperature 0.6
313
+ PARAMETER num_ctx 32768
314
+ ```
315
+
316
+ 2. Create and run the model:
317
+
318
+ ```bash
319
+ ollama create elbaz-prism-glm -f Modelfile.glm
320
+ ollama run elbaz-prism-glm
321
+ ```
322
+
323
+ 3. Use with Ollama's native `tools` API:
324
+
325
+ ```python
326
+ import ollama
327
+
328
+ response = ollama.chat(
329
+ model='elbaz-prism-glm',
330
+ messages=[{'role': 'user', 'content': 'What time is it in Tokyo?'}],
331
+ tools=[{
332
+ 'type': 'function',
333
+ 'function': {
334
+ 'name': 'get_time',
335
+ 'description': 'Get current time in a timezone',
336
+ 'parameters': {
337
+ 'type': 'object',
338
+ 'properties': {
339
+ 'timezone': {'type': 'string', 'description': 'Timezone like Asia/Tokyo'}
340
+ }
341
+ }
342
+ }
343
+ }]
344
+ )
345
+
346
+ # tool_calls field will now be properly populated!
347
+ print(response['message'].get('tool_calls'))
348
+ ```
349
+
350
+ This workaround ensures Ollama's parser correctly populates the `tool_calls` field in API responses.
351
+
352
+ </details>
353
+
354
+
355
  ### Using with LM Studio
356
 
357
  1. Download the GGUF file and place it in your LM Studio models directory