4 6 days ago

this is just a test. Vision seems broken.

vision tools
{{- /* optional beginning-of-sequence token if your caller provides one */ -}}
{{- if .Bos }}{{ .Bos }}{{ end }}
{{- /* SYSTEM + TOOLS block (follows llama4-style guidance for function calls) */ -}}
{{- if or .System .Tools }}<|header_start|>system<|header_end|>
{{- /* Optional enable-thinking preface */ -}}
{{- if .EnableThinking }}
Enable deep thinking subroutine.
{{- end }}
{{- if .System }}
{{ .System }}
{{- end }}
{{- if .Tools }}
You are a helpful assistant and an expert in function composition. You can answer general questions using your internal knowledge OR invoke functions when necessary. Follow these strict guidelines:
1. FUNCTION CALLS:
- ONLY use functions that are EXPLICITLY listed in the function list below
- If NO functions are listed (empty function list []), respond ONLY with internal knowledge or "I don't have access to [Unavailable service] information"
- If a function is not in the list, respond ONLY with internal knowledge or "I don't have access to [Unavailable service] information"
- If ALL required parameters are present AND the query EXACTLY matches a listed function's purpose: output ONLY the function call(s)
- Use exact format: [{"name": "<tool_name_foo>","parameters": {"<param1_name>": "<param1_value>","<param2_name>": "<param2_value>"}}]
Examples:
CORRECT: [{"name": "get_weather","parameters": {"location": "Vancouver"}},{"name": "calculate_route","parameters": {"start": "Boston","end": "New York"}}] <- Only if get_weather and calculate_route are in function list
INCORRECT: [{"name": "population_projections", "parameters": {"country": "United States", "years": 20}}]] <- Bad json format
INCORRECT: Let me check the weather: [{"name": "get_weather","parameters": {"location": "Vancouver"}}]
INCORRECT: [{"name": "get_events","parameters": {"location": "Singapore"}}] <- If function not in list
2. RESPONSE RULES:
- For pure function requests matching a listed function: ONLY output the function call(s)
- For knowledge questions: ONLY output text
- For missing parameters: ONLY request the specific missing parameters
- For unavailable services (not in function list): output ONLY with internal knowledge or "I don't have access to [Unavailable service] information". Do NOT execute a function call.
- If the query asks for information beyond what a listed function provides: output ONLY with internal knowledge about your limitations
- NEVER combine text and function calls in the same response
- NEVER suggest alternative functions when the requested service is unavailable
- NEVER create or invent new functions not listed below
3. STRICT BOUNDARIES:
- ONLY use functions from the list below - no exceptions
- NEVER use a function as an alternative to unavailable information
- NEVER call functions not present in the function list
- NEVER add explanatory text to function calls
- NEVER respond with empty brackets
- Use proper Python/JSON syntax for function calls
- Check the function list carefully before responding
4. TOOL RESPONSE HANDLING:
- When receiving tool responses: provide concise, natural language responses
- Don't repeat tool response verbatim
- Don't add supplementary information
Here is a list of functions in JSON format that you can invoke:
[
{{- range .Tools }}{{ . }}
{{- end }}]
{{- end }}
<|eot|>
{{- end }}
{{- /* Messages loop: we expect .Messages to be an array of {Role, Content, ToolCalls, ToolCallID, ...} */ -}}
{{- range $i, $msg := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 }}
{{- if eq $msg.Role "system" }}{{- /* already represented via .System */ -}}{{- continue }}
{{- else if eq $msg.Role "user" }}<|header_start|>user<|header_end|>
{{ $msg.Content }}
{{- else if eq $msg.Role "assistant" }}<|header_start|>assistant<|header_end|>
{{- if $msg.Content }}{{ $msg.Content }}
{{- else if $msg.ToolCalls }}[
{{- range $tc := $msg.ToolCalls }}
{"name": "{{ $tc.Function.Name }}", "parameters": {{ $tc.Function.Arguments }}}
{{- end }}]
{{- end }}
{{- else if eq $msg.Role "tool" }}<|header_start|>ipython<|header_end|>
{{- /* if the tool returned a call id, package it as JSON similar to original template */ -}}
{{- if $msg.ToolCallID }}{"content": {{ $msg.Content }}, "call_id": "{{ $msg.ToolCallID }}" }
{{- else }}
{{ $msg.Content }}
{{- end }}
{{- end }}
{{- if not $last }}<|eot|>{{ end }}
{{- if and $last (ne $msg.Role "assistant") }}<|header_start|>assistant<|header_end|>
{{- end }}
{{- end }}
{{- /* optional: append generation prompt / thinking tag */ -}}
{{- if .AddGenerationPrompt }}
<|header_start|>assistant<|header_end|>
{{- if .EnableThinking }}
<think>
{{- end }}
{{- end }}