Functionary Small v3.1 is a Llama 3.1 8b based tool calling LLM

Tools 8B

53 Pulls Updated 4 weeks ago

4 weeks ago

c1affc7e6d58 · 4.7GB

model
llama
·
8.03B
·
Q4_0
params
{"stop":["<|start_header_id|>","<|end_header_id|>","<|eot_id|>","<|reserved_special_token"]}
template
{{ if .Messages }} {{- if or .System .Tools }}<|start_header_id|>system<|end_header_id|> {{- if .System }} {{ .System }} {{- end }} {{- if .Tools }} You are a helpful assistant with tool calling capabilities. When you receive a tool call response, use the output to give an answer. {{- end }} {{- end }}<|eot_id|> {{- range $i, $_ := .Messages }} {{- $last := eq (len (slice $.Messages $i)) 1 }} {{- if eq .Role "user" }}<|start_header_id|>user<|end_header_id|> {{- if and $.Tools $last }} Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt. Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables. {{ $.Tools }} {{- end }} {{ .Content }}<|eot_id|>{{ if $last }}<|start_header_id|>assistant<|end_header_id|> {{ end }} {{- else if eq .Role "assistant" }}<|start_header_id|>assistant<|end_header_id|> {{- if .ToolCalls }} {{- range .ToolCalls }}{"name": "{{ .Function.Name }}", "parameters": {{ .Function.Arguments }}}{{ end }} {{- else }} {{ .Content }}{{ if not $last }}<|eot_id|>{{ end }} {{- end }} {{- else if eq .Role "tool" }}<|start_header_id|>ipython<|end_header_id|> {{ .Content }}<|eot_id|>{{ if $last }}<|start_header_id|>assistant<|end_header_id|> {{ end }} {{- end }} {{- end }} {{- else }} {{- if .System }}<|start_header_id|>system<|end_header_id|> {{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|> {{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|> {{ end }}{{ .Response }}{{ if .Response }}<|eot_id|>{{ end }}

Readme


license: mit

Model Card for functionary-small-v3.1

This is not an official port

The system prompt is not correct as of now.

This model was based on meta-llama/Meta-Llama-3.1-8B-Instruct, using Meta’s original prompt template as described in: User-defined Custom tool calling

https://github.com/MeetKai/functionary

Functionary Logo

Functionary is a language model that can interpret and execute functions/plugins.

The model determines when to execute functions, whether in parallel or serially, and can understand their outputs. It only triggers functions as needed. Function definitions are given as JSON Schema Objects, similar to OpenAI GPT function calls.

Key Features

  • Intelligent parallel tool use
  • Able to analyze functions/tools outputs and provide relevant responses grounded in the outputs
  • Able to decide when to not use tools/call functions and provide normal chat response
  • Truly one of the best open-source alternative to GPT-4
  • Support code interpreter

Acknowledgements

Thank you to the meetkai team for training these models and releasing them. Find their github here and the gguf files this is based on here.