ejschwar/
llama3.2-better-prompts:llama3.1-tooling-prompt

110 8 months ago

Llama 3.2 with better templates for tool-calling agents

tools

8 months ago

64db35453544 · 2.0GB

model
llama
·
3.21B
·
Q4_K_M
template
<|start_header_id|>system<|end_header_id|> Environment: ipython Cutting Knowledge Date: December 202
license
LLAMA 3.2 COMMUNITY LICENSE AGREEMENT Llama 3.2 Version Release Date: September 25, 2024 “Agreemen
license
**Llama 3.2** **Acceptable Use Policy** Meta is committed to promoting safe and fair use of its tool
params
{ "stop": [ "<|start_header_id|>", "<|end_header_id|>", "<|eot_id|>"

Readme

For more information, see the base llama-3.2 model’s Readme.

The llama-3.2 model’s prompt template comes from this example prompt on the Llama 3.1 model website. I have found empirically that this template does not work well for tool-calling agents. The default prompt template causes agents to behave unnaturally, such as responding to greetings by calling unrelated tools.

The tags of this model are identical to the llama-3.2:3b model but contain prompt templates that are intended to perform better with tool-calling agents. More information about the creation of these templates is available in this blog post.

Templates

ejschwar/llama3.2-better-prompts:llama3.1-tooling-prompt

This model uses the template for Llama 3.1 documented in the llama-models repository.

ejschwar/llama3.2-better-prompts:llama3.1-tooling-prompt-customized

This model is based on the template for Llama 3.1 documented in the llama-models repository, but with a small change as expressed in the following diff:

-Answer the user's question by making use of the following functions if needed.
-If none of the function can be used, please say so.
+Help and converse with the user.  If and only if the user asks a question that
+is relevant to one of the following functions, make use of them.  If none of
+the functions can be used, answer the query as best you can.

Empirically, I found the change caused the model to more reliably answer questions that were unrelated to the available tools.