MadhuryaPasan/Qwen3-ExpertTools:1.7b-f16

MadhuryaPasan/ Qwen3-ExpertTools:1.7b-f16

66 Downloads Updated 3 months ago

Qwen 3 fine-tuned for expert tool calling.

tools thinking

ollama run MadhuryaPasan/Qwen3-ExpertTools:1.7b-f16

curl http://localhost:11434/api/chat \
  -d '{
    "model": "MadhuryaPasan/Qwen3-ExpertTools:1.7b-f16",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from ollama import chat

response = chat(
    model='MadhuryaPasan/Qwen3-ExpertTools:1.7b-f16',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.message.content)

import ollama from 'ollama'

const response = await ollama.chat({
  model: 'MadhuryaPasan/Qwen3-ExpertTools:1.7b-f16',
  messages: [{role: 'user', content: 'Hello!'}],
})
console.log(response.message.content)

Details

Updated 3 months ago

3 months ago

cb8216739abc · 3.4GB ·

model

archqwen3

·

parameters1.72B

·

quantizationF16

3.4GB

license

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR US

11kB

params

{ "repeat_penalty": 1, "stop": [ "<|im_start|>", "<|im_end|>" ], "te

120B

template

{{- $lastUserIdx := -1 -}} {{- range $idx, $msg := .Messages -}} {{- if eq $msg.Role "user" }}{{ $la

1.7kB

Readme

Training Data

This model was fine-tuned using high-quality function-calling datasets: Salesforce/xlam-function-calling-60k & MadeAgents/xlam-irrelevance-7.5k

This model was finetuned and converted to GGUF format using Unsloth.

Evaluation Results

This model was evaluated using the BFCL (Berkeley Function Calling Leaderboard) evaluation framework to benchmark its tool-calling accuracy.

(Note: Evaluated on the 4-bit quantized (q4_K_M) versions. Full f16 precision may yield slightly higher results).

Evaluation Metric	Base Model (Qwen3 1.7B)	Fine-Tuned (ExpertTools)
Non-Live Overall Acc	29.73%	32.27%
AST Summary	29.73%	32.27%
Simple AST (Overall)	26.92%	33.08%
Python Simple AST	46.75%	52.25%
JavaScript Simple AST	34.00%	46.00%
Java Simple AST	0.00%	1.00%
Multiple AST	32.00%	35.50%
Parallel AST	44.00%	45.50%
Parallel Multiple AST	16.00%	15.00%
Irrelevance Detection	74.58%	61.25%

Note on Irrelevance: As is common when fine-tuning heavily on function-execution datasets, there is a slight trade-off in irrelevance detection (recognizing when not to use a tool), though the model still maintains a strong 61.25% accuracy in this area.

Developer Notes

When the model detects that a user’s prompt does not require any of the provided tools (Irrelevance Detection), its default behavior is to return an empty string ("") or null, rather than hallucinating a fake tool call or generating conversational text.

## Training Data
This model was fine-tuned using high-quality function-calling datasets:
`Salesforce/xlam-function-calling-60k` & `MadeAgents/xlam-irrelevance-7.5k`

This model was finetuned and converted to GGUF format using Unsloth.

## Evaluation Results

This model was evaluated using the BFCL (Berkeley Function Calling Leaderboard) evaluation framework to benchmark its tool-calling accuracy.

*(Note: Evaluated on the 4-bit quantized (`q4_K_M`) versions. Full f16 precision may yield slightly higher results).*

| Evaluation Metric | Base Model (Qwen3 1.7B) | Fine-Tuned (ExpertTools) | 
| :--- | :--- | :--- | 
| **Non-Live Overall Acc** | 29.73% | **32.27%** | 
| **AST Summary** | 29.73% | **32.27%** | 
| **Simple AST (Overall)** | 26.92% | **33.08%** | 
| **Python Simple AST** | 46.75% | **52.25%** |
| **JavaScript Simple AST** | 34.00% | **46.00%** | 
| **Java Simple AST** | 0.00% | **1.00%** | 
| **Multiple AST** | 32.00% | **35.50%** | 
| **Parallel AST** | 44.00% | **45.50%** | 
| **Parallel Multiple AST** | **16.00%** | 15.00% | 
| **Irrelevance Detection** | **74.58%** | 61.25% |

* *Note on Irrelevance:* As is common when fine-tuning heavily on function-execution datasets, there is a slight trade-off in irrelevance detection (recognizing when *not* to use a tool), though the model still maintains a strong 61.25% accuracy in this area.

### Developer Notes

When the model detects that a user's prompt **does not** require any of the provided tools (Irrelevance Detection), its default behavior is to return an empty string (`""`) or null, rather than hallucinating a fake tool call or generating conversational text.

![NoneLive comparison.png](/assets/MadhuryaPasan/Qwen3-ExpertTools/19999204-1a8d-4fb5-afcb-a2848b152991)

Paste, drop or click to upload images (.png, .jpeg, .jpg, .svg, .gif)