Tiny-Agent-α

Introduction

Tiny-Agent-α is an extension of Dria-Agent-a, trained on top of the Qwen2.5-Coder series to be used in edge devices. These models are carefully fine tuned with quantization aware training to minimize performance degradation after quantization.

Tiny-Agent-α employs Pythonic function calling, which is LLMs using blocks of Python code to interact with provided tools and output actions. This method was inspired by many previous work, including but not limited to DynaSaur, RLEF, ADAS and CAMEL. This way of function calling has a few advantages over traditional JSON-based function calling methods:

1. One-shot Parallel Multiple Function Calls: The model can can utilise many synchronous processes in one chat turn to arrive to a solution, which would require other function calling models multiple turns of conversation.
2. Free-form Reasoning and Actions: The model provides reasoning traces freely in natural language and the actions in between ```python ``` blocks, as it already tends to do without special prompting or tuning. This tries to mitigate the possible performance loss caused by imposing specific formats on LLM outputs discussed in Let Me Speak Freely?
3. On-the-fly Complex Solution Generation: The solution provided by the model is essentially a Python program with the exclusion of some “risky” builtins like exec, eval and compile (see full list in Quickstart below). This enables the model to implement custom complex logic with conditionals and synchronous pipelines (using the output of one function in the next function’s arguments) which would not be possible with the current JSON-based function calling methods (as far as we know).

Quickstart

pip install dria_agent

Decorate functions with @tool to expose them to the agent.

from dria_agent import tool

@tool
def check_availability(day: str, start_time: str, end_time: str) -> bool:
    # Your implementation here
    return True

Create an agent with custom tools:

from dria_agent import ToolCallingAgent

agent = ToolCallingAgent(
    tools=[check_availability]
)

Use agent.run(query) to execute tasks with tools.

execution = agent.run("Check my calendar for tomorrow noon", print_results=True)

Where output is:

let me help you check your availability for a 1-hour meditation session       
starting at noon tomorrow.                                                    
                                                                                
Step-by-step reasoning:                                                       
 1. We need to check availability for a specific time slot (noon)              
 2. The duration is 1 hour, so we'll use the same start and end times          
 3. Since it's tomorrow, we should format the date as "YYYY-MM-DD"             
 4. Use the check_availability() function with these parameters                
                                                                                
Here's the code to check your availability:                                   
                                                                                
```python                                                                     
tomorrow = (datetime.now() + timedelta(days=1)).strftime("%Y-%m-%d")          
start_time = "12:00"  # Noon in 24-hour format                                
end_time = "13:00"   # One hour after noon                                    
                                                                                
availability = check_availability(tomorrow, start_time, end_time)             
```                                                                           
                                                                                
The code will:                                                                
- Calculate tomorrow's date using datetime and timedelta                      
- Set the time slot to noon (12:00) for 1 hour duration                       
- Check if this time slot is available using the check_availability function  
                                                                                
The availability variable will contain True if you're available, or False if  
not.    

Evaluation & Performance

We evaluate the model on the Dria-Pythonic-Agent-Benchmark (DPAB): The benchmark we curated with a synthetic data generation +model-based validation + filtering and manual selection to evaluate LLMs on their Pythonic function calling ability, spanning multiple scenarios and tasks. See blog for more information.

Below are the DPAB results:

Current benchmark results for various models (strict):

Citation

@misc{Dria-Agent-a,
      url={https://huggingface.co/blog/andthattoo/dria-agent-a},
      title={Dria-Agent-a},
      author={"andthattoo", "Atakan Tekparmak"}
}