Test script for Q3C / Q3CN / Qwen3.5 tool calls

tool-calls.py

from openai import OpenAI
import json

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="dummy")

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_time",
            "description": "Get the current local time",
            "parameters": {
                "type": "object",
                "properties": {},
                "required": [],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather in a given city and state",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City, e.g. San Francisco"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                    "state": {"type": "string", "description": "State, e.g. CA"},
                },
                "required": ["city"],
            },
        },
    },
]

response = client.chat.completions.create(
    model='dummy',
    messages=[
        {
            "role": "user",
            "content": "What's the weather like in San Francisco, CA, New York, NY, and Toronto, ON? Is it below 70 degree in any of the 3 cities? What's the time now btw?",
        },
    ],
    tools=tools,
    tool_choice="auto",
    logprobs=True,
    top_logprobs=3,
    stream=False,
)

choice = response.choices[0]
message = choice.message
logprobs = choice.logprobs.content

print('reasoning_content:')
print(message.reasoning_content)
print()

print('content:')
print(message.content)
print()

print('tool_calls:')
for tool_call in message.tool_calls:
    print(tool_call.function)
print()

tool_call_started = False
worst_logprob = 0.0
for logprob in logprobs:
    if not tool_call_started and logprob.token == '<tool_call>':
        tool_call_started = True
    if not tool_call_started:
        continue
    if logprob.logprob < worst_logprob:
        worst_logprob = logprob.logprob

print(f'worst logprobs in tool_calls: {worst_logprob:.9f}')

Expected output with a close-to-zero worst logprob in tool calls:

reasoning_content:
The user is asking for weather information in three cities:
1. San Francisco, CA
2. New York, NY
3. Toronto, ON

They also want to know if it's below 70 degrees in any of these cities, and they want to know the current time.

I need to:
1. Get weather for San Francisco, CA
2. Get weather for New York, NY
3. Get weather for Toronto, ON
4. Get the current time

For the weather function, I need to provide:
- city (required)
- state (optional, but provided by user)
- unit (optional, not specified by user)

Since the user didn't specify a temperature unit, I'll use the default or let the system decide. Looking at the function, unit is optional with enum ["celsius", "fahrenheit"]. Since the user mentioned "
70 degree" without specifying, they're likely thinking in Fahrenheit, so I'll use "fahrenheit" for consistency.

Let me make the function calls:


content:
None

tool_calls:
Function(arguments='{"city":"San Francisco","state":"CA","unit":"fahrenheit"}', name='get_weather')
Function(arguments='{"city":"New York","state":"NY","unit":"fahrenheit"}', name='get_weather')
Function(arguments='{"city":"Toronto","state":"ON","unit":"fahrenheit"}', name='get_weather')
Function(arguments='{}', name='get_time')

worst logprobs in tool_calls: -0.000466098

Bad output when strict arguments order is enforced (pre ikawrakow/ik_llama.cpp#1352):

reasoning_content:
The user is asking for weather information in three different cities:
1. San Francisco, CA
2. New York, NY
3. Toronto, ON

They also want to know if it's below 70 degrees in any of these cities, and they're asking for the current time.

I need to:
1. Get weather for San Francisco, CA
2. Get weather for New York, NY
3. Get weather for Toronto, ON
4. Get the current time

For the weather function, I need city (required) and optionally state and unit. Since they're asking about 70 degrees, I should use fahrenheit as the unit.

For Toronto, ON - I need to use "ON" as the state parameter since that's the province code for Ontario.

Let me make these function calls.


content:
None

tool_calls:
Function(arguments='{"city":"San Francisco","state":"CA"}', name='get_weather')
Function(arguments='{"city":"New York","state":"NY"}', name='get_weather')
Function(arguments='{"city":"Toronto","state":"ON"}', name='get_weather')
Function(arguments='{}', name='get_time')

worst logprobs in tool_calls: -29.937694550

The very bad logprob happened when the model wanted to continue with the unit argument, but the grammar didn't allow it.

Bad output when parallel tool calls are not allowed (pre ikawrakow/ik_llama.cpp#1300):

reasoning_content:
The user is asking for:
1. Weather in San Francisco, CA
2. Weather in New York, NY
3. Weather in Toronto, ON
4. Whether it's below 70 degrees in any of these cities
5. Current time

I need to make function calls to get this information. Let me check the available functions:
- get_time: Get the current local time
- get_weather: Get the current weather in a given city and state (requires city, optional state and unit)

For the weather calls, I need to be careful about the parameters:
- San Francisco, CA: city="San Francisco", state="CA"
- New York, NY: city="New York", state="NY"
- Toronto, ON: city="Toronto", state="ON"

I should use fahrenheit since the user mentioned "70 degree" which is typically fahrenheit in the US context.

Let me make all the function calls.

content:
None

tool_calls:
Function(arguments='{"city":"San Francisco","state":"CA"}', name='get_weather')

worst logprobs in tool_calls: -21.598724365

We can change the argument order in the tool definition to be city, state, unit, to bypass the strict ordering issue:

reasoning_content:
The user is asking for weather information for three cities:
1. San Francisco, CA
2. New York, NY  
3. Toronto, ON

They also want to know if it's below 70 degrees in any of these cities, and they want to know the current time.

I need to:
1. Get weather for San Francisco, CA
2. Get weather for New York, NY
3. Get weather for Toronto, ON
4. Get the current time

For the weather function, I need to provide city and state parameters. The user specified:
- San Francisco, CA
- New York, NY
- Toronto, ON

For the unit parameter, since they're asking about 70 degrees, they're likely referring to Fahrenheit (which is common in the US), so I'll use "fahrenheit" as the unit.

Let me make the function calls.

content:
None

tool_calls:
Function(arguments='{"city":"San Francisco","state":"CA","unit":"fahrenheit"}', name='get_weather')

worst logprobs in tool_calls: -23.443851471

The very bad logprob happened when the model wanted to continue with the 2nd tool call, but the grammar didn't allow it.