GuidesTool Calling

Tool Calling

Enable your language models to call functions and interact with external systems. Tool calling works with both on-device and cloud providers.

Basic Tool Calling

Define tools with a schema and implementation:

import { createSession } from 'agentary-js';
 
// Device provider example
const session = await createSession({
  models: [{
    runtime: 'transformers-js',
    model: 'onnx-community/Qwen3-0.6B-ONNX',
    quantization: 'q4',
    engine: 'webgpu'
  }]
});
 
const tools = [{
  definition: {
    name: 'get_weather',
    description: 'Get current weather for a city',
    parameters: {
      type: 'object',
      properties: {
        city: {
          type: 'string',
          description: 'City name'
        },
        units: {
          type: 'string',
          enum: ['celsius', 'fahrenheit'],
          description: 'Temperature units'
        }
      },
      required: ['city']
    }
  },
  implementation: async ({ city, units = 'celsius' }) => {
    // Call your weather API
    const response = await fetch(`https://api.weather.com/${city}`);
    const data = await response.json();
    return JSON.stringify(data);
  }
}];
 
const response = await session.createResponse('onnx-community/Qwen3-0.6B-ONNX', {
  messages: [{ role: 'user', content: 'What is the weather in Tokyo?' }],
  tools
});
 
if (response.type === 'streaming') {
  for await (const chunk of response.stream) {
    process.stdout.write(chunk.token);
  }
}

Tool Calling with Cloud Providers

Cloud providers like Claude and GPT-4 have excellent tool calling support:

const session = await createSession({
  models: [{
    runtime: 'anthropic',
    model: 'claude-3-5-sonnet-20241022',
    proxyUrl: 'https://your-backend.com/api/anthropic',
    modelProvider: 'anthropic'
  }]
});
 
const tools = [{
  definition: {
    name: 'get_weather',
    description: 'Get current weather for a city',
    parameters: {
      type: 'object',
      properties: {
        city: { type: 'string', description: 'City name' }
      },
      required: ['city']
    }
  },
  implementation: async ({ city }) => {
    const data = await fetchWeather(city);
    return JSON.stringify(data);
  }
}];
 
const response = await session.createResponse('claude-3-5-sonnet-20241022', {
  messages: [{ role: 'user', content: 'What is the weather in Tokyo?' }],
  tools
});
 
if (response.type === 'complete') {
  console.log(response.content);
  if (response.toolCalls) {
    console.log('Tools called:', response.toolCalls);
  }
}

Tool Schema

Tools use JSON Schema for parameter validation:

{
  definition: {
    name: 'search_database',
    description: 'Search a database with filters',
    parameters: {
      type: 'object',
      properties: {
        query: {
          type: 'string',
          description: 'Search query'
        },
        limit: {
          type: 'number',
          description: 'Maximum results',
          minimum: 1,
          maximum: 100
        },
        filters: {
          type: 'object',
          properties: {
            category: { type: 'string' },
            minPrice: { type: 'number' },
            maxPrice: { type: 'number' }
          }
        }
      },
      required: ['query']
    }
  },
  implementation: async ({ query, limit = 10, filters = {} }) => {
    // Your database search implementation
    const results = await searchDatabase(query, limit, filters);
    return JSON.stringify(results);
  }
}

Multiple Tools

Provide multiple tools for complex tasks:

const tools = [
  {
    definition: {
      name: 'calculate',
      description: 'Perform mathematical calculations',
      parameters: {
        type: 'object',
        properties: {
          expression: { type: 'string', description: 'Math expression to evaluate' }
        },
        required: ['expression']
      }
    },
    implementation: async ({ expression }) => {
      // Safe math evaluation
      return String(eval(expression));
    }
  },
  {
    definition: {
      name: 'get_date',
      description: 'Get current date and time',
      parameters: {
        type: 'object',
        properties: {
          timezone: { type: 'string', description: 'Timezone (optional)' }
        }
      }
    },
    implementation: async ({ timezone = 'UTC' }) => {
      return new Date().toLocaleString('en-US', { timeZone: timezone });
    }
  }
];

Tool Execution Flow

  1. Model generates tool call - Outputs structured tool call
  2. Automatic execution - Implementation is called with parameters
  3. Result returned - Tool result is added to context
  4. Model continues - Can call more tools or generate final response
// The model can chain multiple tool calls
const response = await session.createResponse(modelId, {
  messages: [{
    role: 'user',
    content: 'Calculate 15% tip on $47.50 and tell me the current time in New York'
  }],
  tools
});
 
if (response.type === 'streaming') {
  // Model will call calculate(), then get_date(), then provide answer
  for await (const chunk of response.stream) {
    process.stdout.write(chunk.token);
  }
} else {
  // Non-streaming response includes tool calls
  console.log(response.content);
  if (response.toolCalls) {
    console.log('Tools called:', response.toolCalls);
  }
}

Agent Session with Tools

For workflows, register tools globally:

import { createAgentSession } from 'agentary-js';
 
const agent = await createAgentSession({
  models: [{
    runtime: 'transformers-js',
    model: 'onnx-community/Qwen3-0.6B-ONNX',
    quantization: 'q4',
    engine: 'webgpu'
  }]
});
 
// Register tools
agent.registerTools([{
  definition: {
    name: 'web_search',
    description: 'Search the web',
    parameters: {
      type: 'object',
      properties: {
        query: { type: 'string', description: 'Search query' }
      },
      required: ['query']
    }
  },
  implementation: async ({ query }) => {
    // Your search implementation
    return searchResults;
  }
}]);
 
// Tools available in all workflow steps
const workflow = {
  id: 'research-workflow',
  steps: [
    {
      id: 'research',
      prompt: 'Research the topic',
      model: 'onnx-community/Qwen3-0.6B-ONNX',
      toolChoice: ['web_search']  // Restrict to specific tools
    }
  ],
  tools: []  // Or specify tools here if not registered globally
};

Tool Calling Best Practices

1. Clear Descriptions

Write clear, specific descriptions:

// ❌ Bad
description: 'Gets data'
 
// ✅ Good
description: 'Retrieves user profile data including name, email, and preferences from the database'

2. Parameter Validation

Use JSON Schema constraints:

parameters: {
  type: 'object',
  properties: {
    email: {
      type: 'string',
      format: 'email',
      description: 'Valid email address'
    },
    age: {
      type: 'number',
      minimum: 0,
      maximum: 150,
      description: 'User age in years'
    }
  }
}

3. Error Handling

Handle errors gracefully in implementations:

implementation: async ({ city }) => {
  try {
    const response = await fetch(`https://api.weather.com/${city}`);
    if (!response.ok) {
      return JSON.stringify({ error: 'City not found' });
    }
    return JSON.stringify(await response.json());
  } catch (error) {
    return JSON.stringify({ error: error.message });
  }
}

4. Return Format

Return JSON strings for complex data:

implementation: async ({ query }) => {
  const results = await search(query);
  return JSON.stringify({
    count: results.length,
    items: results.slice(0, 5),
    hasMore: results.length > 5
  });
}

Monitoring Tool Calls

Use lifecycle events to monitor tool execution:

session.on('tool:call:start', (event) => {
  console.log(`Calling ${event.toolName}:`, event.args);
});
 
session.on('tool:call:complete', (event) => {
  console.log(`${event.toolName} completed in ${event.duration}ms`);
  console.log('Result:', event.result);
});
 
session.on('tool:call:error', (event) => {
  console.error(`${event.toolName} failed:`, event.error);
});

Model Selection for Tools

Different providers have different tool-calling capabilities:

On-Device Models

ModelTool CallingReliability
Qwen3-0.6B✅ SupportedGood

See the Model Support documentation for the latest supported models.

const session = await createSession({
  models: [{
    runtime: 'transformers-js',
    model: 'onnx-community/Qwen3-0.6B-ONNX',
    quantization: 'q4',
    engine: 'webgpu'
  }]
});

Cloud Models

Cloud providers offer excellent tool calling support:

ProviderModelsTool Calling
AnthropicClaude 3.5 Sonnet, Claude 3.5 Haiku✅ Excellent
OpenAIGPT-4o, GPT-4 Turbo✅ Excellent
const session = await createSession({
  models: [{
    runtime: 'anthropic',
    model: 'claude-3-5-sonnet-20241022',
    proxyUrl: 'https://your-backend.com/api/anthropic',
    modelProvider: 'anthropic'
  }]
});

Comparing Device vs Cloud Tool Calling

Device Providers

  • Pros: Privacy (data stays local), no API costs, offline capability
  • Cons: Limited to smaller models, may require more prompting
  • Best for: Simple tools, privacy-sensitive applications, prototype/demo

Cloud Providers

  • Pros: Excellent reliability, handles complex tools, better reasoning
  • Cons: API costs, requires internet, data sent to cloud
  • Best for: Production applications, complex tool chains, high accuracy needs

Hybrid Approach

Use both providers based on the task:

const session = await createSession({
  models: [
    // Device model for simple, frequent tool calls
    {
      runtime: 'transformers-js',
      model: 'onnx-community/Qwen3-0.6B-ONNX',
      quantization: 'q4',
      engine: 'webgpu'
    },
    // Cloud model for complex reasoning
    {
      runtime: 'anthropic',
      model: 'claude-3-5-sonnet-20241022',
      proxyUrl: 'https://your-backend.com/api/anthropic',
      modelProvider: 'anthropic'
    }
  ]
});
 
// Simple tool call - use device model
const quickResponse = await session.createResponse('onnx-community/Qwen3-0.6B-ONNX', {
  messages: [{ role: 'user', content: 'Get weather for Tokyo' }],
  tools: simpleTools
});
 
// Complex multi-tool task - use cloud model
const complexResponse = await session.createResponse('claude-3-5-sonnet-20241022', {
  messages: [{ role: 'user', content: 'Research and analyze market trends' }],
  tools: complexTools
});

Next Steps

Cloud ProviderAgentic Workflows