Självstudie: Skapa en AI-assistent med verktygsanrop

Skapa en AI-assistent som går utöver konversation – den kan anropa funktioner för att utföra åtgärder. Assistenten bestämmer när en funktion behövs, kör den och matar tillbaka resultatet. Allt körs lokalt med Foundry Local SDK.

I den här tutorialen lär du dig följande:

Konfigurera ett projekt och installera Foundry Local SDK
Definiera verktyg som assistenten kan anropa
Skicka ett meddelande som utlöser verktygsanvändning
Kör verktyget och returnera resultat till modellen
Hantera den fullständiga verktygsanropsloopen
Rensa resurser

Förutsättningar

En Windows-, macOS- eller Linux-dator med minst 8 GB RAM-minne.

Exempellagringsplats

Den fullständiga exempelkoden för den här artikeln finns på lagringsplatsen Foundry Local GitHub. Klona lagringsplatsen och gå till exempelanvändningen:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/cs/tutorial-tool-calling

Installera paket

Om du utvecklar eller skickar på Windows väljer du fliken Windows. Windows-paketet integreras med Windows ML-körning – det ger samma API-yta med en bredare bredd av maskinvaruacceleration.

Windows
Plattformsoberoende

dotnet add package Microsoft.AI.Foundry.Local.WinML
dotnet add package OpenAI

dotnet add package Microsoft.AI.Foundry.Local
dotnet add package OpenAI

C#-exemplen på GitHub-lagringsplatsen är förkonfigurerade projekt. Om du skapar från grunden bör du läsa Foundry Local SDK-referensen för mer information om hur du konfigurerar ditt C#-projekt med Foundry Local.

Definiera verktyg

Med verktygsanrop kan modellen begära att koden kör en funktion och returnerar resultatet. Du definierar tillgängliga verktyg som en lista över JSON-scheman som beskriver varje funktions namn, syfte och parametrar.

Öppna Program.cs och lägg till följande verktygsdefinitioner:

// --- Tool definitions ---
List<ToolDefinition> tools =
[
    new ToolDefinition
    {
        Type = "function",
        Function = new FunctionDefinition()
        {
            Name = "get_weather",
            Description = "Get the current weather for a location",
            Parameters = new PropertyDefinition()
            {
                Type = "object",
                Properties = new Dictionary<string, PropertyDefinition>()
                {
                    { "location", new PropertyDefinition() { Type = "string", Description = "The city or location" } },
                    { "unit", new PropertyDefinition() { Type = "string", Description = "Temperature unit (celsius or fahrenheit)" } }
                },
                Required = ["location"]
            }
        }
    },
    new ToolDefinition
    {
        Type = "function",
        Function = new FunctionDefinition()
        {
            Name = "calculate",
            Description = "Perform a math calculation",
            Parameters = new PropertyDefinition()
            {
                Type = "object",
                Properties = new Dictionary<string, PropertyDefinition>()
                {
                    { "expression", new PropertyDefinition() { Type = "string", Description = "The math expression to evaluate" } }
                },
                Required = ["expression"]
            }
        }
    }
];

// --- Tool implementations ---
string ExecuteTool(string functionName, JsonElement arguments)
{
    switch (functionName)
    {
        case "get_weather":
            var location = arguments.GetProperty("location")
                .GetString() ?? "unknown";
            var unit = arguments.TryGetProperty("unit", out var u)
                ? u.GetString() ?? "celsius"
                : "celsius";
            var temp = unit == "celsius" ? 22 : 72;
            return JsonSerializer.Serialize(new
            {
                location,
                temperature = temp,
                unit,
                condition = "Sunny"
            });

        case "calculate":
            var expression = arguments.GetProperty("expression")
                .GetString() ?? "";
            try
            {
                var result = new System.Data.DataTable()
                    .Compute(expression, null);
                return JsonSerializer.Serialize(new
                {
                    expression,
                    result = result?.ToString()
                });
            }
            catch (Exception ex)
            {
                return JsonSerializer.Serialize(new
                {
                    error = ex.Message
                });
            }

        default:
            return JsonSerializer.Serialize(new
            {
                error = $"Unknown function: {functionName}"
            });
    }
}

Varje verktygsdefinition innehåller en name, en description som hjälper modellen att bestämma när den ska användas och ett parameters schema som beskriver förväntade indata.

Lägg till de C#-metoder som implementerar varje verktyg:

// --- Tool definitions ---
List<ToolDefinition> tools =
[
    new ToolDefinition
    {
        Type = "function",
        Function = new FunctionDefinition()
        {
            Name = "get_weather",
            Description = "Get the current weather for a location",
            Parameters = new PropertyDefinition()
            {
                Type = "object",
                Properties = new Dictionary<string, PropertyDefinition>()
                {
                    { "location", new PropertyDefinition() { Type = "string", Description = "The city or location" } },
                    { "unit", new PropertyDefinition() { Type = "string", Description = "Temperature unit (celsius or fahrenheit)" } }
                },
                Required = ["location"]
            }
        }
    },
    new ToolDefinition
    {
        Type = "function",
        Function = new FunctionDefinition()
        {
            Name = "calculate",
            Description = "Perform a math calculation",
            Parameters = new PropertyDefinition()
            {
                Type = "object",
                Properties = new Dictionary<string, PropertyDefinition>()
                {
                    { "expression", new PropertyDefinition() { Type = "string", Description = "The math expression to evaluate" } }
                },
                Required = ["expression"]
            }
        }
    }
];

// --- Tool implementations ---
string ExecuteTool(string functionName, JsonElement arguments)
{
    switch (functionName)
    {
        case "get_weather":
            var location = arguments.GetProperty("location")
                .GetString() ?? "unknown";
            var unit = arguments.TryGetProperty("unit", out var u)
                ? u.GetString() ?? "celsius"
                : "celsius";
            var temp = unit == "celsius" ? 22 : 72;
            return JsonSerializer.Serialize(new
            {
                location,
                temperature = temp,
                unit,
                condition = "Sunny"
            });

        case "calculate":
            var expression = arguments.GetProperty("expression")
                .GetString() ?? "";
            try
            {
                var result = new System.Data.DataTable()
                    .Compute(expression, null);
                return JsonSerializer.Serialize(new
                {
                    expression,
                    result = result?.ToString()
                });
            }
            catch (Exception ex)
            {
                return JsonSerializer.Serialize(new
                {
                    error = ex.Message
                });
            }

        default:
            return JsonSerializer.Serialize(new
            {
                error = $"Unknown function: {functionName}"
            });
    }
}

Modellen kör inte dessa funktioner direkt. Den returnerar en begäran om verktygsanrop med funktionsnamnet och argumenten, och koden kör funktionen.

Skicka ett meddelande som utlöser verktygsanvändning

Initiera Foundry Local SDK, läs in en modell och skicka ett meddelande som modellen kan svara på genom att anropa ett verktyg.

// --- Main application ---
var config = new Configuration
{
    AppName = "foundry_local_samples",
    LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
};

using var loggerFactory = LoggerFactory.Create(builder =>
{
    builder.SetMinimumLevel(
        Microsoft.Extensions.Logging.LogLevel.Information
    );
});
var logger = loggerFactory.CreateLogger<Program>();

await FoundryLocalManager.CreateAsync(config, logger);
var mgr = FoundryLocalManager.Instance;

// Download and register all execution providers.
var currentEp = "";
await mgr.DownloadAndRegisterEpsAsync((epName, percent) =>
{
    if (epName != currentEp)
    {
        if (currentEp != "") Console.WriteLine();
        currentEp = epName;
    }
    Console.Write($"\r  {epName.PadRight(30)}  {percent,6:F1}%");
});
if (currentEp != "") Console.WriteLine();

var catalog = await mgr.GetCatalogAsync();
var model = await catalog.GetModelAsync("qwen2.5-0.5b")
    ?? throw new Exception("Model not found");

await model.DownloadAsync(progress =>
{
    Console.Write($"\rDownloading model: {progress:F2}%");
    if (progress >= 100f) Console.WriteLine();
});

await model.LoadAsync();
Console.WriteLine("Model loaded and ready.");

var chatClient = await model.GetChatClientAsync();
chatClient.Settings.ToolChoice = ToolChoice.Auto;

var messages = new List<ChatMessage>
{
    new ChatMessage
    {
        Role = "system",
        Content = "You are a helpful assistant with access to tools. " +
                  "Use them when needed to answer questions accurately."
    }
};

När modellen fastställer att ett verktyg behövs innehåller ToolCalls svaret i stället för ett vanligt textmeddelande. Nästa steg visar hur du identifierar och hanterar dessa anrop.

Kör verktyget och returnera resultat

När modellen svarar med ett verktygsanrop extraherar du funktionsnamnet och argumenten, kör funktionen och skickar tillbaka resultatet.

Console.WriteLine("\nTool-calling assistant ready! Type 'quit' to exit.\n");

while (true)
{
    Console.Write("You: ");
    var userInput = Console.ReadLine();
    if (string.IsNullOrWhiteSpace(userInput) ||
        userInput.Equals("quit", StringComparison.OrdinalIgnoreCase) ||
        userInput.Equals("exit", StringComparison.OrdinalIgnoreCase))
    {
        break;
    }

    messages.Add(new ChatMessage
    {
        Role = "user",
        Content = userInput
    });

    var response = await chatClient.CompleteChatAsync(
        messages, tools, ct
    );

    var choice = response.Choices[0].Message;

    if (choice.ToolCalls is { Count: > 0 })
    {
        messages.Add(choice);

        foreach (var toolCall in choice.ToolCalls)
        {
            var toolArgs = JsonDocument.Parse(
                toolCall.FunctionCall.Arguments
            ).RootElement;
            Console.WriteLine(
                $"  Tool call: {toolCall.FunctionCall.Name}({toolArgs})"
            );

            var result = ExecuteTool(
                toolCall.FunctionCall.Name, toolArgs
            );
            messages.Add(new ChatMessage
            {
                Role = "tool",
                ToolCallId = toolCall.Id,
                Content = result
            });
        }

        var finalResponse = await chatClient.CompleteChatAsync(
            messages, tools, ct
        );
        var answer = finalResponse.Choices[0].Message.Content ?? "";
        messages.Add(new ChatMessage
        {
            Role = "assistant",
            Content = answer
        });
        Console.WriteLine($"Assistant: {answer}\n");
    }
    else
    {
        var answer = choice.Content ?? "";
        messages.Add(new ChatMessage
        {
            Role = "assistant",
            Content = answer
        });
        Console.WriteLine($"Assistant: {answer}\n");
    }
}

await model.UnloadAsync();
Console.WriteLine("Model unloaded. Goodbye!");

De viktigaste stegen i verktygsanropsloopen är:

Identifiera verktygsanrop – kontrollera response.Choices[0].Message.ToolCalls.
Kör funktionen – parsa argumenten och anropa din lokala funktion.
Returnera resultatet – lägg till ett meddelande med rollen tool och matchande ToolCallId.
Få det slutliga svaret – modellen använder verktygets resultat för att generera ett naturligt svar.

Hantera den fullständiga verktygsanropsloopen

Här är det fullständiga programmet som kombinerar verktygsdefinitioner, SDK-initiering och verktygsanropsloopen till en enda körbar fil.

Ersätt innehållet i Program.cs med följande fullständiga kod:

using System.Text.Json;
using Microsoft.AI.Foundry.Local;
using Betalgo.Ranul.OpenAI.ObjectModels.RequestModels;
using Betalgo.Ranul.OpenAI.ObjectModels.ResponseModels;
using Betalgo.Ranul.OpenAI.ObjectModels.SharedModels;
using Microsoft.Extensions.Logging;

CancellationToken ct = CancellationToken.None;

// --- Tool definitions ---
List<ToolDefinition> tools =
[
    new ToolDefinition
    {
        Type = "function",
        Function = new FunctionDefinition()
        {
            Name = "get_weather",
            Description = "Get the current weather for a location",
            Parameters = new PropertyDefinition()
            {
                Type = "object",
                Properties = new Dictionary<string, PropertyDefinition>()
                {
                    { "location", new PropertyDefinition() { Type = "string", Description = "The city or location" } },
                    { "unit", new PropertyDefinition() { Type = "string", Description = "Temperature unit (celsius or fahrenheit)" } }
                },
                Required = ["location"]
            }
        }
    },
    new ToolDefinition
    {
        Type = "function",
        Function = new FunctionDefinition()
        {
            Name = "calculate",
            Description = "Perform a math calculation",
            Parameters = new PropertyDefinition()
            {
                Type = "object",
                Properties = new Dictionary<string, PropertyDefinition>()
                {
                    { "expression", new PropertyDefinition() { Type = "string", Description = "The math expression to evaluate" } }
                },
                Required = ["expression"]
            }
        }
    }
];

// --- Tool implementations ---
string ExecuteTool(string functionName, JsonElement arguments)
{
    switch (functionName)
    {
        case "get_weather":
            var location = arguments.GetProperty("location")
                .GetString() ?? "unknown";
            var unit = arguments.TryGetProperty("unit", out var u)
                ? u.GetString() ?? "celsius"
                : "celsius";
            var temp = unit == "celsius" ? 22 : 72;
            return JsonSerializer.Serialize(new
            {
                location,
                temperature = temp,
                unit,
                condition = "Sunny"
            });

        case "calculate":
            var expression = arguments.GetProperty("expression")
                .GetString() ?? "";
            try
            {
                var result = new System.Data.DataTable()
                    .Compute(expression, null);
                return JsonSerializer.Serialize(new
                {
                    expression,
                    result = result?.ToString()
                });
            }
            catch (Exception ex)
            {
                return JsonSerializer.Serialize(new
                {
                    error = ex.Message
                });
            }

        default:
            return JsonSerializer.Serialize(new
            {
                error = $"Unknown function: {functionName}"
            });
    }
}

// --- Main application ---
var config = new Configuration
{
    AppName = "foundry_local_samples",
    LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
};

using var loggerFactory = LoggerFactory.Create(builder =>
{
    builder.SetMinimumLevel(
        Microsoft.Extensions.Logging.LogLevel.Information
    );
});
var logger = loggerFactory.CreateLogger<Program>();

await FoundryLocalManager.CreateAsync(config, logger);
var mgr = FoundryLocalManager.Instance;

// Download and register all execution providers.
var currentEp = "";
await mgr.DownloadAndRegisterEpsAsync((epName, percent) =>
{
    if (epName != currentEp)
    {
        if (currentEp != "") Console.WriteLine();
        currentEp = epName;
    }
    Console.Write($"\r  {epName.PadRight(30)}  {percent,6:F1}%");
});
if (currentEp != "") Console.WriteLine();

var catalog = await mgr.GetCatalogAsync();
var model = await catalog.GetModelAsync("qwen2.5-0.5b")
    ?? throw new Exception("Model not found");

await model.DownloadAsync(progress =>
{
    Console.Write($"\rDownloading model: {progress:F2}%");
    if (progress >= 100f) Console.WriteLine();
});

await model.LoadAsync();
Console.WriteLine("Model loaded and ready.");

var chatClient = await model.GetChatClientAsync();
chatClient.Settings.ToolChoice = ToolChoice.Auto;

var messages = new List<ChatMessage>
{
    new ChatMessage
    {
        Role = "system",
        Content = "You are a helpful assistant with access to tools. " +
                  "Use them when needed to answer questions accurately."
    }
};

Console.WriteLine("\nTool-calling assistant ready! Type 'quit' to exit.\n");

while (true)
{
    Console.Write("You: ");
    var userInput = Console.ReadLine();
    if (string.IsNullOrWhiteSpace(userInput) ||
        userInput.Equals("quit", StringComparison.OrdinalIgnoreCase) ||
        userInput.Equals("exit", StringComparison.OrdinalIgnoreCase))
    {
        break;
    }

    messages.Add(new ChatMessage
    {
        Role = "user",
        Content = userInput
    });

    var response = await chatClient.CompleteChatAsync(
        messages, tools, ct
    );

    var choice = response.Choices[0].Message;

    if (choice.ToolCalls is { Count: > 0 })
    {
        messages.Add(choice);

        foreach (var toolCall in choice.ToolCalls)
        {
            var toolArgs = JsonDocument.Parse(
                toolCall.FunctionCall.Arguments
            ).RootElement;
            Console.WriteLine(
                $"  Tool call: {toolCall.FunctionCall.Name}({toolArgs})"
            );

            var result = ExecuteTool(
                toolCall.FunctionCall.Name, toolArgs
            );
            messages.Add(new ChatMessage
            {
                Role = "tool",
                ToolCallId = toolCall.Id,
                Content = result
            });
        }

        var finalResponse = await chatClient.CompleteChatAsync(
            messages, tools, ct
        );
        var answer = finalResponse.Choices[0].Message.Content ?? "";
        messages.Add(new ChatMessage
        {
            Role = "assistant",
            Content = answer
        });
        Console.WriteLine($"Assistant: {answer}\n");
    }
    else
    {
        var answer = choice.Content ?? "";
        messages.Add(new ChatMessage
        {
            Role = "assistant",
            Content = answer
        });
        Console.WriteLine($"Assistant: {answer}\n");
    }
}

await model.UnloadAsync();
Console.WriteLine("Model unloaded. Goodbye!");

Kör verktygssamtalsassistenten:

dotnet run

Du ser utdata som liknar:

Downloading model: 100.00%
Model loaded and ready.

Tool-calling assistant ready! Type 'quit' to exit.

You: What's the weather like today?
  Tool call: get_weather({"location":"current location"})
Assistant: The weather today is sunny with a temperature of 22°C.

You: What is 245 * 38?
  Tool call: calculate({"expression":"245 * 38"})
Assistant: 245 multiplied by 38 equals 9,310.

You: quit
Model unloaded. Goodbye!

Modellen bestämmer när ett verktyg ska anropas baserat på användarens meddelande. För en väderfråga anropas get_weather, för matematik anropas calculate, och för allmänna frågor svarar den direkt utan några verktygsanrop.

Exempellagringsplats

Den fullständiga exempelkoden för den här artikeln finns på lagringsplatsen Foundry Local GitHub. Klona lagringsplatsen och gå till exempelanvändningen:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/js/tutorial-tool-calling

Installera paket

Om du utvecklar eller skickar på Windows väljer du fliken Windows. Windows-paketet integreras med Windows ML-körning – det ger samma API-yta med en bredare bredd av maskinvaruacceleration.

Windows
Plattformsoberoende

npm install foundry-local-sdk-winml openai

npm install foundry-local-sdk openai

Definiera verktyg

Skapa en fil med namnet index.js.

Lägg till följande verktygsdefinitioner:

// --- Tool definitions ---
const tools = [
    {
        type: 'function',
        function: {
            name: 'get_weather',
            description: 'Get the current weather for a location',
            parameters: {
                type: 'object',
                properties: {
                    location: {
                        type: 'string',
                        description: 'The city or location'
                    },
                    unit: {
                        type: 'string',
                        enum: ['celsius', 'fahrenheit'],
                        description: 'Temperature unit'
                    }
                },
                required: ['location']
            }
        }
    },
    {
        type: 'function',
        function: {
            name: 'calculate',
            description: 'Perform a math calculation',
            parameters: {
                type: 'object',
                properties: {
                    expression: {
                        type: 'string',
                        description:
                            'The math expression to evaluate'
                    }
                },
                required: ['expression']
            }
        }
    }
];

// --- Tool implementations ---
function getWeather(location, unit = 'celsius') {
    return {
        location,
        temperature: unit === 'celsius' ? 22 : 72,
        unit,
        condition: 'Sunny'
    };
}

function calculate(expression) {
    // Input is validated against a strict allowlist of numeric/math characters,
    // making this safe from code injection in this tutorial context.
    const allowed = /^[0-9+\-*/(). ]+$/;
    if (!allowed.test(expression)) {
        return { error: 'Invalid expression' };
    }
    try {
        const result = Function(
            `"use strict"; return (${expression})`
        )();
        return { expression, result };
    } catch (err) {
        return { error: err.message };
    }
}

const toolFunctions = {
    get_weather: (args) => getWeather(args.location, args.unit),
    calculate: (args) => calculate(args.expression)
};

Varje verktygsdefinition innehåller en name, en description som hjälper modellen att bestämma när den ska användas och ett parameters schema som beskriver förväntade indata.

Lägg till de JavaScript-funktioner som implementerar varje verktyg:

// --- Tool definitions ---
const tools = [
    {
        type: 'function',
        function: {
            name: 'get_weather',
            description: 'Get the current weather for a location',
            parameters: {
                type: 'object',
                properties: {
                    location: {
                        type: 'string',
                        description: 'The city or location'
                    },
                    unit: {
                        type: 'string',
                        enum: ['celsius', 'fahrenheit'],
                        description: 'Temperature unit'
                    }
                },
                required: ['location']
            }
        }
    },
    {
        type: 'function',
        function: {
            name: 'calculate',
            description: 'Perform a math calculation',
            parameters: {
                type: 'object',
                properties: {
                    expression: {
                        type: 'string',
                        description:
                            'The math expression to evaluate'
                    }
                },
                required: ['expression']
            }
        }
    }
];

// --- Tool implementations ---
function getWeather(location, unit = 'celsius') {
    return {
        location,
        temperature: unit === 'celsius' ? 22 : 72,
        unit,
        condition: 'Sunny'
    };
}

function calculate(expression) {
    // Input is validated against a strict allowlist of numeric/math characters,
    // making this safe from code injection in this tutorial context.
    const allowed = /^[0-9+\-*/(). ]+$/;
    if (!allowed.test(expression)) {
        return { error: 'Invalid expression' };
    }
    try {
        const result = Function(
            `"use strict"; return (${expression})`
        )();
        return { expression, result };
    } catch (err) {
        return { error: err.message };
    }
}

const toolFunctions = {
    get_weather: (args) => getWeather(args.location, args.unit),
    calculate: (args) => calculate(args.expression)
};

Modellen kör inte dessa funktioner direkt. Den returnerar en begäran om verktygsanrop med funktionsnamnet och argumenten, och koden kör funktionen.

Skicka ett meddelande som utlöser verktygsanvändning

Initiera Foundry Local SDK, läs in en modell och skicka ett meddelande som modellen kan svara på genom att anropa ett verktyg.

// --- Main application ---
const manager = FoundryLocalManager.create({
    appName: 'foundry_local_samples',
    logLevel: 'info'
});

// Download and register all execution providers.
let currentEp = '';
await manager.downloadAndRegisterEps((epName, percent) => {
    if (epName !== currentEp) {
        if (currentEp !== '') process.stdout.write('\n');
        currentEp = epName;
    }
    process.stdout.write(`\r  ${epName.padEnd(30)}  ${percent.toFixed(1).padStart(5)}%`);
});
if (currentEp !== '') process.stdout.write('\n');

const model = await manager.catalog.getModel('qwen2.5-0.5b');

await model.download((progress) => {
    process.stdout.write(
        `\rDownloading model: ${progress.toFixed(2)}%`
    );
});
console.log('\nModel downloaded.');

await model.load();
console.log('Model loaded and ready.');

const chatClient = model.createChatClient();

const messages = [
    {
        role: 'system',
        content:
            'You are a helpful assistant with access to tools. ' +
            'Use them when needed to answer questions accurately.'
    }
];

const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout
});

const askQuestion = (prompt) =>
    new Promise((resolve) => rl.question(prompt, resolve));

console.log(
    '\nTool-calling assistant ready! Type \'quit\' to exit.\n'
);

while (true) {
    const userInput = await askQuestion('You: ');
    if (
        userInput.trim().toLowerCase() === 'quit' ||
        userInput.trim().toLowerCase() === 'exit'
    ) {
        break;
    }

    messages.push({ role: 'user', content: userInput });

    const response = await chatClient.completeChat(
        messages, { tools }
    );
    const answer = await processToolCalls(
        messages, response, chatClient
    );

    messages.push({ role: 'assistant', content: answer });
    console.log(`Assistant: ${answer}\n`);
}

await model.unload();
console.log('Model unloaded. Goodbye!');
rl.close();

När modellen fastställer att ett verktyg behövs innehåller tool_calls svaret i stället för ett vanligt textmeddelande. Nästa steg visar hur du identifierar och hanterar dessa anrop.

Kör verktyget och returnera resultat

När modellen svarar med ett verktygsanrop extraherar du funktionsnamnet och argumenten, kör funktionen och skickar tillbaka resultatet.

async function processToolCalls(messages, response, chatClient) {
    let choice = response.choices[0]?.message;

    while (choice?.tool_calls?.length > 0) {
        messages.push(choice);

        for (const toolCall of choice.tool_calls) {
            const functionName = toolCall.function.name;
            const args = JSON.parse(toolCall.function.arguments);
            console.log(
                `  Tool call: ${functionName}` +
                `(${JSON.stringify(args)})`
            );

            const result = toolFunctions[functionName](args);
            messages.push({
                role: 'tool',
                tool_call_id: toolCall.id,
                content: JSON.stringify(result)
            });
        }

        response = await chatClient.completeChat(
            messages, { tools }
        );
        choice = response.choices[0]?.message;
    }

    return choice?.content ?? '';
}

De viktigaste stegen i verktygsanropsloopen är:

Identifiera verktygsanrop – kontrollera response.choices[0]?.message?.tool_calls.
Kör funktionen – parsa argumenten och anropa din lokala funktion.
Returnera resultatet – lägg till ett meddelande med rollen tool och matchande tool_call_id.
Få det slutliga svaret – modellen använder verktygets resultat för att generera ett naturligt svar.

Hantera den fullständiga verktygsanropsloopen

Här är det fullständiga programmet som kombinerar verktygsdefinitioner, SDK-initiering och verktygsanropsloopen till en enda körbar fil.

Skapa en fil med namnet index.js och lägg till följande fullständiga kod:

import { FoundryLocalManager } from 'foundry-local-sdk';
import * as readline from 'readline';

// --- Tool definitions ---
const tools = [
    {
        type: 'function',
        function: {
            name: 'get_weather',
            description: 'Get the current weather for a location',
            parameters: {
                type: 'object',
                properties: {
                    location: {
                        type: 'string',
                        description: 'The city or location'
                    },
                    unit: {
                        type: 'string',
                        enum: ['celsius', 'fahrenheit'],
                        description: 'Temperature unit'
                    }
                },
                required: ['location']
            }
        }
    },
    {
        type: 'function',
        function: {
            name: 'calculate',
            description: 'Perform a math calculation',
            parameters: {
                type: 'object',
                properties: {
                    expression: {
                        type: 'string',
                        description:
                            'The math expression to evaluate'
                    }
                },
                required: ['expression']
            }
        }
    }
];

// --- Tool implementations ---
function getWeather(location, unit = 'celsius') {
    return {
        location,
        temperature: unit === 'celsius' ? 22 : 72,
        unit,
        condition: 'Sunny'
    };
}

function calculate(expression) {
    // Input is validated against a strict allowlist of numeric/math characters,
    // making this safe from code injection in this tutorial context.
    const allowed = /^[0-9+\-*/(). ]+$/;
    if (!allowed.test(expression)) {
        return { error: 'Invalid expression' };
    }
    try {
        const result = Function(
            `"use strict"; return (${expression})`
        )();
        return { expression, result };
    } catch (err) {
        return { error: err.message };
    }
}

const toolFunctions = {
    get_weather: (args) => getWeather(args.location, args.unit),
    calculate: (args) => calculate(args.expression)
};

async function processToolCalls(messages, response, chatClient) {
    let choice = response.choices[0]?.message;

    while (choice?.tool_calls?.length > 0) {
        messages.push(choice);

        for (const toolCall of choice.tool_calls) {
            const functionName = toolCall.function.name;
            const args = JSON.parse(toolCall.function.arguments);
            console.log(
                `  Tool call: ${functionName}` +
                `(${JSON.stringify(args)})`
            );

            const result = toolFunctions[functionName](args);
            messages.push({
                role: 'tool',
                tool_call_id: toolCall.id,
                content: JSON.stringify(result)
            });
        }

        response = await chatClient.completeChat(
            messages, { tools }
        );
        choice = response.choices[0]?.message;
    }

    return choice?.content ?? '';
}

// --- Main application ---
const manager = FoundryLocalManager.create({
    appName: 'foundry_local_samples',
    logLevel: 'info'
});

// Download and register all execution providers.
let currentEp = '';
await manager.downloadAndRegisterEps((epName, percent) => {
    if (epName !== currentEp) {
        if (currentEp !== '') process.stdout.write('\n');
        currentEp = epName;
    }
    process.stdout.write(`\r  ${epName.padEnd(30)}  ${percent.toFixed(1).padStart(5)}%`);
});
if (currentEp !== '') process.stdout.write('\n');

const model = await manager.catalog.getModel('qwen2.5-0.5b');

await model.download((progress) => {
    process.stdout.write(
        `\rDownloading model: ${progress.toFixed(2)}%`
    );
});
console.log('\nModel downloaded.');

await model.load();
console.log('Model loaded and ready.');

const chatClient = model.createChatClient();

const messages = [
    {
        role: 'system',
        content:
            'You are a helpful assistant with access to tools. ' +
            'Use them when needed to answer questions accurately.'
    }
];

const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout
});

const askQuestion = (prompt) =>
    new Promise((resolve) => rl.question(prompt, resolve));

console.log(
    '\nTool-calling assistant ready! Type \'quit\' to exit.\n'
);

while (true) {
    const userInput = await askQuestion('You: ');
    if (
        userInput.trim().toLowerCase() === 'quit' ||
        userInput.trim().toLowerCase() === 'exit'
    ) {
        break;
    }

    messages.push({ role: 'user', content: userInput });

    const response = await chatClient.completeChat(
        messages, { tools }
    );
    const answer = await processToolCalls(
        messages, response, chatClient
    );

    messages.push({ role: 'assistant', content: answer });
    console.log(`Assistant: ${answer}\n`);
}

await model.unload();
console.log('Model unloaded. Goodbye!');
rl.close();

Kör assistenten för verktygsanrop:

node index.js

Du ser utdata som liknar:

Downloading model: 100.00%
Model downloaded.
Model loaded and ready.

Tool-calling assistant ready! Type 'quit' to exit.

You: What's the weather like today?
  Tool call: get_weather({"location":"current location"})
Assistant: The weather today is sunny with a temperature of 22°C.

You: What is 245 * 38?
  Tool call: calculate({"expression":"245 * 38"})
Assistant: 245 multiplied by 38 equals 9,310.

You: quit
Model unloaded. Goodbye!

Exempellagringsplats

Den fullständiga exempelkoden för den här artikeln finns på lagringsplatsen Foundry Local GitHub. Klona lagringsplatsen och gå till exempelanvändningen:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/python/tutorial-tool-calling

Installera paket

Om du utvecklar eller skickar på Windows väljer du fliken Windows. Windows-paketet integreras med Windows ML-körning – det ger samma API-yta med en bredare bredd av maskinvaruacceleration.

Windows
Plattformsoberoende

pip install foundry-local-sdk-winml openai

pip install foundry-local-sdk openai

Definiera verktyg

Skapa en fil med namnet main.py och lägg till följande verktygsdefinitioner:

# --- Tool definitions ---
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city or location"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["location"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "calculate",
            "description": "Perform a math calculation",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {
                        "type": "string",
                        "description": (
                            "The math expression to evaluate"
                        )
                    }
                },
                "required": ["expression"]
            }
        }
    }
]


# --- Tool implementations ---
def get_weather(location, unit="celsius"):
    """Simulate a weather lookup."""
    return {
        "location": location,
        "temperature": 22 if unit == "celsius" else 72,
        "unit": unit,
        "condition": "Sunny"
    }


def calculate(expression):
    """Evaluate a math expression safely."""
    allowed = set("0123456789+-*/(). ")
    if not all(c in allowed for c in expression):
        return {"error": "Invalid expression"}
    try:
        result = eval(expression)
        return {"expression": expression, "result": result}
    except Exception as e:
        return {"error": str(e)}


tool_functions = {
    "get_weather": get_weather,
    "calculate": calculate
}

Varje verktygsdefinition innehåller en name, en description som hjälper modellen att bestämma när den ska användas och ett parameters schema som beskriver förväntade indata.

Lägg sedan till de Python funktioner som implementerar varje verktyg:

# --- Tool definitions ---
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city or location"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["location"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "calculate",
            "description": "Perform a math calculation",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {
                        "type": "string",
                        "description": (
                            "The math expression to evaluate"
                        )
                    }
                },
                "required": ["expression"]
            }
        }
    }
]


# --- Tool implementations ---
def get_weather(location, unit="celsius"):
    """Simulate a weather lookup."""
    return {
        "location": location,
        "temperature": 22 if unit == "celsius" else 72,
        "unit": unit,
        "condition": "Sunny"
    }


def calculate(expression):
    """Evaluate a math expression safely."""
    allowed = set("0123456789+-*/(). ")
    if not all(c in allowed for c in expression):
        return {"error": "Invalid expression"}
    try:
        result = eval(expression)
        return {"expression": expression, "result": result}
    except Exception as e:
        return {"error": str(e)}


tool_functions = {
    "get_weather": get_weather,
    "calculate": calculate
}

Modellen kör inte dessa funktioner direkt. Den returnerar en begäran om verktygsanrop med funktionsnamnet och argumenten, och koden kör funktionen.

Skicka ett meddelande som utlöser verktygsanvändning

Initiera Foundry Local SDK, läs in en modell och skicka ett meddelande som modellen kan svara på genom att anropa ett verktyg.

def main():
    # Initialize the Foundry Local SDK
    config = Configuration(app_name="foundry_local_samples")
    FoundryLocalManager.initialize(config)
    manager = FoundryLocalManager.instance

    # Download and register all execution providers.
    current_ep = ""
    def ep_progress(ep_name: str, percent: float):
        nonlocal current_ep
        if ep_name != current_ep:
            if current_ep:
                print()
            current_ep = ep_name
        print(f"\r  {ep_name:<30}  {percent:5.1f}%", end="", flush=True)

    manager.download_and_register_eps(progress_callback=ep_progress)
    if current_ep:
        print()

    # Select and load a model
    model = manager.catalog.get_model("qwen2.5-0.5b")
    model.download(
        lambda progress: print(
            f"\rDownloading model: {progress:.2f}%",
            end="",
            flush=True
        )
    )
    print()
    model.load()
    print("Model loaded and ready.")

    # Get a chat client
    client = model.get_chat_client()

    # Conversation with a system prompt
    messages = [
        {
            "role": "system",
            "content": "You are a helpful assistant with access to tools. "
                       "Use them when needed to answer questions accurately."
        }
    ]

    print("\nTool-calling assistant ready! Type 'quit' to exit.\n")

    while True:
        user_input = input("You: ")
        if user_input.strip().lower() in ("quit", "exit"):
            break

        messages.append({"role": "user", "content": user_input})

        response = client.complete_chat(messages, tools=tools)
        answer = process_tool_calls(messages, response, client)

        messages.append({"role": "assistant", "content": answer})
        print(f"Assistant: {answer}\n")

    # Clean up
    model.unload()
    print("Model unloaded. Goodbye!")

När modellen fastställer att ett verktyg behövs innehåller tool_calls svaret i stället för ett vanligt textmeddelande. Nästa steg visar hur du identifierar och hanterar dessa anrop.

Kör verktyget och returnera resultat

När modellen svarar med ett verktygsanrop extraherar du funktionsnamnet och argumenten, kör funktionen och skickar tillbaka resultatet.

def process_tool_calls(messages, response, client):
    """Handle tool calls in a loop until the model produces a final answer."""
    choice = response.choices[0].message

    while choice.tool_calls:
        # Convert the assistant message to a dict for the SDK
        assistant_msg = {
            "role": "assistant",
            "content": choice.content,
            "tool_calls": [
                {
                    "id": tc.id,
                    "type": tc.type,
                    "function": {
                        "name": tc.function.name,
                        "arguments": tc.function.arguments,
                    },
                }
                for tc in choice.tool_calls
            ],
        }
        messages.append(assistant_msg)

        for tool_call in choice.tool_calls:
            function_name = tool_call.function.name
            arguments = json.loads(tool_call.function.arguments)
            print(f"  Tool call: {function_name}({arguments})")

            # Execute the function and add the result
            func = tool_functions[function_name]
            result = func(**arguments)
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": json.dumps(result)
            })

        # Send the updated conversation back
        response = client.complete_chat(messages, tools=tools)
        choice = response.choices[0].message

    return choice.content

De viktigaste stegen i verktygsanropsloopen är:

Identifiera verktygsanrop – kontrollera response.choices[0].message.tool_calls.
Kör funktionen – parsa argumenten och anropa din lokala funktion.
Returnera resultatet – lägg till ett meddelande med rollen tool och matchande tool_call_id.
Få det slutliga svaret – modellen använder verktygets resultat för att generera ett naturligt svar.

Hantera den fullständiga verktygsanropsloopen

Här är det fullständiga programmet som kombinerar verktygsdefinitioner, SDK-initiering och verktygsanropsloopen till en enda körbar fil.

Skapa en fil med namnet main.py och lägg till följande fullständiga kod:

import json
from foundry_local_sdk import Configuration, FoundryLocalManager


# --- Tool definitions ---
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city or location"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["location"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "calculate",
            "description": "Perform a math calculation",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {
                        "type": "string",
                        "description": (
                            "The math expression to evaluate"
                        )
                    }
                },
                "required": ["expression"]
            }
        }
    }
]


# --- Tool implementations ---
def get_weather(location, unit="celsius"):
    """Simulate a weather lookup."""
    return {
        "location": location,
        "temperature": 22 if unit == "celsius" else 72,
        "unit": unit,
        "condition": "Sunny"
    }


def calculate(expression):
    """Evaluate a math expression safely."""
    allowed = set("0123456789+-*/(). ")
    if not all(c in allowed for c in expression):
        return {"error": "Invalid expression"}
    try:
        result = eval(expression)
        return {"expression": expression, "result": result}
    except Exception as e:
        return {"error": str(e)}


tool_functions = {
    "get_weather": get_weather,
    "calculate": calculate
}


def process_tool_calls(messages, response, client):
    """Handle tool calls in a loop until the model produces a final answer."""
    choice = response.choices[0].message

    while choice.tool_calls:
        # Convert the assistant message to a dict for the SDK
        assistant_msg = {
            "role": "assistant",
            "content": choice.content,
            "tool_calls": [
                {
                    "id": tc.id,
                    "type": tc.type,
                    "function": {
                        "name": tc.function.name,
                        "arguments": tc.function.arguments,
                    },
                }
                for tc in choice.tool_calls
            ],
        }
        messages.append(assistant_msg)

        for tool_call in choice.tool_calls:
            function_name = tool_call.function.name
            arguments = json.loads(tool_call.function.arguments)
            print(f"  Tool call: {function_name}({arguments})")

            # Execute the function and add the result
            func = tool_functions[function_name]
            result = func(**arguments)
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": json.dumps(result)
            })

        # Send the updated conversation back
        response = client.complete_chat(messages, tools=tools)
        choice = response.choices[0].message

    return choice.content


def main():
    # Initialize the Foundry Local SDK
    config = Configuration(app_name="foundry_local_samples")
    FoundryLocalManager.initialize(config)
    manager = FoundryLocalManager.instance

    # Download and register all execution providers.
    current_ep = ""
    def ep_progress(ep_name: str, percent: float):
        nonlocal current_ep
        if ep_name != current_ep:
            if current_ep:
                print()
            current_ep = ep_name
        print(f"\r  {ep_name:<30}  {percent:5.1f}%", end="", flush=True)

    manager.download_and_register_eps(progress_callback=ep_progress)
    if current_ep:
        print()

    # Select and load a model
    model = manager.catalog.get_model("qwen2.5-0.5b")
    model.download(
        lambda progress: print(
            f"\rDownloading model: {progress:.2f}%",
            end="",
            flush=True
        )
    )
    print()
    model.load()
    print("Model loaded and ready.")

    # Get a chat client
    client = model.get_chat_client()

    # Conversation with a system prompt
    messages = [
        {
            "role": "system",
            "content": "You are a helpful assistant with access to tools. "
                       "Use them when needed to answer questions accurately."
        }
    ]

    print("\nTool-calling assistant ready! Type 'quit' to exit.\n")

    while True:
        user_input = input("You: ")
        if user_input.strip().lower() in ("quit", "exit"):
            break

        messages.append({"role": "user", "content": user_input})

        response = client.complete_chat(messages, tools=tools)
        answer = process_tool_calls(messages, response, client)

        messages.append({"role": "assistant", "content": answer})
        print(f"Assistant: {answer}\n")

    # Clean up
    model.unload()
    print("Model unloaded. Goodbye!")


if __name__ == "__main__":
    main()

Kör verktygsanropsassistenten:

python main.py

Du ser utdata som liknar:

Downloading model: 100.00%
Model loaded and ready.

Tool-calling assistant ready! Type 'quit' to exit.

You: What's the weather like today?
  Tool call: get_weather({'location': 'current location'})
Assistant: The weather today is sunny with a temperature of 22°C.

You: What is 245 * 38?
  Tool call: calculate({'expression': '245 * 38'})
Assistant: 245 multiplied by 38 equals 9,310.

You: quit
Model unloaded. Goodbye!

Exempellagringsplats

Den fullständiga exempelkoden för den här artikeln finns på lagringsplatsen Foundry Local GitHub. Klona lagringsplatsen och gå till exempelanvändningen:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/rust/tutorial-tool-calling

Installera paket

Om du utvecklar eller skickar på Windows väljer du fliken Windows. Windows-paketet integreras med Windows ML-körning – det ger samma API-yta med en bredare bredd av maskinvaruacceleration.

Windows
Plattformsoberoende

cargo add foundry-local-sdk --features winml
cargo add tokio --features full
cargo add tokio-stream anyhow

cargo add foundry-local-sdk
cargo add tokio --features full
cargo add tokio-stream anyhow

Definiera verktyg

Lägg till beroendet serde_json för JSON-hantering:
```
cargo add serde_json
```

Öppna src/main.rs och lägg till följande verktygsdefinitioner:

// --- Tool definitions ---
let tools: Vec<ChatCompletionTools> = serde_json::from_value(json!([
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description":
                "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description":
                            "The city or location"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["location"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "calculate",
            "description": "Perform a math calculation",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {
                        "type": "string",
                        "description":
                            "The math expression to evaluate"
                    }
                },
                "required": ["expression"]
            }
        }
    }
]))?;

Varje verktygsdefinition innehåller en name, en description som hjälper modellen att bestämma när den ska användas och ett parameters schema som beskriver förväntade indata.

Lägg till de Rust-funktioner som implementerar varje verktyg:

// --- Tool definitions ---
let tools: Vec<ChatCompletionTools> = serde_json::from_value(json!([
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description":
                "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description":
                            "The city or location"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["location"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "calculate",
            "description": "Perform a math calculation",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {
                        "type": "string",
                        "description":
                            "The math expression to evaluate"
                    }
                },
                "required": ["expression"]
            }
        }
    }
]))?;

Modellen kör inte dessa funktioner direkt. Den returnerar en begäran om verktygsanrop med funktionsnamnet och argumenten, och koden kör funktionen.

Skicka ett meddelande som utlöser verktygsanvändning

Initiera Foundry Local SDK, läs in en modell och skicka ett meddelande som modellen kan svara på genom att anropa ett verktyg.

// Initialize the Foundry Local SDK
let manager = FoundryLocalManager::create(
    FoundryLocalConfig::new("tool-calling-app"),
)?;

// Download and register all execution providers.
manager
    .download_and_register_eps_with_progress(None, {
        let mut current_ep = String::new();
        move |ep_name: &str, percent: f64| {
            if ep_name != current_ep {
                if !current_ep.is_empty() {
                    println!();
                }
                current_ep = ep_name.to_string();
            }
            print!("\r  {:<30}  {:5.1}%", ep_name, percent);
            io::stdout().flush().ok();
        }
    })
    .await?;
println!();

// Select and load a model
let model = manager
    .catalog()
    .get_model("qwen2.5-0.5b")
    .await?;

if !model.is_cached().await? {
    println!("Downloading model...");
    model
        .download(Some(|progress: f64| {
            print!("\r  {progress:.1}%");
            io::stdout().flush().ok();
        }))
        .await?;
    println!();
}

model.load().await?;
println!("Model loaded and ready.");

// Create a chat client
let client = model
    .create_chat_client()
    .temperature(0.7)
    .max_tokens(512)
    .tool_choice(ChatToolChoice::Auto);

// Conversation with a system prompt
let mut messages: Vec<ChatCompletionRequestMessage> = vec![
    ChatCompletionRequestSystemMessage::from(
        "You are a helpful assistant with access to tools. \
         Use them when needed to answer questions accurately.",
    )
    .into(),
];

När modellen fastställer att ett verktyg behövs innehåller tool_calls svaret i stället för ett vanligt textmeddelande. Nästa steg visar hur du identifierar och hanterar dessa anrop.

Kör verktyget och returnera resultat

När modellen svarar med ett verktygsanrop extraherar du funktionsnamnet och argumenten, kör funktionen och skickar tillbaka resultatet.

println!(
    "\nTool-calling assistant ready! Type 'quit' to exit.\n"
);

let stdin = io::stdin();
loop {
    print!("You: ");
    io::stdout().flush()?;

    let mut input = String::new();
    stdin.lock().read_line(&mut input)?;
    let input = input.trim();

    if input.eq_ignore_ascii_case("quit")
        || input.eq_ignore_ascii_case("exit")
    {
        break;
    }

    messages.push(
        ChatCompletionRequestUserMessage::from(input).into(),
    );

    let mut response = client
        .complete_chat(&messages, Some(&tools))
        .await?;

    // Process tool calls in a loop
    while response.choices[0].message.tool_calls.is_some() {
        let tool_calls = response.choices[0]
            .message
            .tool_calls
            .as_ref()
            .unwrap();

        // Append the assistant's tool_calls message via JSON
        let assistant_msg: ChatCompletionRequestMessage =
            serde_json::from_value(json!({
                "role": "assistant",
                "content": null,
                "tool_calls": tool_calls,
            }))?;
        messages.push(assistant_msg);

        for tc_enum in tool_calls {
            let tool_call = match tc_enum {
                ChatCompletionMessageToolCalls::Function(
                    tc,
                ) => tc,
                _ => continue,
            };
            let function_name =
                &tool_call.function.name;
            let arguments: Value =
                serde_json::from_str(
                    &tool_call.function.arguments,
                )?;
            println!(
                "  Tool call: {}({})",
                function_name, arguments
            );

            let result =
                execute_tool(function_name, &arguments);
            messages.push(
                ChatCompletionRequestToolMessage {
                    content: result.to_string().into(),
                    tool_call_id: tool_call.id.clone(),
                }
                .into(),
            );
        }

        response = client
            .complete_chat(&messages, Some(&tools))
            .await?;
    }

    let answer = response.choices[0]
        .message
        .content
        .as_deref()
        .unwrap_or("");
    let assistant_msg: ChatCompletionRequestMessage =
        serde_json::from_value(json!({
            "role": "assistant",
            "content": answer,
        }))?;
    messages.push(assistant_msg);
    println!("Assistant: {}\n", answer);
}

// Clean up
model.unload().await?;
println!("Model unloaded. Goodbye!");

De viktigaste stegen i verktygsanropsloopen är:

Identifiera verktygsanrop – kontrollera response.choices[0].message.tool_calls.
Kör funktionen – parsa argumenten och anropa din lokala funktion.
Returnera resultatet – lägg till ett meddelande med rollen tool och matchande verktygsanrops-ID.
Få det slutliga svaret – modellen använder verktygets resultat för att generera ett naturligt svar.

Hantera den fullständiga verktygsanropsloopen

Här är det fullständiga programmet som kombinerar verktygsdefinitioner, SDK-initiering och verktygsanropsloopen till en enda körbar fil.

Ersätt innehållet i src/main.rs med följande fullständiga kod:

use foundry_local_sdk::{
    ChatCompletionRequestMessage,
    ChatCompletionRequestSystemMessage,
    ChatCompletionRequestToolMessage,
    ChatCompletionRequestUserMessage,
    ChatCompletionMessageToolCalls,
    ChatCompletionTools, ChatToolChoice,
    FoundryLocalConfig, FoundryLocalManager,
};
use serde_json::{json, Value};
use std::io::{self, BufRead, Write};

// --- Tool implementations ---
fn execute_tool(
    name: &str,
    arguments: &Value,
) -> Value {
    match name {
        "get_weather" => {
            let location = arguments["location"]
                .as_str()
                .unwrap_or("unknown");
            let unit = arguments["unit"]
                .as_str()
                .unwrap_or("celsius");
            let temp = if unit == "celsius" { 22 } else { 72 };
            json!({
                "location": location,
                "temperature": temp,
                "unit": unit,
                "condition": "Sunny"
            })
        }
        "calculate" => {
            let expression = arguments["expression"]
                .as_str()
                .unwrap_or("");
            let is_valid = expression
                .chars()
                .all(|c| "0123456789+-*/(). ".contains(c));
            if !is_valid {
                return json!({"error": "Invalid expression"});
            }
            match eval_expression(expression) {
                Ok(result) => json!({
                    "expression": expression,
                    "result": result
                }),
                Err(e) => json!({"error": e}),
            }
        }
        _ => json!({"error": format!("Unknown function: {}", name)}),
    }
}

fn eval_expression(expr: &str) -> Result<f64, String> {
    let expr = expr.replace(' ', "");
    let chars: Vec<char> = expr.chars().collect();
    let mut pos = 0;
    let result = parse_add(&chars, &mut pos)?;
    if pos < chars.len() {
        return Err("Unexpected character".to_string());
    }
    Ok(result)
}

fn parse_add(
    chars: &[char],
    pos: &mut usize,
) -> Result<f64, String> {
    let mut result = parse_mul(chars, pos)?;
    while *pos < chars.len()
        && (chars[*pos] == '+' || chars[*pos] == '-')
    {
        let op = chars[*pos];
        *pos += 1;
        let right = parse_mul(chars, pos)?;
        result = if op == '+' {
            result + right
        } else {
            result - right
        };
    }
    Ok(result)
}

fn parse_mul(
    chars: &[char],
    pos: &mut usize,
) -> Result<f64, String> {
    let mut result = parse_atom(chars, pos)?;
    while *pos < chars.len()
        && (chars[*pos] == '*' || chars[*pos] == '/')
    {
        let op = chars[*pos];
        *pos += 1;
        let right = parse_atom(chars, pos)?;
        result = if op == '*' {
            result * right
        } else {
            result / right
        };
    }
    Ok(result)
}

fn parse_atom(
    chars: &[char],
    pos: &mut usize,
) -> Result<f64, String> {
    if *pos < chars.len() && chars[*pos] == '(' {
        *pos += 1;
        let result = parse_add(chars, pos)?;
        if *pos < chars.len() && chars[*pos] == ')' {
            *pos += 1;
        }
        return Ok(result);
    }
    let start = *pos;
    while *pos < chars.len()
        && (chars[*pos].is_ascii_digit() || chars[*pos] == '.')
    {
        *pos += 1;
    }
    if start == *pos {
        return Err("Expected number".to_string());
    }
    let num_str: String = chars[start..*pos].iter().collect();
    num_str.parse::<f64>().map_err(|e| e.to_string())
}

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // --- Tool definitions ---
    let tools: Vec<ChatCompletionTools> = serde_json::from_value(json!([
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description":
                    "Get the current weather for a location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description":
                                "The city or location"
                        },
                        "unit": {
                            "type": "string",
                            "enum": ["celsius", "fahrenheit"],
                            "description": "Temperature unit"
                        }
                    },
                    "required": ["location"]
                }
            }
        },
        {
            "type": "function",
            "function": {
                "name": "calculate",
                "description": "Perform a math calculation",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "expression": {
                            "type": "string",
                            "description":
                                "The math expression to evaluate"
                        }
                    },
                    "required": ["expression"]
                }
            }
        }
    ]))?;

    // Initialize the Foundry Local SDK
    let manager = FoundryLocalManager::create(
        FoundryLocalConfig::new("tool-calling-app"),
    )?;

    // Download and register all execution providers.
    manager
        .download_and_register_eps_with_progress(None, {
            let mut current_ep = String::new();
            move |ep_name: &str, percent: f64| {
                if ep_name != current_ep {
                    if !current_ep.is_empty() {
                        println!();
                    }
                    current_ep = ep_name.to_string();
                }
                print!("\r  {:<30}  {:5.1}%", ep_name, percent);
                io::stdout().flush().ok();
            }
        })
        .await?;
    println!();

    // Select and load a model
    let model = manager
        .catalog()
        .get_model("qwen2.5-0.5b")
        .await?;

    if !model.is_cached().await? {
        println!("Downloading model...");
        model
            .download(Some(|progress: f64| {
                print!("\r  {progress:.1}%");
                io::stdout().flush().ok();
            }))
            .await?;
        println!();
    }

    model.load().await?;
    println!("Model loaded and ready.");

    // Create a chat client
    let client = model
        .create_chat_client()
        .temperature(0.7)
        .max_tokens(512)
        .tool_choice(ChatToolChoice::Auto);

    // Conversation with a system prompt
    let mut messages: Vec<ChatCompletionRequestMessage> = vec![
        ChatCompletionRequestSystemMessage::from(
            "You are a helpful assistant with access to tools. \
             Use them when needed to answer questions accurately.",
        )
        .into(),
    ];

    println!(
        "\nTool-calling assistant ready! Type 'quit' to exit.\n"
    );

    let stdin = io::stdin();
    loop {
        print!("You: ");
        io::stdout().flush()?;

        let mut input = String::new();
        stdin.lock().read_line(&mut input)?;
        let input = input.trim();

        if input.eq_ignore_ascii_case("quit")
            || input.eq_ignore_ascii_case("exit")
        {
            break;
        }

        messages.push(
            ChatCompletionRequestUserMessage::from(input).into(),
        );

        let mut response = client
            .complete_chat(&messages, Some(&tools))
            .await?;

        // Process tool calls in a loop
        while response.choices[0].message.tool_calls.is_some() {
            let tool_calls = response.choices[0]
                .message
                .tool_calls
                .as_ref()
                .unwrap();

            // Append the assistant's tool_calls message via JSON
            let assistant_msg: ChatCompletionRequestMessage =
                serde_json::from_value(json!({
                    "role": "assistant",
                    "content": null,
                    "tool_calls": tool_calls,
                }))?;
            messages.push(assistant_msg);

            for tc_enum in tool_calls {
                let tool_call = match tc_enum {
                    ChatCompletionMessageToolCalls::Function(
                        tc,
                    ) => tc,
                    _ => continue,
                };
                let function_name =
                    &tool_call.function.name;
                let arguments: Value =
                    serde_json::from_str(
                        &tool_call.function.arguments,
                    )?;
                println!(
                    "  Tool call: {}({})",
                    function_name, arguments
                );

                let result =
                    execute_tool(function_name, &arguments);
                messages.push(
                    ChatCompletionRequestToolMessage {
                        content: result.to_string().into(),
                        tool_call_id: tool_call.id.clone(),
                    }
                    .into(),
                );
            }

            response = client
                .complete_chat(&messages, Some(&tools))
                .await?;
        }

        let answer = response.choices[0]
            .message
            .content
            .as_deref()
            .unwrap_or("");
        let assistant_msg: ChatCompletionRequestMessage =
            serde_json::from_value(json!({
                "role": "assistant",
                "content": answer,
            }))?;
        messages.push(assistant_msg);
        println!("Assistant: {}\n", answer);
    }

    // Clean up
    model.unload().await?;
    println!("Model unloaded. Goodbye!");

    Ok(())
}

Kör verktygssamtalsassistenten:

cargo run

Du ser utdata som liknar:

Downloading model: 100.00%
Model loaded and ready.

Tool-calling assistant ready! Type 'quit' to exit.

You: What's the weather like today?
  Tool call: get_weather({"location":"current location"})
Assistant: The weather today is sunny with a temperature of 22°C.

You: What is 245 * 38?
  Tool call: calculate({"expression":"245 * 38"})
Assistant: 245 multiplied by 38 equals 9,310.

You: quit
Model unloaded. Goodbye!

Rensa resurser

Modellvikterna finns kvar i din lokala cache efter att du har avladdat en modell. Det innebär att nästa gång du kör programmet utelämnas nedladdningssteget och modellen läses in snabbare. Ingen extra rensning krävs om du inte vill frigöra diskutrymme.

Feedback

Var den här sidan till hjälp?

Last updated on 2026-04-09

Dela via

Självstudie: Skapa en AI-assistent med verktygsanrop

Förutsättningar

Exempellagringsplats

Installera paket

Definiera verktyg

Skicka ett meddelande som utlöser verktygsanvändning

Kör verktyget och returnera resultat

Hantera den fullständiga verktygsanropsloopen

Exempellagringsplats

Installera paket

Definiera verktyg

Skicka ett meddelande som utlöser verktygsanvändning

Kör verktyget och returnera resultat

Hantera den fullständiga verktygsanropsloopen

Exempellagringsplats

Installera paket

Definiera verktyg

Skicka ett meddelande som utlöser verktygsanvändning

Kör verktyget och returnera resultat

Hantera den fullständiga verktygsanropsloopen

Exempellagringsplats

Installera paket

Definiera verktyg

Skicka ett meddelande som utlöser verktygsanvändning

Kör verktyget och returnera resultat

Hantera den fullständiga verktygsanropsloopen

Rensa resurser

Relaterat innehåll

Feedback

Ytterligare resurser