Självstudie: Skapa en röst-till-text-anteckningstagare

Installera paket

Exempellagringsplats

Den fullständiga exempelkoden för den här artikeln finns på lagringsplatsen Foundry Local GitHub. Klona lagringsplatsen och gå till exempelanvändningen:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/cs/tutorial-voice-to-text

Om du utvecklar eller skickar på Windows väljer du fliken Windows. Windows-paketet integreras med Windows ML-körning – det ger samma API-yta med en bredare bredd av maskinvaruacceleration.

dotnet add package Microsoft.AI.Foundry.Local.WinML
dotnet add package OpenAI

dotnet add package Microsoft.AI.Foundry.Local
dotnet add package OpenAI

C#-exemplen på GitHub-lagringsplatsen är förkonfigurerade projekt. Om du skapar från grunden bör du läsa Foundry Local SDK-referensen för mer information om hur du konfigurerar ditt C#-projekt med Foundry Local.

Transkribera en ljudfil

I det här steget läser du in en tal-till-text-modell och transkriberar en ljudfil. Foundry Local SDK använder modellaliaset whisper för att välja den bästa Whisper-varianten för din maskinvara.

• Öppna Program.cs och ersätt innehållet med följande kod för att initiera SDK, läsa in talmodellen och transkribera en ljudfil:

  // Load the speech-to-text model
  var speechModel = await catalog.GetModelAsync("whisper-tiny")
      ?? throw new Exception("Speech model not found");
  
  await speechModel.DownloadAsync(progress =>
  {
      Console.Write($"\rDownloading speech model: {progress:F2}%");
      if (progress >= 100f) Console.WriteLine();
  });
  
  await speechModel.LoadAsync();
  Console.WriteLine("Speech model loaded.");
  
  // Transcribe the audio file
  var audioClient = await speechModel.GetAudioClientAsync();
  var transcriptionText = new StringBuilder();
  
  Console.WriteLine("\nTranscription:");
  var audioResponse = audioClient
      .TranscribeAudioStreamingAsync("meeting-notes.wav", ct);
  await foreach (var chunk in audioResponse)
  {
      Console.Write(chunk.Text);
      transcriptionText.Append(chunk.Text);
  }
  Console.WriteLine();
  
  // Unload the speech model to free memory
  await speechModel.UnloadAsync();
  

  Metoden GetAudioClientAsync returnerar en klient för ljudåtgärder. Metoden TranscribeAudioStreamingAsync strömmar transkriptionssegment när de blir tillgängliga. Du ackumulerar texten så att du kan skicka den till chattmodellen i nästa steg.

Sammanfatta transkriptionen

Använd nu en chattmodell för att organisera den råa transkriptionen i strukturerade anteckningar. qwen2.5-0.5b Läs in modellen och skicka transkriptionen som kontext med en systemprompt som instruerar modellen att skapa rena, sammanfattade anteckningar.

Lägg till följande kod efter transkriptionssteget:

// Load the chat model for summarization
var chatModel = await catalog.GetModelAsync("qwen2.5-0.5b")
    ?? throw new Exception("Chat model not found");

await chatModel.DownloadAsync(progress =>
{
    Console.Write($"\rDownloading chat model: {progress:F2}%");
    if (progress >= 100f) Console.WriteLine();
});

await chatModel.LoadAsync();
Console.WriteLine("Chat model loaded.");

// Summarize the transcription into organized notes
var chatClient = await chatModel.GetChatClientAsync();
var messages = new List<ChatMessage>
{
    new ChatMessage
    {
        Role = "system",
        Content = "You are a note-taking assistant. Summarize " +
                  "the following transcription into organized, " +
                  "concise notes with bullet points."
    },
    new ChatMessage
    {
        Role = "user",
        Content = transcriptionText.ToString()
    }
};

var chatResponse = await chatClient.CompleteChatAsync(messages, ct);
var summary = chatResponse.Choices[0].Message.Content;
Console.WriteLine($"\nSummary:\n{summary}");

// Clean up
await chatModel.UnloadAsync();
Console.WriteLine("\nDone. Models unloaded.");

Systemprompten formar modellens utdataformat. Genom att instruera den att skapa "organiserade, koncisa anteckningar med punktlistor" får du strukturerat innehåll snarare än en rå parafras.

Kombinera till en komplett app

Ersätt innehållet i Program.cs med följande fullständiga kod som transkriberar en ljudfil och sammanfattar transkriptionen:

using Microsoft.AI.Foundry.Local;
using Betalgo.Ranul.OpenAI.ObjectModels.RequestModels;
using Microsoft.Extensions.Logging;
using System.Text;

CancellationToken ct = CancellationToken.None;

var config = new Configuration
{
    AppName = "foundry_local_samples",
    LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
};

using var loggerFactory = LoggerFactory.Create(builder =>
{
    builder.SetMinimumLevel(
        Microsoft.Extensions.Logging.LogLevel.Information
    );
});
var logger = loggerFactory.CreateLogger<Program>();

// Initialize the singleton instance
await FoundryLocalManager.CreateAsync(config, logger);
var mgr = FoundryLocalManager.Instance;

// Download and register all execution providers.
var currentEp = "";
await mgr.DownloadAndRegisterEpsAsync((epName, percent) =>
{
    if (epName != currentEp)
    {
        if (currentEp != "") Console.WriteLine();
        currentEp = epName;
    }
    Console.Write($"\r  {epName.PadRight(30)}  {percent,6:F1}%");
});
if (currentEp != "") Console.WriteLine();

var catalog = await mgr.GetCatalogAsync();

// Load the speech-to-text model
var speechModel = await catalog.GetModelAsync("whisper-tiny")
    ?? throw new Exception("Speech model not found");

await speechModel.DownloadAsync(progress =>
{
    Console.Write($"\rDownloading speech model: {progress:F2}%");
    if (progress >= 100f) Console.WriteLine();
});

await speechModel.LoadAsync();
Console.WriteLine("Speech model loaded.");

// Transcribe the audio file
var audioClient = await speechModel.GetAudioClientAsync();
var transcriptionText = new StringBuilder();

Console.WriteLine("\nTranscription:");
var audioResponse = audioClient
    .TranscribeAudioStreamingAsync("meeting-notes.wav", ct);
await foreach (var chunk in audioResponse)
{
    Console.Write(chunk.Text);
    transcriptionText.Append(chunk.Text);
}
Console.WriteLine();

// Unload the speech model to free memory
await speechModel.UnloadAsync();

// Load the chat model for summarization
var chatModel = await catalog.GetModelAsync("qwen2.5-0.5b")
    ?? throw new Exception("Chat model not found");

await chatModel.DownloadAsync(progress =>
{
    Console.Write($"\rDownloading chat model: {progress:F2}%");
    if (progress >= 100f) Console.WriteLine();
});

await chatModel.LoadAsync();
Console.WriteLine("Chat model loaded.");

// Summarize the transcription into organized notes
var chatClient = await chatModel.GetChatClientAsync();
var messages = new List<ChatMessage>
{
    new ChatMessage
    {
        Role = "system",
        Content = "You are a note-taking assistant. Summarize " +
                  "the following transcription into organized, " +
                  "concise notes with bullet points."
    },
    new ChatMessage
    {
        Role = "user",
        Content = transcriptionText.ToString()
    }
};

var chatResponse = await chatClient.CompleteChatAsync(messages, ct);
var summary = chatResponse.Choices[0].Message.Content;
Console.WriteLine($"\nSummary:\n{summary}");

// Clean up
await chatModel.UnloadAsync();
Console.WriteLine("\nDone. Models unloaded.");

Kör anteckningstagaren:

dotnet run

Du ser utdata som liknar:

Downloading speech model: 100.00%
Speech model loaded.

Transcription:
OK so let's get started with the weekly sync. First, the backend
API is nearly done. Sarah finished the authentication endpoints
yesterday. We still need to add rate limiting before we go to
staging. On the frontend, the dashboard redesign is about seventy
percent complete. Jake, can you walk us through the new layout?
Great. The charts look good. I think we should add a filter for
date range though. For testing, we have about eighty percent code
coverage on the API. We need to write integration tests for the
new auth flow before Friday. Let's plan to do a full regression
test next Tuesday before the release. Any blockers? OK, sounds
like we are in good shape. Let's wrap up.

Downloading chat model: 100.00%
Chat model loaded.

Summary:
- **Backend API**: Authentication endpoints complete. Rate limiting
  still needed before staging deployment.
- **Frontend**: Dashboard redesign 70% complete. New chart layout
  reviewed. Action item: add a date range filter.
- **Testing**: API code coverage at 80%. Integration tests for the
  auth flow due Friday. Full regression test scheduled for next
  Tuesday before release.
- **Status**: No blockers reported. Team is on track.

Done. Models unloaded.

Programmet transkriberar först ljudinnehållet med strömmande utdata och skickar sedan den ackumulerade texten till en chattmodell som extraherar viktiga punkter och organiserar dem i strukturerade anteckningar.

Installera paket

Exempellagringsplats

Den fullständiga exempelkoden för den här artikeln finns på lagringsplatsen Foundry Local GitHub. Klona lagringsplatsen och gå till exempelanvändningen:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/js/tutorial-voice-to-text

Om du utvecklar eller skickar på Windows väljer du fliken Windows. Windows-paketet integreras med Windows ML-körning – det ger samma API-yta med en bredare bredd av maskinvaruacceleration.

npm install foundry-local-sdk-winml openai

npm install foundry-local-sdk openai

Transkribera en ljudfil

I det här steget läser du in en tal-till-text-modell och transkriberar en ljudfil. Foundry Local SDK använder modellaliaset whisper för att välja den bästa Whisper-varianten för din maskinvara.

1. Skapa en fil med namnet app.js.

2. Lägg till följande kod för att initiera SDK: et, läsa in talmodellen och transkribera en ljudfil:

   // Load the speech-to-text model
   const speechModel = await manager.catalog.getModel('whisper-tiny');
   await speechModel.download((progress) => {
       process.stdout.write(
           `\rDownloading speech model: ${progress.toFixed(2)}%`
       );
   });
   console.log('\nSpeech model downloaded.');
   
   await speechModel.load();
   console.log('Speech model loaded.');
   
   // Transcribe the audio file
   const audioClient = speechModel.createAudioClient();
   const transcription = await audioClient.transcribe(
       path.join(__dirname, 'meeting-notes.wav')
   );
   console.log(`\nTranscription:\n${transcription.text}`);
   
   // Unload the speech model to free memory
   await speechModel.unload();
   

   Metoden createAudioClient returnerar en klient för ljudåtgärder. Metoden transcribe accepterar en filsökväg och returnerar ett objekt med en text egenskap som innehåller det transkriberade innehållet.

Sammanfatta transkriptionen

Använd nu en chattmodell för att organisera den råa transkriptionen i strukturerade anteckningar. qwen2.5-0.5b Läs in modellen och skicka transkriptionen som kontext med en systemprompt som instruerar modellen att skapa rena, sammanfattade anteckningar.

Lägg till följande kod efter transkriptionssteget:

// Load the chat model for summarization
const chatModel = await manager.catalog.getModel('qwen2.5-0.5b');
await chatModel.download((progress) => {
    process.stdout.write(
        `\rDownloading chat model: ${progress.toFixed(2)}%`
    );
});
console.log('\nChat model downloaded.');

await chatModel.load();
console.log('Chat model loaded.');

// Summarize the transcription into organized notes
const chatClient = chatModel.createChatClient();
const messages = [
    {
        role: 'system',
        content: 'You are a note-taking assistant. Summarize ' +
                 'the following transcription into organized, ' +
                 'concise notes with bullet points.'
    },
    {
        role: 'user',
        content: transcription.text
    }
];

const response = await chatClient.completeChat(messages);
const summary = response.choices[0]?.message?.content;
console.log(`\nSummary:\n${summary}`);

// Clean up
await chatModel.unload();
console.log('\nDone. Models unloaded.');

Systemprompten formar modellens utdataformat. Genom att instruera den att skapa "organiserade, koncisa anteckningar med punktlistor" får du strukturerat innehåll snarare än en rå parafras.

Kombinera till en komplett app

Skapa en fil med namnet app.js och lägg till följande fullständiga kod som transkriberar en ljudfil och sammanfattar transkriptionen:

import { FoundryLocalManager } from 'foundry-local-sdk';
import { fileURLToPath } from 'url';
import path from 'path';

const __dirname = path.dirname(fileURLToPath(import.meta.url));

// Initialize the Foundry Local SDK
const manager = FoundryLocalManager.create({
    appName: 'foundry_local_samples',
    logLevel: 'info'
});

// Download and register all execution providers.
let currentEp = '';
await manager.downloadAndRegisterEps((epName, percent) => {
    if (epName !== currentEp) {
        if (currentEp !== '') process.stdout.write('\n');
        currentEp = epName;
    }
    process.stdout.write(`\r  ${epName.padEnd(30)}  ${percent.toFixed(1).padStart(5)}%`);
});
if (currentEp !== '') process.stdout.write('\n');

// Load the speech-to-text model
const speechModel = await manager.catalog.getModel('whisper-tiny');
await speechModel.download((progress) => {
    process.stdout.write(
        `\rDownloading speech model: ${progress.toFixed(2)}%`
    );
});
console.log('\nSpeech model downloaded.');

await speechModel.load();
console.log('Speech model loaded.');

// Transcribe the audio file
const audioClient = speechModel.createAudioClient();
const transcription = await audioClient.transcribe(
    path.join(__dirname, 'meeting-notes.wav')
);
console.log(`\nTranscription:\n${transcription.text}`);

// Unload the speech model to free memory
await speechModel.unload();

// Load the chat model for summarization
const chatModel = await manager.catalog.getModel('qwen2.5-0.5b');
await chatModel.download((progress) => {
    process.stdout.write(
        `\rDownloading chat model: ${progress.toFixed(2)}%`
    );
});
console.log('\nChat model downloaded.');

await chatModel.load();
console.log('Chat model loaded.');

// Summarize the transcription into organized notes
const chatClient = chatModel.createChatClient();
const messages = [
    {
        role: 'system',
        content: 'You are a note-taking assistant. Summarize ' +
                 'the following transcription into organized, ' +
                 'concise notes with bullet points.'
    },
    {
        role: 'user',
        content: transcription.text
    }
];

const response = await chatClient.completeChat(messages);
const summary = response.choices[0]?.message?.content;
console.log(`\nSummary:\n${summary}`);

// Clean up
await chatModel.unload();
console.log('\nDone. Models unloaded.');

Kör anteckningstagaren:

node app.js

Du ser utdata som liknar:

Downloading speech model: 100.00%
Speech model downloaded.
Speech model loaded.

Transcription:
OK so let's get started with the weekly sync. First, the backend
API is nearly done. Sarah finished the authentication endpoints
yesterday. We still need to add rate limiting before we go to
staging. On the frontend, the dashboard redesign is about seventy
percent complete. Jake, can you walk us through the new layout?
Great. The charts look good. I think we should add a filter for
date range though. For testing, we have about eighty percent code
coverage on the API. We need to write integration tests for the
new auth flow before Friday. Let's plan to do a full regression
test next Tuesday before the release. Any blockers? OK, sounds
like we are in good shape. Let's wrap up.

Downloading chat model: 100.00%
Chat model downloaded.
Chat model loaded.

Summary:
- **Backend API**: Authentication endpoints complete. Rate limiting
  still needed before staging deployment.
- **Frontend**: Dashboard redesign 70% complete. New chart layout
  reviewed. Action item: add a date range filter.
- **Testing**: API code coverage at 80%. Integration tests for the
  auth flow due Friday. Full regression test scheduled for next
  Tuesday before release.
- **Status**: No blockers reported. Team is on track.

Done. Models unloaded.

Programmet transkriberar först ljudinnehållet och skickar sedan texten till en chattmodell som extraherar viktiga punkter och organiserar dem i strukturerade anteckningar.

Installera paket

Exempellagringsplats

Den fullständiga exempelkoden för den här artikeln finns på lagringsplatsen Foundry Local GitHub. Klona lagringsplatsen och gå till exempelanvändningen:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/python/tutorial-voice-to-text

Om du utvecklar eller skickar på Windows väljer du fliken Windows. Windows-paketet integreras med Windows ML-körning – det ger samma API-yta med en bredare bredd av maskinvaruacceleration.

pip install foundry-local-sdk-winml openai

pip install foundry-local-sdk openai

Transkribera en ljudfil

I det här steget läser du in en tal-till-text-modell och transkriberar en ljudfil. Foundry Local SDK använder modellaliaset whisper för att välja den bästa Whisper-varianten för din maskinvara.

1. Skapa en fil med namnet app.py.

2. Lägg till följande kod för att initiera SDK: et, läsa in talmodellen och transkribera en ljudfil:

   # Load the speech-to-text model
   speech_model = manager.catalog.get_model("whisper-tiny")
   speech_model.download(
       lambda progress: print(
           f"\rDownloading speech model: {progress:.2f}%",
           end="",
           flush=True,
       )
   )
   print()
   speech_model.load()
   print("Speech model loaded.")
   
   # Transcribe the audio file
   audio_client = speech_model.get_audio_client()
   transcription = audio_client.transcribe("meeting-notes.wav")
   print(f"\nTranscription:\n{transcription.text}")
   
   # Unload the speech model to free memory
   speech_model.unload()
   

   Metoden get_audio_client returnerar en klient för ljudåtgärder. Metoden transcribe accepterar en filsökväg och returnerar ett objekt med en text egenskap som innehåller det transkriberade innehållet.

Sammanfatta transkriptionen

Använd nu en chattmodell för att organisera den råa transkriptionen i strukturerade anteckningar. qwen2.5-0.5b Läs in modellen och skicka transkriptionen som kontext med en systemprompt som instruerar modellen att skapa rena, sammanfattade anteckningar.

Lägg till följande kod efter transkriptionssteget:

# Load the chat model for summarization
chat_model = manager.catalog.get_model("qwen2.5-0.5b")
chat_model.download(
    lambda progress: print(
        f"\rDownloading chat model: {progress:.2f}%",
        end="",
        flush=True,
    )
)
print()
chat_model.load()
print("Chat model loaded.")

# Summarize the transcription into organized notes
client = chat_model.get_chat_client()
messages = [
    {
        "role": "system",
        "content": "You are a note-taking assistant. "
                   "Summarize the following transcription "
                   "into organized, concise notes with "
                   "bullet points.",
    },
    {"role": "user", "content": transcription.text},
]

response = client.complete_chat(messages)
summary = response.choices[0].message.content
print(f"\nSummary:\n{summary}")

# Clean up
chat_model.unload()
print("\nDone. Models unloaded.")

Systemprompten formar modellens utdataformat. Genom att instruera den att skapa "organiserade, koncisa anteckningar med punktlistor" får du strukturerat innehåll snarare än en rå parafras.

Kombinera till en komplett app

Skapa en fil med namnet app.py och lägg till följande fullständiga kod som transkriberar en ljudfil och sammanfattar transkriptionen:

from foundry_local_sdk import Configuration, FoundryLocalManager


def main():
    # Initialize the Foundry Local SDK
    config = Configuration(app_name="foundry_local_samples")
    FoundryLocalManager.initialize(config)
    manager = FoundryLocalManager.instance

    # Download and register all execution providers.
    current_ep = ""
    def ep_progress(ep_name: str, percent: float):
        nonlocal current_ep
        if ep_name != current_ep:
            if current_ep:
                print()
            current_ep = ep_name
        print(f"\r  {ep_name:<30}  {percent:5.1f}%", end="", flush=True)

    manager.download_and_register_eps(progress_callback=ep_progress)
    if current_ep:
        print()

    # Load the speech-to-text model
    speech_model = manager.catalog.get_model("whisper-tiny")
    speech_model.download(
        lambda progress: print(
            f"\rDownloading speech model: {progress:.2f}%",
            end="",
            flush=True,
        )
    )
    print()
    speech_model.load()
    print("Speech model loaded.")

    # Transcribe the audio file
    audio_client = speech_model.get_audio_client()
    transcription = audio_client.transcribe("meeting-notes.wav")
    print(f"\nTranscription:\n{transcription.text}")

    # Unload the speech model to free memory
    speech_model.unload()

    # Load the chat model for summarization
    chat_model = manager.catalog.get_model("qwen2.5-0.5b")
    chat_model.download(
        lambda progress: print(
            f"\rDownloading chat model: {progress:.2f}%",
            end="",
            flush=True,
        )
    )
    print()
    chat_model.load()
    print("Chat model loaded.")

    # Summarize the transcription into organized notes
    client = chat_model.get_chat_client()
    messages = [
        {
            "role": "system",
            "content": "You are a note-taking assistant. "
                       "Summarize the following transcription "
                       "into organized, concise notes with "
                       "bullet points.",
        },
        {"role": "user", "content": transcription.text},
    ]

    response = client.complete_chat(messages)
    summary = response.choices[0].message.content
    print(f"\nSummary:\n{summary}")

    # Clean up
    chat_model.unload()
    print("\nDone. Models unloaded.")


if __name__ == "__main__":
    main()

Kör anteckningsprogrammet:

python app.py

Du ser utdata som liknar:

Downloading speech model: 100.00%
Speech model loaded.

Transcription:
OK so let's get started with the weekly sync. First, the backend
API is nearly done. Sarah finished the authentication endpoints
yesterday. We still need to add rate limiting before we go to
staging. On the frontend, the dashboard redesign is about seventy
percent complete. Jake, can you walk us through the new layout?
Great. The charts look good. I think we should add a filter for
date range though. For testing, we have about eighty percent code
coverage on the API. We need to write integration tests for the
new auth flow before Friday. Let's plan to do a full regression
test next Tuesday before the release. Any blockers? OK, sounds
like we are in good shape. Let's wrap up.

Downloading chat model: 100.00%
Chat model loaded.

Summary:
- **Backend API**: Authentication endpoints complete. Rate limiting
  still needed before staging deployment.
- **Frontend**: Dashboard redesign 70% complete. New chart layout
  reviewed. Action item: add a date range filter.
- **Testing**: API code coverage at 80%. Integration tests for the
  auth flow due Friday. Full regression test scheduled for next
  Tuesday before release.
- **Status**: No blockers reported. Team is on track.

Done. Models unloaded.

Programmet transkriberar först ljudinnehållet och skickar sedan texten till en chattmodell som extraherar viktiga punkter och organiserar dem i strukturerade anteckningar.

Installera paket

Exempellagringsplats

Den fullständiga exempelkoden för den här artikeln finns på lagringsplatsen Foundry Local GitHub. Klona lagringsplatsen och gå till exempelanvändningen:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/rust/tutorial-voice-to-text

Om du utvecklar eller skickar på Windows väljer du fliken Windows. Windows-paketet integreras med Windows ML-körning – det ger samma API-yta med en bredare bredd av maskinvaruacceleration.

cargo add foundry-local-sdk --features winml
cargo add tokio --features full
cargo add tokio-stream anyhow

cargo add foundry-local-sdk
cargo add tokio --features full
cargo add tokio-stream anyhow

Transkribera en ljudfil

I det här steget läser du in en tal-till-text-modell och transkriberar en ljudfil. Foundry Local SDK använder modellaliaset whisper för att välja den bästa Whisper-varianten för din maskinvara.

• Öppna src/main.rs och ersätt innehållet med följande kod för att initiera SDK, läsa in talmodellen och transkribera en ljudfil:

  // Load the speech-to-text model
  let speech_model = manager
      .catalog()
      .get_model("whisper-tiny")
      .await?;
  
  if !speech_model.is_cached().await? {
      println!("Downloading speech model...");
      speech_model
          .download(Some(|progress: f64| {
              print!("\r  {progress:.1}%");
              io::stdout().flush().ok();
          }))
          .await?;
      println!();
  }
  
  speech_model.load().await?;
  println!("Speech model loaded.");
  
  // Transcribe the audio file
  let audio_client = speech_model.create_audio_client();
  let transcription = audio_client
      .transcribe("meeting-notes.wav")
      .await?;
  println!("\nTranscription:\n{}", transcription.text);
  
  // Unload the speech model to free memory
  speech_model.unload().await?;
  

  Metoden create_audio_client returnerar en klient för ljudåtgärder. Metoden transcribe accepterar en filsökväg och returnerar ett objekt med ett text fält som innehåller det transkriberade innehållet.

Sammanfatta transkriptionen

Använd nu en chattmodell för att organisera den råa transkriptionen i strukturerade anteckningar. qwen2.5-0.5b Läs in modellen och skicka transkriptionen som kontext med en systemprompt som instruerar modellen att skapa rena, sammanfattade anteckningar.

Lägg till följande kod efter transkriptionssteget, inuti funktionen main:

// Load the chat model for summarization
let chat_model = manager
    .catalog()
    .get_model("qwen2.5-0.5b")
    .await?;

if !chat_model.is_cached().await? {
    println!("Downloading chat model...");
    chat_model
        .download(Some(|progress: f64| {
            print!("\r  {progress:.1}%");
            io::stdout().flush().ok();
        }))
        .await?;
    println!();
}

chat_model.load().await?;
println!("Chat model loaded.");

// Summarize the transcription into organized notes
let client = chat_model
    .create_chat_client()
    .temperature(0.7)
    .max_tokens(512);

let messages: Vec<ChatCompletionRequestMessage> = vec![
    ChatCompletionRequestSystemMessage::from(
        "You are a note-taking assistant. Summarize \
         the following transcription into organized, \
         concise notes with bullet points.",
    )
    .into(),
    ChatCompletionRequestUserMessage::from(
        transcription.text.as_str(),
    )
    .into(),
];

let response = client
    .complete_chat(&messages, None)
    .await?;
let summary = response.choices[0]
    .message
    .content
    .as_deref()
    .unwrap_or("");
println!("\nSummary:\n{}", summary);

// Clean up
chat_model.unload().await?;
println!("\nDone. Models unloaded.");

Systemprompten formar modellens utdataformat. Genom att instruera den att skapa "organiserade, koncisa anteckningar med punktlistor" får du strukturerat innehåll snarare än en rå parafras.

Kombinera till en komplett app

Ersätt innehållet i src/main.rs med följande fullständiga kod som transkriberar en ljudfil och sammanfattar transkriptionen:

use foundry_local_sdk::{
    ChatCompletionRequestMessage,
    ChatCompletionRequestSystemMessage,
    ChatCompletionRequestUserMessage,
    FoundryLocalConfig, FoundryLocalManager,
};
use std::io::{self, Write};

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // Initialize the Foundry Local SDK
    let manager = FoundryLocalManager::create(
        FoundryLocalConfig::new("note-taker"),
    )?;

    // Download and register all execution providers.
    manager
        .download_and_register_eps_with_progress(None, {
            let mut current_ep = String::new();
            move |ep_name: &str, percent: f64| {
                if ep_name != current_ep {
                    if !current_ep.is_empty() {
                        println!();
                    }
                    current_ep = ep_name.to_string();
                }
                print!("\r  {:<30}  {:5.1}%", ep_name, percent);
                io::stdout().flush().ok();
            }
        })
        .await?;
    println!();

    // Load the speech-to-text model
    let speech_model = manager
        .catalog()
        .get_model("whisper-tiny")
        .await?;

    if !speech_model.is_cached().await? {
        println!("Downloading speech model...");
        speech_model
            .download(Some(|progress: f64| {
                print!("\r  {progress:.1}%");
                io::stdout().flush().ok();
            }))
            .await?;
        println!();
    }

    speech_model.load().await?;
    println!("Speech model loaded.");

    // Transcribe the audio file
    let audio_client = speech_model.create_audio_client();
    let transcription = audio_client
        .transcribe("meeting-notes.wav")
        .await?;
    println!("\nTranscription:\n{}", transcription.text);

    // Unload the speech model to free memory
    speech_model.unload().await?;

    // Load the chat model for summarization
    let chat_model = manager
        .catalog()
        .get_model("qwen2.5-0.5b")
        .await?;

    if !chat_model.is_cached().await? {
        println!("Downloading chat model...");
        chat_model
            .download(Some(|progress: f64| {
                print!("\r  {progress:.1}%");
                io::stdout().flush().ok();
            }))
            .await?;
        println!();
    }

    chat_model.load().await?;
    println!("Chat model loaded.");

    // Summarize the transcription into organized notes
    let client = chat_model
        .create_chat_client()
        .temperature(0.7)
        .max_tokens(512);

    let messages: Vec<ChatCompletionRequestMessage> = vec![
        ChatCompletionRequestSystemMessage::from(
            "You are a note-taking assistant. Summarize \
             the following transcription into organized, \
             concise notes with bullet points.",
        )
        .into(),
        ChatCompletionRequestUserMessage::from(
            transcription.text.as_str(),
        )
        .into(),
    ];

    let response = client
        .complete_chat(&messages, None)
        .await?;
    let summary = response.choices[0]
        .message
        .content
        .as_deref()
        .unwrap_or("");
    println!("\nSummary:\n{}", summary);

    // Clean up
    chat_model.unload().await?;
    println!("\nDone. Models unloaded.");

    Ok(())
}

Kör anteckningstagaren:

cargo run

Du ser utdata som liknar:

Downloading speech model: 100.00%
Speech model loaded.

Transcription:
OK so let's get started with the weekly sync. First, the backend
API is nearly done. Sarah finished the authentication endpoints
yesterday. We still need to add rate limiting before we go to
staging. On the frontend, the dashboard redesign is about seventy
percent complete. Jake, can you walk us through the new layout?
Great. The charts look good. I think we should add a filter for
date range though. For testing, we have about eighty percent code
coverage on the API. We need to write integration tests for the
new auth flow before Friday. Let's plan to do a full regression
test next Tuesday before the release. Any blockers? OK, sounds
like we are in good shape. Let's wrap up.

Downloading chat model: 100.00%
Chat model loaded.

Summary:
- **Backend API**: Authentication endpoints complete. Rate limiting
  still needed before staging deployment.
- **Frontend**: Dashboard redesign 70% complete. New chart layout
  reviewed. Action item: add a date range filter.
- **Testing**: API code coverage at 80%. Integration tests for the
  auth flow due Friday. Full regression test scheduled for next
  Tuesday before release.
- **Status**: No blockers reported. Team is on track.

Done. Models unloaded.

Programmet transkriberar först ljudinnehållet och skickar sedan texten till en chattmodell som extraherar viktiga punkter och organiserar dem i strukturerade anteckningar.