Fundamentals & Environment Setup

In this module, we explore how to build applications using Large Language Models (LLMs) and Semantic Kernel, Microsoft’s open-source orchestration framework for AI.

LLMs and Semantic Kernel

Semantic Kernel enables you to:

Build AI-powered applications using LLMs
Create intelligent agents that can understand and respond to user queries
Integrate with tools to extend agent capabilities
Manage conversations with built-in chat history support

Once you understand the fundamentals, you can:

Create conversational agents that understand context
Use tools and functions to extend agent capabilities
Build production-ready applications with proper architecture
Implement conversation history and state management

We don’t cover LLMs in great depth throughout the labs. If you’re interested in learning more, please read the reference documentation.

Semantic Kernel has a specific architectural pattern that makes it exceptionally strong for building agents. You can learn more about this pattern in the reference documentation.

Labs

This hands-on workshop gradually builds understanding and experience with LLMs and Semantic Kernel.

Lab 1: Setting up the lab environment

Goal: Learn how to configure Semantic Kernel

Before we start coding our first agent, we need to talk about the lab environment. The labs for the workshop are located in the labs folder.

Exercise: Configure the project

Copy the starter files for the lab to a new directory. Each of the labs has a starter and final folder. Make sure to copy the starter folder somewhere so you can edit the contents of the lab. You can find the files for this lab in code/module-01.
Configure the connection string for the LLM. After you’ve copied over the contents of the lab, run the following commands in a terminal to configure the connection string for the language model:
Terminal window
```
cd hosting/InfoSupport.AgentWorkshop.AppHost
dotnet user-secrets set "ConnectionStrings:languagemodel" "<connection string>"
```
Replace the <connection string> placeholder with the connection string you received in the email from us! The connection string has the format Endpoint=...;Key=..... The Endpoint refers to the URL of the Azure OpenAI resource. The Key refers to the API key.
Configure the Semantic Kernel package. Run the following command from apps/chat/InfoSupport.AgentWorkshop.Chat to add the Semantic Kernel package:
Terminal window
```
dotnet add package Microsoft.SemanticKernel
```
Configure the Kernel object in the chat application. Add the following code to apps/chat/InfoSupport.AgentWorkshop.Chat/Program.cs below the call to builder.AddServiceDefaults():
```
builder.AddAzureOpenAIClient("languagemodel");

builder.Services.AddKernel()
    .AddAzureOpenAIChatCompletion(builder.Configuration["LanguageModel:ChatCompletionDeploymentName"]!)
    .AddAzureOpenAITextEmbeddingGeneration(builder.Configuration["LanguageModel:TextEmbeddingDeploymentName"]!);
```
We use Aspire to orchestrate the different parts of the application. This saves us a lot of time.
The call to AddAzureOpenAIClient uses a connection string that we configured in the application host project. The Aspire application host project automatically injects this connection string in the correct places based on how you configured the Azure OpenAI service resource in the application host.
If you’re interested in learning more about the reasons why we use Aspire, make sure to read the reference documentation

Lab 2: Creating Your First Agent

Goal: Learn how to create a basic agent using Semantic Kernel.

In this lab, we’ll build a basic conversational agent using Semantic Kernel. We’ll start with a simple system prompt and gradually add more capabilities.

Exercise: Configure agent instructions

Open apps/chat/InfoSupport.AgentWorkshop.Chat/instructions.txt to understand the agent’s instructions.

The instructions should look like this:
```
You're a friendly AI. Your name is Mike.
```
Right now, the instructions are pretty basic. Don’t worry, we’ll expand these in the next labs to a more fully featured set of agent instructions. The agent instructions determine how the agent will respond to user queries.
Add the following code at the start of the AgentCompletionService class in apps/chat/InfoSupport.AgentWorkshop.Chat/Services/AgentCompletionService.cs to define the agent:
```
private readonly ChatCompletionAgent _chatCompletionAgent = new()
{
    Name = "Mike",
    Instructions = EmbeddedResource.Read("instructions.txt"),
    Kernel = kernel
};
```
This code defines an agent with a name, instructions, and the kernel instance. The agent loads its instructions from the instructions.txt file we just opened.

Exercise: Generating a response

Next, we need to implement the GenerateResponseAsync method. This method is responsible for generating a streaming response to a question from the user.

First, we need to retrieve a conversation or create a new one.

When you chat with a digital assistant, you typically want to do that in the context of a conversation. We’re using a Conversation class located in apps/chat/InfoSupport.AgentWorkshop.Chat/Models with a collection of messages and other metadata. Before generating an actual response, we should look up an existing conversation or create one if there’s none with the specified threadId.

Add the following code to the GenerateResponseAsync method:
```
var conversation = await applicationDbContext.Conversations
   .SingleOrDefaultAsync(x => x.ThreadId == request.ThreadId)
   .ConfigureAwait(false);

if (conversation is null)
{
   conversation = new Conversation(request.ThreadId);
   await applicationDbContext.AddAsync(conversation);
}
```
The frontend proactively generates an identifier for each conversation that you start even if you never submit a prompt to the server. Creating a new identifier doesn’t mean a conversation exists, it is only created once you submit the first prompt to the server. The server records the generated thread ID in the database so the frontend can make the assumption that there’s a conversation with the specified thread ID.
After looking up the conversation data, we need to create a ChatHistoryAgentThread to contain the collection of messages that the language model should generate a response for. The LLM doesn’t know conversations at all. It only knows a sequence of tokens. The ChatHistoryAgentThread is a way for us to transport a sequence of chat messages and get back a response sequence.

Add the following lines to the GenerateResponseAsync method to create the agent thread:
```
var chatHistory = BuildChatHistory(conversation);
ChatHistoryAgentThread? thread = new ChatHistoryAgentThread(chatHistory);
```
This code uses a helper method BuildChatHistory to fill the thread with data.
Add the following code to the end of the AgentCompletionService class to implement the BuildChatHistory method:
```
private ChatHistory BuildChatHistory(Conversation conversation)
{
    var chatHistory = new ChatHistory();

    foreach (var message in conversation.Messages.OrderBy(x => x.Timestamp))
    {
        if (message.Author == ConversationMessageRole.Assistant)
        {
            chatHistory.AddAssistantMessage(message.Content);
        }
        else
        {
            chatHistory.AddUserMessage(message.Content);
        }
    }

    return chatHistory;
}
```
Agents require an agent thread. When you use a service like Azure AI Agents, the server maintains the thread for you. Since we’re building a chat history agent, we must maintain the thread content ourselves.
The ChatHistoryAgentThread uses the ChatHistory class to store the messages in the thread. You can think of the ChatHistory class as a collection of messages ordered by timestamp. The ChatHistory class is a simple wrapper around a list of messages.
If you’re interested in using Azure AI Agents, check out the documentation.
After creating the thread for the agent, we can record the submitted prompt with the conversation and invoke the agent.

Add the following lines to the GenerateResponseAsync method:
```
conversation.AppendUserMessage(request.Prompt);

var responseStream = _chatCompletionAgent.InvokeStreamingAsync(request.Prompt, thread);
```
The response stream here is an IAsyncEnumerable — an enumerable stream of LLM-generated content. If you don’t like streaming you can use InvokeAsync instead. For this workshop, we use streaming as it provides a nicer user experience.

Add the following code to the GenerateResponseAsync method to record the assistant response.

Our agent doesn’t track chat history, so we need to do that ourselves. Add the following code to record the response and forward the chunks to the caller:

var assistantResponseMessageBuilder = new StringBuilder();

await foreach (var chunk in responseStream)
{
    assistantResponseMessageBuilder.Append(chunk.Message.Content);

    // The agent sometimes produces empty chunks, these add no value and we should
    // filter them out here to prevent bad things from happening in the frontend.
    if (!string.IsNullOrEmpty(chunk.Message.Content))
    {
        yield return chunk;
    }
}

conversation.AppendAssistantResponse(assistantResponseMessageBuilder.ToString());

Add the following code to the GenerateResponseAsync method to save the conversation with the new user prompt and record the assistant response.
```
await applicationDbContext.SaveChangesAsync().ConfigureAwait(false);
```

The agent can now talk to us. If you start the application host in hosting/InfoSupport.AgentWorkshop.AppHost with dotnet run you can access the frontend at http://localhost:3000 and talk to the agent!

Summary and next steps

In this module, we covered the basic concepts of configuring Semantic Kernel and connecting to an LLM. We then covered how to build a basic agent with custom instructions and how to handle a conversation.

In the next module, we’ll extend the basic agent with the capacity to index documents that the agent can later search through to give answers to questions that users may have about Semantic Kernel.