How can an OpenAI Chatbot Solution be built into Teams - The Architectual Structure

Authors: Rongwei Ma and Saska Saarioja

Following the introduction to company-specific AI Chatbots built into Teams , this second part delves into the architectural structures of AI Chatbots, evaluating the pros and cons of each design option provided by Microsoft.

Architecture Structure

Microsoft presents various architectural choices for integrating OpenAI Chatbot solutions with Teams. Cloud1 focuses primarily on architectures 2 and 3 as we are more interested in exploring and understanding the value and possibilities of Azure OpenAI in AI Chatbot. We briefly introduce Option 1 here and more detailed analysis centers on Options 2 and 3, with evaluating their potential benefits and limitations.

OpenAIChatbotblog2

Microsoft Azure AI Chatbot Architecture Options [1]

Option 1: Microsoft Copilot Studio + AI Builder

This option provides a convenient approach using Microsoft Copilot Studio, Power Automate and AI Builder. The AI Builder provides several ready-packaged models such as key phrase extracting model, category classification model, extracting text in photos and PDF documents (OCR), sentiment analysis model and text recognition/translation model [2]. This option would be best fit for the use cases particularly interested in those readymade models and build the AI Chatbot based on them.

Option 2: Microsoft Copilot Studio + Power Automate

Option 2 is the approach based on Microsoft Copilot Studio, Power Automate, and Azure OpenAI. Microsoft Copilot Studio extends the capabilities to build the bot using low-code and drag-and-drop tools. In addition, it offers a platform that enables the creation of diverse AI-powered chatbots, ranging from simple Q&A to complex conversations. PVA is simple to deploy into the Teams chat and there are different channels provided by PVA such as mobile apps, Facebook, Microsoft Teams, or any channel supported by the Azure Bot Framework [3].

The chatbot logic is defined in PVA. The PVA can Call an action of the Power Automate flow [3], Azure OpenAI or Azure AI Search can be called here as an HTTP request. Through Teams chat, users ask questions and move to PVA and further enter to Power Automate flow with Azure OpenAI and send the result back to the user. If the user would like to access the company internally, Azure AI Search would be needed before Azure OpenAI so that the search index can find corresponding information to Azure OpenAI.

This approach in general is convenient and easy to implement. The drag-and-drop is user-friendly and fast to develop. For people who don’t know the code can quickly learn and build a chatbot. Knowing the logic would be the key factor here. When the needs are complicated or demand various functionalities, Power Automate can connect to programming code to develop sophisticated functions. With Office 365 licenses (E1, E3, or E5), the chatbot is free to use through the Teams channel. However, the chatbot may still require the designer to know coding complicated functions, as well as this option may have slightly higher monthly costs compared to option 3.

A screenshot of a computer

Description automatically generated

Cloud1 Copilot Studio example

Option 3: Micorosft Bot Framework - Python/C# Custom Development

The Microsoft Bot Framework, coupled with Azure AI Bot Service, offers a comprehensive environment for building and managing AI bots, supporting popular programming languages and various channels. Microsoft Bot Framework offers an open-source, free-to-use framework for building enterprise-grade conversation AI experiences that allow secure control of the data. Along with Azure AI Bot Service, it provides tools to build, test, deploy, and manage intelligent bots.

The open-source software development kit (SDK) and tools can easily connect the bot to several popular channels like Teams, Alexa, Facebook, Skype, Slack, Telegram, Web chat, etc. The components include Bot framework, web search and Azure OpenAI and Teams. This Bot framework allows developing the Chatbot by C#, Java, JavaScript or Python languager[5].

The architecture is based on the Bot Framework SDK and uses Azure Web App Service to host the HTTP-based web application. Azure OpenAI or Azure AI Search are embedded into Bot Framework. The Bot Framework Emulator allows local testing and after deploying the bot into Azure Cloud, further tests can be done in Azure Bot service “Test in Web Chat” function and integrate into Microsoft Teams in “Channels”. The bot identity can be managed in Azure in a few different ways.

As a user-assigned managed identity (C# and JavaScript support)
As a single-tenant app. (C# and JavaScript support)
As a multi-tenant app. (C#, JavaScript, Java and Python support)

Microsoft Bot Framework SDK offers a collection of libraries, tools, and services to let the user continue developing based on the existing framework. The SDK framework offers flexibility and is easy to attach with different libraries like Langchain/Semantic kernel to do further development. It also offers a cheap price to use monthly. On the other hand, this option would require more programming understanding and take more time to develop.

image-png-Dec-20-2023-08-01-09-6573-AM

Test in Web Chat Azure Bot

In this two-part series exploring the integration and utilization of AI Chatbots in business environments, an understanding of the transformative role of AI in technology, particularly through Azure OpenAI was formed as well as a detailed analysis of various architectural structures for an AI Chatbot integration in Microsoft Teams provided.

In the first part, we emphasized the potential of Azure OpenAI, including GPT-4, in revolutionizing business interactions with technology. The implementation of an AI Chatbot Proof of Concept (PoC) tailored for specific departmental needs highlighted the versatility and security these chatbots bring to internal communications and data access. The second part offered a deep dive into the architectural choices provided by Microsoft for integrating OpenAI Chatbot solutions with Teams. With a primary focus on architecture 2 and 3, we assessed their pros and cons. The options ranged from using basic Azure AI services to more complex structures involving Microsoft Copilot Studio, Power Automate, and the Microsoft Bot Framework.

Based on Cloud1’s experience, we understand that each architectural option comes with its unique set of strengths and challenges. The choice of architecture depends on specific business requirements and technical capabilities. This series aimed to empower businesses with the necessary knowledge to make informed decisions in adopting AI Chatbot technologies, ensuring they align with the organizational goals and enhance operational efficiency.