Multi-Agents Chatbot

GPT-trainer is the first and most powerful LLM-native conversational AI platform that implements a multi-agent architecture. Multiple AI Agents work together in a coordinated orchestration to enable you to create solutions with unprecedented versatility, speed, and automation.

We present this guide to help you understand how our Agents work together, as well as share with you some best practices surrounding multi-agent chatbot design.

GPT-trainer has 2 types of Agents:

• User-facing: Agent directly interacts with users conversationally in a Q&A fashion. Only a single user-facing Agent engages with the user when a new query is input into the chatbot.

• Background: Agent never interacts with users directly and instead monitors the conversation in an ongoing fashion. All background Agents are run whenever the user submits a new query to the chatbot.

All Agents have two states:

• Active: Agent is “Connected”. It is live and running.
• Inactive: Agent is “Disabled”. It does not do anything.

Since only a single user-facing Agent engages with the user at any given time, our system must choose the best user-facing Agent based on their respective specializations. To do this, we have our own AI Supervisor working in the background to conduct intent classification.

Since the only reliable source of information our AI Supervisor has when determining which Agent is best suited to perform which tasks is the “Agent Description”, it is absolutely crucial that you provide an explicit and clear description for every User Facing Agent.

Good Agent descriptions should answer the question “What type of queries should the Agent handle?” More specifically, Agent descriptions should encompass all relevant “user intents” that it is supposed to handle. Here are some examples of good Agent descriptions:

Agent descriptions should not include details about “how the Agent performs its job”. That part should be reserved for the Agent’s base prompt. Text such as “Agent provides informed answers based on knowledge from the source library.” is not needed and can be distracting to the AI Supervisor.

If you have specific “user intents” that you’d like an Agent to handle, mention it explicitly inside the Agent description. Intents can look like follows:

• Seeking clarification on the classification of a specific item

• Seeking advice or recommendations

• Requesting creative content

• Asking about future features or capabilities

• Seeking information regarding product pricing

• Seeking information on a technical concept

• Requesting assistance with academic or educational materials

• Providing contact information

• Updating contact information

Depending what you design the Agent to handle, your own custom list of intents may look very different.

So what makes for a good multi-agent chatbot design? How can I ensure that all my intended user intents are handled by the appropriate Agent?

When designing a multi-agent setup, we recommend keeping to a “MECE” approach. “MECE” refers to - Mutually Exclusive, Collectively Exhaustive. In other words, no gaps, no overlaps.

Ideally, you want the situation where:

User-facing Agent A handles:

• intent class 1
• intent class 2
• intent class 3
• etc.

User-facing Agent B handles:

• intent class 10
• intent class 11
• intent class 12
• etc.

and:

There is minimal overlap in intents among Agents. All your Agents combined should cover all possible intents your users might have.

But since you can’t ever account for all intents in the universe, and given the fact that intents are presented in natural language which can have multiple permutations to express the same meaning, there is no deterministic way to get 100% consistency. This is true for all chatbots that rely on “intent classification” workflows, regardless of whether they are LLM-powered or take a traditional “flowchart” approach.

In our AI Supervisor routing design, we use an algorithm that balances latency, consistency, and comprehensiveness. It will not always give you 100% accuracy, but you can get pretty close if you design your chatbots effectively.

​

Debugging the chatbot

If you notice that the chatbot is not responding to your expectations, then there may be two causes.

1. The wrong Agent may have been assigned to handle the incoming user query.
2. The Agent itself may not be optimally configured.

To find out whether your intended Agent has been selected to handle the user’s query, go to “Inbox” and turn on debug mode.

Under “Active Agent”, you will see the Agent that generated the response. Please note that you will only see “Active Agent” metadata if you have more than one active user-facing Agents.

If the Agent that handled the user query was not the correct one, you can return to Agent configuration and update its description. You should also include, as explicitly as possible, the intent associated with the user query within the description of the Agent that should have handled it. This will ensure that in the future, queries of this type would be more consistently funned to the correct Agent.

If the correct Agent handled the user query but the response was not what you expected, then the issue resides within the Agent itself. You will then need to update the Agent configuration. The most straightforward way to fix a response is by revising the AI’s answer directly:

This ensures that the chatbot will generate an answer based on what you revised every time it encounters the question in the future, assuming the same Agent was chosen to respond to the query.

If you’d like to make the Agent more robust and consistent in general, you can curate the training data, choose a more powerful LLM, or be more explicit in your base prompt. To learn some best practices for training the chatbot, see best practices for preparing training data

Happy building!