Introduction

In part 1 of this article, I described how you can use Adaptive Cards to collect end-user feedback in a non-annoying way. The end user can choose to give feedback only if they want, but can just as easily not do so without affecting the flow of the conversation with the Copilot Studio agent.

The description in part 1 applies if you have Generative AI set to Classic. If your agent is in Norwegian this will be the case, as the Generative AI setting is only available in English for now.

NB

Microsoft have changed something that makes links to citations not work properly in Microsoft teams when following this article.
Please see comment from jelle here for a change that makes citations in teams work again.

If your agent uses English and has turned on generative orchestration instead of Classic, then the procedure is slightly different for collecting user feedback.

When generative orchestration is turned on, the agent uses AI to determine which combination of topics, actions, and knowledge can best solve what the user is asking for, also called the Plan.

Generative orchestration gives us as building agents in Copilot Studio three new topic triggers that can be used to connect to this orchestration process to adjust or modify the process, for example to connect to feedback forms from this article.

The three new topic triggers are:

Triggered by Agent:

Triggered by Agent trigger right after the Generative Orchestrator gets the message from the user. Here you can describe in natural language when you want this topic to trigger, it can be for example when you want to control how the agent should respond to a certain topic, this can be opening hours, booking tickets or any topic you want to control beyond what generative AI answers

Plan complete:

Plan complete trigger after generative orchestration has executed its plan. This is before the response to the user is generated. This gives you an opportunity to add some middleware, for example, you can log information, or set up variables for later use, etc.

AI Response Generated:

AI Response Generated trigger after generative orchestration has executed the plan, but before the response has been sent to the user. Here you can insert some logic before the response is sent to the user, this provides great opportunities to enrich the response before sending it to the end user. For instance, here you can use the AI prompt and remove personal information like names, addresses, etc., from the response. This is also the topic trigger we will use when we enter the “Do you want to give feedback on the chatbot?” button in this article.

Create a Feedback button topic

We create a new topic, name it “Feedback button”. Change the trigger to “AI response generated”.

This is our topic that will trigger before a response is sent to the user. Here we will cancel the response, add an adaptive card button to send the user to a form for feedback.

Add a new node, “Set variable value” and select the system variable “ContinueResponse” and set the value to “false”.

This will cancel the response so that we can add our button.

Add a new node “Send a message”

Add the system variable “Resonse.FormattedText” to the message field, this is the response from AI that we interrupted in the previous step. The reason we do this is to make the adaptive card button appear below the response of the AI instead of above.

Click Add in the “Send a message” node and add an “Adaptive card”, select “Formula card” and paste the adaptive card code below.

{
 type: "AdaptiveCard",
 '$schema': "https://adaptivecards.io/schemas/adaptive-card.json",
 version: "1.5",
 Body: [
    {
 type: "ActionSet",
 actions: [
        {
 type: "Action.Submit",
 title: "Would you like to give a feedback on the chatbot?",
 data: {
 MessageEvent: "FeedbackResponse",
 Prompt: System. LastMessage.Text,
 Response: System. Response.FormattedText
            
          }
        }
      ]
    }
  ]
}

This adaptive card shows a link with the text “Do you want to give feedback on the chatbot?” when the user clicks on the link, a MessageEvent named “FeedbackResponse” is sent, this contains the user’s question in the “Prompt” property and generative AI’s response in the “Response” property

Create Feedback topic

We have now created the link that the user can click on to give feedback on the chatbot. Now we will create a new topic or topic where the user will get a dialog box they can write their feedback in.

Create a new topic manually and name it “Feedback.” In order for this topic to be triggered, we need to change the trigger to “message received”. Click on the symbol with the arrows and change the trigger to “message received”. Now you have a topic that will trigger on all messages sent from the user, we want to filter so that this topic is only triggered when they click on our “Do you want to give feedback on the chatbot?” link.

Click Edit on the “message received” trigger, select the “activity type” Message, set Condition to Formula and enter the following PowerFX script on “Function”:

! IsBlank(System.Activity.Value.MessageEvent) && Text(System.Activity.Value.MessageEvent) = "FeedbackResponse"

This means that this topic is only triggered when the user clicks on our link that sends a MessageEvent named “FeedbackResponse”.

Add a new node, select “Parse value” under “Variable management”

Here we will “parse” the system variable “Activity.Value”, it will contain the entire message we sent from our “Would you like to give a feedback on the chatbot?” link, the question the user asked and the answer generative AI gave.

In “Parse Value”, select “Activity.Value” under system and in Data type, select “from sample data”. Select “Get schema from sample JSON” and paste the following JSON:

{
"MessageEvent": "MyFeedback";
"Prompt": "my prompt";
"Response": "Response from gene AI"

}

In “Save as”, insert a new variable, call this VarFeedback. We can use this later so that we can see what questions and answers the user is giving feedback on.

Then add another node of type “Ask with adaptive card”. Paste the following adaptive card JSON:

{
"type": "AdaptiveCard",
'$schema': 'http://adaptivecards.io/schemas/adaptive-card.json';
"version": "1.3",
"body": [
{
"type": "TextBlock",
"text": "feedback";
"size": "large",
"wrap": true,
"weight": "Bolder"
            
},
{
"type": "Input.Text",
"placeholder": "Post your feedback comments here",
"isMultiline": true,
"id": "12",
"height": "stretch"
}
],
"Actions": [
{
"type": "Action.Submit",
"title": "Send feedback",
"associatedInputs": "auto",
"id": "11"
}
]
}

This will provide users with a text box where they can write their feedback. Create a new variable for the output and call it VarUsersFeedback.

This variable will contain what the user writes in the feedback box.

Summary

Now we’ve created an adaptive card with a link that says “Would you like to give feedback on the chatbot?” that pops up every time generative AI answers a question. The user can choose to click on the link to post a feedback or choose not to.

If the user clicks on the link, the user is shown an adaptive card where it is possible to enter feedback and click send.

In this article, we have not used this information for anything yet, but here you are free to save the information where you want. For example, you can store your feedback in Dataverse using Power Automate, or you can send the information to Azure Application Insights for further analysis.

For now, we’re just adding a node to print the information in chat so you can see that everything is working as intended.

After the feedback adaptive card, add a node of type “Send a message”. In it, we print the variables “VarFeedback.Prompt” and “VarFeedback.Response”, which contain the question the user asked and answered generative AI.

In addition, we print “VarUsersFeedback” which will be the feedback the user gave.

If you now save a topic and run a test, you will see that we have created a solution where the user can give feedback in an easy way if they want.

More Information

(3) Understanding Generative Orchestration Topic Triggers in Copilot Studio – YouTube

Ask with Adaptive Cards – Microsoft Copilot Studio | Microsoft Learn

Orchestrate agent behavior with generative AI – Microsoft Copilot Studio | Microsoft Learn