Hi @Luka Dujmovic
Fred here from the Automated and Proactive Support Team at Intercom.
Thank you for your question on conversation analysis.
It seems like you have explored many of the options available within Intercom and would like a bulk view of well organized and structured insights - Almost like a pivot table that can help with analyzing improvements. The reality is that this technology is so new within the Customer Support space, we’re still figuring it out ourselves!
Although there are ways to automate and focus on specific areas that can be improved (AI helps with that), the best analysis still requires some human input to review and assess Fin performance. Here are some of the effective strategies we employ at Intercom:
- The Optimize tab in Fin AI Agent - This feature allows you to see unresolved questions which are grouped by AI into topics and can be organized by volume, abandonment rate, or routed to team rate. These topics are a great starting point for deeper analysis. For example, you could focus on a topic area that was driving the most volume, see the questions customers are asking, and then improve the sources Fin is using to answer those questions.
- Leveraging custom reports:
- Use the “Unresolved conversations” metric.
- Segment by conversation topics. If you already use conversation topics you can enhance your reporting by segmenting your report by ‘topics’. This approach helps to organize the unanswered customer questions into relevant groups/product areas. If you haven't set up topics yet, consider creating them based on FAQs or specific product areas that your company offers end users. This will help categorize conversations and facilitate easier management.
- Tip: chart drill-in and exporting chart data can be used for different use cases to assess the conversation transcripts.
- Reviewing Fin AI Agent CSAT performance and filtering by poor ratings to see specific conversations where the customer experience could be improved either through workflow setup or AI content.
- Examining resolution rates for content by checking the ‘Resolved’ column found in Fin AI Agent > Content. For example, you can search and filter for new content you’ve created and assess how each individual source is performing. If its ‘Resolved’ rate is over 50% you’re off to a good start! If it’s less than 50% click on the source to optimize it for AI. If it’s an article, you can also see conversations customers started after viewing this source to find out where they still needed help.
- External in-house script built to assess, summarize and group Fin AI Agent performance based on different Product Areas.
We hope these practices will assist you in efficiently managing and enhancing AI interactions with your customers. In the meantime, I have sent this thread over to the relevant product teams who are continuously looking at new ways to leverage AI within the product for analysis and optimization recommendations, stay tuned for exciting additions in the future!
If you have any further questions or need additional assistance, please feel free to reach out.