We are currently monitoring all customer conversations where Fin is involved to ensure an excellent customer experience. While a report already exists that pulls all conversations in which Fin has participated, there is no mechanism to leave the evaluation of the quality of Fin’s responses with real customers.
I propose introducing a simple rating system within Intercom that allows us to assess Fin’s performance with three categories: Good, Acceptable, or Poor. This would work in a similar way to the evaluation logic currently used in the Test module.
Implementing this functionality would enable us to:
-
Track the accuracy and helpfulness of Fin’s contributions.
-
Monitor Fin’s “health” over time.
-
Compare Fin’s performance across different topics and contexts.
This improvement would provide valuable insights into Fin’s effectiveness and ensure we continuously deliver a high-quality customer experience.