Fin AI Preview doesn't match live behavior | Community
Skip to main content

Hi all, we’re pretty new to Fin AI & chatbot setup in general, so we might be missing something, however, I’m seeing massive differences in how Fin responds to questions in the Preview environment versus the actual live environment on our website.

As an example – one of our users asked a fairly basic question about our service, and Fin AI responded with “Sorry, I couldn’t find information related to your question.” However, when I pasted the same question into the Fin AI preview (logged in as an organizational user within the Intercom admin dashboard), it returned a comprehensive response and fully answered the user’s question.

Currently, we have Fin AI enabled for all Users, Leads, and Visitors via Intercom Messenger only, and have it trained on all of our published support articles, as prompted via the Simple Deploy process.

I’ve scoured all available Fin AI settings within Intercom to no avail, and I’m not able to find another community thread about this topic. Can anyone offer me assistance with this? Thank you!

@Jesse Bobick  Sorry you're having this issue I have seen some discrepancies between how Finn responds in preview and how Finn responds in the wild but usually it's not drastic unless there's some other nuances and that's one of the things that sometimes hard to pin down. 

 

Without seeing more about your setup I would say one thing to look at is check your knowledge sources and make sure that they're all turned on for Finn AI involvement to the public cuz sometimes things are turned on for co-pilot that aren't turned on for the public and also make sure that all the knowledge sources are available to the audiences that you are connecting with. 

 

Also remember the nuances of your guidance like if you have guidance for certain audiences or if you have your users and leads aren't being detected correctly for some reason that might impact how it's working as well and if you're using Fin tasks for example and fantast picks it up it could also create some differences  in my experience.

 

I recommend turning on the show all conversation events in the conversation view and like looking at where AI is engaging and any guidance that's being applied and things like that that'll help you get a better understanding of what is happening and what isn't happening in the process.

 

These are the kinds of things that I will be talking about soon on the  intercom community meetup and we'll be able to dig into some of the stuff together in the meantime, Feel free to DM me and I will be happy to kind of take a look at it with you cuz I'm curious about what's going on It helps me learn a little bit more about where people are stuck. 

 

Apologies this is actually voice to text so if are there any mistakes I'll hopefully fix them when I get a chance to edit it but I wanted to send you a quick response.