Fin AI Preview doesn't match live behavior | Community
Skip to main content
Answered

Fin AI Preview doesn't match live behavior

  • October 24, 2025
  • 2 replies
  • 127 views
J

Hi all, we’re pretty new to Fin AI & chatbot setup in general, so we might be missing something, however, I’m seeing massive differences in how Fin responds to questions in the Preview environment versus the actual live environment on our website.

As an example – one of our users asked a fairly basic question about our service, and Fin AI responded with “Sorry, I couldn’t find information related to your question.” However, when I pasted the same question into the Fin AI preview (logged in as an organizational user within the Intercom admin dashboard), it returned a comprehensive response and fully answered the user’s question.

Currently, we have Fin AI enabled for all Users, Leads, and Visitors via Intercom Messenger only, and have it trained on all of our published support articles, as prompted via the Simple Deploy process.

I’ve scoured all available Fin AI settings within Intercom to no avail, and I’m not able to find another community thread about this topic. Can anyone offer me assistance with this? Thank you!

Best answer by Nathan Sudds

​@Jesse Bobick  Sorry you're having this issue I have seen some discrepancies between how Fin responds in preview and how Fin responds in the wild but usually it's not drastic unless there's some other nuances and that's one of the things that sometimes hard to pin down. 

 

Without seeing more about your setup I would say one thing to look at is check your knowledge sources and make sure that they're all turned on for Fin AI involvement to the public because sometimes things are turned on for co-pilot that aren't turned on for the public and also make sure that all the knowledge sources are available to the audiences that you are connecting with. 

 

Also remember the nuances of your guidance if you have guidance for certain audiences or if your users and leads aren't being detected correctly for some reason that might impact how it's working as well and if you're using Fin tasks for example and a task picks it up it could also create some differences  in my experience.

 

I recommend turning on the show all conversation events in the conversation view and like looking at where Fin is engaging, which workflows are involved and any guidance that's being applied and that'll help you get a better understanding of what is happening and what isn't happening in the process.

 

These are the kinds of things that I will be talking about soon on the  Intercom Community Meetup and we'll be able to dig into some of the stuff together.  In the meantime, I hope this is helpful and feel free to DM me or book a time on my calendar and I will be happy to take a look at it with you because now I'm curious about what's going on.

 

Apologies this was voice to text originally but I made some updates,  if are there any mistakes, I just wanted to make a note for reference because I was trying to reply quickly while on the road. 

 

 

2 replies

Nathan Sudds
Top Expert ✨
Forum|alt.badge.img+7
  • Nathan Sudds
  • Top Expert
  • 381 replies
  • Answer
  • October 25, 2025

​@Jesse Bobick  Sorry you're having this issue I have seen some discrepancies between how Fin responds in preview and how Fin responds in the wild but usually it's not drastic unless there's some other nuances and that's one of the things that sometimes hard to pin down. 

 

Without seeing more about your setup I would say one thing to look at is check your knowledge sources and make sure that they're all turned on for Fin AI involvement to the public because sometimes things are turned on for co-pilot that aren't turned on for the public and also make sure that all the knowledge sources are available to the audiences that you are connecting with. 

 

Also remember the nuances of your guidance if you have guidance for certain audiences or if your users and leads aren't being detected correctly for some reason that might impact how it's working as well and if you're using Fin tasks for example and a task picks it up it could also create some differences  in my experience.

 

I recommend turning on the show all conversation events in the conversation view and like looking at where Fin is engaging, which workflows are involved and any guidance that's being applied and that'll help you get a better understanding of what is happening and what isn't happening in the process.

 

These are the kinds of things that I will be talking about soon on the  Intercom Community Meetup and we'll be able to dig into some of the stuff together.  In the meantime, I hope this is helpful and feel free to DM me or book a time on my calendar and I will be happy to take a look at it with you because now I'm curious about what's going on.

 

Apologies this was voice to text originally but I made some updates,  if are there any mistakes, I just wanted to make a note for reference because I was trying to reply quickly while on the road. 

 

 

Nathan Sudds, Intercom Consultant | activelabs.ca | Book a free call: cal.com/nathansudds/intercom. Case study: https://www.intercom.com/customers/jukebox
CharlotteMartin34567
Forum|alt.badge.img

It sounds like Fin’s live environment isn’t accessing the same data sources as the Preview. Double-check that your live Messenger workspace is connected to the same Help Center (and visibility settings) as the one used in Preview. Also ensure the articles are published and visible to “Everyone,” not just logged-in users—Fin can’t surface restricted content in the live chat. Re-syncing the content under Fin → Data sources → Help Center often fixes this mismatch.


Terms & ConditionsAccessibility statement