Skip to main content
Question

Fin's flawed assumed resolved & pricing design


bosbeest

Hi everyone,

 

I'd like to get your two cents on the way Fin handles “assuming” it resolved an issue at the moment. Sorry for the essay 😅

TL;DR: Fin assumes it has resolved an issue when you step in before a customers presses “Speak to Human”. I do this when Fin is completely wrong and I want to help the customer ASAP as they're in big trouble and I can help them out of it quickly. 

 

My situation:
I work a company which makes photography hardware for event venues, museums, theme parks, etc. with which you can take a picture, replace backgrounds with themed ones (Friends, Harry Potter, F1, Bluey, etc.) and print them on location. Our system works on our inhouse SaaS platform. Locations that use our hardware are often so packed that there are long queues to take a pic. 

Supporting customers:
When something goes wrong, hardware or software related, people on location instantly start “losing” money and they're stressed about it. Unsatisfied queues of visitors make the situation worse. 
When they contact us through intercom, Fin picks it up first. We have a resolve rate of about 12%. However, many times we see that Fin gives a completely incorrect answer - one with multiple steps to check / resolve. If we see this, we step in as we want to help the customer up and running ASAP. 

The issue:
I saw that we were paying for many more Fin resolved issues than I could see. So I contacted Intercom twice and we finally figured out; Fin “assumes” it resolved an issue way too often - i.e. when we step in before a customer clicks “Speak to Human”. Fin is designed like this because Intercom assumes the worst of their customers. A bad actor could potentially step in every single time Fin gives an answer which is correct, and prevent their customer from clicking “That helped”. Thus preventing a Fin resolve and saving $0.99.

 

Honestly I understand the design in a way, but this is the opposite of customer-friendliness. I now have to watch how a stressed out customer asks for technical help, get an incorrect answer, try all the incorrect steps, get more stressed and frustrated whilst doing so as they're nonsense, having complaining customers around them, and then after many minutes click the “Speak to Human” button. After which I can step in and help them. 

It means I get:

  • Lower customer satisfaction.
  • Longer resolve times.
  • Incorrect Fin statistics.
  • More expensive invoices.

 

Has anyone encountered this as well? Or solved it in some way? 

11 replies

Christian S12

I’ve noticed the same behavior with Fin taking credit for interactions when someone intervenes and also when a timeout happens, and the customer ends up being passed to a teammate. I’ve brought this up to Intercom’s product team before. While I can understand why it’s set up this way, my argument is: can Fin recognize when the interaction happens after the timeout and not take credit for it? For example, similar to how CSAT surveys are only sent after 250 characters, maybe Fin could exclude interactions where a timeout occurs, and the agent handles most of the resolution after a certain point or a certain amount of characters occur between a teammate and customer. 


Trevor
Innovator ✨
Forum|alt.badge.img+2
  • Innovator ✨
  • 28 replies
  • January 28, 2025

@bosbeest According to Intercom, they DO NOT count assumed resolutions, and do not bill you for those if the same user reaches out and talks to a teammate within 24 hours, on the same ticket or another ticket. I was told this on a call with them months ago, but I haven’t done too much looking and investigating with that to confirm. But if this is the case, you probably aren’t being billed for those like you’re assuming? 
I too wish we had some additional options to help with those drop-offs though. Especially when I’m not super happy with Fin’s answer. 

I’ve been putting a lot of effort into fixing all of Fin’s answers with more robust knowledge content, and overall it’s been a great experience for us!


Anjelica
  • New Participant
  • 1 reply
  • January 28, 2025

Not a quick fix by any means! but in my company we do robust chat audit of 100’s of replies from Fin which were not great. We go through the chats and pick up on trends and topics that Fin repeatedly fails at. We then add or update the content. This is not going to immediately help with your problem of Fin thinking its answered something correctly, however, over time it definitely reduces the number of chats answered incorrectly. I do find that most issues my Fin bot has encountered can be somewhat fixed by really doubling down on the content. 


Forum|alt.badge.img
  • Connector
  • 5 replies
  • January 28, 2025

We experience this as well and I can tell you what we’ve done. 

 

  1. The hard truth is, yes – your customers are going to get frustrated at first until you train Fin better. We segmented our customer base to who we were willing to take that risk with and filtered out our VIPs. It was hard for our team to watch at first, but we asked that they wait until the customer click “talk to a human” before they intervened. Then, we made sure to “Add a snippet” to correct the misinformation once the issue was resolved. We’re now at close to a 30% Confirmed Resolution rate and have filtered out customers we know only write in with requests that Fin can’t handle. 
  2. I agree that “Assumed Resolution” is a misnomer. We do still have agents stepping in before the customer asks to speak to a person because the guidance provided was wrong. I don’t think you are be billed for those resolutions, but the category name should change to better describe the behavior we’re seeing. 
  3. If you haven’t yet, you *need* to revise your knowledge base. Fin can only be as good as the information you provide it, so making sure your knowledge base is up to date and easy to understand will greatly improve your performance. 

  • New Participant
  • 3 replies
  • January 28, 2025

Here is a post on Reddit may provide you some options: https://www.reddit.com/r/SaaS/comments/1ibeigd/ai_automation_options_on_intercom/


Forum|alt.badge.img
  • Connector
  • 5 replies
  • January 30, 2025

Something that drastically improved our Confirmed Resolutions and overall accuracy was turning on “Content From Conversations”. It scanned all of our previous conversations and generated it’s own snippets from there. 

I think that took us from 15% to almost 30% Confirmed Resolutions in a very short timeframe. 


bosbeest
  • Author
  • New Participant
  • 4 replies
  • February 3, 2025
Trevor wrote:

@bosbeest According to Intercom, they DO NOT count assumed resolutions, and do not bill you for those if the same user reaches out and talks to a teammate within 24 hours, on the same ticket or another ticket.

Hi Trevor, if I understand you correctly - in your case a customer reaches out again. In my case there is no second interaction with the same client. We simply answer the customer's question before they can click “Talk to Human” because we see it's wrong. This is counted as an assumed resolution and this is most definitely being billed. 

 

Steve Flynn wrote:

Something that drastically improved our Confirmed Resolutions and overall accuracy was turning on “Content From Conversations”. It scanned all of our previous conversations and generated it’s own snippets from there. 

I think that took us from 15% to almost 30% Confirmed Resolutions in a very short timeframe. 

Wow I could have sworn this was already taken into account by default! I thought this was one of Fin's greatest selling points and I thought I'd seen it use older conversations before 🤷‍♂️ I've enabled it now and it will be learning from the conversations of the past 30 days. Wish that could be the past year though!

 

 

It's clear that more, strong content is the way to go for now. Still don't like the way they count nonsense resolutions though. 

 


bosbeest
  • Author
  • New Participant
  • 4 replies
  • February 24, 2025

This weekend brought a new situation to light where Intercom's “assuming” of AI resolutions is badly designed and unfair (for us, not for them $$$). 

 

We had an outage in certain region of the world, which caused multiple customers to send messages stating things weren’t working. 

According to Intercom, I should have ALL of these customers try nonsensical suggestions from Fin first, even though we know it's a server side issue. So I can't send messages to all these people until they've tried all the completely useless solutions - unless I'm fine with paying $1 per reply. 

 

Does Intercom have anything to say about this?


Julian Murray
Forum|alt.badge.img

@bosbeest curious what your ideal solution would look like in these scenarios? Did you put a banner in the chat tool during the outage or was that not practical due to it being out of hours or over weekend? Would the new beta Fin guidance ability, allowing you to tell Fin what to do with outages of unknown origin, help for specifying flow and Fin’s responses ahead of the next time this happens? https://www.intercom.com/help/en/articles/10210126-fin-guidance-beta  Or did you have some other ideal solution in mind? Curious what you envisage the best fix to be, product or commercial or other? 


bosbeest
  • Author
  • New Participant
  • 4 replies
  • February 24, 2025

Hi Julian,

 

I'll try to state it in the shortest possible way:

My ideal scenario is one where Intercom comes up with an alternative to their current way of billing or assumption of resolution. They've set it up in the current way because from the very first stage of design, they mistrust their customers and assume we will all try to circumvent their billing structure. That's why interrupting Fin will count as an “assumed resolution”. These are Intercom's words, not mine. 

 

To answer your other questions: 

I have not used a banner as I wasn't aware of the functionality. However, this would not mitigate the issue at hand. 

I have coincidentally played around with the guidance ability and it seems very powerful. However, you can't set up rules for an AI to understand that there is something wrong with for example a certificate for a specific region of the world. Nor for any other back-end malfunction for that matter. So unfortunately, this would also not be able to mitigate the issue at hand. 

 


Julian Murray
Forum|alt.badge.img

Thanks for the insights. I was hoping FIN guidance would help with some of these issues. The product and commercial model is rapidly evolving so hopefully you get a model that works for your full needs soon. J 


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings