Skip to main content
Answered

Fin's flawed assumed resolved & pricing design


bosbeest

Hi everyone,

 

I'd like to get your two cents on the way Fin handles “assuming” it resolved an issue at the moment. Sorry for the essay 😅

TL;DR: Fin assumes it has resolved an issue when you step in before a customers presses “Speak to Human”. I do this when Fin is completely wrong and I want to help the customer ASAP as they're in big trouble and I can help them out of it quickly. 

 

My situation:
I work a company which makes photography hardware for event venues, museums, theme parks, etc. with which you can take a picture, replace backgrounds with themed ones (Friends, Harry Potter, F1, Bluey, etc.) and print them on location. Our system works on our inhouse SaaS platform. Locations that use our hardware are often so packed that there are long queues to take a pic. 

Supporting customers:
When something goes wrong, hardware or software related, people on location instantly start “losing” money and they're stressed about it. Unsatisfied queues of visitors make the situation worse. 
When they contact us through intercom, Fin picks it up first. We have a resolve rate of about 12%. However, many times we see that Fin gives a completely incorrect answer - one with multiple steps to check / resolve. If we see this, we step in as we want to help the customer up and running ASAP. 

The issue:
I saw that we were paying for many more Fin resolved issues than I could see. So I contacted Intercom twice and we finally figured out; Fin “assumes” it resolved an issue way too often - i.e. when we step in before a customer clicks “Speak to Human”. Fin is designed like this because Intercom assumes the worst of their customers. A bad actor could potentially step in every single time Fin gives an answer which is correct, and prevent their customer from clicking “That helped”. Thus preventing a Fin resolve and saving $0.99.

 

Honestly I understand the design in a way, but this is the opposite of customer-friendliness. I now have to watch how a stressed out customer asks for technical help, get an incorrect answer, try all the incorrect steps, get more stressed and frustrated whilst doing so as they're nonsense, having complaining customers around them, and then after many minutes click the “Speak to Human” button. After which I can step in and help them. 

It means I get:

  • Lower customer satisfaction.
  • Longer resolve times.
  • Incorrect Fin statistics.
  • More expensive invoices.

 

Has anyone encountered this as well? Or solved it in some way? 

Best answer by Trevor

@bosbeest According to Intercom, they DO NOT count assumed resolutions, and do not bill you for those if the same user reaches out and talks to a teammate within 24 hours, on the same ticket or another ticket. I was told this on a call with them months ago, but I haven’t done too much looking and investigating with that to confirm. But if this is the case, you probably aren’t being billed for those like you’re assuming? 
I too wish we had some additional options to help with those drop-offs though. Especially when I’m not super happy with Fin’s answer. 

I’ve been putting a lot of effort into fixing all of Fin’s answers with more robust knowledge content, and overall it’s been a great experience for us!

View original
Did this topic help you find an answer to your question?

7 replies

Christian S12

I’ve noticed the same behavior with Fin taking credit for interactions when someone intervenes and also when a timeout happens, and the customer ends up being passed to a teammate. I’ve brought this up to Intercom’s product team before. While I can understand why it’s set up this way, my argument is: can Fin recognize when the interaction happens after the timeout and not take credit for it? For example, similar to how CSAT surveys are only sent after 250 characters, maybe Fin could exclude interactions where a timeout occurs, and the agent handles most of the resolution after a certain point or a certain amount of characters occur between a teammate and customer. 


Trevor
Forum|alt.badge.img+1
  • Active User
  • 23 replies
  • Answer
  • January 28, 2025

@bosbeest According to Intercom, they DO NOT count assumed resolutions, and do not bill you for those if the same user reaches out and talks to a teammate within 24 hours, on the same ticket or another ticket. I was told this on a call with them months ago, but I haven’t done too much looking and investigating with that to confirm. But if this is the case, you probably aren’t being billed for those like you’re assuming? 
I too wish we had some additional options to help with those drop-offs though. Especially when I’m not super happy with Fin’s answer. 

I’ve been putting a lot of effort into fixing all of Fin’s answers with more robust knowledge content, and overall it’s been a great experience for us!


Anjelica
  • New Participant
  • 1 reply
  • January 28, 2025

Not a quick fix by any means! but in my company we do robust chat audit of 100’s of replies from Fin which were not great. We go through the chats and pick up on trends and topics that Fin repeatedly fails at. We then add or update the content. This is not going to immediately help with your problem of Fin thinking its answered something correctly, however, over time it definitely reduces the number of chats answered incorrectly. I do find that most issues my Fin bot has encountered can be somewhat fixed by really doubling down on the content. 


Forum|alt.badge.img
  • Connector
  • 5 replies
  • January 28, 2025

We experience this as well and I can tell you what we’ve done. 

 

  1. The hard truth is, yes – your customers are going to get frustrated at first until you train Fin better. We segmented our customer base to who we were willing to take that risk with and filtered out our VIPs. It was hard for our team to watch at first, but we asked that they wait until the customer click “talk to a human” before they intervened. Then, we made sure to “Add a snippet” to correct the misinformation once the issue was resolved. We’re now at close to a 30% Confirmed Resolution rate and have filtered out customers we know only write in with requests that Fin can’t handle. 
  2. I agree that “Assumed Resolution” is a misnomer. We do still have agents stepping in before the customer asks to speak to a person because the guidance provided was wrong. I don’t think you are be billed for those resolutions, but the category name should change to better describe the behavior we’re seeing. 
  3. If you haven’t yet, you *need* to revise your knowledge base. Fin can only be as good as the information you provide it, so making sure your knowledge base is up to date and easy to understand will greatly improve your performance. 

  • New Participant
  • 2 replies
  • January 28, 2025

Here is a post on Reddit may provide you some options: https://www.reddit.com/r/SaaS/comments/1ibeigd/ai_automation_options_on_intercom/


Forum|alt.badge.img
  • Connector
  • 5 replies
  • January 30, 2025

Something that drastically improved our Confirmed Resolutions and overall accuracy was turning on “Content From Conversations”. It scanned all of our previous conversations and generated it’s own snippets from there. 

I think that took us from 15% to almost 30% Confirmed Resolutions in a very short timeframe. 


bosbeest
  • Author
  • New Participant
  • 2 replies
  • February 3, 2025
Trevor wrote:

@bosbeest According to Intercom, they DO NOT count assumed resolutions, and do not bill you for those if the same user reaches out and talks to a teammate within 24 hours, on the same ticket or another ticket.

Hi Trevor, if I understand you correctly - in your case a customer reaches out again. In my case there is no second interaction with the same client. We simply answer the customer's question before they can click “Talk to Human” because we see it's wrong. This is counted as an assumed resolution and this is most definitely being billed. 

 

Steve Flynn wrote:

Something that drastically improved our Confirmed Resolutions and overall accuracy was turning on “Content From Conversations”. It scanned all of our previous conversations and generated it’s own snippets from there. 

I think that took us from 15% to almost 30% Confirmed Resolutions in a very short timeframe. 

Wow I could have sworn this was already taken into account by default! I thought this was one of Fin's greatest selling points and I thought I'd seen it use older conversations before 🤷‍♂️ I've enabled it now and it will be learning from the conversations of the past 30 days. Wish that could be the past year though!

 

 

It's clear that more, strong content is the way to go for now. Still don't like the way they count nonsense resolutions though. 

 


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings