I'd love to hear your feedback so far!

A couple of suggestions for Fin’s behavior moving forward after 1 week into the trial:

It would be very helpful to provide ‘internal snippets’ to provide context to Fin but not allow it to source that content in its answers. For example, users might refer to their account settings as preferences, setup, etc. Rather than updating the Help Center content to use all the relevant words, I’d like to train Fin to understand these terms could be used interchangeably. This is not practical with current snippets, because I don’t want Fin to recall this information to users. I just want it to use this context to find the right resources. 

It would also be great to tie Fin into URLs and/or events. If we can associate a set of URLs with a specific feature, Fin can prioritize those articles first. Same with events, once there has been action taken towards an event or if the user has an attribute that indicates they use a feature often, Fin should prioritize referencing these articles first. This would be great for products that have a variety of features with similar terms. 

• In Intercom, this could look like a user asking “How do I share a message” and Fin responding with “It looks like you have begun creating a Post, to share a Post do XZY”.

Finally, in the inbox ‘Improve this answer’ section, ideally there is a way to indicate if the articles sourced were relevant through a thumbs up/thumps down type of selection. Also would like to associate a relevant article that wasn’t sourced. During our trial I’ve found most users do not respond regarding satisfaction. This would help improve the accuracy of fins responses and help us feel more confident in Fin’s soft resolution rate. 

Hey @Julia G I don’t know if this group is still active, but I hope it is. 

So, today I started testing FIN, but I honestly had a bit of bigger expectations. 

1. We are a startup, hence we do not have the time to generate a lot of support content, but we have a lot of documents / documentations and many technical documentations as PDFs. 
2. I was happy to see that FIN can be trained using snippets, PDFs, and external links. So, I did feed FIN with some of those information. 
3. Then I tested it. On the superficial simple questions it did well. However, when I started asking questions from the PDF content, FIN failed to give me simple answers from the PDFs. 
4. whereas, when I used the same document using AskyourPDF and ChatGPT 4, I was able to get the information I needed. 

I still do not understand how FIN works in general, but our current need is to be able to train FIN on a bunch of data, that is not fully optimized / suitable for regular reading. Hoping that FIN is able to crunch the information (similarly to ChatGPT 4) and generate answers to the users. 

Any particular clarity, or timeline, or even ways of improvement that I can do? 

PS: the documents I am sharing are technical documents. 

PS2: I went through the FIN academy. 

Thanks, 

Hi @Ruth O apologies for my delayed response. For some reason, the notification dropped into my spam! 

I’m very open for a chat about taking part in some of this testing for sure! 

Let me know what’s the best way to connect. 

Looking forward to hearing back from you, and once more, apologies for the delayed response. 

Hi 👋

I would like to share a funny experience with Fin. We are running our trial (not very amazed so far). Fin correctly recognized the language as Brazilian Portuguese. But then he suddenly provided an answer even though he was not asked any questions. When I clicked “Improve answer”, I found out that he simply hallucinated the question to have something to answer.😑

Hi 👋

I would like to share a funny experience with Fin. We are running our trial (not very amazed so far). Fin correctly recognized the language as Brazilian Portuguese. But then he suddenly provided an answer even though he was not asked any questions. When I clicked “Improve answer”, I found out that he simply hallucinated the question to have something to answer.😑

 

That’s why more testing is required. I think that we need to do some intensive testing before we actually release this to clients. 

Hey @Julia G 

Been testing FIN a few weeks and so far FIN has not met the expectations set out in the launch videos and demos.

The biggest thing that were demonstrated in those demos that isn’t happening is:

1. Asking clarifying questions: In 95% of cases FIN doesn’t do this where it needs to. Instead it hallucinates OR generalizes completely (e.g. linking the customer to a few generic articles on our site). The worst place this happens for us is where a client asks about a product e.g. “How can I take this product?” and FIN doesn’t ask them which product before answering.

Hope this helps.

I’m running some tests on Fin. I’ve had Fin learn a bunch of articles and just a handful of URLs. All these content sources are pretty lengthy and in-depth. In my testing so far, Fin does provide the right information for the most part. However, Fin dumps it all into one GIANT paragraph with no line breaks of any kind. This means the customer receives a VERY long, very thorough essay response but it’s all smashed into one intimidating block of text. Is there any way to get Fin to use paragraphs? 

I’ve also noticed Fin sometimes tends to provide too wordy answers as @Matt Hsu  and other users mentioned.  The right way to go would be for it to ask clarifying questions if the user’s initial query is too broad (I still haven’t been able to find the pattern when it happens and when it doesn’t, because sometimes it does ask clarifying questions).

The answer debugging tool is useful for creating new snippets, but not so much for training the bot on what type of answer is considered a good answer.