Hey @craig​, I did a lot of work in helping our Support team roll out its own iteration of Resolution Bot (back when it first launched as Answer Bot), so I might be able to offer some insight here.

My first question is - within the variations of the question you're using to train Resolution Bot, are you clustering? "Clustering" refers to using multiple examples of each variation within your training.

For example, for a question on "How do I reset my password?", you'd want to include three examples of "How do I change my password?", three of "How do I get a new password?", three of "How do I reset my password?" etc.

Love the weekly review process, could you share some insight on that?

When you say you're seeing a "small number" of answers sending, what number are we talking here?

What kind of resolution rate are you seeing, on average, for your top-performing Answers?

Yeah so using variations of;

do, could, should, would etc

Where it gets tricky is words like "Channel" for us which is a partner you connect to and you could add, connect, map, remove etc and they all read very similar sans that key word (which we have also added for exact matches).

Small numbers as in less than 3 or never sent.

Res rate at about 30%, send rate below 5%

Right, sounds to me like upping the sending rate is the key to success here. If a sending rate is low, it means that Resolution Bot is having a tough time matching the answers you've shared to conversations from your customers. Do you use qualifiers when setting up Answers?

Correct but probably more work required there

Hey @craig​ 

Have you been able to optimize Resolution Bot? We tinkered with it at the start of 2019 but were getting similar results and having similar struggles, and we decided to stop it. I've been wondering about trying to train it up again, but our app is fairly complex. I'm suspecting this might be more useful for simpler user experiences, such as ecommerce websites.

Hey Kevin, not to the level I am happy with. I am struggling to get basic, let alone advanced answers land even when training the bots with 15-20+ variations of the question.

I find the following challenges;

• Where complex scenarios require end user explanations (or streams of consciousness)
• Where terminology is shared across common features / scenarios
• Where the first answer, generally then paths towards other answers - currently you cannot link answers or trigger further answers and would need to point to a knowledge base / other content

I still feel with consistent use of bots across the journey, there is scale and value to be had but not until they actually send and land in the correct context.

Am currently trying to talk to some experts @ Intercom on this to see what we are doing wrong, or what is wrong with the bot(s).

That makes sense. Thank you for the update.

Terminology being shared across features / scenarios was a large issue for us too -- particularly since popular misnomers from our users carry across too! I think another challenge here is that if you have a great product team, common questions will increasingly be answered in the product, such that over time the bot will be relegated to answer the more obtuse questions that are challenging to train against.

If you ever do have an "ah ha" moment that significantly increases the effectiveness / accuracy of the bot, I hope you update this thread or group.

I suspect this feature will be useful to our team in the future as Intercom improves its learning and accuracy. I'll give it another whirl one day.

@eric f11​ shout out mate, who can you loop in to help us all build our own SKYNETS and help start the resobot revolution (also cannot believe there is no robot emoji)

Hey @craig​ @kevin​ - my own experience with Answer Bot is that it's best served answering basic, "binary" questions, like "How do I reset my password?" or "How do I change my payment details?"

When Resolution Bot first launched (or Answer Bot, as it was known then), I was heavily involved in helping our support team build out our first 100 answers. It was challenging - in truth, Intercom is its own worst customer when it comes to Reso Bot due to the complex nature of inbound queries we regularly get.

I wrote a blog post about the experience with my colleague Leanne Harte - I hope it's insightful, but happy to answer any other questions you might have!