Product Wishlist | Community
Skip to main content

    Idea Pipeline

    Filter by idea status

    Browse by category

    3560 Ideas

    Jordan CantorNew Participant

    Feature request: A built-in QA sandbox for Fin AI Agent and OutcomesSubmitted

    Building a reliable Fin AI agent and Outcomes means testing more than the happy path. We need to know whether Fin follows the right procedure, asks for missing information, handles exceptions, and hands off when it should. We also need to know whether a change that fixes one workflow breaks another.Today, Preview is useful for debugging a single conversation, but running tests one by one takes too much time. Evals help with batch testing, but their scope and run limits make it hard to test several workflows together, give each workflow its own success criteria, and rerun the full set after every change.I’d like Intercom to add a built-in sandbox enviornment where teams can:Test safely. Run draft configurations against simulated conversations without contacting customers or triggering live actions. Simulate tool responses, failures, and handoffs. Batch test across workflows. Run refunds, onboarding, and escalations in the same batch, with separate success criteria for each scenario. For example, a refund test should pass only if Fin verifies eligibility before attempting the refund. Run QA automatically and without limits. Generate test cases from Procedures, save them as reusable suites, and rerun them on demand, on a schedule, or whenever the configuration changes. Sandbox runs should be free and unlimited, with no monthly quota on simulations or evaluations. See what passed, what failed, and why. Show success rates by workflow and criterion, compare results with the published configuration, and link each failure to the conversation and the step where Fin went off track.The goal is to make QA part of building rather than another feature that has a financial barrier. Before publishing a change, I want to run the full regression suite and see whether Fin follows our protocols and achieves the intended outcomes. We should be able to do that after every meaningful change, as often as we need without limits.

    Eric PluskotaNew Participant

    add guidance controls for Procedures in Intercom (Fin)Submitted

    SummaryGuidance in Intercom lets us control which content and sources Fin prioritises in specific situations. There is no equivalent for Procedures. We’d like similar guidance for procedures so we can control when and how Fin chooses to run one.Current behaviourGuidance covers content and sources, so we can steer Fin toward the right articles or sources for a given situation. Procedures have no comparable prioritisation or situational control.ProblemCertain types of questions have many variables that can confuse Fin. Fin also prioritises the core question rather than providing an A-Z answer. This makes it hard to:Keep Fin’s behaviour predictable and consistent. Make sure the right procedure runs in the right context. Stop Fin from answering with content when a procedure should run, or the other way around.Requested solutionAdd procedure guidance that works like content and sources guidance. It should let us:Prioritise specific procedures in defined situations (for example by topic, customer type, or conversation context). Set when a procedure should be preferred over content or sources, and vice versa. Help Fin handle multi-variable questions by guiding it through the full flow, rather than only answering the core question. Test and review how this guidance affects Fin’s behaviour, as with existing guidance.Expected impactMore accurate and consistent procedure usage. More complete answers on complex, multi-variable questions. Less manual workaround, such as over-tuning procedure descriptions or content to influence routing. More confidence in rolling out procedures at scale.