Summary:
When a scorecard's attributes are updated, unreviewed conversations already captured by a Monitor should reflect the new attributes, either as incomplete/pending fields, or via a manual "re-apply scorecard" option.
The problem:
Scorecards evolve. When you refine attributes to better capture quality signals, only future conversations benefit from those changes. Existing conversations in the Monitor, especially unreviewed ones, are locked to the old scorecard structure, making it impossible to apply your improved rubric retroactively without manual workarounds.
This is particularly frustrating when a Monitor has been running for weeks and you haven't yet reviewed the backlog. Updating the scorecard mid-cycle effectively creates two incompatible datasets within the same Monitor.
What I'd expect instead:
At minimum, unreviewed conversations in a Monitor should have the new or updated attributes added as empty/pending fields when a scorecard is changed, preserving any existing scores while surfacing the new ones for review. Reviewed conversations could reasonably remain unchanged.
Ideally, there would also be a "Re-apply scorecard to unreviewed conversations" option, a reset or refresh button scoped to conversations that haven't been manually evaluated yet.
Impact:
Without this, teams have to either (a) accept inconsistent data across a Monitor's history, or (b) or manually re-process old conversations. All of these options undermine the value of Monitors as a continuous QA tool.