AI Learning & A/B Testing
Let your AI get smarter over time with sentiment tracking, empathy learning, and procedure experiments.
What is AI Learning?
AI Learning brings together two powerful optimization features: sentiment tracking (how well your AI recovers upset callers) and A/B testing (comparing different versions of procedures to see which performs better). Together, they help your AI continuously improve.
Sentiment Tracking
When your AI detects that a caller is frustrated, it can automatically add empathetic language to its responses. The AI Learning page shows you how effective these interventions are.
Key metrics
| Metric | What It Means |
|---|---|
| Total Interventions | How many times the AI acted because a caller sounded frustrated: an empathy phrase put in front of a Flow Builder procedure's reply, or an offer to connect the caller to a person. |
| Recovery Rate | Of the interventions the caller spoke again after, the percentage where they sounded calmer on that next turn (Irate to Upset counts). Higher is better. |
| By Sentiment Level | Interventions split between "Upset" (mild frustration) and "Irate" (strong frustration). |
A/B Testing (Experiments)
A/B testing lets you compare two versions of a procedure to see which one performs better. For example, you might test a shorter greeting against a longer one, or a different question order.
How to run an experiment
Open a procedure in Flow Builder and click "+ Add Variant." This creates a copy you can modify.
Edit the variant with your proposed improvements. Change wording, reorder steps, or try different approaches.
When you publish, BLEUM automatically splits incoming traffic between the original and the variant.
Watch the A/B Testing page to see which version has better completion rates, shorter handle times, or higher satisfaction.
Understanding results
BLEUM tracks statistical confidence to help you make decisions. Wait until you have enough data (usually at least 100 calls per variant) before drawing conclusions. The system will show you when results are statistically significant.
Common Questions
BLEUM assigns each call at random to the original or a variant, in the traffic shares set for the experiment (50/50 for the original and one variant). Every call is assigned afresh, so someone who calls back during the experiment may get a different version from last time.
Yes, you can run experiments on different procedures simultaneously. However, avoid running multiple experiments on the same procedure at the same time, as it makes results harder to interpret.