Skip to main content

Improvement proposals for an automation

Each automation reads its own results and proposes a change only when the numbers support it: promote an A/B version that won, or put back the version before a change that measured worse. A person applies it, and it can be undone.

Steps

  1. Open an automation. Under "Improvement proposals" you see what its own results suggest. Each card shows the evidence (people, conversions, rates and the test's p-value), what it was measured against, exactly what would change, and the other options.
  2. "A/B winner" appears when one version of a test is most likely the best, with enough people in every version: at least 30, or more when the test sets its own minimum (100 unless changed). If the test has a holdout and the leading version does not beat it, you see "No lift over the holdout" instead, and promoting it is not proposed.
  3. "Back to the previous version" compares the current version with the last earlier one whose content differs (a version that only turned it back on, or saved it unchanged, is counted together with the one it copies), counting only people whose goal window has closed, so a new version is not judged on days it has not had yet. It is proposed only when both versions have at least 30 such people and the difference is unlikely to be chance.
  4. A card marked "Small sample" is there to follow, not to act on: it has no Apply button. "For your information" cards (a step that keeps failing, an automation with no goal) explain and point you to the run log or the goal panel.
  5. Apply, then confirm. The change is saved as a new version; people already in a run keep the version they started on. Dismiss hides that proposal until the automation changes or it recommends something else. Every decision is listed with who made it, and an applied change is measured against the version it replaced. "Return to version" puts the earlier one back.

Questions

What never changes by itself?
Nothing in the automation changes without a person: not the steps, the messages, the goal, the limits or who it reaches. Proposals are computed when you open the panel, there is no background job and no AI model behind them. The only automatic promotion is the one a split step opts into with its own auto-promote setting.
Does a better number prove the change worked?
No. The panel compares versions and arms; it does not prove cause. A holdout, where one is set, is the closest thing to a fair comparison, and it says so beside each result.
Who can apply a proposal?
The people who may change a live automation: in Brand My Inbox an owner or an admin, in EmailPro a sender role or above. Editors and senders can dismiss one (in Brand My Inbox, the roles that may change deals); everyone else can read. Through MCP an assistant can read the proposals (flow_proposals, crm_automation_proposals) but cannot apply or dismiss one.

Updated 2026-10-25 · llms-full.txt