Skip to content

Pause, roll out, end and other test actions

What each action in a test's menu does, from pausing and rolling out a winner to archiving, deleting and resetting data.

On this page

Each row on A/B tests has a ⋯ menu, and each test page has buttons at the top plus More actions. What you see depends on the test's status: Draft, Live, Paused, Ended or Rolled out. Rollouts have their own tab and actions (see below).

Launch, pause and resume

  • Launch test (drafts): starts splitting traffic.
  • Pause: visitors see the original page until you resume. Data is kept, but paused tests split no traffic, so resume or end them soon to keep results clean.
  • Resume: starts splitting traffic again.

End test

Visitors go back to the original page and results stop updating. If Crovio has called a winner, it's recorded with the test and in Learnings. Ended tests can't be restarted, but you can duplicate them.

Roll out a winner

Rolling out shows one group to every visitor the test targets, without counting them. Use it to keep a winning change live while you add it to your theme for good.

  1. On a running test, click Roll out… next to End test (or More actions → Roll out a winner). On an ended test, click Roll out on the group's row in Test groups.
  2. Pick the group. The one Crovio called is marked Results pick.
  3. Confirm. A running test ends; the test shows [Group] rolled out and moves to the Rollouts tab of the list.

Pick Control (keep the original) to end the test without rolling anything out. Use Change rolled-out group or Stop rollout later; stopping sends everyone back to your original page.

Roll out a change without testing

Sure about a change already (a winner from an earlier test, a seasonal banner, a fix)? Roll it out without a test. Nothing is measured against the original, so there's no waiting for a result.

  1. Click New test, then Roll out a change under What do you want to do?
  2. Pick the kind of change. Shipping rates are test only: set your rates in Shopify.
  3. Set up the change on its tab, then choose Who sees it: pages, devices, visitors and the Share of visitors (content, code, template, theme and split URL rollouts can go to part of your visitors; offers, shipping, checkout, prices and subscriptions always reach everyone who matches).
  4. Preview it on QA & preview, then click Go live.

A live rollout sits under Rollouts in the list with its share ("Live · 30%"). Pause shows everyone the original until you resume; Stop rollout ends it for good (duplicate it to roll it out again). A price rollout writes the new prices to your products in Shopify, and a subscription rollout copies the offer to the plan, so stopping them doesn't undo that.

To find out whether it actually helps, choose Turn into an A/B test: a new draft with your original against the change, split 50/50.

Duplicate

Creates a new draft with the same groups, targeting and goals. Use it to run an ended test again or to try a follow-up.

A duplicated price test starts from today's prices in Shopify (so a rolled-out winner is the new starting point) and without hidden copies: it makes its own when you launch it, or when you click Make copies now.

Archive

Hides an ended or draft test from the list. Archived tests keep their results under the Archived tab; Unarchive brings one back.

Delete

Removes the test and all its results for good (and stops a rollout). Pause or end a live test first. Archive instead if you want to keep the results.

Deleting a price test that has run swaps every cart that holds a test price back to the original product at your normal price, and its hidden copies are deleted 48 hours later. A draft's copies are deleted right away.

Reset data

On a running test, More actions → Reset data deletes the visitors, orders, goals and events collected so far. The test keeps running with the same groups and results count again from now. Use it after fixing a broken variant.

Rename

Click the test's name at the top of its page.

Was this page helpful?