A/B Kit

A/B Kit

Simple A/B testing for WordPress

A/B Kit lets you test different versions of your pages to see which one gets more people to take action—whether that's filling out a form, making a purchase, or clicking a button.

Instead of guessing what works, you show version A to half your visitors and version B to the other half. After enough people visit, the plugin tells you which version performed better.

Getting Started

What you need:

  • WordPress 5.8 or newer
  • PHP 7.4 or newer
  • Admin access to your site

To install:

  1. Download the plugin from your CraftedPath account
  2. In WordPress, go to Plugins → Add New → Upload Plugin
  3. Upload the zip file and click Activate
  4. Go to CraftedPath → License and enter your license key

That's it. The plugin creates its database tables automatically.

Creating a Test

  1. Go to any page on your site (while logged in as admin)
  2. Click "A/B Test" in the black admin bar at the top
  3. Give your test a name — something like "Homepage headline test"
  4. Set your Success URL — this is the page people land on after they convert (like /thank-you/ or /order-complete/)
  5. Click "+ Add Variation" to create something to test against your original
  6. Click "Edit" on your new variation
  7. Click any element on the page to change it — you can edit text, swap images, hide things, or change styles
  8. Click "Start Test" when you're ready

The plugin automatically splits traffic evenly between your original page (called "Control") and your variation.

What you can change:

  • Text content
  • Images (picks from your media library)
  • CSS styles
  • HTML structure
  • Or hide elements entirely

How It Works

Once your test is running:

  1. Each visitor gets randomly assigned to see either Control or your variation
  2. They always see the same version on return visits (tracked via a cookie)
  3. If they reach your Success URL, that counts as a conversion
  4. The plugin calculates which version is performing better

The four test states:

State What's happening
Draft Test isn't live yet. Edit freely.
Running Collecting data. Can't edit variations.
Paused Stopped temporarily. Everyone sees Control.
Completed Test ended. Everyone sees the winner.

Important: You can't edit variations while a test is running. This protects your data—if you changed things mid-test, you wouldn't know what actually caused the results.

Reading Your Results

Check your test in CraftedPath → A/B Kit. The status tells you where things stand:

Status What it means
Waiting for visitors No one has seen the test yet
Need X+ more sessions Still gathering data
Too early to tell Not enough difference yet
Early signs One version is slightly ahead (60-75% confidence)
Looking promising Clear leader emerging (75-95% confidence)
Winner found We're 95%+ confident this version is better
Original is winning Your current page beats the variation

How long will it take?

Depends on your traffic:

  • Busy sites (1,000+ visitors/day): 1-2 weeks
  • Medium sites (100-1,000/day): 2-4 weeks
  • Smaller sites (under 100/day): 1-3 months

The plugin needs enough data to be confident the results aren't just random chance.

When to End a Test

Wait for "Winner found" before ending your test.

It's tempting to stop early when one version is ahead, but early leads often flip. The statistics need time to be sure.

Once you see "Winner found":

  1. Click End Test
  2. Pick the winner (or let the plugin pick based on the stats)
  3. Everyone now sees the winning version

Want to keep improving? Click "Create New Test from Winner" to use your winner as the new Control and test something else against it.

Troubleshooting

"I don't see variations changing"

  • Make sure your test status is "Running" (check the badge in the admin bar)
  • Clear any page caching
  • Try in an incognito window
  • Check your browser console for JavaScript errors

"Conversions aren't being tracked"

  • Double-check your Success URL matches exactly
  • The visitor must have seen the test page first, then reached the Success URL
  • Success URL matching is based on the path (e.g., /thank-you/), not the full URL

"My license won't activate"

  • Copy the key carefully from your CraftedPath account
  • Make sure you haven't used all your site activations
  • Check that your server can make outbound HTTPS requests

"The test has been running for weeks with no winner"

This happens when:

  • Traffic is low (just need more time)
  • Your variations are too similar (try a bolder change)
  • Your conversion rate is very low (test a higher-traffic page)

Technical Details

Cookie: cpab_visitor — stores a random ID (not personally identifiable), expires after 30 days

Database tables:

  • wp_cpab_tests — your tests
  • wp_cpab_variations — the variations for each test
  • wp_cpab_sessions — visitor sessions and conversions

Statistics: Uses Bayesian probability with Monte Carlo simulation. The plugin calculates how likely each variation is to be the true winner, rather than just comparing raw numbers.

Privacy: Only stores a hashed visitor ID. No personal information. If you're under GDPR, mention A/B testing in your privacy policy and note that you use first-party cookies.

To completely uninstall:

  1. Deactivate and delete the plugin
  2. Tables stay in your database by default
  3. To remove everything: drop the three wp_cpab_* tables and delete the cpab_db_version option