A/B testing is a common practice in UX design and information architecture in which website traffic is split between two versions of a webpage to see which performs better. In an eCommerce context, A/B testing can help designers determine which of two or more page variants results in the best conversion rates.

While Noibu's bread and butter is error tracking, our script records every session, and you can leverage our Sessions as a tool to analyze A/B test results. Through the NoibuJS SDK, you can configure and collect custom ID values that correspond with each variant in the A/B test, which gives you an easy way to sort sessions on each variant. Once you've identified sessions for each page variant, you can watch session recordings to see how shoppers interact with the UX, and note which of the two options led to the most issues and which led to the most completed transactions.

Configuring the SDK for A/B Testing

Through the NoibuJS software development kit–or SDK–you can configure tracking of custom attributes in session recordings to make it easier to filter for sessions with specific characteristics. Usually, the page variant shown to a shopper in an A/B test is controlled by an appVersion or variant ID, and you can instruct the SDK to track this identifier.

For example, if you're comparing conversion rates on a checkout page running two versions–appVersion 123 vs appVersion 124, for example–instruct the SDK to track appVersion as a custom ID name. Once the test is complete, you can filter for sessions with an appVersion value of 123, and compare those results to sessions with an appVersion of 124.

To keep your results as focussed as possible, we recommend enabling the version or variant tracking shortly before the test begins, and disabling it afterwards.

Analyzing A/B Test Results

Beyond simply watching session recordings, you can use Sessions to analyze shopper friction and conversion rates on each page variant. Some data points to consider include:

• What is the average Friction Factor of sessions on each page variant?
• Does a given variant have recurring Session Symptoms? This may indicate a bug or UX issue.
• How long is the average session on each page variant? Does it take longer for shoppers to complete their journey on one version?
• How many shoppers in each variant reached the Checkout Completed funnel step, or the last intended step in the flow you're examining?

Learn more about Manual Testing with Help Code and Monitoring Webpage Health with Sessions.