Prepare Your Environment for A/B Testing

Prepare your environment

Environment - disk space and actions to take

If you want to test different index settings here are the steps:

• Ensure you have enough space on your account to duplicate index A
• Create an alternative index (B) using the replica feature
• Modify some index settings on B

Analytics

You need to implement Click Analytics to capture click and conversion events, because we use click and conversion rates to do comparative A/B testing. You don’t have to test both click and conversion events, only one of them is enough. See below for more on testing strategies.

Unique user identifier

Make sure every search is associated with a unique user identifier. This serves two purposes:

• It ensures randomness. The engine creates two sets of users - one for variant A and another for variant B. Whenever a new user arrives, they’re randomly distributed into A or B, based on a hash of their unique user identifier.
• It maintains the same distribution. If a user was assigned to variant A once, they continue with A and never B. This “user-to-variant stickiness” is mandatory for the A/B test conclusion to be reliable.

A unique user token is the mechanism by which Algolia ensures that the same user is always directed to the same variant. By default, the user token is the user’s IP address. However, you should not rely on this behavior. We recommend that you explicitly create a unique user identifier that you can keep track of in your own application. The user identifier can be sent to Algolia with the userToken parameter, or with the X-Algolia-UserToken request header.

A/B Testing

Once these steps are all done, you can then follow the Dashboard’s process to start, stop, and view your tests.