A walkthrough of the A/B testing journey in Live Chat, covering how experiments and variants are assigned, how long assignments remain sticky for authenticated and anonymous users, when and why users are re‑assigned, and what the experience looks like when multiple experiments run simultaneously or when none apply.

Before starting, review the Glossary to understand the key terms used throughout.

How Experiments and Variants Are Assigned

Whenever a user lands on a page where Live Chat is enabled or whenever the user is updated on that page, such as an anonymous user logging in or an authenticated user logging out and becoming a guest, the system decides whether the user should see the default chat experience or a variant from one of the running experiments.

Step-by-Step Assignment

1. On page load, and again whenever the user is updated, the system evaluates the target audience filters of all currently live experiments on that Live Chat application.

2. The system identifies which of those experiments the user is eligible for.

3. If the user is eligible for one or more experiments, one experiment is selected randomly.

4. Within the selected experiment, the user is randomly assigned to one variant, either the control version or one of the test variants, based on the configured traffic split.

What this means for the user

• The user always sees one consistent experience, either the default chat application or one variant of a single experiment.

• The user never sees a blended experience mixing pieces of multiple variants.

• The assignment decision happens before the chat widget renders, so the user does not see any flicker or version swap unless the user is updated.

Stickiness: Why the Same User Keeps Seeing the Same Variant

Once a user has been assigned a variant, the same assignment is reused on future page loads and user updates as long as the experiment is still running and the user still satisfies its target audience. This is what makes the experience consistent for the customer and the data clean for reporting.

How stickiness is scoped

What is re-evaluated on every page load

• Whether the previously assigned experiment is still live.

• Whether the user still satisfies that experiment's target audience filters on the current page and with their current identity.

• If both conditions are true, the same experiment and the same variant are reused, and no new random pick happens.

• If either is false, the user is treated as needing a fresh assignment (see Section 4).

How a User Can Fall Out of an Experiment

A user can stop seeing the variant they were assigned to. This is always intentional; re-evaluation only happens on a new page load or user update.

Reasons a user falls out

• The experiment itself ends.

• The user navigates to a page or state where they no longer match the experiment's target audience filters or no longer qualifies under the new identity.

What happens next

• On the next page load, the system evaluates all live experiments again.

• If the user is eligible for another experiment, they are assigned to one and randomly given a variant.

• If the user is not eligible for any live experiment, they see the default chat configuration.

No flicker guarantee

• All re-evaluation happens at page load or when the user is updated on login or logout.

• The same user never sees their old variant briefly replaced by a new one on the same page.

When the User is Eligible for Multiple Experiments

More than one experiment can be live at the same time on the same Live Chat application. If a user happens to match the target audience filters of more than one of them, the system assigns the user to a random experiment. A user is always in at most one experiment at a time.

If the user is already part of an active experiment and continues to meet the targeting criteria on a later page load, they remain assigned to the same experiment. This helps maintain a consistent user experience and ensures each variant receives enough users.

Worked Example: How the assignment rule plays Out

The example below shows the same user moving through different contexts while two experiments are running concurrently.

Setup

Two experiments are live on the same Live Chat application.

Step 1: User A opens URL X on laptop

User A is on a desktop browser, so they do not match Experiment 1's audience (mobile only). They are on URL X, so they do match Experiment 2's audience. The user is assigned to Experiment 2 and randomly placed in one of its variants according to 90-10 traffic split. They see that variant consistently on URL X.

Step 2: User A then opens URL Y on mobile

Now the user is on mobile but not on URL X. They no longer match Experiment 2. They do match Experiment 1 (mobile users). On this page load, they are assigned to Experiment 1 and randomly placed in one of its variants with a 50% probability of getting assigned either of control or variant A.

Step 3: User A then moves to URL X on mobile

Now the user matches the audience of both experiments at the same time (mobile and URL X). Because Experiment 1 was the latest experiment assigned to this user, they are assigned to Experiment 1 on this page load, not Experiment 2. They see the same variant of Experiment 1 consistently.

When No Experiment is Running (or User is Eligible for None)

If there are no live experiments on the application, or the user does not match the audience of any live experiment, the system shows the default chat configuration.

UX Scenarios: What the User Actually Sees

These scenarios describe the visible behaviour from the user's point of view. Each scenario assumes one or more experiments are running on the Live Chat application.

Scenario 1: Fresh visitor lands on an eligible page

The user lands on a page that matches the target audience of a running experiment. They are randomly assigned to one variant of that experiment and immediately see that variant. There is no flicker or transition from a default version.

Scenario 2: Authenticated user returns later

An authenticated user who was previously assigned to Variant A returns the next day. The experiment is still live and they still satisfy its audience. They see Variant A again, consistently with their previous session.

Scenario 3: Anonymous user navigates across pages in the same session

An anonymous user is assigned to a variant on their first eligible page load. As they navigate to other pages where the same experiment audience still applies, the same experiment and variant are reused. They never see a different variant in the same session.

Scenario 4: User moves to a page that no longer matches the experiment audience

On the new page, the user's previous experiment no longer applies. The system evaluates again on that page load — and would do the same on a user update. If another live experiment matches, they are assigned to one variant of it. If nothing matches, they see the default chat configuration.

Scenario 5: Experiment is stopped while the user is mid-session

The user continues to see the current rendered experience on the current page. They will be seeing the experiment variant until the next page load or user update, when the system evaluates again. If the user is now eligible for a different live experiment, they are placed into that one. If not, they see the default chat configuration.

Scenario 6: No live experiment applies to the user

The user sees the default chat configuration. The Live Chat experience behaves exactly as it would if A/B testing were not enabled.

Scenario 7: Anonymous user clears cookies or opens an incognito window

Because anonymous stickiness is tied to the browser session, the user is treated as new on the next page load or user update. The assignment is recalculated from scratch- they may or may not land in the same experiment or variant they had before.

Scenario 8: Anonymous user logs in mid-journey

As soon as the user logs in and the authenticated identity is passed to Live Chat, this counts as a user update and the system re-evaluates assignment using the authenticated identity — without waiting for the next page load. If they are now eligible for an experiment, they will start seeing a variant of that experiment. The same applies in reverse when an authenticated user logs out and becomes a guest.

Glossary

The terms below are used throughout this document with the specific meanings described here.