Sanity
Learn
CoursesA/B TestingImplementing an A/B test

A/B Testing

Lesson
3

Implementing an A/B test

How to query for data and how to setup an A/B test on a front end
Log in to mark your progress for each Lesson and Task

Depending on how content is rendered will influence how you fetch data to be used on the page.

If your page is statically generated, changing its content is not achievable at build time. In this instance, you might want to apply a routing-based approach with middleware.

For server-side rendering, pass the values for which variant to use as parameters to your query. This might require middleware to set the user group and store it in a cookie.

For client-side rendering you can add the parameters to the fetch—or you can fetch all variants and then filter the results client-side.

To ensure a consistent experience for a user it is a good idea to set a cookie to store the value of the user group they have been assigned.

For filtering on the query it may look something like this: 

*[
    _type == "event" &&
    slug.current == $slug
  ][0]{
  ...,
  "name": coalesce(
    newName.variants[
    experimentId == $experiment 
    && variantId == $variant
  ][0].value,
    newName.default,
    name
  ),
  "date": coalesce(date, now()),
  "doorsOpen": coalesce(doorsOpen, 0),
  headline->,
  venue->
}

This query uses the coalesce function to get the correct variant based on the experiment and the variant. If those are not present or do not match you'll get the default value, and then fallback to the existing name field incase the data has not been migrated across.

You will need to pass along values for the $experiment and $variant query params. These should come from an external service or a cookie set on the user.

If you're unable to perform filtering server-side in the query, you may instead query for all variants, and then write a function like this which will look through all of the returned variants and return only one.

const getExperimentContent = (field, experimentId, variantId) => {
  return (
    field.variants.find(
      (variant) =>
        variant.experimentId === experimentId && 
        variant.variantId === variantId
    )?.value || field.default
  );
};

You also need to consider occasions where A/B tests are not running, and some users might be excluded from an experiment, so we need to ensure that we get a fallback value if this is the case.

The following details how to implement A/B tests in Next.js, but the same principles and implementation patterns could be repeated in any framework.

Let's try implementing the A/B test created in the Studio in the Next.js application. For this you'll use the hardcoded experiment that was initially added.

In order to get a working A/B test you'll need some way of assigning an ID and group to a user—using middleware—and retrieving the variant that user should see.

Learn more about Next.js middleware in their documentation.

You'll also need a way to track if a user has viewed an experiment. This tracking would typically be done on an external A/B testing service like Google analytics, Segment, or others that could be linked up to a A/B testing service.

To be able to analyze the experiment we will need to know if a user has been included in an experiment and which variant they saw. What we can do is get the userID and userGroup and use this in a client component to send an event when the user has viewed the page with an experiment.

Create functions for getting variants, userId and setting user group
import { v4 } from "uuid";
import { cookies } from "next/headers";
import type { NextRequest, NextResponse } from "next/server";


type Experiment = Record<
  string,
  { label: string; variants: { id: string; label: string }[] }
>;


const EXPERIMENTS: Experiment = {
  "event-name": {
    label: "Event Name",
    variants: [
      {
        id: "control",
        label: "Control",
      },
      {
        id: "variant",
        label: "Variant",
      },
    ],
  },
};


const getTestCookie = async () => {
  const cookieStore = await cookies();
  return cookieStore.get("ab-test")?.value;
};


export const getUserGroup = async () => {
  const testCookie = await getTestCookie();
  return testCookie ? JSON.parse(testCookie)?.userGroup : "control";
};


// mocking a fetch to an external service for getting an experiment variant
export const getExperimentValue = async (experimentName: string) => {
  const userGroup = await getUserGroup();
  return {
    variant: EXPERIMENTS[experimentName].variants.find(
      (variant) => variant.id === userGroup
    ),
  };
};


export const setCookiesValue = (
  request: NextRequest,
  response: NextResponse
) => {
  if (!request.cookies.has("ab-test")) {
    // randomly assign a user to a group
    const userGroup = Math.random() > 0.5 ? "control" : "variant";
    // create a user ID
    const userId = v4();
    // Setting cookies on the response using the `ResponseCookies` API
    response.cookies.set("ab-test", JSON.stringify({ userGroup, userId }));
  }


  return response;
};


// If use is part of any experiments, get the tracking call data
// This is passed into the <Tracking> client component
export const getDeferredTrackingData = async (): Promise<
  | {
      userGroup: string;
      userId: string;
    }
  | undefined
> => {
  const testCookie = await getTestCookie();
  const data = testCookie ? JSON.parse(testCookie) : undefined;
  return data;
};
Create middleware to incepted the page request and storing the user group and user id as a cookie
import { NextResponse } from "next/server";
import type { NextRequest } from "next/server";
import { setCookiesValue } from "./lib/experiments";


export function middleware(request: NextRequest) {
  let response = NextResponse.next();
  response = setCookiesValue(request, response);


  return response;
}


export const config = {
  matcher: [
    /*
     * Match all request paths except for the ones starting with:
     * - api (API routes)
     * - _next/static (static files)
     * - _next/image (image optimization files)
     * - favicon.ico, sitemap.xml, robots.txt (metadata files)
     */
    "/((?!api|_next/static|_next/image|favicon.ico|sitemap.xml|robots.txt).*)",
  ],
};

A new component is required to send back events when a user has been assigned an ID and a group and has partaken in an experiment.

The example code below only logs these events to the console. You would need to replace this with the integration of your choice for measuring experiments.

Create client component for sending tracking data
"use client";


import { useEffect } from "react";


// Helper component to track experiment views from server components
export function Tracking({
  userGroup,
  userId,
}: {
  userGroup: string;
  userId: string;
}) {
  useEffect(() => {
    // TODO: track with Google Analytics, Segment, etc.
    console.log("Viewed Experiment, send tracking", {
      userGroup: userGroup,
      userId: userId,
    });
  }, [userId, userGroup]);


  return null;
}

Update the route for a single event to query for the correct variant as well as include the Tracking component.

Update the single event page route
import { getExperimentValue } from "@/lib/experiments";
import { client } from "@/sanity/client";
import { sanityFetch } from "@/sanity/live";
import imageUrlBuilder from "@sanity/image-url";
import { SanityImageSource } from "@sanity/image-url/lib/types/types";
import { defineQuery, PortableText } from "next-sanity";
import Image from "next/image";
import Link from "next/link";
import { notFound } from "next/navigation";


const EVENT_QUERY = defineQuery(`*[
    _type == "event" &&
    slug.current == $slug
  ][0]{
  ...,
  "name": coalesce(newName.variants[experimentId == $experiment && variantId == $variant][0].value, newName.default, name),
  "date": coalesce(date, now()),
  "doorsOpen": coalesce(doorsOpen, 0),
  headline->,
  venue->
}`);


const { projectId, dataset } = client.config();
const urlFor = (source: SanityImageSource) =>
  projectId && dataset
    ? imageUrlBuilder({ projectId, dataset }).image(source)
    : null;


export default async function EventPage({
  params,
}: {
  params: Promise<{ slug: string }>;
}) {
  const { slug } = await params;


  const { variant } = await getExperimentValue("event-name");
  const trackingData = await getDeferredTrackingData();


  const queryParams = {
    slug,
    experiment: "event-name",
    variant: variant?.id || "",
  };


  const { data: event } = await sanityFetch({
    query: EVENT_QUERY,
    params: queryParams,
  });


  if (!event) {
    notFound();
  }
  const {
    name,
    date,
    headline,
    image,
    details,
    eventType,
    doorsOpen,
    venue,
    tickets,
  } = event;
  const eventImageUrl = image
    ? urlFor(image)?.width(550).height(310).url()
    : null;
  const eventDate = new Date(date).toDateString();
  const eventTime = new Date(date).toLocaleTimeString();
  const doorsOpenTime = new Date(
    new Date(date).getTime() - doorsOpen * 60000
  ).toLocaleTimeString();


  return (
    <main className="container mx-auto grid gap-12 p-12">
      <div className="mb-4">
        <Link href="/">← Back to events</Link>
      </div>
      <div className="grid items-top gap-12 sm:grid-cols-2">
        <Image
          src={eventImageUrl || "https://placehold.co/550x310/png"}
          alt={name || "Event"}
          className="mx-auto aspect-video overflow-hidden rounded-xl object-cover object-center sm:w-full"
          height="310"
          width="550"
        />
        <div className="flex flex-col justify-center space-y-4">
          <div className="space-y-4">
            {eventType ? (
              <div className="inline-block rounded-lg bg-gray-100 px-3 py-1 text-sm dark:bg-gray-800 capitalize">
                {eventType.replace("-", " ")}
              </div>
            ) : null}
            {name ? (
              <h1 className="text-4xl font-bold tracking-tighter mb-8">
                {name}
              </h1>
            ) : null}
            {headline?.name ? (
              <dl className="grid grid-cols-2 gap-1 text-sm font-medium sm:gap-2 lg:text-base">
                <dd className="font-semibold">Artist</dd>
                <dt>{headline?.name}</dt>
              </dl>
            ) : null}
            <dl className="grid grid-cols-2 gap-1 text-sm font-medium sm:gap-2 lg:text-base">
              <dd className="font-semibold">Date</dd>
              <div>
                {eventDate && <dt>{eventDate}</dt>}
                {eventTime && <dt>{eventTime}</dt>}
              </div>
            </dl>
            {doorsOpenTime ? (
              <dl className="grid grid-cols-2 gap-1 text-sm font-medium sm:gap-2 lg:text-base">
                <dd className="font-semibold">Doors Open</dd>
                <div className="grid gap-1">
                  <dt>Doors Open</dt>
                  <dt>{doorsOpenTime}</dt>
                </div>
              </dl>
            ) : null}
            {venue?.name ? (
              <dl className="grid grid-cols-2 gap-1 text-sm font-medium sm:gap-2 lg:text-base">
                <div className="flex items-start">
                  <dd className="font-semibold">Venue</dd>
                </div>
                <div className="grid gap-1">
                  <dt>{venue.name}</dt>
                </div>
              </dl>
            ) : null}
          </div>
          {details && details.length > 0 && (
            <div className="prose max-w-none">
              <PortableText value={details} />
            </div>
          )}
          {tickets && (
            <a
              className="flex items-center justify-center rounded-md bg-blue-500 p-4 text-white"
              href={tickets}
            >
              Buy Tickets
            </a>
          )}
        </div>
      </div>
      {trackingData && (
        <Tracking
          userGroup={trackingData.userGroup}
          userId={trackingData.userId}
        />
      )}
    </main>
  );
}

With this done you will now see that a cookie is set when you visit an event page, and that based on that cookie and the content in your Sanity Studio the name of the event will show 1 of 3 options (the default if no experiment matches, the control for an experiment, or the variant).

If you remove the cookie and refresh the page you will see a new cookie is added and there is a random chance it will be assigned a different group, thus potentially showing you a different name.

Let's test everything you've learned in the final lesson with a quiz.

You have 4 uncompleted tasks in this lesson
0 of 4