Hypothesis testing (a.k.a. A/B testing) is the gold standard in many quantitative and qualitative research fields. A/B tests are typically used when you want to compare data between two conditions that differ on a key independent variable. For example, if you want feedback on how a new logo compares with your current logo you might run an A/B test where half of the participants provide feedback on the existing logo and then the other half provide feedback on the new logo. In both conditions, the same questions and measures are used so that the two groups can be directly compared.
Why Can’t the Same Participants Provide Feedback on Both Logos?
Asking the same participants to provide feedback on both levels of your independent variable is an example of a “within-subject method”, and introduces complications like carry-over effects, where feedback on one logo influences feedback on the other logo. In this case, you would want to ensure that the initial logo that participants provide feedback on is randomized between the two variants.
A/B testing with random assignment ensures that you collect data that is easily interpretable and, more importantly, unbiased.
Option 1: Use Our A/B Test Template
Our A/B test template is the perfect solution for creating a quick A/B test with audio response question types.
Step 1: Select the A/B Test Template
Start by creating a new survey and selecting the A/B test template. This will open a new survey draft with four pre-populated questions. The A/B test template is set up so that half of your participants are randomly assigned to respond to the first two questions (the “A” group) and the other half are randomly assigned to the last two questions (the “B” group).
Step 2: Customize the Template
You can customize the template by editing the question text, uploading your own stimuli, and changing the question type. You can also add additional questions to the survey, but be careful of where additional questions are added; built-in survey logic maintains the random assignment functionality. Questions for group A must be added after the first question in the template, while questions for group B must be added after the third question in the template.
Step 3: Run It!
And that’s it! Click “Deploy” and you’re ready to run your A/B test with random condition assignment. After you’re done collecting responses, you can easily check which condition participants were assigned to by exporting your data as a CSV file.
Option 2: Build Your A/B Test From Scratch
Building your A/B test from scratch is a good option for surveys that you have already started to build or if you need to test more than 2 conditions.
Step 1: Build Your Survey in Blocks
Building a simple A/B test in Phonic (i.e., two groups) is easy. Start by building your survey in the survey builder. You will want to create two “blocks” of questions: one for “A” participants and one for “B” participants. Build the survey such that all Block A questions are grouped together and all Block B questions are grouped together. For example, in a 10 question survey, questions 1-5 would belong to Block A while questions 6-10 would belong to Block B.
Step 2: Create Random Conditions
Once your survey is created, hop over to the survey settings tab. In survey settings, you will see “Random Conditions”. Applying random conditions to a survey randomly assigns participants to predefined groups with some associated likelihood. In this case, we will have “Group A” and “Group B”, and half of the participants will be assigned to Group A and half will be assigned to Group B.
Step 3: Add Survey Logic
After your random conditions have been created, head back to the questions tab. Here you will reference your random conditions in survey logic, to display the two blocks accordingly. Survey logic should be applied on the first question of each block. On the first question of Block A, logic needs to be set so that Group B individuals skip forward to the beginning of Block B.
Survey logic should also be added on the first question of Block B, such that those in Group A skip forward to the End of Survey.
Step 4: Run It!
And that’s it! Click “Deploy” and you’re ready to run your A/B test with random condition assignment. After you’re done collecting responses, you can easily check which condition participants were assigned to by exporting your data as a CSV file.
Need assistance building a survey? The Phonic team is here to help! Get in touch with us at contact@phonic.ai.
