Skip to main content

A/B testing the covid19 vaccine

Hi MD,

Vaccines for covid19 were first developed a year ago and finally they're here. What took them so long?

The answer to this question points to the most significant A/B test conducted in recent history: the clinical trials for making sure vaccines are safe and effective against the covid19 virus.

Clinical trials for COVID19 vaccines

From Vaccine Clinical Trials 101

These clinical trials take so long because they're divided into three phases, with each phase testing a different aspect of the vaccine. This 3 stage design has taken shape over multiple decades with thousands of scientists informing how it should be run.

Because human lives are at stake, clinical trials of vaccines are done very carefully. Be too hasty, and an unsafe vaccine can get released. But be too careful, and many more lives can be lost from the unchecked pandemic.

This tradeoff leads to thoughtful design of clinical trials of vaccines and hence for the conversion optimization industry, there's a lot to learn from it.

So, let's see the 3 stages in which vaccines are tested and what we can learn from them for our own A/B tests.

Phase 1: Test for side-effects

Once developed in the lab and tested on animal models, in the first phase, a vaccine is tested on a small group of humans. The primary objective of this phase is to ensure safety of the vaccine and to check if there are any side effects from it.

Translating it to the A/B testing world, if you have a large change to make (say a homepage change, or a massive marketing campaign), the first thing you need to check for is not the positive impact of the change but for potential negative impact. A variation that increases your clickthrough rate but ultimately decreases your business is no good. Hence, you should always have a set of business metrics (called guardrail metrics) which you never want to see getting reduced.

Phase 2: Test for effect-size

Once phase 1 is over, researchers enroll a bigger group and divide this group into multiple groups, with each group getting a different dosage level, and one group getting no dosage at all (and instead getting a placebo). You can think of this as an A/B/C/D test where the control means no change while different variations correspond to different dosage levels.

Volunteers belonging to different groups are then monitored over time to see whether the disease got cured or prevented, while ensuring there are no major side effects.

The variation with minimum side effects and maximum benefits is declared a winner, and then progressed to final phase 3 trials.

Phase 3: Large scale confirmatory test

By phase 2, there's clear evidence that the vaccine seems to work and researchers have discovered an appropriate dosage for it as well. However, the work isn't done.

Researchers want to be doubly sure that the vaccine will work when released to the masses, and hence in phase 3 significantly expand the number of people included in the test. Researchers actively seek out people from various previous conditions and age groups to ensure the sample they have is a good representation of the entire population.

This larger group is divided into two groups and the final A/B test happens with one placebo and the winner dosage from the previous phase. This is the phase that takes the most amount of time and effort.

Summing up: A/B testing works!

The tradeoffs relating to human lives seen for testing vaccines aren't likely to arrive in our businesses, but we all can learn from the rigorous manner in which scientists figure out things. Just like us, they're humans too and want so badly for vaccines and other treatments to work. But they also know that the cost of a wrong vaccine can be devastating.

In the same spirit, I think it's prudent to realize that even though we want our ideas to succeed so badly, we need to be careful about the possibility of such ideas making things worse for the business. A/B testing is the only way to ensure that they don't.

Regards,
Paras Chopra
Founder and Chairman, VWO

@paraschopra on Twitter

PS: These letters are getting cross-posted on our website too. Read my past letters here.



VWO (A Wingify product) 1104, 11th Floor, KLJ Tower North B-5, Netaji Subhash Place, Pitampura, Delhi – 110034, India
This email was sent to bablu.sahil786.rockmoney@blogger.com. Update your email preferences to choose the types of emails you receive.

Comments

Popular posts from this blog

ria money transfer- Online Money Transfer ria money

Riya Money Transfer is one of the largest money transfer companies in the world. With more than 317,000 locations in 146 countries around the world and an easy-to-use online dashboard, Rhea is convenient for both sender and recipient. Founded in New York in 1987, Rhea began with the simple goal of providing simple and reliable shipping services to people who ship home with an emphasis on friendly service. Over the years, Rhea has grown as a global company, expanding (and continuing to expand) to its worldwide location. In addition to remittance services, Riya Money Transfer offers bill pay, mobile recharge, prepaid debit cards, check cashing, money orders, and more. With a strong presence on almost every continent in the world, Riya is a brand that millions of people trust. Finally, as a company that places great value on the safety of its customers, you know that your money is always in good hands. The good thing about RIA Money Transfer $ 0 transfer fee when you use a bank account to

Youtube to Video Mp3 Mp4 Converter Download

Youtube Link to Video Mp3 Converter Download.  1. Youtube to mp3 Dowloader Converter.  You can download app here,  csc Best downloader app.  Youtube to mp3 converter This fast and easy to use converter allows you,  Download YouTube video mp3 for free wherever you are,  G0. Just one YouL ube link coPy, rewind and get ready to Listen to your mp3 file. So fast and easy! Instruction,  The main thing you need to do before getting ready,  The to-use mp3 has to be copied to a link first. Then go to our site 2conv.com and paste this link Special area in tne. Convert press to mp3. Your MP3 is ready to listen and enjoy! Don't forget to share this amazing ,opportunity Your friend. They will be very grateful to you! Enjoy Music with 2conv - Your Online YOu l ube to mp3 converter,  Are you tired of trying a fast and sociable to mp3, YouTube converter music that will allow you,  Download videos and favorite music tracks? And yours,  Tries to find an easy option to get your files,  Tre

SBI Clerk Exam Results

  SBI Clerk Results 2021 Out, Clerk prelims Marks out.  Table of Contents SBI Clerk Score Card 2021 Out SBI Clerk Prelims Score Card 2021 SBI Clerk Score Card & Marks 2021 SBI Clerk Score Card 2021 Out SBI Clerk Prelims Score Card 2021: State Bank of India has released the SBI Clerk Prelims Score Card 2021 along with the release of SBI Clerk Prelims Result on 21st September 2021. The candidates who have appeared in the prelims exam on 10th, 11th, 12th, 13th July and 29th August 2021 can check their SBI Clerk Score Card from the official website of SBI @sbi.co.in or directly from the below link mentioned in the article. SBI Clerk Prelims Score Card 2021 SBI Clerk Prelims Score Card 2021 Organisation     State Bank of India Exam Name     SBI Clerk 2021 Post Name     Junior Associates (Clerical Cadre) Vacancies     5454 SBI Clerk Exam Date     10th, 11th, 12th, 13th July and 29th August 2021 SBI Clerk Prelims Score Card     21st September 2021 SBI Clerk Prelims Result     21