In this chapter, we'll see the concept of A/B testing. We'll go through the t-test, the t-statistic, and the p-value, all useful tools for determining whether a result is actually real or a result of random variation. We'll dive into some real examples and get our hands dirty with some Python code and compute the t-statistics and p-values.

Following that, we'll look into how long you should run an experiment for before reaching a conclusion. Finally, we'll discuss the potential issues that can harm the results of your experiment and may cause you to reach the wrong conclusion.

We'll cover the following topics:

• A/B testing concepts
• T-test and p-value
• Measuring t-statistics and p-values using Python
• Determining how long to run an experiment
• A/B test gotchas