T-test and p-value

How do you know if a change resulting from an A/B test is actually a real result of what you changed, or if it's just random variation? Well, there are a couple of statistical tools at our disposal called the t-test or t-statistic, and the p-value. Let's learn more about what those are and how they can help you determine whether an experiment is good or not.

The aim is to figure out if a result is real or not. Was this just a result of random variance that's inherent in the data itself, or are we seeing an actual, statistically significant change in behavior between our control group and our test group? T-tests and p-values are a way to compute that.

Remember, statistically significant doesn't really have a specific meaning. At the end of the day it has to be a judgment call. You have to pick a probability value that you're going to accept of a result being real or not. But there's always going to be a chance that it's still a result of random variation, and you have to make sure your stakeholders understand that.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.116.36.56