Hypothesis test: use the p value to accept or reject your hypothesis


Compile | VK
Source | analytics vidhya


Test is one of the most basic concepts in statistics. Not only in data science, hypothesis testing is very important in various fields. Want to know how to do it? Let’s take an example. Now there’s a lifebuoy shower gel.

Shower Gel manufacturers claim it kills 99.9% of bacteria. How can they say that? There has to be a test technique to prove that. So hypothesis testing is used to prove a claim or any hypothesis.


  1. Definition of hypothesis testing

  2. Null sum substitution hypothesis test

  3. Simple hypothesis test and compound hypothesis test

  4. Single tailed and double tailed tests

  5. Critical region

  6. Type I and type II errors.

  7. statistical significance

  8. Confidence level

  9. Importance

  10. P value

This blog breaks down these concepts into small pieces so you can understand their motivations and uses. When you finish reading this blog, the basics of hypothesis testing will be clear!!

Definition of hypothesis testing

Hypothesis is a statement, hypothesis or proposition about parameter values (mean, variance, median, etc.).

A hypothesis is a valid guess about something in the world around you. It should be tested by experiment or observation.

For example, if we say “Doni is the best Indian captain of all time,” it’s an assumption, based on the team’s average wins and losses during his time as captain. We can test the statement against all the matching data.

Null hypothesis and alternative hypothesis test

The null hypothesis is to test whether the hypothesis can be rejected if the hypothesis is true. Similar to the concept of innocence. We assume innocence until we have enough evidence to prove the suspect guilty.

In short, we can think of the null hypothesis as an accepted statement, for example, that the sky is blue. We have accepted this statement.

Denoted by H0.

The alternative hypothesis complements the null hypothesis. It is contrary to the original hypothesis, which covers all possible values of population parameters together with the original hypothesis.

Denoted by H1.

Let’s use an example to understand this:

A soap company claims that its products kill 99% of the bacteria on average. To test the company’s claims, we will propose a zero sum alternative hypothesis.

Zero hypothesis (H0): the average is equal to 99%

Alternative hypothesis (H1): the average value is not equal to 99%.

Note: when we test a hypothesis, we assume that the original hypothesis is true until there is enough evidence in the sample to prove it is false. In this case, we reject the original hypothesis and support the alternative hypothesis.

If the sample does not provide enough evidence for us to reject the null hypothesis, we cannot say that the null hypothesis is true because it is only based on sample data. The zero hypothesis needs to study the overall data.

Simple hypothesis test and compound hypothesis test

When an assumption specifies the exact value of a parameter, it is a simple assumption. If it specifies a range of values, it is called a compound assumption. for example

  • A motorcycle company claims that the average mileage per liter of a certain model is 100 km. This is a simple hypothetical case.

  • The average age of a class is over 20 years old. This is a composite hypothesis.

One tailed and two tailed hypothesis tests

If the substitution hypothesis gives the substitution of the parameter values specified in the null hypothesis in two directions (less than and greater than), it is called the double tailed test.

If the substitution hypothesis only gives the substitution of the parameter value specified in the null hypothesis in one direction (less than or greater than), it is called single tailed test. for example

  • H0: the average value is 100
  • H1: the average value is not equal to 100

According to H1, the average can be greater than or less than 100. This is an example of a two tailed test


  • H0: average value > = 100
  • H1: average < 100

Here, the average is less than 100. This is called the one tailed test.

Reject domain

The rejection field is the rejection region in the sample space. If the calculated value is in it, we reject the null hypothesis.

Let’s use an example to understand this:

Suppose you want to rent an apartment. You list all available apartments from different real country websites. Your budget is 15000 rupees per month. You can’t spend that much more. Your list of apartments ranges from 7000 / month to 30000 / month.

You randomly select an apartment from the list and assume the following assumptions:

  • H0: you want to rent this apartment.

  • H1: you won’t rent this apartment.

Now, since your budget is 15000, you have to turn down all apartments that are above that price.

Here all prices over 15000 become your rejection domain. If the price of a random apartment is in this area, you must reject your null hypothesis. If the price of an apartment is not in this area, you cannot reject your null hypothesis.

According to the alternative hypothesis, the rejection region is located on one or two tails of the probability distribution curve. The rejection region is a predefined region corresponding to the cut-off value in the probability distribution curve. It is represented by α.

A threshold is a value that separates values that support or reject the null hypothesis and is calculated based on alpha.

We’ll see more examples later, and we’ll clearly know how to choose alpha.

According to another hypothesis, there are three cases of rejection domain

Case 1)This is a double tailed test.

Case 2)This condition is also known as the left tail test.

Case 3)This condition is also known as the right tail test.

Type I and type II errors

Therefore, the first and second types of errors are one of the important topics of hypothesis testing. Let’s simplify the topic by breaking it down into smaller parts.

A false positive example (type I error)——When you reject a real zero hypothesis.

False negativity (type II error)——When you accept a false zero hypothesis.

  • The probability of type I error (false positive example) is equal to the significance level or size of critical region α.

    α = P [reject H0 when H0 is true]

  • The probability of type II error (false negative) is equal to β.

    β = P [do not reject H0 when H1 is true]


The man was arrested for burglary. A jury of judges must find guilty or innocent.

H0People are innocent

H1Man is guilty

The first type of error is if the jury finds someone guilty [refuses to accept H0], even though the person is innocent [H0 is true].

The second type of error would be when the jury releases the person [does not reject H0] although the person is guilty [H1 is true].

statistical significance

To understand this topic, let’s consider an example: suppose a confectionery produces 500 grams of candy a day. The factory no longer claimed that 500 grams of candy was lost after the factory or one more day was lost.

So, how can this worker claim this mistake? So, where should we draw a line to determine the weight of a candy bar? This decision / threshold is statistically significant.

confidence level

As the name suggests, how confident we are: how confident we are in making decisions. The LOC should be greater than 95%. Confidence levels below 95% are not acceptable.

Significance level (α)

Significance level, in the simplest terms, is the critical probability of mistakenly rejecting the zero hypothesis when it is actually true. This is also known as the type I error rate.

This is the probability of type I errors. It is also the size of the reject domain.

Generally speaking, in the test, it is very low, such as 0.05 (5%) or 0.01 (1%).

If H0 is not rejected at the significance level of 5%, then we can say that our null hypothesis is correct, with 95% confidence.

P value

Suppose we test hypotheses at a significance level of 1%.

H0: average value(we just assume a one tailed test.)

We get the threshold (based on the type of test we use) and find that our test statistic is greater than the threshold. Therefore, we have to reject the null hypothesis here because it is in the reject domain.

If the null hypothesis is rejected at 1%, it is certain that it will be rejected at a higher level of significance, such as 5% or 10%.

If our significance level is less than 1%, do we also have to reject our hypothesis?

Yes, that’s possible, and the “P value” is working.

P value is the minimum significance level that can reject the null hypothesis.

That’s why many tests now give a p value, and it’s more popular because it gives more information than the threshold.

  • For the right tail test:

    P value = P [test statistic > = observed value of test statistic]

  • For the left tail test:

    P value = P [test statistic < = observed value of test statistic]

  • For the double tailed test:

    P value = 2 * P [test statistic > = | observation value of test statistic |]

P-value decision making

We compared the p value with the significance level (alpha) to make a decision on the null hypothesis.

  • If P is greater than alpha, we do not reject the null hypothesis.

  • If the value of P is less than alpha, we reject the null hypothesis.

Link to the original text: https://www.analyticsvidhya.com/blog/2020/07/hypothesis-testing-68351/

Welcome to visit pan Chuang AI blog station:

Sklearn machine learning Chinese official document:

Welcome to pay attention to pan Chuang blog resource collection station: