definition of probability and hypothesis testing

school Campus Bookshelves
menu_book Bookshelves
perm_media Learning Objects
login Login
how_to_reg Request Instructor Account
hub Instructor Commons
Download Page (PDF)
Download Full Book (PDF)
Periodic Table
Physics Constants
Scientific Calculator
Reference & Cite
Tools expand_more
Readability

selected template will load here

This action is not available.

9.1: Introduction to Hypothesis Testing

Last updated
Save as PDF
Page ID 10211

Kyle Siegrist
University of Alabama in Huntsville via Random Services

Basic Theory

Preliminaries.

As usual, our starting point is a random experiment with an underlying sample space and a probability measure $\P$. In the basic statistical model, we have an observable random variable $\bs{X}$ taking values in a set $S$. In general, $\bs{X}$ can have quite a complicated structure. For example, if the experiment is to sample $n$ objects from a population and record various measurements of interest, then \[ \bs{X} = (X_1, X_2, \ldots, X_n) \] where $X_i$ is the vector of measurements for the $i$th object. The most important special case occurs when $(X_1, X_2, \ldots, X_n)$ are independent and identically distributed. In this case, we have a random sample of size $n$ from the common distribution.

The purpose of this section is to define and discuss the basic concepts of statistical hypothesis testing . Collectively, these concepts are sometimes referred to as the Neyman-Pearson framework, in honor of Jerzy Neyman and Egon Pearson, who first formalized them.

A statistical hypothesis is a statement about the distribution of $\bs{X}$. Equivalently, a statistical hypothesis specifies a set of possible distributions of $\bs{X}$: the set of distributions for which the statement is true. A hypothesis that specifies a single distribution for $\bs{X}$ is called simple ; a hypothesis that specifies more than one distribution for $\bs{X}$ is called composite .

In hypothesis testing , the goal is to see if there is sufficient statistical evidence to reject a presumed null hypothesis in favor of a conjectured alternative hypothesis . The null hypothesis is usually denoted $H_0$ while the alternative hypothesis is usually denoted $H_1$.

An hypothesis test is a statistical decision ; the conclusion will either be to reject the null hypothesis in favor of the alternative, or to fail to reject the null hypothesis. The decision that we make must, of course, be based on the observed value $\bs{x}$ of the data vector $\bs{X}$. Thus, we will find an appropriate subset $R$ of the sample space $S$ and reject $H_0$ if and only if $\bs{x} \in R$. The set $R$ is known as the rejection region or the critical region . Note the asymmetry between the null and alternative hypotheses. This asymmetry is due to the fact that we assume the null hypothesis, in a sense, and then see if there is sufficient evidence in $\bs{x}$ to overturn this assumption in favor of the alternative.

An hypothesis test is a statistical analogy to proof by contradiction, in a sense. Suppose for a moment that $H_1$ is a statement in a mathematical theory and that $H_0$ is its negation. One way that we can prove $H_1$ is to assume $H_0$ and work our way logically to a contradiction. In an hypothesis test, we don't prove anything of course, but there are similarities. We assume $H_0$ and then see if the data $\bs{x}$ are sufficiently at odds with that assumption that we feel justified in rejecting $H_0$ in favor of $H_1$.

Often, the critical region is defined in terms of a statistic $w(\bs{X})$, known as a test statistic , where $w$ is a function from $S$ into another set $T$. We find an appropriate rejection region $R_T \subseteq T$ and reject $H_0$ when the observed value $w(\bs{x}) \in R_T$. Thus, the rejection region in $S$ is then $R = w^{-1}(R_T) = \left\{\bs{x} \in S: w(\bs{x}) \in R_T\right\}$. As usual, the use of a statistic often allows significant data reduction when the dimension of the test statistic is much smaller than the dimension of the data vector.

The ultimate decision may be correct or may be in error. There are two types of errors, depending on which of the hypotheses is actually true.

Types of errors:

A type 1 error is rejecting the null hypothesis $H_0$ when $H_0$ is true.
A type 2 error is failing to reject the null hypothesis $H_0$ when the alternative hypothesis $H_1$ is true.

Similarly, there are two ways to make a correct decision: we could reject $H_0$ when $H_1$ is true or we could fail to reject $H_0$ when $H_0$ is true. The possibilities are summarized in the following table:

Of course, when we observe $\bs{X} = \bs{x}$ and make our decision, either we will have made the correct decision or we will have committed an error, and usually we will never know which of these events has occurred. Prior to gathering the data, however, we can consider the probabilities of the various errors.

If $H_0$ is true (that is, the distribution of $\bs{X}$ is specified by $H_0$), then $\P(\bs{X} \in R)$ is the probability of a type 1 error for this distribution. If $H_0$ is composite, then $H_0$ specifies a variety of different distributions for $\bs{X}$ and thus there is a set of type 1 error probabilities.

The maximum probability of a type 1 error, over the set of distributions specified by $ H_0 $, is the significance level of the test or the size of the critical region.

The significance level is often denoted by $\alpha$. Usually, the rejection region is constructed so that the significance level is a prescribed, small value (typically 0.1, 0.05, 0.01).

If $H_1$ is true (that is, the distribution of $\bs{X}$ is specified by $H_1$), then $\P(\bs{X} \notin R)$ is the probability of a type 2 error for this distribution. Again, if $H_1$ is composite then $H_1$ specifies a variety of different distributions for $\bs{X}$, and thus there will be a set of type 2 error probabilities. Generally, there is a tradeoff between the type 1 and type 2 error probabilities. If we reduce the probability of a type 1 error, by making the rejection region $R$ smaller, we necessarily increase the probability of a type 2 error because the complementary region $S \setminus R$ is larger.

The extreme cases can give us some insight. First consider the decision rule in which we never reject $H_0$, regardless of the evidence $\bs{x}$. This corresponds to the rejection region $R = \emptyset$. A type 1 error is impossible, so the significance level is 0. On the other hand, the probability of a type 2 error is 1 for any distribution defined by $H_1$. At the other extreme, consider the decision rule in which we always rejects $H_0$ regardless of the evidence $\bs{x}$. This corresponds to the rejection region $R = S$. A type 2 error is impossible, but now the probability of a type 1 error is 1 for any distribution defined by $H_0$. In between these two worthless tests are meaningful tests that take the evidence $\bs{x}$ into account.

If $H_1$ is true, so that the distribution of $\bs{X}$ is specified by $H_1$, then $\P(\bs{X} \in R)$, the probability of rejecting $H_0$ is the power of the test for that distribution.

Thus the power of the test for a distribution specified by $ H_1 $ is the probability of making the correct decision.

Suppose that we have two tests, corresponding to rejection regions $R_1$ and $R_2$, respectively, each having significance level $\alpha$. The test with region $R_1$ is uniformly more powerful than the test with region $R_2$ if \[ \P(\bs{X} \in R_1) \ge \P(\bs{X} \in R_2) \text{ for every distribution of } \bs{X} \text{ specified by } H_1 \]

Naturally, in this case, we would prefer the first test. Often, however, two tests will not be uniformly ordered; one test will be more powerful for some distributions specified by $H_1$ while the other test will be more powerful for other distributions specified by $H_1$.

If a test has significance level $\alpha$ and is uniformly more powerful than any other test with significance level $\alpha$, then the test is said to be a uniformly most powerful test at level $\alpha$.

Clearly a uniformly most powerful test is the best we can do.

$P$-value

In most cases, we have a general procedure that allows us to construct a test (that is, a rejection region $R_\alpha$) for any given significance level $\alpha \in (0, 1)$. Typically, $R_\alpha$ decreases (in the subset sense) as $\alpha$ decreases.

The $P$-value of the observed value $\bs{x}$ of $\bs{X}$, denoted $P(\bs{x})$, is defined to be the smallest $\alpha$ for which $\bs{x} \in R_\alpha$; that is, the smallest significance level for which $H_0$ is rejected, given $\bs{X} = \bs{x}$.

Knowing $P(\bs{x})$ allows us to test $H_0$ at any significance level for the given data $\bs{x}$: If $P(\bs{x}) \le \alpha$ then we would reject $H_0$ at significance level $\alpha$; if $P(\bs{x}) \gt \alpha$ then we fail to reject $H_0$ at significance level $\alpha$. Note that $P(\bs{X})$ is a statistic . Informally, $P(\bs{x})$ can often be thought of as the probability of an outcome as or more extreme than the observed value $\bs{x}$, where extreme is interpreted relative to the null hypothesis $H_0$.

Analogy with Justice Systems

There is a helpful analogy between statistical hypothesis testing and the criminal justice system in the US and various other countries. Consider a person charged with a crime. The presumed null hypothesis is that the person is innocent of the crime; the conjectured alternative hypothesis is that the person is guilty of the crime. The test of the hypotheses is a trial with evidence presented by both sides playing the role of the data. After considering the evidence, the jury delivers the decision as either not guilty or guilty . Note that innocent is not a possible verdict of the jury, because it is not the point of the trial to prove the person innocent. Rather, the point of the trial is to see whether there is sufficient evidence to overturn the null hypothesis that the person is innocent in favor of the alternative hypothesis of that the person is guilty. A type 1 error is convicting a person who is innocent; a type 2 error is acquitting a person who is guilty. Generally, a type 1 error is considered the more serious of the two possible errors, so in an attempt to hold the chance of a type 1 error to a very low level, the standard for conviction in serious criminal cases is beyond a reasonable doubt .

Tests of an Unknown Parameter

Hypothesis testing is a very general concept, but an important special class occurs when the distribution of the data variable $\bs{X}$ depends on a parameter $\theta$ taking values in a parameter space $\Theta$. The parameter may be vector-valued, so that $\bs{\theta} = (\theta_1, \theta_2, \ldots, \theta_n)$ and $\Theta \subseteq \R^k$ for some $k \in \N_+$. The hypotheses generally take the form \[ H_0: \theta \in \Theta_0 \text{ versus } H_1: \theta \notin \Theta_0 \] where $\Theta_0$ is a prescribed subset of the parameter space $\Theta$. In this setting, the probabilities of making an error or a correct decision depend on the true value of $\theta$. If $R$ is the rejection region, then the power function $ Q $ is given by \[ Q(\theta) = \P_\theta(\bs{X} \in R), \quad \theta \in \Theta \] The power function gives a lot of information about the test.

The power function satisfies the following properties:

$Q(\theta)$ is the probability of a type 1 error when $\theta \in \Theta_0$.
$\max\left\{Q(\theta): \theta \in \Theta_0\right\}$ is the significance level of the test.
$1 - Q(\theta)$ is the probability of a type 2 error when $\theta \notin \Theta_0$.
$Q(\theta)$ is the power of the test when $\theta \notin \Theta_0$.

If we have two tests, we can compare them by means of their power functions.

Suppose that we have two tests, corresponding to rejection regions $R_1$ and $R_2$, respectively, each having significance level $\alpha$. The test with rejection region $R_1$ is uniformly more powerful than the test with rejection region $R_2$ if $ Q_1(\theta) \ge Q_2(\theta)$ for all $ \theta \notin \Theta_0 $.

Most hypothesis tests of an unknown real parameter $\theta$ fall into three special cases:

Suppose that $ \theta $ is a real parameter and $ \theta_0 \in \Theta $ a specified value. The tests below are respectively the two-sided test , the left-tailed test , and the right-tailed test .

$H_0: \theta = \theta_0$ versus $H_1: \theta \ne \theta_0$
$H_0: \theta \ge \theta_0$ versus $H_1: \theta \lt \theta_0$
$H_0: \theta \le \theta_0$ versus $H_1: \theta \gt \theta_0$

Thus the tests are named after the conjectured alternative. Of course, there may be other unknown parameters besides $\theta$ (known as nuisance parameters ).

Equivalence Between Hypothesis Test and Confidence Sets

There is an equivalence between hypothesis tests and confidence sets for a parameter $\theta$.

Suppose that $C(\bs{x})$ is a $1 - \alpha$ level confidence set for $\theta$. The following test has significance level $\alpha$ for the hypothesis $ H_0: \theta = \theta_0 $ versus $ H_1: \theta \ne \theta_0 $: Reject $H_0$ if and only if $\theta_0 \notin C(\bs{x})$

By definition, $\P[\theta \in C(\bs{X})] = 1 - \alpha$. Hence if $H_0$ is true so that $\theta = \theta_0$, then the probability of a type 1 error is $P[\theta \notin C(\bs{X})] = \alpha$.

Equivalently, we fail to reject $H_0$ at significance level $\alpha$ if and only if $\theta_0$ is in the corresponding $1 - \alpha$ level confidence set. In particular, this equivalence applies to interval estimates of a real parameter $\theta$ and the common tests for $\theta$ given above .

In each case below, the confidence interval has confidence level $1 - \alpha$ and the test has significance level $\alpha$.

Suppose that $\left[L(\bs{X}, U(\bs{X})\right]$ is a two-sided confidence interval for $\theta$. Reject $H_0: \theta = \theta_0$ versus $H_1: \theta \ne \theta_0$ if and only if $\theta_0 \lt L(\bs{X})$ or $\theta_0 \gt U(\bs{X})$.
Suppose that $L(\bs{X})$ is a confidence lower bound for $\theta$. Reject $H_0: \theta \le \theta_0$ versus $H_1: \theta \gt \theta_0$ if and only if $\theta_0 \lt L(\bs{X})$.
Suppose that $U(\bs{X})$ is a confidence upper bound for $\theta$. Reject $H_0: \theta \ge \theta_0$ versus $H_1: \theta \lt \theta_0$ if and only if $\theta_0 \gt U(\bs{X})$.

Pivot Variables and Test Statistics

Recall that confidence sets of an unknown parameter $\theta$ are often constructed through a pivot variable , that is, a random variable $W(\bs{X}, \theta)$ that depends on the data vector $\bs{X}$ and the parameter $\theta$, but whose distribution does not depend on $\theta$ and is known. In this case, a natural test statistic for the basic tests given above is $W(\bs{X}, \theta_0)$.

Statistics Made Easy

Introduction to Hypothesis Testing

A statistical hypothesis is an assumption about a population parameter .

For example, we may assume that the mean height of a male in the U.S. is 70 inches.

The assumption about the height is the statistical hypothesis and the true mean height of a male in the U.S. is the population parameter .

A hypothesis test is a formal statistical test we use to reject or fail to reject a statistical hypothesis.

The Two Types of Statistical Hypotheses

To test whether a statistical hypothesis about a population parameter is true, we obtain a random sample from the population and perform a hypothesis test on the sample data.

There are two types of statistical hypotheses:

The null hypothesis , denoted as H 0 , is the hypothesis that the sample data occurs purely from chance.

The alternative hypothesis , denoted as H 1 or H a , is the hypothesis that the sample data is influenced by some non-random cause.

Hypothesis Tests

A hypothesis test consists of five steps:

1. State the hypotheses.

State the null and alternative hypotheses. These two hypotheses need to be mutually exclusive, so if one is true then the other must be false.

2. Determine a significance level to use for the hypothesis.

Decide on a significance level. Common choices are .01, .05, and .1.

3. Find the test statistic.

Find the test statistic and the corresponding p-value. Often we are analyzing a population mean or proportion and the general formula to find the test statistic is: (sample statistic – population parameter) / (standard deviation of statistic)

4. Reject or fail to reject the null hypothesis.

Using the test statistic or the p-value, determine if you can reject or fail to reject the null hypothesis based on the significance level.

The p-value tells us the strength of evidence in support of a null hypothesis. If the p-value is less than the significance level, we reject the null hypothesis.

5. Interpret the results.

Interpret the results of the hypothesis test in the context of the question being asked.

The Two Types of Decision Errors

There are two types of decision errors that one can make when doing a hypothesis test:

Type I error: You reject the null hypothesis when it is actually true. The probability of committing a Type I error is equal to the significance level, often called alpha , and denoted as α.

Type II error: You fail to reject the null hypothesis when it is actually false. The probability of committing a Type II error is called the Power of the test or Beta , denoted as β.

One-Tailed and Two-Tailed Tests

A statistical hypothesis can be one-tailed or two-tailed.

A one-tailed hypothesis involves making a “greater than” or “less than ” statement.

For example, suppose we assume the mean height of a male in the U.S. is greater than or equal to 70 inches. The null hypothesis would be H0: µ ≥ 70 inches and the alternative hypothesis would be Ha: µ < 70 inches.

A two-tailed hypothesis involves making an “equal to” or “not equal to” statement.

For example, suppose we assume the mean height of a male in the U.S. is equal to 70 inches. The null hypothesis would be H0: µ = 70 inches and the alternative hypothesis would be Ha: µ ≠ 70 inches.

Note: The “equal” sign is always included in the null hypothesis, whether it is =, ≥, or ≤.

Related: What is a Directional Hypothesis?

Types of Hypothesis Tests

There are many different types of hypothesis tests you can perform depending on the type of data you’re working with and the goal of your analysis.

The following tutorials provide an explanation of the most common types of hypothesis tests:

Introduction to the One Sample t-test Introduction to the Two Sample t-test Introduction to the Paired Samples t-test Introduction to the One Proportion Z-Test Introduction to the Two Proportion Z-Test

Published by Zach

Unit 12: Significance tests (hypothesis testing)

About this unit, the idea of significance tests.

Simple hypothesis testing (Opens a modal)
Idea behind hypothesis testing (Opens a modal)
Examples of null and alternative hypotheses (Opens a modal)
P-values and significance tests (Opens a modal)
Comparing P-values to different significance levels (Opens a modal)
Estimating a P-value from a simulation (Opens a modal)
Using P-values to make conclusions (Opens a modal)
Simple hypothesis testing Get 3 of 4 questions to level up!
Writing null and alternative hypotheses Get 3 of 4 questions to level up!
Estimating P-values from simulations Get 3 of 4 questions to level up!

Error probabilities and power

Introduction to Type I and Type II errors (Opens a modal)
Type 1 errors (Opens a modal)
Examples identifying Type I and Type II errors (Opens a modal)
Introduction to power in significance tests (Opens a modal)
Examples thinking about power in significance tests (Opens a modal)
Consequences of errors and significance (Opens a modal)
Type I vs Type II error Get 3 of 4 questions to level up!
Error probabilities and power Get 3 of 4 questions to level up!

Tests about a population proportion

Constructing hypotheses for a significance test about a proportion (Opens a modal)
Conditions for a z test about a proportion (Opens a modal)
Reference: Conditions for inference on a proportion (Opens a modal)
Calculating a z statistic in a test about a proportion (Opens a modal)
Calculating a P-value given a z statistic (Opens a modal)
Making conclusions in a test about a proportion (Opens a modal)
Writing hypotheses for a test about a proportion Get 3 of 4 questions to level up!
Conditions for a z test about a proportion Get 3 of 4 questions to level up!
Calculating the test statistic in a z test for a proportion Get 3 of 4 questions to level up!
Calculating the P-value in a z test for a proportion Get 3 of 4 questions to level up!
Making conclusions in a z test for a proportion Get 3 of 4 questions to level up!

Tests about a population mean

Writing hypotheses for a significance test about a mean (Opens a modal)
Conditions for a t test about a mean (Opens a modal)
Reference: Conditions for inference on a mean (Opens a modal)
When to use z or t statistics in significance tests (Opens a modal)
Example calculating t statistic for a test about a mean (Opens a modal)
Using TI calculator for P-value from t statistic (Opens a modal)
Using a table to estimate P-value from t statistic (Opens a modal)
Comparing P-value from t statistic to significance level (Opens a modal)
Free response example: Significance test for a mean (Opens a modal)
Writing hypotheses for a test about a mean Get 3 of 4 questions to level up!
Conditions for a t test about a mean Get 3 of 4 questions to level up!
Calculating the test statistic in a t test for a mean Get 3 of 4 questions to level up!
Calculating the P-value in a t test for a mean Get 3 of 4 questions to level up!
Making conclusions in a t test for a mean Get 3 of 4 questions to level up!

Hypothesis Testing

Hypothesis testing is a tool for making statistical inferences about the population data. It is an analysis tool that tests assumptions and determines how likely something is within a given standard of accuracy. Hypothesis testing provides a way to verify whether the results of an experiment are valid.

A null hypothesis and an alternative hypothesis are set up before performing the hypothesis testing. This helps to arrive at a conclusion regarding the sample obtained from the population. In this article, we will learn more about hypothesis testing, its types, steps to perform the testing, and associated examples.

What is Hypothesis Testing in Statistics?

Hypothesis testing uses sample data from the population to draw useful conclusions regarding the population probability distribution . It tests an assumption made about the data using different types of hypothesis testing methodologies. The hypothesis testing results in either rejecting or not rejecting the null hypothesis.

Hypothesis Testing Definition

Hypothesis testing can be defined as a statistical tool that is used to identify if the results of an experiment are meaningful or not. It involves setting up a null hypothesis and an alternative hypothesis. These two hypotheses will always be mutually exclusive. This means that if the null hypothesis is true then the alternative hypothesis is false and vice versa. An example of hypothesis testing is setting up a test to check if a new medicine works on a disease in a more efficient manner.

Null Hypothesis

The null hypothesis is a concise mathematical statement that is used to indicate that there is no difference between two possibilities. In other words, there is no difference between certain characteristics of data. This hypothesis assumes that the outcomes of an experiment are based on chance alone. It is denoted as $H_{0}$. Hypothesis testing is used to conclude if the null hypothesis can be rejected or not. Suppose an experiment is conducted to check if girls are shorter than boys at the age of 5. The null hypothesis will say that they are the same height.

Alternative Hypothesis

The alternative hypothesis is an alternative to the null hypothesis. It is used to show that the observations of an experiment are due to some real effect. It indicates that there is a statistical significance between two possible outcomes and can be denoted as $H_{1}$ or $H_{a}$. For the above-mentioned example, the alternative hypothesis would be that girls are shorter than boys at the age of 5.

Hypothesis Testing P Value

In hypothesis testing, the p value is used to indicate whether the results obtained after conducting a test are statistically significant or not. It also indicates the probability of making an error in rejecting or not rejecting the null hypothesis.This value is always a number between 0 and 1. The p value is compared to an alpha level, $\alpha$ or significance level. The alpha level can be defined as the acceptable risk of incorrectly rejecting the null hypothesis. The alpha level is usually chosen between 1% to 5%.

Hypothesis Testing Critical region

All sets of values that lead to rejecting the null hypothesis lie in the critical region. Furthermore, the value that separates the critical region from the non-critical region is known as the critical value.

Hypothesis Testing Formula

Depending upon the type of data available and the size, different types of hypothesis testing are used to determine whether the null hypothesis can be rejected or not. The hypothesis testing formula for some important test statistics are given below:

z = $\frac{\overline{x}-\mu}{\frac{\sigma}{\sqrt{n}}}$. $\overline{x}$ is the sample mean, $\mu$ is the population mean, $\sigma$ is the population standard deviation and n is the size of the sample.
t = $\frac{\overline{x}-\mu}{\frac{s}{\sqrt{n}}}$. s is the sample standard deviation.
$\chi ^{2} = \sum \frac{(O_{i}-E_{i})^{2}}{E_{i}}$. $O_{i}$ is the observed value and $E_{i}$ is the expected value.

We will learn more about these test statistics in the upcoming section.

Types of Hypothesis Testing

Selecting the correct test for performing hypothesis testing can be confusing. These tests are used to determine a test statistic on the basis of which the null hypothesis can either be rejected or not rejected. Some of the important tests used for hypothesis testing are given below.

Hypothesis Testing Z Test

A z test is a way of hypothesis testing that is used for a large sample size (n ≥ 30). It is used to determine whether there is a difference between the population mean and the sample mean when the population standard deviation is known. It can also be used to compare the mean of two samples. It is used to compute the z test statistic. The formulas are given as follows:

One sample: z = $\frac{\overline{x}-\mu}{\frac{\sigma}{\sqrt{n}}}$.
Two samples: z = $\frac{(\overline{x_{1}}-\overline{x_{2}})-(\mu_{1}-\mu_{2})}{\sqrt{\frac{\sigma_{1}^{2}}{n_{1}}+\frac{\sigma_{2}^{2}}{n_{2}}}}$.

Hypothesis Testing t Test

The t test is another method of hypothesis testing that is used for a small sample size (n < 30). It is also used to compare the sample mean and population mean. However, the population standard deviation is not known. Instead, the sample standard deviation is known. The mean of two samples can also be compared using the t test.

One sample: t = $\frac{\overline{x}-\mu}{\frac{s}{\sqrt{n}}}$.
Two samples: t = $\frac{(\overline{x_{1}}-\overline{x_{2}})-(\mu_{1}-\mu_{2})}{\sqrt{\frac{s_{1}^{2}}{n_{1}}+\frac{s_{2}^{2}}{n_{2}}}}$.

Hypothesis Testing Chi Square

The Chi square test is a hypothesis testing method that is used to check whether the variables in a population are independent or not. It is used when the test statistic is chi-squared distributed.

One Tailed Hypothesis Testing

One tailed hypothesis testing is done when the rejection region is only in one direction. It can also be known as directional hypothesis testing because the effects can be tested in one direction only. This type of testing is further classified into the right tailed test and left tailed test.

Right Tailed Hypothesis Testing

The right tail test is also known as the upper tail test. This test is used to check whether the population parameter is greater than some value. The null and alternative hypotheses for this test are given as follows:

$H_{0}$: The population parameter is ≤ some value

$H_{1}$: The population parameter is > some value.

If the test statistic has a greater value than the critical value then the null hypothesis is rejected

Left Tailed Hypothesis Testing

The left tail test is also known as the lower tail test. It is used to check whether the population parameter is less than some value. The hypotheses for this hypothesis testing can be written as follows:

$H_{0}$: The population parameter is ≥ some value

$H_{1}$: The population parameter is < some value.

The null hypothesis is rejected if the test statistic has a value lesser than the critical value.

Two Tailed Hypothesis Testing

In this hypothesis testing method, the critical region lies on both sides of the sampling distribution. It is also known as a non - directional hypothesis testing method. The two-tailed test is used when it needs to be determined if the population parameter is assumed to be different than some value. The hypotheses can be set up as follows:

$H_{0}$: the population parameter = some value

$H_{1}$: the population parameter ≠ some value

The null hypothesis is rejected if the test statistic has a value that is not equal to the critical value.

Hypothesis Testing Steps

Hypothesis testing can be easily performed in five simple steps. The most important step is to correctly set up the hypotheses and identify the right method for hypothesis testing. The basic steps to perform hypothesis testing are as follows:

Step 1: Set up the null hypothesis by correctly identifying whether it is the left-tailed, right-tailed, or two-tailed hypothesis testing.
Step 2: Set up the alternative hypothesis.
Step 3: Choose the correct significance level, $\alpha$, and find the critical value.
Step 4: Calculate the correct test statistic (z, t or $\chi$) and p-value.
Step 5: Compare the test statistic with the critical value or compare the p-value with $\alpha$ to arrive at a conclusion. In other words, decide if the null hypothesis is to be rejected or not.

Hypothesis Testing Example

The best way to solve a problem on hypothesis testing is by applying the 5 steps mentioned in the previous section. Suppose a researcher claims that the mean average weight of men is greater than 100kgs with a standard deviation of 15kgs. 30 men are chosen with an average weight of 112.5 Kgs. Using hypothesis testing, check if there is enough evidence to support the researcher's claim. The confidence interval is given as 95%.

Step 1: This is an example of a right-tailed test. Set up the null hypothesis as $H_{0}$: $\mu$ = 100.

Step 2: The alternative hypothesis is given by $H_{1}$: $\mu$ > 100.

Step 3: As this is a one-tailed test, $\alpha$ = 100% - 95% = 5%. This can be used to determine the critical value.

1 - $\alpha$ = 1 - 0.05 = 0.95

0.95 gives the required area under the curve. Now using a normal distribution table, the area 0.95 is at z = 1.645. A similar process can be followed for a t-test. The only additional requirement is to calculate the degrees of freedom given by n - 1.

Step 4: Calculate the z test statistic. This is because the sample size is 30. Furthermore, the sample and population means are known along with the standard deviation.

z = $\frac{\overline{x}-\mu}{\frac{\sigma}{\sqrt{n}}}$.

$\mu$ = 100, $\overline{x}$ = 112.5, n = 30, $\sigma$ = 15

z = $\frac{112.5-100}{\frac{15}{\sqrt{30}}}$ = 4.56

Step 5: Conclusion. As 4.56 > 1.645 thus, the null hypothesis can be rejected.

Hypothesis Testing and Confidence Intervals

Confidence intervals form an important part of hypothesis testing. This is because the alpha level can be determined from a given confidence interval. Suppose a confidence interval is given as 95%. Subtract the confidence interval from 100%. This gives 100 - 95 = 5% or 0.05. This is the alpha value of a one-tailed hypothesis testing. To obtain the alpha value for a two-tailed hypothesis testing, divide this value by 2. This gives 0.05 / 2 = 0.025.

Probability and Statistics
Data Handling

Important Notes on Hypothesis Testing

Hypothesis testing is a technique that is used to verify whether the results of an experiment are statistically significant.
It involves the setting up of a null hypothesis and an alternate hypothesis.
There are three types of tests that can be conducted under hypothesis testing - z test, t test, and chi square test.
Hypothesis testing can be classified as right tail, left tail, and two tail tests.

Examples on Hypothesis Testing

Example 1: The average weight of a dumbbell in a gym is 90lbs. However, a physical trainer believes that the average weight might be higher. A random sample of 5 dumbbells with an average weight of 110lbs and a standard deviation of 18lbs. Using hypothesis testing check if the physical trainer's claim can be supported for a 95% confidence level. Solution: As the sample size is lesser than 30, the t-test is used. $H_{0}$: $\mu$ = 90, $H_{1}$: $\mu$ > 90 $\overline{x}$ = 110, $\mu$ = 90, n = 5, s = 18. $\alpha$ = 0.05 Using the t-distribution table, the critical value is 2.132 t = $\frac{\overline{x}-\mu}{\frac{s}{\sqrt{n}}}$ t = 2.484 As 2.484 > 2.132, the null hypothesis is rejected. Answer: The average weight of the dumbbells may be greater than 90lbs
Example 2: The average score on a test is 80 with a standard deviation of 10. With a new teaching curriculum introduced it is believed that this score will change. On random testing, the score of 38 students, the mean was found to be 88. With a 0.05 significance level, is there any evidence to support this claim? Solution: This is an example of two-tail hypothesis testing. The z test will be used. $H_{0}$: $\mu$ = 80, $H_{1}$: $\mu$ ≠ 80 $\overline{x}$ = 88, $\mu$ = 80, n = 36, $\sigma$ = 10. $\alpha$ = 0.05 / 2 = 0.025 The critical value using the normal distribution table is 1.96 z = $\frac{\overline{x}-\mu}{\frac{\sigma}{\sqrt{n}}}$ z = $\frac{88-80}{\frac{10}{\sqrt{36}}}$ = 4.8 As 4.8 > 1.96, the null hypothesis is rejected. Answer: There is a difference in the scores after the new curriculum was introduced.
Example 3: The average score of a class is 90. However, a teacher believes that the average score might be lower. The scores of 6 students were randomly measured. The mean was 82 with a standard deviation of 18. With a 0.05 significance level use hypothesis testing to check if this claim is true. Solution: The t test will be used. $H_{0}$: $\mu$ = 90, $H_{1}$: $\mu$ < 90 $\overline{x}$ = 110, $\mu$ = 90, n = 6, s = 18 The critical value from the t table is -2.015 t = $\frac{\overline{x}-\mu}{\frac{s}{\sqrt{n}}}$ t = $\frac{82-90}{\frac{18}{\sqrt{6}}}$ t = -1.088 As -1.088 > -2.015, we fail to reject the null hypothesis. Answer: There is not enough evidence to support the claim.

go to slide go to slide go to slide

definition of probability and hypothesis testing

Book a Free Trial Class

FAQs on Hypothesis Testing

What is hypothesis testing.

Hypothesis testing in statistics is a tool that is used to make inferences about the population data. It is also used to check if the results of an experiment are valid.

What is the z Test in Hypothesis Testing?

The z test in hypothesis testing is used to find the z test statistic for normally distributed data . The z test is used when the standard deviation of the population is known and the sample size is greater than or equal to 30.

What is the t Test in Hypothesis Testing?

The t test in hypothesis testing is used when the data follows a student t distribution . It is used when the sample size is less than 30 and standard deviation of the population is not known.

What is the formula for z test in Hypothesis Testing?

The formula for a one sample z test in hypothesis testing is z = $\frac{\overline{x}-\mu}{\frac{\sigma}{\sqrt{n}}}$ and for two samples is z = $\frac{(\overline{x_{1}}-\overline{x_{2}})-(\mu_{1}-\mu_{2})}{\sqrt{\frac{\sigma_{1}^{2}}{n_{1}}+\frac{\sigma_{2}^{2}}{n_{2}}}}$.

What is the p Value in Hypothesis Testing?

The p value helps to determine if the test results are statistically significant or not. In hypothesis testing, the null hypothesis can either be rejected or not rejected based on the comparison between the p value and the alpha level.

What is One Tail Hypothesis Testing?

When the rejection region is only on one side of the distribution curve then it is known as one tail hypothesis testing. The right tail test and the left tail test are two types of directional hypothesis testing.

What is the Alpha Level in Two Tail Hypothesis Testing?

To get the alpha level in a two tail hypothesis testing divide $\alpha$ by 2. This is done as there are two rejection regions in the curve.

Open topic with navigation

The P value, or calculated probability, is the probability of finding the observed, or more extreme, results when the null hypothesis (H 0 ) of a study question is true – the definition of ‘extreme’ depends on how the hypothesis is being tested. P is also described in terms of rejecting H 0 when it is actually true, however, it is not a direct probability of this state.

The null hypothesis is usually an hypothesis of "no difference" e.g. no difference between blood pressures in group A and group B. Define a null hypothesis for each study question clearly before the start of your study.

The only situation in which you should use a one sided P value is when a large change in an unexpected direction would have absolutely no relevance to your study. This situation is unusual; if you are in any doubt then use a two sided P value.

The term significance level (alpha) is used to refer to a pre-chosen probability and the term "P value" is used to indicate a probability that you calculate after a given study.

The alternative hypothesis (H 1 ) is the opposite of the null hypothesis; in plain language terms this is usually the hypothesis you set out to investigate. For example, question is "is there a significant (not due to chance) difference in blood pressures between groups A and B if we give group A the test drug and group B a sugar pill?" and alternative hypothesis is " there is a difference in blood pressures between groups A and B if we give group A the test drug and group B a sugar pill".

If your P value is less than the chosen significance level then you reject the null hypothesis i.e. accept that your sample gives reasonable evidence to support the alternative hypothesis. It does NOT imply a "meaningful" or "important" difference; that is for you to decide when considering the real-world relevance of your result.

The choice of significance level at which you reject H 0 is arbitrary. Conventionally the 5% (less than 1 in 20 chance of being wrong), 1% and 0.1% (P < 0.05, 0.01 and 0.001) levels have been used. These numbers can give a false sense of security.

In the ideal world, we would be able to define a "perfectly" random sample, the most appropriate test and one definitive conclusion. We simply cannot. What we can do is try to optimise all stages of our research to minimise sources of uncertainty. When presenting P values some groups find it helpful to use the asterisk rating system as well as quoting the P value:

P < 0.05 *

P < 0.01 **

P < 0.001

Most authors refer to statistically significant as P < 0.05 and statistically highly significant as P < 0.001 (less than one in a thousand chance of being wrong).

The asterisk system avoids the woolly term "significant". Please note, however, that many statisticians do not like the asterisk rating system when it is used without showing P values. As a rule of thumb, if you can quote an exact P value then do. You might also want to refer to a quoted exact P value as an asterisk in text narrative or tables of contrasts elsewhere in a report.

At this point, a word about error. Type I error is the false rejection of the null hypothesis and type II error is the false acceptance of the null hypothesis. As an aid memoir: think that our cynical society rejects before it accepts.

The significance level (alpha) is the probability of type I error. The power of a test is one minus the probability of type II error (beta). Power should be maximised when selecting statistical methods. If you want to estimate sample sizes then you must understand all of the terms mentioned here.

The following table shows the relationship between power and error in hypothesis testing:

If you are interested in further details of probability and sampling theory at this point then please refer to one of the general texts listed in the reference section .

You must understand confidence intervals if you intend to quote P values in reports and papers. Statistical referees of scientific journals expect authors to quote confidence intervals with greater prominence than P values.

Notes about Type I error :

is the incorrect rejection of the null hypothesis
maximum probability is set in advance as alpha
is not affected by sample size as it is set in advance
increases with the number of tests or end points (i.e. do 20 rejections of H 0 and 1 is likely to be wrongly significant for alpha = 0.05)

Notes about Type II error :

is the incorrect acceptance of the null hypothesis
probability is beta
beta depends upon sample size and alpha
can't be estimated except as a function of the true population effect
beta gets smaller as the sample size gets larger
beta gets smaller as the number of tests or end points increases

Hypothesis Testing Framework

Now that we've seen an example and explored some of the themes for hypothesis testing, let's specify the procedure that we will follow.

Hypothesis Testing Steps

The formal framework and steps for hypothesis testing are as follows:

Identify and define the parameter of interest
Define the competing hypotheses to test
Set the evidence threshold, formally called the significance level
Generate or use theory to specify the sampling distribution and check conditions
Calculate the test statistic and p-value
Evaluate your results and write a conclusion in the context of the problem.

We'll discuss each of these steps below.

Identify Parameter of Interest

First, I like to specify and define the parameter of interest. What is the population that we are interested in? What characteristic are we measuring?

By defining our population of interest, we can confirm that we are truly using sample data. If we find that we actually have population data, our inference procedures are not needed. We could proceed by summarizing our population data.

By identifying and defining the parameter of interest, we can confirm that we use appropriate methods to summarize our variable of interest. We can also focus on the specific process needed for our parameter of interest.

In our example from the last page, the parameter of interest would be the population mean time that a host has been on Airbnb for the population of all Chicago listings on Airbnb in March 2023. We could represent this parameter with the symbol $\mu$. It is best practice to fully define $\mu$ both with words and symbol.

Define the Hypotheses

For hypothesis testing, we need to decide between two competing theories. These theories must be statements about the parameter. Although we won't have the population data to definitively select the correct theory, we will use our sample data to determine how reasonable our "skeptic's theory" is.

The first hypothesis is called the null hypothesis, $H_0$. This can be thought of as the "status quo", the "skeptic's theory", or that nothing is happening.

Examples of null hypotheses include that the population proportion is equal to 0.5 ($p = 0.5$), the population median is equal to 12 ($M = 12$), or the population mean is equal to 14.5 ($\mu = 14.5$).

The second hypothesis is called the alternative hypothesis, $H_a$ or $H_1$. This can be thought of as the "researcher's hypothesis" or that something is happening. This is what we'd like to convince the skeptic to believe. In most cases, the desired outcome of the researcher is to conclude that the alternative hypothesis is reasonable to use moving forward.

Examples of alternative hypotheses include that the population proportion is greater than 0.5 ($p > 0.5$), the population median is less than 12 ($M < 12$), or the population mean is not equal to 14.5 ($\mu \neq 14.5$).

There are a few requirements for the hypotheses:

the hypotheses must be about the same population parameter,
the hypotheses must have the same null value (provided number to compare to),
the null hypothesis must have the equality (the equals sign must be in the null hypothesis),
the alternative hypothesis must not have the equality (the equals sign cannot be in the alternative hypothesis),
there must be no overlap between the null and alternative hypothesis.

You may have previously seen null hypotheses that include more than an equality (e.g. $p \le 0.5$). As long as there is an equality in the null hypothesis, this is allowed. For our purposes, we will simplify this statement to ($p = 0.5$).

To summarize from above, possible hypotheses statements are:

$H_0: p = 0.5$ vs. $H_a: p > 0.5$

$H_0: M = 12$ vs. $H_a: M < 12$

$H_0: \mu = 14.5$ vs. $H_a: \mu \neq 14.5$

In our second example about Airbnb hosts, our hypotheses would be:

$H_0: \mu = 2100$ vs. $H_a: \mu > 2100$.

Set Threshold (Significance Level)

There is one more step to complete before looking at the data. This is to set the threshold needed to convince the skeptic. This threshold is defined as an $\alpha$ significance level. We'll define exactly what the $\alpha$ significance level means later. For now, smaller $\alpha$s correspond to more evidence being required to convince the skeptic.

A few common $\alpha$ levels include 0.1, 0.05, and 0.01.

For our Airbnb hosts example, we'll set the threshold as 0.02.

Determine the Sampling Distribution of the Sample Statistic

The first step (as outlined above) is the identify the parameter of interest. What is the best estimate of the parameter of interest? Typically, it will be the sample statistic that corresponds to the parameter. This sample statistic, along with other features of the distribution will prove especially helpful as we continue the hypothesis testing procedure.

However, we do have a decision at this step. We can choose to use simulations with a resampling approach or we can choose to rely on theory if we are using proportions or means. We then also need to confirm that our results and conclusions will be valid based on the available data.

Required Condition

The one required assumption, regardless of approach (resampling or theory), is that the sample is random and representative of the population of interest. In other words, we need our sample to be a reasonable sample of data from the population.

Using Simulations and Resampling

If we'd like to use a resampling approach, we have no (or minimal) additional assumptions to check. This is because we are relying on the available data instead of assumptions.

We do need to adjust our data to be consistent with the null hypothesis (or skeptic's claim). We can then rely on our resampling approach to estimate a plausible sampling distribution for our sample statistic.

Recall that we took this approach on the last page. Before simulating our estimated sampling distribution, we adjusted the mean of the data so that it matched with our skeptic's claim, shown in the code below.

We'll see a few more examples on the next page.

Using Theory

On the other hand, we could rely on theory in order to estimate the sampling distribution of our desired statistic. Recall that we had a few different options to rely on:

the CLT for the sampling distribution of a sample mean
the binomial distribution for the sampling distribution of a proportion (or count)
the Normal approximation of a binomial distribution (using the CLT) for the sampling distribution of a proportion

If relying on the CLT to specify the underlying sampling distribution, you also need to confirm:

having a random sample and
having a sample size that is less than 10% of the population size if the sampling is done without replacement
having a Normally distributed population for a quantitative variable OR
having a large enough sample size (usually at least 25) for a quantitative variable
having a large enough sample size for a categorical variable (defined by $np$ and $n(1-p)$ being at least 10)

If relying on the binomial distribution to specify the underlying sampling distribution, you need to confirm:

having a set number of trials, $n$
having the same probability of success, $p$ for each observation

After determining the appropriate theory to use, we should check our conditions and then specify the sampling distribution for our statistic.

For the Airbnb hosts example, we have what we've assumed to be a random sample. It is not taken with replacement, so we also need to assume that our sample size (700) is less than 10% of our population size. In other words, we need to assume that the population of Chicago Airbnbs in March 2023 was at least 7000. Since we do have our (presumed) population data available, we can confirm that there were at least 7000 Chicago Airbnbs in the population in 2023.

Additionally, we can confirm that normality of the sampling distribution applies for the CLT to apply. Our sample size is more than 25 and the parameter of interest is a mean, so this meets our necessary criteria for the normality condition to be valid.

With the conditions now met, we can estimate our sampling distribution. From the CLT, we know that the distribution for the sample mean should be $\bar{X} \sim N(\mu, \frac{\sigma}{\sqrt{n}})$.

Now, we face our next challenge -- what to plug in as the mean and standard error for this distribution. Since we are adopting the skeptic's point of view for the purpose of this approach, we can plug in the value of $\mu_0 = 2100$. We also know that the sample size $n$ is 700. But what should we plug in for the population standard deviation $\sigma$?

When we don't know the value of a parameter, we will generally plug in our best estimate for the parameter. In this case, that corresponds to plugging in $\hat{\sigma}$, or our sample standard deviation.

Now, our estimated sampling distribution based on the CLT is: $\bar{X} \sim N(2100, 41.4045)$.

If we compare to our corresponding skeptic's sampling distribution on the last page, we can confirm that the theoretical sampling distribution is similar to the simulated sampling distribution based on resampling.

Assumptions not met

What do we do if the necessary conditions aren't met for the sampling distribution? Because the simulation-based resampling approach has minimal assumptions, we should be able to use this approach to produce valid results as long as the provided data is representative of the population.

The theory-based approach has more conditions, and we may not be able to meet all of the necessary conditions. For example, if our parameter is something other than a mean or proportion, we may not have appropriate theory. Additionally, we may not have a large enough sample size.

First, we could consider changing approaches to the simulation-based one.
Second, we might look at how we could meet the necessary conditions better. In some cases, we may be able to redefine groups or make adjustments so that the setup of the test is closer to what is needed.
As a last resort, we may be able to continue following the hypothesis testing steps. In this case, your calculations may not be valid or exact; however, you might be able to use them as an estimate or an approximation. It would be crucial to specify the violation and approximation in any conclusions or discussion of the test.

Calculate the evidence with statistics and p-values

Now, it's time to calculate how much evidence the sample contains to convince the skeptic to change their mind. As we saw above, we can convince the skeptic to change their mind by demonstrating that our sample is unlikely to occur if their theory is correct.

How do we do this? We do this by calculating a probability associated with our observed value for the statistic.

For example, for our situation, we want to convince the skeptic that the population mean is actually greater than 2100 days. We do that by calculating the probability that a sample mean would be as large or larger than what we observed in our actual sample, which was 2188 days. Why do we need the larger portion? We use the larger portion because a sample mean of 2200 days also provides evidence that the population mean is larger than 2100 days; it isn't limited to exactly what we observed in our sample. We call this specific probability the p-value.

That is, the p-value is the probability of observing a test statistic as extreme or more extreme (as determined by the alternative hypothesis), assuming the null hypothesis is true.

Our observed p-value for the Airbnb host example demonstrates that the probability of getting a sample mean host time of 2188 days (the value from our sample) or more is 1.46%, assuming that the true population mean is 2100 days.

Test statistic

Notice that the formal definition of a p-value mentions a test statistic . In most cases, this word can be replaced with "statistic" or "sample" for an equivalent statement.

Oftentimes, we'll see that our sample statistic can be used directly as the test statistic, as it was above. We could equivalently adjust our statistic to calculate a test statistic. This test statistic is often calculated as:

$\text{test statistic} = \frac{\text{estimate} - \text{hypothesized value}}{\text{standard error of estimate}}$

P-value Calculation Options

Note also that the p-value definition includes a probability associated with a test statistic being as extreme or more extreme (as determined by the alternative hypothesis . How do we determine the area that we consider when calculating the probability. This decision is determined by the inequality in the alternative hypothesis.

For example, when we were trying to convince the skeptic that the population mean is greater than 2100 days, we only considered those sample means that we at least as large as what we observed -- 2188 days or more.

If instead we were trying to convince the skeptic that the population mean is less than 2100 days ($H_a: \mu < 2100$), we would consider all sample means that were at most what we observed - 2188 days or less. In this case, our p-value would be quite large; it would be around 99.5%. This large p-value demonstrates that our sample does not support the alternative hypothesis. In fact, our sample would encourage us to choose the null hypothesis instead of the alternative hypothesis of $\mu < 2100$, as our sample directly contradicts the statement in the alternative hypothesis.

If we wanted to convince the skeptic that they were wrong and that the population mean is anything other than 2100 days ($H_a: \mu \neq 2100$), then we would want to calculate the probability that a sample mean is at least 88 days away from 2100 days. That is, we would calculate the probability corresponding to 2188 days or more or 2012 days or less. In this case, our p-value would be roughly twice the previously calculated p-value.

We could calculate all of those probabilities using our sampling distributions, either simulated or theoretical, that we generated in the previous step. If we chose to calculate a test statistic as defined in the previous section, we could also rely on standard normal distributions to calculate our p-value.

Evaluate your results and write conclusion in context of problem

Once you've gathered your evidence, it's now time to make your final conclusions and determine how you might proceed.

In traditional hypothesis testing, you often make a decision. Recall that you have your threshold (significance level $\alpha$) and your level of evidence (p-value). We can compare the two to determine if your p-value is less than or equal to your threshold. If it is, you have enough evidence to persuade your skeptic to change their mind. If it is larger than the threshold, you don't have quite enough evidence to convince the skeptic.

Common formal conclusions (if given in context) would be:

I have enough evidence to reject the null hypothesis (the skeptic's claim), and I have sufficient evidence to suggest that the alternative hypothesis is instead true.
I do not have enough evidence to reject the null hypothesis (the skeptic's claim), and so I do not have sufficient evidence to suggest the alternative hypothesis is true.

The only decision that we can make is to either reject or fail to reject the null hypothesis (we cannot "accept" the null hypothesis). Because we aren't actively evaluating the alternative hypothesis, we don't want to make definitive decisions based on that hypothesis. However, when it comes to making our conclusion for what to use going forward, we frame this on whether we could successfully convince someone of the alternative hypothesis.

A less formal conclusion might look something like:

Based on our sample of Chicago Airbnb listings, it seems as if the mean time since a host has been on Airbnb (for all Chicago Airbnb listings) is more than 5.75 years.

Significance Level Interpretation

We've now seen how the significance level $\alpha$ is used as a threshold for hypothesis testing. What exactly is the significance level?

The significance level $\alpha$ has two primary definitions. One is that the significance level is the maximum probability required to reject the null hypothesis; this is based on how the significance level functions within the hypothesis testing framework. The second definition is that this is the probability of rejecting the null hypothesis when the null hypothesis is true; in other words, this is the probability of making a specific type of error called a Type I error.

Why do we have to be comfortable making a Type I error? There is always a chance that the skeptic was originally correct and we obtained a very unusual sample. We don't want to the skeptic to be so convinced of their theory that no evidence can convince them. In this case, we need the skeptic to be convinced as long as the evidence is strong enough . Typically, the probability threshold will be low, to reduce the number of errors made. This also means that a decent amount of evidence will be needed to convince the skeptic to abandon their position in favor of the alternative theory.

p-value Limitations and Misconceptions

In comparison to the $\alpha$ significance level, we also need to calculate the evidence against the null hypothesis with the p-value.

The p-value is the probability of getting a test statistic as extreme or more extreme (in the direction of the alternative hypothesis), assuming the null hypothesis is true.

Recently, p-values have gotten some bad press in terms of how they are used. However, that doesn't mean that p-values should be abandoned, as they still provide some helpful information. Below, we'll describe what p-values don't mean, and how they should or shouldn't be used to make decisions.

Factors that affect a p-value

What features affect the size of a p-value?

the null value, or the value assumed under the null hypothesis
the effect size (the difference between the null value under the null hypothesis and the true value of the parameter)
the sample size

More evidence against the null hypothesis will be obtained if the effect size is larger and if the sample size is larger.

Misconceptions

We gave a definition for p-values above. What are some examples that p-values don't mean?

A p-value is not the probability that the null hypothesis is correct
A p-value is not the probability that the null hypothesis is incorrect
A p-value is not the probability of getting your specific sample
A p-value is not the probability that the alternative hypothesis is correct
A p-value is not the probability that the alternative hypothesis is incorrect
A p-value does not indicate the size of the effect

Our p-value is a way of measuring the evidence that your sample provides against the null hypothesis, assuming the null hypothesis is in fact correct.

Using the p-value to make a decision

Why is there bad press for a p-value? You may have heard about the standard $\alpha$ level of 0.05. That is, we would be comfortable with rejecting the null hypothesis once in 20 attempts when the null hypothesis is really true. Recall that we reject the null hypothesis when the p-value is less than or equal to the significance level.

Consider what would happen if you have two different p-values: 0.049 and 0.051.

In essence, these two p-values represent two very similar probabilities (4.9% vs. 5.1%) and very similar levels of evidence against the null hypothesis. However, when we make our decision based on our threshold, we would make two different decisions (reject and fail to reject, respectively). Should this decision really be so simplistic? I would argue that the difference shouldn't be so severe when the sample statistics are likely very similar. For this reason, I (and many other experts) strongly recommend using the p-value as a measure of evidence and including it with your conclusion.

Putting too much emphasis on the decision (and having a significant result) has created a culture of misusing p-values. For this reason, understanding your p-value itself is crucial.

Searching for p-values

The other concern with setting a definitive threshold of 0.05 is that some researchers will begin performing multiple tests until finding a p-value that is small enough. However, with a p-value of 0.05, we know that we will have a p-value less than 0.05 1 time out of every 20 times, even when the null hypothesis is true.

This means that if researchers start hunting for p-values that are small (sometimes called p-hacking), then they are likely to identify a small p-value every once in a while by chance alone. Researchers might then publish that result, even though the result is actually not informative. For this reason, it is recommended that researchers write a definitive analysis plan to prevent performing multiple tests in search of a result that occurs by chance alone.

Best Practices

With all of this in mind, what should we do when we have our p-value? How can we prevent or reduce misuse of a p-value?

Report the p-value along with the conclusion
Specify the effect size (the value of the statistic)
Define an analysis plan before looking at the data
Interpret the p-value clearly to specify what it indicates
Consider using an alternate statistical approach, the confidence interval, discussed next, when appropriate

Search Search Please fill out this field.
Fundamental Analysis

Hypothesis to Be Tested: Definition and 4 Steps for Testing with Example

What Is Hypothesis Testing?

Hypothesis testing, sometimes called significance testing, is an act in statistics whereby an analyst tests an assumption regarding a population parameter. The methodology employed by the analyst depends on the nature of the data used and the reason for the analysis.

Hypothesis testing is used to assess the plausibility of a hypothesis by using sample data. Such data may come from a larger population, or from a data-generating process. The word "population" will be used for both of these cases in the following descriptions.

Key Takeaways

Hypothesis testing is used to assess the plausibility of a hypothesis by using sample data.
The test provides evidence concerning the plausibility of the hypothesis, given the data.
Statistical analysts test a hypothesis by measuring and examining a random sample of the population being analyzed.
The four steps of hypothesis testing include stating the hypotheses, formulating an analysis plan, analyzing the sample data, and analyzing the result.

How Hypothesis Testing Works

In hypothesis testing, an analyst tests a statistical sample, with the goal of providing evidence on the plausibility of the null hypothesis.

Statistical analysts test a hypothesis by measuring and examining a random sample of the population being analyzed. All analysts use a random population sample to test two different hypotheses: the null hypothesis and the alternative hypothesis.

The null hypothesis is usually a hypothesis of equality between population parameters; e.g., a null hypothesis may state that the population mean return is equal to zero. The alternative hypothesis is effectively the opposite of a null hypothesis (e.g., the population mean return is not equal to zero). Thus, they are mutually exclusive , and only one can be true. However, one of the two hypotheses will always be true.

The null hypothesis is a statement about a population parameter, such as the population mean, that is assumed to be true.

4 Steps of Hypothesis Testing

All hypotheses are tested using a four-step process:

The first step is for the analyst to state the hypotheses.
The second step is to formulate an analysis plan, which outlines how the data will be evaluated.
The third step is to carry out the plan and analyze the sample data.
The final step is to analyze the results and either reject the null hypothesis, or state that the null hypothesis is plausible, given the data.

Real-World Example of Hypothesis Testing

If, for example, a person wants to test that a penny has exactly a 50% chance of landing on heads, the null hypothesis would be that 50% is correct, and the alternative hypothesis would be that 50% is not correct.

Mathematically, the null hypothesis would be represented as Ho: P = 0.5. The alternative hypothesis would be denoted as "Ha" and be identical to the null hypothesis, except with the equal sign struck-through, meaning that it does not equal 50%.

A random sample of 100 coin flips is taken, and the null hypothesis is then tested. If it is found that the 100 coin flips were distributed as 40 heads and 60 tails, the analyst would assume that a penny does not have a 50% chance of landing on heads and would reject the null hypothesis and accept the alternative hypothesis.

If, on the other hand, there were 48 heads and 52 tails, then it is plausible that the coin could be fair and still produce such a result. In cases such as this where the null hypothesis is "accepted," the analyst states that the difference between the expected results (50 heads and 50 tails) and the observed results (48 heads and 52 tails) is "explainable by chance alone."

Some staticians attribute the first hypothesis tests to satirical writer John Arbuthnot in 1710, who studied male and female births in England after observing that in nearly every year, male births exceeded female births by a slight proportion. Arbuthnot calculated that the probability of this happening by chance was small, and therefore it was due to “divine providence.”

What is Hypothesis Testing?

Hypothesis testing refers to a process used by analysts to assess the plausibility of a hypothesis by using sample data. In hypothesis testing, statisticians formulate two hypotheses: the null hypothesis and the alternative hypothesis. A null hypothesis determines there is no difference between two groups or conditions, while the alternative hypothesis determines that there is a difference. Researchers evaluate the statistical significance of the test based on the probability that the null hypothesis is true.

What are the Four Key Steps Involved in Hypothesis Testing?

Hypothesis testing begins with an analyst stating two hypotheses, with only one that can be right. The analyst then formulates an analysis plan, which outlines how the data will be evaluated. Next, they move to the testing phase and analyze the sample data. Finally, the analyst analyzes the results and either rejects the null hypothesis or states that the null hypothesis is plausible, given the data.

What are the Benefits of Hypothesis Testing?

Hypothesis testing helps assess the accuracy of new ideas or theories by testing them against data. This allows researchers to determine whether the evidence supports their hypothesis, helping to avoid false claims and conclusions. Hypothesis testing also provides a framework for decision-making based on data rather than personal opinions or biases. By relying on statistical analysis, hypothesis testing helps to reduce the effects of chance and confounding variables, providing a robust framework for making informed conclusions.

What are the Limitations of Hypothesis Testing?

Hypothesis testing relies exclusively on data and doesn’t provide a comprehensive understanding of the subject being studied. Additionally, the accuracy of the results depends on the quality of the available data and the statistical methods used. Inaccurate data or inappropriate hypothesis formulation may lead to incorrect conclusions or failed tests. Hypothesis testing can also lead to errors, such as analysts either accepting or rejecting a null hypothesis when they shouldn’t have. These errors may result in false conclusions or missed opportunities to identify significant patterns or relationships in the data.

The Bottom Line

Hypothesis testing refers to a statistical process that helps researchers and/or analysts determine the reliability of a study. By using a well-formulated hypothesis and set of statistical tests, individuals or businesses can make inferences about the population that they are studying and draw conclusions based on the data presented. There are different types of hypothesis testing, each with their own set of rules and procedures. However, all hypothesis testing methods have the same four step process, which includes stating the hypotheses, formulating an analysis plan, analyzing the sample data, and analyzing the result. Hypothesis testing plays a vital part of the scientific process, helping to test assumptions and make better data-based decisions.

Sage. " Introduction to Hypothesis Testing. " Page 4.

Elder Research. " Who Invented the Null Hypothesis? "

Formplus. " Hypothesis Testing: Definition, Uses, Limitations and Examples. "

Terms of Service
Editorial Policy
Privacy Policy
Your Privacy Choices

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

Knowledge Base

Test statistics | Definition, Interpretation, and Examples

Published on July 17, 2020 by Rebecca Bevans . Revised on June 22, 2023.

The test statistic is a number calculated from a statistical test of a hypothesis. It shows how closely your observed data match the distribution expected under the null hypothesis of that statistical test.

The test statistic is used to calculate the p value of your results, helping to decide whether to reject your null hypothesis.

What exactly is a test statistic, types of test statistics, interpreting test statistics, reporting test statistics, other interesting articles, frequently asked questions about test statistics.

A test statistic describes how closely the distribution of your data matches the distribution predicted under the null hypothesis of the statistical test you are using.

The distribution of data is how often each observation occurs, and can be described by its central tendency and variation around that central tendency. Different statistical tests predict different types of distributions, so it’s important to choose the right statistical test for your hypothesis.

The test statistic summarizes your observed data into a single number using the central tendency, variation, sample size, and number of predictor variables in your statistical model.

Generally, the test statistic is calculated as the pattern in your data (i.e., the correlation between variables or difference between groups) divided by the variance in the data (i.e., the standard deviation ).

Null hypothesis ( H 0 ): There is no correlation between temperature and flowering date.
Alternate hypothesis ( H A or H 1 ): There is a correlation between temperature and flowering date.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

Academic style
Vague sentences
Style consistency

See an example

Below is a summary of the most common test statistics, their hypotheses, and the types of statistical tests that use them.

Different statistical tests will have slightly different ways of calculating these test statistics, but the underlying hypotheses and interpretations of the test statistic stay the same.

In practice, you will almost always calculate your test statistic using a statistical program (R, SPSS, Excel, etc.), which will also calculate the p value of the test statistic. However, formulas to calculate these statistics by hand can be found online.

a regression coefficient of 0.36
a t value comparing that coefficient to the predicted range of regression coefficients under the null hypothesis of no relationship

The t value of the regression test is 2.36 – this is your test statistic.

For any combination of sample sizes and number of predictor variables, a statistical test will produce a predicted distribution for the test statistic. This shows the most likely range of values that will occur if your data follows the null hypothesis of the statistical test.

The more extreme your test statistic – the further to the edge of the range of predicted test values it is – the less likely it is that your data could have been generated under the null hypothesis of that statistical test.

The agreement between your calculated test statistic and the predicted values is described by the p value . The smaller the p value, the less likely your test statistic is to have occurred under the null hypothesis of the statistical test.

Because the test statistic is generated from your observed data, this ultimately means that the smaller the p value, the less likely it is that your data could have occurred if the null hypothesis was true.

Test statistics can be reported in the results section of your research paper along with the sample size, p value of the test, and any characteristics of your data that will help to put these results into context.

Whether or not you need to report the test statistic depends on the type of test you are reporting.

By surveying a random subset of 100 trees over 25 years we found a statistically significant ( p < 0.01) positive correlation between temperature and flowering dates ( R 2 = 0.36, SD = 0.057).

In our comparison of mouse diet A and mouse diet B, we found that the lifespan on diet A ( M = 2.1 years; SD = 0.12) was significantly shorter than the lifespan on diet B ( M = 2.6 years; SD = 0.1), with an average difference of 6 months ( t (80) = -12.75; p < 0.01).

Prevent plagiarism. Run a free check.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

Confidence interval
Descriptive statistics
Measures of central tendency
Correlation coefficient

Methodology

Cluster sampling
Stratified sampling
Types of interviews
Cohort study
Thematic analysis

Research bias

Implicit bias
Cognitive bias
Survivorship bias
Availability heuristic
Nonresponse bias
Regression to the mean

A test statistic is a number calculated by a statistical test . It describes how far your observed data is from the null hypothesis of no relationship between variables or no difference among sample groups.

The test statistic tells you how different two or more groups are from the overall population mean , or how different a linear slope is from the slope predicted by a null hypothesis . Different test statistics are used in different statistical tests.

The formula for the test statistic depends on the statistical test being used.

Generally, the test statistic is calculated as the pattern in your data (i.e. the correlation between variables or difference between groups) divided by the variance in the data (i.e. the standard deviation ).

The test statistic you use will be determined by the statistical test.

You can choose the right statistical test by looking at what type of data you have collected and what type of relationship you want to test.

The test statistic will change based on the number of observations in your data, how variable your observations are, and how strong the underlying patterns in the data are.

For example, if one data set has higher variability while another has lower variability, the first data set will produce a test statistic closer to the null hypothesis , even if the true correlation between two variables is the same in either data set.

Statistical significance is a term used by researchers to state that it is unlikely their observations could have occurred under the null hypothesis of a statistical test . Significance is usually denoted by a p -value , or probability value.

Statistical significance is arbitrary – it depends on the threshold, or alpha value, chosen by the researcher. The most common threshold is p < 0.05, which means that the data is likely to occur less than 5% of the time under the null hypothesis .

When the p -value falls below the chosen alpha value, then we say the result of the test is statistically significant.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bevans, R. (2023, June 22). Test statistics | Definition, Interpretation, and Examples. Scribbr. Retrieved April 8, 2024, from https://www.scribbr.com/statistics/test-statistic/

Is this article helpful?

Rebecca Bevans

Other students also liked, understanding p values | definition and examples, choosing the right statistical test | types & examples, what is effect size and why does it matter (examples), what is your plagiarism score.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
Ind Psychiatry J
v.18(1); Jan-Jun 2009

Probability, clinical decision making and hypothesis testing

A. banerjee.

Department of Community Medicine, D. Y. Patil Medical College, Pune - 411018, India

S. L. Jadhav

J. s. bhawalkar.

Few clinicians grasp the true concept of probability expressed in the ‘ P value.’ For most, a statistically significant P value is the end of the search for truth. In fact, the opposite is the case. The present paper attempts to put the P value in proper perspective by explaining different types of probabilities, their role in clinical decision making, medical research and hypothesis testing.

The clinician who wishes to remain abreast with the results of medical research needs to develop a statistical sense. He reads a number of journal articles; and constantly, he must ask questions such as, “Am I convinced that lack of mental activity predisposes to Alzheimer’s? Or “Do I believe that a particular drug cures more patients than the drug I use currently?”

The results of most studies are quantitative; and in earlier times, the reader made up his mind whether or not to accept the results of a particular study by merely looking at the figures. For instance, if 25 out of 30 patients were cured with a new drug compared with 15 out of the 30 on placebo, the superiority of the new drug was readily accepted.

In recent years, the presentation of medical research has undergone much transformation. Nowadays, no respectable journal will accept a paper if the results have not been subjected to statistical significance tests. The use of statistics has accelerated with the ready availability of statistical software. It has now become fashionable to organize workshops on research methodology and biostatistics. No doubt, this development was long overdue and one concedes that the methodologies of most medical papers have considerably improved in recent years. But at the same time, a new problem has arisen. The reading of medical journals today presupposes considerable statistical knowledge; however, those doctors who are not familiar with statistical theory tend to interpret the results of significance tests uncritically or even incorrectly.

It is often overlooked that the results of a statistical test depend not only on the observed data but also on the choice of statistical model. The statistician doing analysis of the data has a choice between several tests which are based on different models and assumptions. Unfortunately, many research workers who know little about statistics leave the statistical analysis to statisticians who know little about medicine; and the end result may well be a series of meaningless calculations.

Many readers of medical journals do not know the correct interpretation of ‘ P values,’ which are the results of significance tests. Usually, it is only stated whether the P value is below 5% ( P < .05) or above 5% ( P > .05). According to convention, the results of P < .05 are said to be statistically significant, and those with P > .05 are said to be statistically nonsignificant. These expressions are taken so seriously by most that it is almost considered ‘unscientific’ to believe in a nonsignificant result or not to believe in a ‘significant’ result. It is taken for granted that a ‘significant’ difference is a true difference and that a ‘nonsignificant’ difference is a chance finding and does not merit further exploration. Nothing can be further from the truth.

The present paper endeavors to explain the meaning of probability, its role in everyday clinical practice and the concepts behind hypothesis testing.

WHAT IS PROBABILITY?

Probability is a recurring theme in medical practice. No doctor who returns home from a busy day at the hospital is spared the nagging feeling that some of his diagnoses may turn out to be wrong, or some of his treatments may not lead to the expected cure. Encountering the unexpected is an occupational hazard in clinical practice. Doctors after some experience in their profession reconcile to the fact that diagnosis and prognosis always have varying degrees of uncertainty and at best can be stated as probable in a particular case.

Critical appraisal of medical journals also leads to the same gut feeling. One is bombarded with new research results, but experience dictates that well-established facts of today may be refuted in some other scientific publication in the following weeks or months. When a practicing clinician reads that some new treatment is superior to the conventional one, he will assess the evidence critically, and at best he will conclude that probably it is true.

Two types of probabilities

The statistical probability concept is so widely prevalent that almost everyone believes that probability is a frequency . It is not, of course, an ordinary frequency which can be estimated by simple observations, but it is the ideal or truth in the universe , which is reflected by the observed frequency. For example, when we want to determine the probability of obtaining an ace from a pack of cards (which, let us assume has been tampered with by a dishonest gambler), we proceed by drawing a card from the pack a large number of times, as we know in the long run, the observed frequency will approach the true probability or truth in the universe. Mathematicians often state that a probability is a long-run frequency, and a probability that is defined in this way is called a frequential probability . The exact magnitude of a frequential probability will remain elusive as we cannot make an infinite number of observations; but when we have made a decent number of observations (adequate sample size), we can calculate the confidence intervals, which are likely to include the true frequential probability. The width of the confidence interval depends on the number of observations (sample size).

The frequential probability concept is so prevalent that we tend to overlook terms like chance, risk and odds, in which the term probability implies a different meaning. Few hypothetical examples will make this clear. Consider the statement, “The cure for Alzheimer’s disease will probably be discovered in the coming decade.” This statement does not indicate the basis of this expectation or belief as in frequential probability, where a number of repeated observations provide the foundation for probability calculation. However, it may be based on the present state of research in Alzheimer’s. A probabilistic statement incorporates some amount of uncertainty, which may be quantified as follows: A politician may state that there is a fifty-fifty chance of winning the next election, a bookie may say that the odds of India winning the next one-day cricket game is four to one, and so on. At first glance, such probabilities may appear frequential ones, but a little reflection will reveal the contrary. We are concerned with unique events, i.e., the likely cure of a disease in the future, the next particular election, the next particular one-day game — and it makes no sense to apply the statistical idea that these types of probabilities are long-run frequencies. Further reflection will illustrate that these statements about probabilities of the election and one-day game are no different from the one about the cure for Alzheimer’s, apart from the fact that in the latter cases an attempt has been made to quantify the magnitude of belief in the occurrence of the event.

It follows from the above deliberations that we have 2 types of probability concepts. In the jargon of statistics, a probability is ideal or truth in the universe which lies beneath an observed frequency — such probabilities may be called frequential probabilities. In literary language, a probability is a measure of our subjective belief in the occurrence of a particular event or truth of a hypothesis. Such probabilities, which may be quantified that they look like frequential ones, are called subjective probabilities. Bayesian statistical theory also takes into account subjective probabilities (Lindley, 1973; Winkler, 1972). The following examples will try to illustrate these (rather confusing) concepts.

A young man is brought to the psychiatry OPD with history of withdrawal. He also gives history of talking to himself and giggling without cause. There is also a positive family history of schizophrenia. The consulting psychiatrist who examines the patient concludes that there is a 90% probability that this patient suffers from schizophrenia.

We ask the psychiatrist what makes him make such a statement. He may not be able to say that he knows from experience that 90% of such patients suffer from schizophrenia. The statement therefore may not be based on observed frequency. Instead, the psychiatrist states his probability based on his knowledge of the natural history of disease and the available literature regarding signs and symptoms in schizophrenia and positive family history. From this knowledge, the psychiatrist concludes that his belief in the diagnosis of schizophrenia in that particular patient is as strong as his belief in picking a black ball from a box containing 10 white and 90 black balls. The probability in this case is certainly subjective probability .

Let us consider another example: A 26-year-old married female patient who suffered from severe abdominal pain is referred to a hospital. She is also having amenorrhea for the past 4 months. The pain is located in the left lower abdomen. The gynecologist who examines her concludes that there is a 30% probability that the patient is suffering from ectopic pregnancy.

As before, we ask the gynecologist to explain on what basis the diagnosis of ectopic pregnancy is suspected. In this case the gynecologist states that he has studied a large number of successive patients with this symptom complex of lower abdominal pain with amenorrhea, and that a subsequent laparotomy revealed an ectopic pregnancy in 30% of the cases.

If we accept that the study cited is large enough to make us assume that the possibility of the observed frequency of ectopic pregnancy did not differ from the true frequential probability, it is natural to conclude that the gynecologist’s probability claim is more ‘evidence based’ than that of the psychiatrist, but again this is debatable.

In order to grasp this in proper perspective, it is necessary to note that the gynecologist stated that the probability of ectopic pregnancy in that particular patient was 30%. Therefore, we are concerned with a unique event just as the politician’s next election or India’s next one-day match. So in this case also, the probability is a subjective probability which was based on an observed frequency . One might also argue that even this probability is not good enough. We might ask the gynecologist to base his belief on a group of patients who also had the same age, height, color of hair and social background; and in the end, the reference group would be so restrictive that even the experience from a very large study would not provide the necessary information. If we went even further and required that he must base his belief on patients who in all respects resembled this particular patient, the probabilistic problem would vanish as we will be dealing with a certainty rather than a probability.

The clinician’s belief in a particular diagnosis in an individual patient may be based on the recorded experience in a group of patients, but it is still a subjective probability. It reflects not only the observed frequency of the disease in a reference group but also the clinician’s theoretical knowledge which determines the choice of reference group (Wulff, Pedersen and Rosenberg, 1986). Recorded experience is never the sole basis of clinical decision making.

GAP BETWEEN THEORY AND PRACTICE

The two situations described above are relatively straightforward. The physician observed a patient with a particular set of signs and symptoms and assessed the subjective probability about the diagnosis in each case. Such probabilities have been termed diagnostic probabilities (Wulff, Pedersen and Rosenberg, 1986). In practice, however, clinicians make diagnosis in a more complex manner which they themselves may be unable to analyze logically.

For instance, suppose the clinician suspects one of his patients is suffering from a rare disease named ‘D.’ He requests a suitable test to confirm the diagnosis, and suppose the test is positive for disease ‘D.’ He now wishes to assess the probability of the diagnosis being positive on the basis of this information, but perhaps the medical literature only provides the information that a positive test is seen in 70% of the patients with disease ‘D.’ However, it is also positive in 2% of patients without disease ‘D.’ How to tackle this doctor’s dilemma? First a formal analysis may be attempted, and then we can return to everyday clinical thinking. The frequential probability which the doctor found in the literature may be written in the statistical notation as follows:

P (S/D+) = .70, i.e., the probability of the presence of this particular sign (or test) given this particular disease is 70%.

P (S/D–) = .02, i.e., the probability of this particular sign given the absence of this particular disease is 2%.

However, such probabilities are of little clinical relevance. The clinical relevance is in the ‘opposite’ probability. In clinical practice, one would like to know the P (D/S), i.e., the probability of the disease in a particular patient given this positive sign. This can be estimated by means of Bayes’ Theorem (Papoulis, 1984; Lindley, 1973; Winkler, 1972). The formula of Bayes’ Theorem is reproduced below, from which it will be evident that to calculate P(D/S), we must also know the prior probability of the presence and the absence of the disease, i.e., P (D+) and P (D–).

P (D/S) = P (S/D+) P (D+) ÷ P (S/D+) P (D+) + P (S/D–) P (D–)

In the example of the disease ‘D’ above, let us assume that we estimate that prior probability of the disease being present, i.e., P (D+), is 25%; and therefore, prior probability of the absence of disease, i.e., P (D–), is 75%. Using the Bayes’ Theorem formula, we can calculate that the probability of the disease given a positive sign, i.e., P (D/S), is 92%.

We of course do not suggest that clinicians should always make calculations of this sort when confronted with a diagnostic dilemma, but they must in an intuitive way think along these lines. Clinical knowledge is to a large extent based on textbook knowledge, and ordinary textbooks do not tell the reader much about the probabilities of different diseases given different symptoms. Bayes’ Theorem guides a clinician how to use textbook knowledge for practical clinical purposes.

The practical significance of this point is illustrated by the European doctor who accepted a position at a hospital in tropical Africa. In order to prepare himself for the new job, he bought himself a large textbook of tropical medicine and studied in great detail the clinical pictures of a large number of exotic diseases. However, for several months after his arrival at the tropical hospital, his diagnostic performance was very poor, as he knew nothing about the relative frequency of all these diseases. He had to acquaint himself with the prior probability, P (D +), of the diseases in the catchment area of the hospital before he could make precise diagnoses.

The same thing happens on a smaller scale when a doctor trained at a university hospital establishes himself in general practice. At the beginning, he will suspect his patients of all sorts of rare diseases (which are common at the university hospital), but after a while he will learn to assess correctly the frequency of different diseases in the general population.

PROBABILITY AND HYPOTHESIS TESTING

Besides predictions on individual patients, the doctor is also concerned in generalizations to the population at large or the target population. We may say that probably there may have been life at Mars. We may even quantify our belief and mention that there is 95% probability that depression responds more quickly during treatment with a particular antidepressant than during treatment with a placebo. These probabilities are again subjective probabilities rather than frequential probabilities . The last statement does not imply that 95% of depression cases respond to the particular antidepressant or that 95% of the published reports mention that the particular antidepressant is the best. It simply means that our belief in the truth of the statement is the same as our belief in picking up a red ball from a box containing 95 red balls and 5 white balls. It means that we are, however, almost not totally convinced that the average recovery time during treatment with a particular antidepressant is shorter than during placebo treatment.

The purpose of hypothesis testing is to aid the clinician in reaching a conclusion concerning the universe by examining a sample from that universe. A hypothesis may be defined as a presumption or statement about the truth in the universe. For example, a clinician may hypothesize that a certain drug may be effective in 80% of the cases of schizophrenia. It is frequently concerned about the parameters in the population about which the presumption or statement is made. It is the basis for motivating the research project. There are two types of hypotheses, research hypothesis and statistical hypothesis (Daniel, 2000; Guyatt et al ., 1995).

Genesis of research hypothesis

Hypothesis may be generated by deduction from anatomical, physiological facts or from clinical observations.

Statistical hypothesis

Statistical hypotheses are hypotheses that are stated in such a way that they may be evaluated by appropriate statistical techniques.

Pre-requisites for hypothesis testing

Nature of data.

The types of data that form the basis of hypothesis testing procedures must be understood, since these dictate the choice of statistical test.

Presumptions

These presumptions are the normality of the population distribution, equality of the standard deviations, random samples.

There are 2 statistical hypotheses involved in hypothesis testing. These should be stated a priori and explicitly. The null hypothesis is the hypothesis to be tested. It is denoted by the symbol H 0 . It is also known as the hypothesis of no difference . The null hypothesis is set up with the sole purpose of efforts to knock it down. In the testing of hypothesis, the null hypothesis is either rejected (knocked down) or not rejected (upheld). If the null hypothesis is not rejected, the interpretation is that the data is not sufficient evidence to cause rejection. If the testing process rejects the null hypothesis, the inference is that the data available to us is not compatible with the null hypothesis and by default we accept the alternative hypothesis , which in most cases is the research hypothesis. The alternative hypothesis is designated with the symbol H A .

Limitations

Neither hypothesis testing nor statistical tests lead to proof. It merely indicates whether the hypothesis is supported or not supported by the available data. When we reject a null hypothesis, we do not mean it is not true but that it may be true. By default when we do not reject the null hypothesis, we should have this limitation in mind and should not convey the impression that this implies proof.

Test statistic

The test statistic is the statistic that is derived from the data from the sample. Evidently, many possible values of the test statistic can be computed depending on the particular sample selected. The test statistic serves as a decision maker, nothing more, nothing less, rather than proof or lack of it. The decision to reject or not to reject the null hypothesis depends on the magnitude of the test statistic.

Types of decision errors

The error committed when a true null hypothesis is rejected is called the type I error or α error . When a false null hypothesis is not rejected, we commit type II error, or β error . When we reject a null hypothesis, there is always the risk (howsoever small it may be) of committing a type I error, i.e., rejecting a true null hypothesis. On the other hand, whenever we fail to reject a null hypothesis, the risk of failing to reject a false null hypothesis, or committing a type II error, will always be present. Put in other words, the test statistic does not eliminate uncertainty (as many tend to believe); it only quantifies our uncertainty.

Calculation of test statistic

From the data contained in the sample, we compute a value of the test statistic and compare it with the rejection and non-rejection regions, which have to be specified in advance.

Statistical decision

The statistical decision consists of rejecting or of not rejecting the null hypothesis. It is rejected if the computed value of the test statistic falls in the rejection region, and it is not rejected if the value falls in the non-rejection region.

If H 0 is rejected, we conclude that H A is true. If H 0 is not rejected, we conclude that H 0 may be true.

The P value is a number that tells us how unlikely our sample values are, given that the null hypothesis is true. A P value indicating that the sample results are not likely to have occurred, if the null hypothesis is true, provides reason for doubting the truth of the null hypothesis.

We must remember that, when the null hypothesis is not rejected, one should not say the null hypothesis is accepted. We should mention that the null hypothesis is “not rejected.” We avoid using the word accepted in this case because we may have committed a type II error. Since, frequently, the probability of committing error can be quite high (particularly with small sample sizes), we should not commit ourselves to accepting the null hypothesis.

INTERPRETATIONS

With the above discussion on probability, clinical decision making and hypothesis testing in mind, let us reconsider the meaning of P values. When we come across the statement that there is statistically significant difference between two treatment regimes with P < .05, we should not interpret that there is less than 5% probability that there is no difference, and that there is 95% probability that a difference exists, as many uninformed readers tend to do. The statement that there is difference between the cure rates of two treatments is a general one, and we have already discussed that the probability of the truth of a general statement (hypothesis) is subjective , whereas the probabilities which are calculated by statisticians are frequential ones. The hypothesis that one treatment is better than the other is either true or false and cannot be interpreted in frequential terms.

To explain this further, suppose someone claims that 20 (80%) of 25 patients who received drug A were cured, compared to 12 (48%) of 25 patients who received drug B. In this case, there are two possibilities, either the null hypothesis is true, which means that the two treatments are equally effective and the observed difference arose by chance; or the null hypothesis is not true (and we accept the alternative hypothesis by default), which means that one treatment is better than the other. The clinician wants to make up his mind to what extent he believes in the truth of the alternative hypothesis (or the falsehood of the null hypothesis ). To resolve this issue, he needs the aid of statistical analysis. However, it is essential to note that the P value does not provide a direct answer. Let us assume in this case the statistician does a significance test and gets a P value = .04, meaning that the difference is statistically significant ( P < .05). But as explained earlier, this does not mean that there is a 4% probability that the null hypothesis is true and 96% chance that the alternative hypothesis is true. The P value is a frequential probability and it provides the information that there is a 4% probability of obtaining such a difference between the cure rates, if the null hypothesis is true . In other words, the statistician asks us to assume that the null hypothesis is true and to imagine that we do a large number of trials. In that case, the long-run frequency of trials which show a difference between the cure rates like the one we found, or even a larger one, will be 4%.

Prior belief and interpretation of the P value

In order to elucidate the implications of the correct statistical definition of the P value, let us imagine that the patients who took part in the above trial suffered from depression, and that drug A was gentamycin, while drug B was a placebo. Our theoretical knowledge gives us no grounds for believing that gentamycin has any affect whatsoever in the cure of depression. For this reason, our prior confidence in the truth of the null hypothesis is immense (say, 99.99%), whereas our prior confidence in the alternative hypothesis is minute (0.01%). We must take these prior probabilities into account when we assess the result of the trial. We have the following choice. Either we accept the null hypothesis in spite of the fact that the probability of the trial result is fairly low at 4% ( P < .05) given the null hypothesis is true, or we accept the alternative hypothesis by rejecting the null hypothesis in spite of the fact that the subjective probability of that hypothesis is extremely low in the light of our prior knowledge.

It will be evident that the choice is a difficult one, as both hypotheses, each in its own way, may be said to be unlikely, but any clinician who reasons along these lines will choose that hypothesis which is least unacceptable: He will accept the null hypothesis and claim that the difference between the cure rates arose by chance (however small it may be), as he does not feel that the evidence from this single trial is sufficient to shake his prior belief in the null hypothesis.

Misinterpretation of P values is extremely common. One of the reasons may be that those who teach research methods do not themselves appreciate the problem. The P value is the probability of obtaining a value of the test statistic as large as or larger than the one computed from the data when in reality there is no difference between the different treatments. In other words, the P value is the probability of being wrong when asserting that a difference exists.

Lastly, we must remember we do not establish proof by hypothesis testing, and uncertainty will always remain in empirical research; at the most, we can only quantify our uncertainty.

Source of Support: Nil

Conflict of Interest: None declared.

User Preferences

Content preview.

Arcu felis bibendum ut tristique et egestas quis:

Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
Duis aute irure dolor in reprehenderit in voluptate
Excepteur sint occaecat cupidatat non proident

Keyboard Shortcuts

25.1 - definition of power.

Let's start our discussion of statistical power by recalling two definitions we learned when we first introduced to hypothesis testing:

A Type I error occurs if we reject the null hypothesis $H_0$ (in favor of the alternative hypothesis $H_A$) when the null hypothesis $H_0$ is true. We denote $\alpha=P(\text{Type I error})$.
A Type II error occurs if we fail to reject the null hypothesis $H_0$ when the alternative hypothesis $H_A$ is true. We denote $\beta=P(\text{Type II error})$.

You'll certainly need to know these two definitions inside and out, as you'll be thinking about them a lot in this lesson, and at any time in the future when you need to calculate a sample size either for yourself or for someone else.

Example 25-1 Section

The Brinell hardness scale is one of several definitions used in the field of materials science to quantify the hardness of a piece of metal. The Brinell hardness measurement of a certain type of rebar used for reinforcing concrete and masonry structures was assumed to be normally distributed with a standard deviation of 10 kilograms of force per square millimeter. Using a random sample of $n=25$ bars, an engineer is interested in performing the following hypothesis test:

the null hypothesis $H_0:\mu=170$
against the alternative hypothesis $H_A:\mu>170$

If the engineer decides to reject the null hypothesis if the sample mean is 172 or greater, that is, if $\bar{X} \ge 172 $, what is the probability that the engineer commits a Type I error?

In this case, the engineer commits a Type I error if his observed sample mean falls in the rejection region, that is, if it is 172 or greater, when the true (unknown) population mean is indeed 170. Graphically, $\alpha$, the engineer's probability of committing a Type I error looks like this:

Now, we can calculate the engineer's value of $\alpha$ by making the transformation from a normal distribution with a mean of 170 and a standard deviation of 10 to that of $Z$, the standard normal distribution using:

$Z= \frac{\bar{X}-\mu}{\sigma / \sqrt{n}} $

Doing so, we get:

So, calculating the engineer's probability of committing a Type I error reduces to making a normal probability calculation. The probability is 0.1587 as illustrated here:

$\alpha = P(\bar{X} \ge 172 \text { if } \mu = 170) = P(Z \ge 1.00) = 0.1587 $

A probability of 0.1587 is a bit high. We'll learn in this lesson how the engineer could reduce his probability of committing a Type I error.

If, unknown to engineer, the true population mean were $\mu=173$, what is the probability that the engineer commits a Type II error?

In this case, the engineer commits a Type II error if his observed sample mean does not fall in the rejection region, that is, if it is less than 172, when the true (unknown) population mean is 173. Graphically, $\beta$, the engineer's probability of committing a Type II error looks like this:

Again, we can calculate the engineer's value of $\beta$ by making the transformation from a normal distribution with a mean of 173 and a standard deviation of 10 to that of $Z$, the standard normal distribution. Doing so, we get:

So, calculating the engineer's probability of committing a Type II error again reduces to making a normal probability calculation. The probability is 0.3085 as illustrated here:

$\beta= P(\bar{X} < 172 \text { if } \mu = 173) = P(Z < -0.50) = 0.3085 $

A probability of 0.3085 is a bit high. We'll learn in this lesson how the engineer could reduce his probability of committing a Type II error.

The power of a hypothesis test is the probability of making the correct decision if the alternative hypothesis is true. That is, the power of a hypothesis test is the probability of rejecting the null hypothesis $H_0$ when the alternative hypothesis $H_A$ is the hypothesis that is true.

Let's return to our engineer's problem to see if we can instead look at the glass as being half full!

Example 25-1 (continued) Section

If, unknown to the engineer, the true population mean were $\mu=173$, what is the probability that the engineer makes the correct decision by rejecting the null hypothesis in favor of the alternative hypothesis?

In this case, the engineer makes the correct decision if his observed sample mean falls in the rejection region, that is, if it is greater than 172, when the true (unknown) population mean is 173. Graphically, the power of the engineer's hypothesis test looks like this:

That makes the power of the engineer's hypothesis test 0.6915 as illustrated here:

$\text{Power } = P(\bar{X} \ge 172 \text { if } \mu = 173) = P(Z \ge -0.50) = 0.6915 $

which of course could have alternatively been calculated by simply subtracting the probability of committing a Type II error from 1, as shown here:

$\text{Power } = 1 - \beta = 1 - 0.3085 = 0.6915 $

At any rate, if the unknown population mean were 173, the engineer's hypothesis test would be at least a bit better than flipping a fair coin, in which he'd have but a 50% chance of choosing the correct hypothesis. In this case, he has a 69.15% chance. He could still do a bit better.

In general, for every hypothesis test that we conduct, we'll want to do the following:

Minimize the probability of committing a Type I error. That, is minimize $\alpha=P(\text{Type I Error})$. Typically, a significance level of $\alpha\le 0.10$ is desired.

Maximize the power (at a value of the parameter under the alternative hypothesis that is scientifically meaningful). Typically, we desire power to be 0.80 or greater. Alternatively, we could minimize $\beta=P(\text{Type II Error})$, aiming for a type II error rate of 0.20 or less.

By the way, in the second point, what exactly does "at a value of the parameter under the alternative hypothesis that is scientifically meaningful" mean? Well, let's suppose that a medical researcher is interested in testing the null hypothesis that the mean total blood cholesterol in a population of patients is 200 mg/dl against the alternative hypothesis that the mean total blood cholesterol is greater than 200 mg/dl . Well, the alternative hypothesis contains an infinite number of possible values of the mean. Under the alternative hypothesis, the mean of the population could be, among other values, 201, 202, or 210. Suppose the medical researcher rejected the null hypothesis, because the mean was 201. Whoopdy-do...would that be a rocking conclusion? No, probably not. On the other hand, suppose the medical researcher rejected the null hypothesis, because the mean was 215. In that case, the mean is substantially different enough from the assumed mean under the null hypothesis, that we'd probably get excited about the result. In summary, in this example, we could probably all agree to consider a mean of 215 to be "scientifically meaningful," whereas we could not do the same for a mean of 201.

Now, of course, all of this talk is a bit if gibberish, because we'd never really know whether the true unknown population mean were 201 or 215, otherwise, we wouldn't have to be going through the process of conducting a hypothesis test about the mean. We can do something though. We can plan our scientific studies so that our hypothesis tests have enough power to reject the null hypothesis in favor of values of the parameter under the alternative hypothesis that are scientifically meaningful.

Tutorial Playlist

Statistics tutorial, everything you need to know about the probability density function in statistics, the best guide to understand central limit theorem, an in-depth guide to measures of central tendency : mean, median and mode, the ultimate guide to understand conditional probability.

A Comprehensive Look at Percentile in Statistics

The Best Guide to Understand Bayes Theorem

Everything you need to know about the normal distribution, an in-depth explanation of cumulative distribution function, a complete guide to chi-square test, a complete guide on hypothesis testing in statistics, understanding the fundamentals of arithmetic and geometric progression, the definitive guide to understand spearman’s rank correlation, a comprehensive guide to understand mean squared error, all you need to know about the empirical rule in statistics, the complete guide to skewness and kurtosis, a holistic look at bernoulli distribution.

All You Need to Know About Bias in Statistics

A Complete Guide to Get a Grasp of Time Series Analysis

The Key Differences Between Z-Test Vs. T-Test

The Complete Guide to Understand Pearson's Correlation

A complete guide on the types of statistical studies, everything you need to know about poisson distribution, your best guide to understand correlation vs. regression, the most comprehensive guide for beginners on what is correlation, what is hypothesis testing in statistics types and examples.

Lesson 10 of 24 By Avijeet Biswal

A Complete Guide on Hypothesis Testing in Statistics

In today’s data-driven world , decisions are based on data all the time. Hypothesis plays a crucial role in that process, whether it may be making business decisions, in the health sector, academia, or in quality improvement. Without hypothesis & hypothesis tests, you risk drawing the wrong conclusions and making bad decisions. In this tutorial, you will look at Hypothesis Testing in Statistics.

What Is Hypothesis Testing in Statistics?

Hypothesis Testing is a type of statistical analysis in which you put your assumptions about a population parameter to the test. It is used to estimate the relationship between 2 statistical variables.

Let's discuss few examples of statistical hypothesis from real-life -

A teacher assumes that 60% of his college's students come from lower-middle-class families.
A doctor believes that 3D (Diet, Dose, and Discipline) is 90% effective for diabetic patients.

Now that you know about hypothesis testing, look at the two types of hypothesis testing in statistics.

Hypothesis Testing Formula

Z = ( x̅ – μ0 ) / (σ /√n)

Here, x̅ is the sample mean,
μ0 is the population mean,
σ is the standard deviation,
n is the sample size.

How Hypothesis Testing Works?

An analyst performs hypothesis testing on a statistical sample to present evidence of the plausibility of the null hypothesis. Measurements and analyses are conducted on a random sample of the population to test a theory. Analysts use a random population sample to test two hypotheses: the null and alternative hypotheses.

The null hypothesis is typically an equality hypothesis between population parameters; for example, a null hypothesis may claim that the population means return equals zero. The alternate hypothesis is essentially the inverse of the null hypothesis (e.g., the population means the return is not equal to zero). As a result, they are mutually exclusive, and only one can be correct. One of the two possibilities, however, will always be correct.

Your Dream Career is Just Around The Corner!

Null Hypothesis and Alternate Hypothesis

The Null Hypothesis is the assumption that the event will not occur. A null hypothesis has no bearing on the study's outcome unless it is rejected.

H0 is the symbol for it, and it is pronounced H-naught.

The Alternate Hypothesis is the logical opposite of the null hypothesis. The acceptance of the alternative hypothesis follows the rejection of the null hypothesis. H1 is the symbol for it.

Let's understand this with an example.

A sanitizer manufacturer claims that its product kills 95 percent of germs on average.

To put this company's claim to the test, create a null and alternate hypothesis.

H0 (Null Hypothesis): Average = 95%.

Alternative Hypothesis (H1): The average is less than 95%.

Another straightforward example to understand this concept is determining whether or not a coin is fair and balanced. The null hypothesis states that the probability of a show of heads is equal to the likelihood of a show of tails. In contrast, the alternate theory states that the probability of a show of heads and tails would be very different.

Become a Data Scientist with Hands-on Training!

Hypothesis Testing Calculation With Examples

Let's consider a hypothesis test for the average height of women in the United States. Suppose our null hypothesis is that the average height is 5'4". We gather a sample of 100 women and determine that their average height is 5'5". The standard deviation of population is 2.

To calculate the z-score, we would use the following formula:

z = ( x̅ – μ0 ) / (σ /√n)

z = (5'5" - 5'4") / (2" / √100)

z = 0.5 / (0.045)

We will reject the null hypothesis as the z-score of 11.11 is very large and conclude that there is evidence to suggest that the average height of women in the US is greater than 5'4".

Steps of Hypothesis Testing

Step 1: specify your null and alternate hypotheses.

It is critical to rephrase your original research hypothesis (the prediction that you wish to study) as a null (Ho) and alternative (Ha) hypothesis so that you can test it quantitatively. Your first hypothesis, which predicts a link between variables, is generally your alternate hypothesis. The null hypothesis predicts no link between the variables of interest.

Step 2: Gather Data

For a statistical test to be legitimate, sampling and data collection must be done in a way that is meant to test your hypothesis. You cannot draw statistical conclusions about the population you are interested in if your data is not representative.

Step 3: Conduct a Statistical Test

Other statistical tests are available, but they all compare within-group variance (how to spread out the data inside a category) against between-group variance (how different the categories are from one another). If the between-group variation is big enough that there is little or no overlap between groups, your statistical test will display a low p-value to represent this. This suggests that the disparities between these groups are unlikely to have occurred by accident. Alternatively, if there is a large within-group variance and a low between-group variance, your statistical test will show a high p-value. Any difference you find across groups is most likely attributable to chance. The variety of variables and the level of measurement of your obtained data will influence your statistical test selection.

Step 4: Determine Rejection Of Your Null Hypothesis

Your statistical test results must determine whether your null hypothesis should be rejected or not. In most circumstances, you will base your judgment on the p-value provided by the statistical test. In most circumstances, your preset level of significance for rejecting the null hypothesis will be 0.05 - that is, when there is less than a 5% likelihood that these data would be seen if the null hypothesis were true. In other circumstances, researchers use a lower level of significance, such as 0.01 (1%). This reduces the possibility of wrongly rejecting the null hypothesis.

Step 5: Present Your Results

The findings of hypothesis testing will be discussed in the results and discussion portions of your research paper, dissertation, or thesis. You should include a concise overview of the data and a summary of the findings of your statistical test in the results section. You can talk about whether your results confirmed your initial hypothesis or not in the conversation. Rejecting or failing to reject the null hypothesis is a formal term used in hypothesis testing. This is likely a must for your statistics assignments.

Types of Hypothesis Testing

To determine whether a discovery or relationship is statistically significant, hypothesis testing uses a z-test. It usually checks to see if two means are the same (the null hypothesis). Only when the population standard deviation is known and the sample size is 30 data points or more, can a z-test be applied.

A statistical test called a t-test is employed to compare the means of two groups. To determine whether two groups differ or if a procedure or treatment affects the population of interest, it is frequently used in hypothesis testing.

Chi-Square

You utilize a Chi-square test for hypothesis testing concerning whether your data is as predicted. To determine if the expected and observed results are well-fitted, the Chi-square test analyzes the differences between categorical variables from a random sample. The test's fundamental premise is that the observed values in your data should be compared to the predicted values that would be present if the null hypothesis were true.

Hypothesis Testing and Confidence Intervals

Both confidence intervals and hypothesis tests are inferential techniques that depend on approximating the sample distribution. Data from a sample is used to estimate a population parameter using confidence intervals. Data from a sample is used in hypothesis testing to examine a given hypothesis. We must have a postulated parameter to conduct hypothesis testing.

Bootstrap distributions and randomization distributions are created using comparable simulation techniques. The observed sample statistic is the focal point of a bootstrap distribution, whereas the null hypothesis value is the focal point of a randomization distribution.

A variety of feasible population parameter estimates are included in confidence ranges. In this lesson, we created just two-tailed confidence intervals. There is a direct connection between these two-tail confidence intervals and these two-tail hypothesis tests. The results of a two-tailed hypothesis test and two-tailed confidence intervals typically provide the same results. In other words, a hypothesis test at the 0.05 level will virtually always fail to reject the null hypothesis if the 95% confidence interval contains the predicted value. A hypothesis test at the 0.05 level will nearly certainly reject the null hypothesis if the 95% confidence interval does not include the hypothesized parameter.

Simple and Composite Hypothesis Testing

Depending on the population distribution, you can classify the statistical hypothesis into two types.

Simple Hypothesis: A simple hypothesis specifies an exact value for the parameter.

Composite Hypothesis: A composite hypothesis specifies a range of values.

A company is claiming that their average sales for this quarter are 1000 units. This is an example of a simple hypothesis.

Suppose the company claims that the sales are in the range of 900 to 1000 units. Then this is a case of a composite hypothesis.

One-Tailed and Two-Tailed Hypothesis Testing

The One-Tailed test, also called a directional test, considers a critical region of data that would result in the null hypothesis being rejected if the test sample falls into it, inevitably meaning the acceptance of the alternate hypothesis.

In a one-tailed test, the critical distribution area is one-sided, meaning the test sample is either greater or lesser than a specific value.

In two tails, the test sample is checked to be greater or less than a range of values in a Two-Tailed test, implying that the critical distribution area is two-sided.

If the sample falls within this range, the alternate hypothesis will be accepted, and the null hypothesis will be rejected.

Become a Data Scientist With Real-World Experience

Right Tailed Hypothesis Testing

If the larger than (>) sign appears in your hypothesis statement, you are using a right-tailed test, also known as an upper test. Or, to put it another way, the disparity is to the right. For instance, you can contrast the battery life before and after a change in production. Your hypothesis statements can be the following if you want to know if the battery life is longer than the original (let's say 90 hours):

The null hypothesis is (H0 <= 90) or less change.
A possibility is that battery life has risen (H1) > 90.

The crucial point in this situation is that the alternate hypothesis (H1), not the null hypothesis, decides whether you get a right-tailed test.

Left Tailed Hypothesis Testing

Alternative hypotheses that assert the true value of a parameter is lower than the null hypothesis are tested with a left-tailed test; they are indicated by the asterisk "<".

Suppose H0: mean = 50 and H1: mean not equal to 50

According to the H1, the mean can be greater than or less than 50. This is an example of a Two-tailed test.

In a similar manner, if H0: mean >=50, then H1: mean <50

Here the mean is less than 50. It is called a One-tailed test.

Type 1 and Type 2 Error

A hypothesis test can result in two types of errors.

Type 1 Error: A Type-I error occurs when sample results reject the null hypothesis despite being true.

Type 2 Error: A Type-II error occurs when the null hypothesis is not rejected when it is false, unlike a Type-I error.

Suppose a teacher evaluates the examination paper to decide whether a student passes or fails.

H0: Student has passed

H1: Student has failed

Type I error will be the teacher failing the student [rejects H0] although the student scored the passing marks [H0 was true].

Type II error will be the case where the teacher passes the student [do not reject H0] although the student did not score the passing marks [H1 is true].

Level of Significance

The alpha value is a criterion for determining whether a test statistic is statistically significant. In a statistical test, Alpha represents an acceptable probability of a Type I error. Because alpha is a probability, it can be anywhere between 0 and 1. In practice, the most commonly used alpha values are 0.01, 0.05, and 0.1, which represent a 1%, 5%, and 10% chance of a Type I error, respectively (i.e. rejecting the null hypothesis when it is in fact correct).

Future-Proof Your AI/ML Career: Top Dos and Don'ts

A p-value is a metric that expresses the likelihood that an observed difference could have occurred by chance. As the p-value decreases the statistical significance of the observed difference increases. If the p-value is too low, you reject the null hypothesis.

Here you have taken an example in which you are trying to test whether the new advertising campaign has increased the product's sales. The p-value is the likelihood that the null hypothesis, which states that there is no change in the sales due to the new advertising campaign, is true. If the p-value is .30, then there is a 30% chance that there is no increase or decrease in the product's sales. If the p-value is 0.03, then there is a 3% probability that there is no increase or decrease in the sales value due to the new advertising campaign. As you can see, the lower the p-value, the chances of the alternate hypothesis being true increases, which means that the new advertising campaign causes an increase or decrease in sales.

Why is Hypothesis Testing Important in Research Methodology?

Hypothesis testing is crucial in research methodology for several reasons:

Provides evidence-based conclusions: It allows researchers to make objective conclusions based on empirical data, providing evidence to support or refute their research hypotheses.
Supports decision-making: It helps make informed decisions, such as accepting or rejecting a new treatment, implementing policy changes, or adopting new practices.
Adds rigor and validity: It adds scientific rigor to research using statistical methods to analyze data, ensuring that conclusions are based on sound statistical evidence.
Contributes to the advancement of knowledge: By testing hypotheses, researchers contribute to the growth of knowledge in their respective fields by confirming existing theories or discovering new patterns and relationships.

Limitations of Hypothesis Testing

Hypothesis testing has some limitations that researchers should be aware of:

It cannot prove or establish the truth: Hypothesis testing provides evidence to support or reject a hypothesis, but it cannot confirm the absolute truth of the research question.
Results are sample-specific: Hypothesis testing is based on analyzing a sample from a population, and the conclusions drawn are specific to that particular sample.
Possible errors: During hypothesis testing, there is a chance of committing type I error (rejecting a true null hypothesis) or type II error (failing to reject a false null hypothesis).
Assumptions and requirements: Different tests have specific assumptions and requirements that must be met to accurately interpret results.

After reading this tutorial, you would have a much better understanding of hypothesis testing, one of the most important concepts in the field of Data Science . The majority of hypotheses are based on speculation about observed behavior, natural phenomena, or established theories.

If you are interested in statistics of data science and skills needed for such a career, you ought to explore Simplilearn’s Post Graduate Program in Data Science.

If you have any questions regarding this ‘Hypothesis Testing In Statistics’ tutorial, do share them in the comment section. Our subject matter expert will respond to your queries. Happy learning!

1. What is hypothesis testing in statistics with example?

Hypothesis testing is a statistical method used to determine if there is enough evidence in a sample data to draw conclusions about a population. It involves formulating two competing hypotheses, the null hypothesis (H0) and the alternative hypothesis (Ha), and then collecting data to assess the evidence. An example: testing if a new drug improves patient recovery (Ha) compared to the standard treatment (H0) based on collected patient data.

2. What is hypothesis testing and its types?

Hypothesis testing is a statistical method used to make inferences about a population based on sample data. It involves formulating two hypotheses: the null hypothesis (H0), which represents the default assumption, and the alternative hypothesis (Ha), which contradicts H0. The goal is to assess the evidence and determine whether there is enough statistical significance to reject the null hypothesis in favor of the alternative hypothesis.

Types of hypothesis testing:

One-sample test: Used to compare a sample to a known value or a hypothesized value.
Two-sample test: Compares two independent samples to assess if there is a significant difference between their means or distributions.
Paired-sample test: Compares two related samples, such as pre-test and post-test data, to evaluate changes within the same subjects over time or under different conditions.
Chi-square test: Used to analyze categorical data and determine if there is a significant association between variables.
ANOVA (Analysis of Variance): Compares means across multiple groups to check if there is a significant difference between them.

3. What are the steps of hypothesis testing?

The steps of hypothesis testing are as follows:

Formulate the hypotheses: State the null hypothesis (H0) and the alternative hypothesis (Ha) based on the research question.
Set the significance level: Determine the acceptable level of error (alpha) for making a decision.
Collect and analyze data: Gather and process the sample data.
Compute test statistic: Calculate the appropriate statistical test to assess the evidence.
Make a decision: Compare the test statistic with critical values or p-values and determine whether to reject H0 in favor of Ha or not.
Draw conclusions: Interpret the results and communicate the findings in the context of the research question.

4. What are the 2 types of hypothesis testing?

One-tailed (or one-sided) test: Tests for the significance of an effect in only one direction, either positive or negative.
Two-tailed (or two-sided) test: Tests for the significance of an effect in both directions, allowing for the possibility of a positive or negative effect.

The choice between one-tailed and two-tailed tests depends on the specific research question and the directionality of the expected effect.

5. What are the 3 major types of hypothesis?

The three major types of hypotheses are:

Null Hypothesis (H0): Represents the default assumption, stating that there is no significant effect or relationship in the data.
Alternative Hypothesis (Ha): Contradicts the null hypothesis and proposes a specific effect or relationship that researchers want to investigate.
Nondirectional Hypothesis: An alternative hypothesis that doesn't specify the direction of the effect, leaving it open for both positive and negative possibilities.

Find our Data Analyst Online Bootcamp in top cities:

About the author.

Avijeet is a Senior Research Analyst at Simplilearn. Passionate about Data Analytics, Machine Learning, and Deep Learning, Avijeet is also interested in politics, cricket, and football.

Recommended Resources

Free eBook: Top Programming Languages For A Data Scientist

Normality Test in Minitab: Minitab with Statistics

Machine Learning Career Guide: A Playbook to Becoming a Machine Learning Engineer

PMP, PMI, PMBOK, CAPM, PgMP, PfMP, ACP, PBA, RMP, SP, and OPM3 are registered marks of the Project Management Institute, Inc.

IMAGES

Hypothesis Testing- Meaning, Types & Steps
Statistical Hypothesis Testing: Step by Step
PPT
PPT
What is Hypothesis Testing? Types and Methods
Introduction to Hypothesis Testing in R

VIDEO

Hypothesis-Testing regarding
Probability
Probability
Conducting Hypothesis Test Using The Traditional Method (The z-test Formula)
Probability
EXPLORING MORE ELEMENTS OF HYPOTHESIS TESTING

COMMENTS

9.1: Introduction to Hypothesis Testing
In hypothesis testing, the goal is to see if there is sufficient statistical evidence to reject a presumed null hypothesis in favor of a conjectured alternative hypothesis.The null hypothesis is usually denoted $H_0$ while the alternative hypothesis is usually denoted $H_1$. An hypothesis test is a statistical decision; the conclusion will either be to reject the null hypothesis in favor ...
Statistical hypothesis test
Statistical hypothesis testing is a key technique of both frequentist inference and Bayesian inference, although the two types of inference have notable differences. Statistical hypothesis tests define a procedure that controls (fixes) the probability of incorrectly deciding that a default position (null hypothesis) is incorrect. The procedure ...
Hypothesis Testing
There are 5 main steps in hypothesis testing: State your research hypothesis as a null hypothesis and alternate hypothesis (H o) and (H a or H 1 ). Collect data in a way designed to test the hypothesis. Perform an appropriate statistical test. Decide whether to reject or fail to reject your null hypothesis. Present the findings in your results ...
Statistics
Hypothesis testing. Hypothesis testing is a form of statistical inference that uses data from a sample to draw conclusions about a population parameter or a population probability distribution.First, a tentative assumption is made about the parameter or distribution. This assumption is called the null hypothesis and is denoted by H 0.An alternative hypothesis (denoted H a), which is the ...
Introduction to Hypothesis Testing
A hypothesis test consists of five steps: 1. State the hypotheses. State the null and alternative hypotheses. These two hypotheses need to be mutually exclusive, so if one is true then the other must be false. 2. Determine a significance level to use for the hypothesis. Decide on a significance level.
Statistical Hypothesis Testing Overview
Statistical power is the probability that a hypothesis test correctly infers that a sample effect exists in the population. In other words, the test correctly rejects a false null hypothesis. ... The p-value has a fairly convoluted definition. It is the probability of obtaining the effect observed in a sample, or more extreme, if the null ...
An Introduction to Statistics: Understanding Hypothesis Testing and
HYPOTHESIS TESTING. A clinical trial begins with an assumption or belief, and then proceeds to either prove or disprove this assumption. In statistical terms, this belief or assumption is known as a hypothesis. Counterintuitively, what the researcher believes in (or is trying to prove) is called the "alternate" hypothesis, and the opposite ...
Significance tests (hypothesis testing)
Significance tests give us a formal process for using sample data to evaluate the likelihood of some claim about a population value. Learn how to conduct significance tests and calculate p-values to see how likely a sample result is to occur by random chance. You'll also see how we use p-values to make conclusions about hypotheses.
Hypothesis Testing
Hypothesis testing is a technique that is used to verify whether the results of an experiment are statistically significant. It involves the setting up of a null hypothesis and an alternate hypothesis. There are three types of tests that can be conducted under hypothesis testing - z test, t test, and chi square test.
P Values (Calculated Probability) and Hypothesis Testing
P Values. The P value, or calculated probability, is the probability of finding the observed, or more extreme, results when the null hypothesis (H0) of a study question is true - the definition of 'extreme' depends on how the hypothesis is being tested. P is also described in terms of rejecting H0 when it is actually true, however, it is ...
Hypothesis Testing, P Values, Confidence Intervals, and Significance
Definition/Introduction. Medical providers often rely on evidence-based medicine to guide decision-making in practice. Often a research hypothesis is tested with results provided, typically with p values, confidence intervals, or both. Additionally, statistical or research significance is estimated or determined by the investigators.
Hypothesis Testing Framework
One is that the significance level is the maximum probability required to reject the null hypothesis; this is based on how the significance level functions within the hypothesis testing framework. The second definition is that this is the probability of rejecting the null hypothesis when the null hypothesis is true; in other words, this is the ...
Understanding P-values
The p value is a number, calculated from a statistical test, that describes how likely you are to have found a particular set of observations if the null hypothesis were true. P values are used in hypothesis testing to help decide whether to reject the null hypothesis. The smaller the p value, the more likely you are to reject the null hypothesis.
S.3.2 Hypothesis Testing (P-Value Approach)
Two-Tailed. In our example concerning the mean grade point average, suppose again that our random sample of n = 15 students majoring in mathematics yields a test statistic t* instead of equaling -2.5.The P-value for conducting the two-tailed test H 0: μ = 3 versus H A: μ ≠ 3 is the probability that we would observe a test statistic less than -2.5 or greater than 2.5 if the population mean ...
Hypothesis to Be Tested: Definition and 4 Steps for Testing with Example
Hypothesis testing is an act in statistics whereby an analyst tests an assumption regarding a population parameter. The methodology employed by the analyst depends on the nature of the data used ...
Test statistics
The test statistic is a number calculated from a statistical test of a hypothesis. It shows how closely your observed data match the distribution expected under the null hypothesis of that statistical test. The test statistic is used to calculate the p value of your results, helping to decide whether to reject your null hypothesis.
Probability, clinical decision making and hypothesis testing
The present paper attempts to put the P value in proper perspective by explaining different types of probabilities, their role in clinical decision making, medical research and hypothesis testing. Keywords: Hypothesis testing, P value, Probability. The clinician who wishes to remain abreast with the results of medical research needs to develop ...
25.1
The power of a hypothesis test is the probability of making the correct decision if the alternative hypothesis is true. That is, the power of a hypothesis test is the probability of rejecting the null hypothesis H 0 when the alternative hypothesis H A is the hypothesis that is true. Let's return to our engineer's problem to see if we can ...
Probability distributions, hypothesis testing, and analysis
Probability and statistics are overlapping but conceptually distinct notions. Probability can be seen as a special aspect of logical reasoning, and it is useful for understanding and interpretation of forensic (and statistical) evidence. Statistics are based on the collection and analysis of empirical data. Probabilistic reasoning is deductive ...
What is Hypothesis Testing in Statistics? Types and Examples
Hypothesis testing is a statistical method used to determine if there is enough evidence in a sample data to draw conclusions about a population. It involves formulating two competing hypotheses, the null hypothesis (H0) and the alternative hypothesis (Ha), and then collecting data to assess the evidence.

9.1: Introduction to Hypothesis Testing

Basic Theory

\(P\)-value

Analogy with Justice Systems

Tests of an Unknown Parameter

Equivalence Between Hypothesis Test and Confidence Sets

Pivot Variables and Test Statistics

Introduction to Hypothesis Testing

The Two Types of Statistical Hypotheses

Hypothesis Tests

The Two Types of Decision Errors

One-Tailed and Two-Tailed Tests

Types of Hypothesis Tests

Published by Zach

Unit 12: Significance tests (hypothesis testing)

Error probabilities and power

Tests about a population proportion

Tests about a population mean

More significance testing videos

Hypothesis Testing

What is Hypothesis Testing in Statistics?

Hypothesis Testing Definition

Null Hypothesis

Alternative Hypothesis

Hypothesis Testing P Value

Hypothesis Testing Critical region

Hypothesis Testing Formula

Types of Hypothesis Testing

Hypothesis Testing Z Test

Hypothesis Testing t Test

Hypothesis Testing Chi Square

One Tailed Hypothesis Testing

Two Tailed Hypothesis Testing

Hypothesis Testing Steps

Hypothesis Testing Example

Hypothesis Testing and Confidence Intervals

Examples on Hypothesis Testing

FAQs on Hypothesis Testing

What is the z Test in Hypothesis Testing?

What is the t Test in Hypothesis Testing?

What is the formula for z test in Hypothesis Testing?

What is the p Value in Hypothesis Testing?

What is One Tail Hypothesis Testing?

What is the Alpha Level in Two Tail Hypothesis Testing?

Hypothesis Testing Framework

Hypothesis Testing Steps

Identify Parameter of Interest

Define the Hypotheses

Set Threshold (Significance Level)

Determine the Sampling Distribution of the Sample Statistic

Required Condition

Using Simulations and Resampling

Using Theory

Assumptions not met

Calculate the evidence with statistics and p-values

Test statistic

P-value Calculation Options

Evaluate your results and write conclusion in context of problem

Significance Level Interpretation

p-value Limitations and Misconceptions

Factors that affect a p-value

Misconceptions

Using the p-value to make a decision

Searching for p-values

Best Practices

Hypothesis to Be Tested: Definition and 4 Steps for Testing with Example

What Is Hypothesis Testing?

Key Takeaways

How Hypothesis Testing Works

4 Steps of Hypothesis Testing

Real-World Example of Hypothesis Testing

What is Hypothesis Testing?

What are the Four Key Steps Involved in Hypothesis Testing?

What are the Benefits of Hypothesis Testing?

What are the Limitations of Hypothesis Testing?

The Bottom Line

Have a language expert improve your writing

Test statistics | Definition, Interpretation, and Examples

Table of contents

Receive feedback on language, structure, and formatting