composite hypothesis statistics

Simple Hypothesis and Composite Hypothesis

A simple hypothesis is one in which all parameters of the distribution are specified. For example, the heights of college students are normally distributed with $${\sigma ^2} = 4$$, and the hypothesis that its mean $$\mu $$ is, say, $$62”$$; that is, $${H_o}:\mu = 62$$. So we have stated a simple hypothesis, as the mean and variance together specify a normal distribution completely. A simple hypothesis, in general, states that $$\theta = {\theta _o}$$ where $${\theta _o}$$ is the specified value of a parameter $$\theta $$, ($$\theta $$ may represent $$\mu ,p,{\mu _1} – {\mu _2}$$ etc).

A hypothesis which is not simple (i.e. in which not all of the parameters are specified) is called a composite hypothesis. For instance, if we hypothesize that $${H_o}:\mu > 62$$ (and $${\sigma ^2} = 4$$) or$${H_o}:\mu = 62$$ and $${\sigma ^2} < 4$$, the hypothesis becomes a composite hypothesis because we cannot know the exact distribution of the population in either case. Obviously, the parameters $$\mu > 62”$$ and$${\sigma ^2} < 4$$ have more than one value and no specified values are being assigned. The general form of a composite hypothesis is $$\theta \leqslant {\theta _o}$$ or $$\theta \geqslant {\theta _o}$$; that is, the parameter $$\theta $$ does not exceed or does not fall short of a specified value $${\theta _o}$$. The concept of simple and composite hypotheses applies to both the null hypothesis and alternative hypothesis.

Hypotheses may also be classified as exact and inexact. A hypothesis is said to be an exact hypothesis if it selects a unique value for the parameter, such as $${H_o}:\mu = 62$$ or $$p > 0.5$$. A hypothesis is called an inexact hypothesis when it indicates more than one possible value for the parameter, such as $${H_o}:\mu \ne 62$$ or $${H_o}:p = 62$$. A simple hypothesis must be exact while an exact hypothesis is not necessarily a simple hypothesis. An inexact hypothesis is a composite hypothesis.

One Comment

Etini August 5 @ 7:28 pm

How can i design a sequential test for the shape parameters of the beta distribution

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Definition: Simple and composite hypothesis

Definition: Let $H$ be a statistical hypothesis . Then,

$H$ is called a simple hypothesis, if it completely specifies the population distribution; in this case, the sampling distribution of the test statistic is a function of sample size alone.

$H$ is called a composite hypothesis, if it does not completely specify the population distribution; for example, the hypothesis may only specify one parameter of the distribution and leave others unspecified.

• Wikipedia (2021): "Exclusion of the null hypothesis" ; in: Wikipedia, the free encyclopedia , retrieved on 2021-03-19 ; URL: https://en.wikipedia.org/wiki/Exclusion_of_the_null_hypothesis#Terminology .

What does "Composite Hypothesis" mean?

Definition of Composite Hypothesis in the context of A/B testing (online controlled experiments).

What is a Composite Hypothesis?

In hypothesis testing a composite hypothesis is a hypothesis which covers a set of values from the parameter space. For example, if the entire parameter space covers -∞ to +∞ a composite hypothesis could be μ ≤ 0. It could be any other number as well, such 1, 2 or 3,1245. The alternative hypothesis is always a composite hypothesis : either one-sided hypothesis if the null is composite or a two-sided one if the null is a point null. The "composite" part means that such a hypothesis is the union of many simple point hypotheses.

In a Null Hypothesis Statistical Test only the null hypothesis can be a point hypothesis. Also, a composite hypothesis usually spans from -∞ to zero or some value of practical significance or from such a value to +∞.

Testing a composite null is what is most often of interest in an A/B testing scenario as we are usually interested in detecting and estimating effects in only one direction: either an increase in conversion rate or average revenue per user, or a decrease in unsubscribe events would be of interest and not its opposite. In fact, running a test so long as to detect a statistically significant negative outcome can result in significant business harm.

Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.

Articles on Composite Hypothesis

One-tailed vs Two-tailed Tests of Significance in A/B Testing blog.analytics-toolkit.com

Related A/B Testing terms

Statistical Methods in Online A/B Testing

Take your A/B testing program to the next level with the most comprehensive book on user testing statistics in e-commerce.

Glossary index by letter

Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.

Biostatistics

• Data Science
• Programming
• Social Science
• Certificates
• Undergraduate
• For Businesses
• FAQs and Knowledge Base

Test Yourself

• Instructors

Composite Hypothesis

Composite Hypothesis:

A statistical hypothesis which does not completely specify the distribution of a random variable is referred to as a composite hypothesis.

Browse Other Glossary Entries

Planning on taking an introductory statistics course, but not sure if you need to start at the beginning? Review the course description for each of our introductory statistics courses and estimate which best matches your level, then take the self test for that course. If you get all or almost all the questions correct, move on and take the next test.

Data Analytics

Considering becoming adata scientist, customer analyst or our data science certificate program?

Advanced Statistics Quiz

Looking at statistics for graduate programs or to enhance your foundational knowledge?

Regression Quiz

Entering the biostatistics field? Test your skill here.

Stay Informed

Read up on our latest blogs

Learn about our certificate programs

Find the right course for you

We'd love to answer your questions

Our mentors and academic advisors are standing by to help guide you towards the courses or program that makes the most sense for you and your goals.

300 W Main St STE 301, Charlottesville, VA 22903

(434) 973-7673

[email protected]

By submitting your information, you agree to receive email communications from Statistics.com. All information submitted is subject to our privacy policy . You may opt out of receiving communications at any time.

Tutorial Playlist

Statistics tutorial, everything you need to know about the probability density function in statistics, the best guide to understand central limit theorem, an in-depth guide to measures of central tendency : mean, median and mode, the ultimate guide to understand conditional probability.

A Comprehensive Look at Percentile in Statistics

The Best Guide to Understand Bayes Theorem

Everything you need to know about the normal distribution, an in-depth explanation of cumulative distribution function, a complete guide to chi-square test, a complete guide on hypothesis testing in statistics, understanding the fundamentals of arithmetic and geometric progression, the definitive guide to understand spearman’s rank correlation, a comprehensive guide to understand mean squared error, all you need to know about the empirical rule in statistics, the complete guide to skewness and kurtosis, a holistic look at bernoulli distribution.

All You Need to Know About Bias in Statistics

A Complete Guide to Get a Grasp of Time Series Analysis

The Key Differences Between Z-Test Vs. T-Test

The Complete Guide to Understand Pearson's Correlation

A complete guide on the types of statistical studies, everything you need to know about poisson distribution, your best guide to understand correlation vs. regression, the most comprehensive guide for beginners on what is correlation, what is hypothesis testing in statistics types and examples.

Lesson 10 of 24 By Avijeet Biswal

Table of Contents

In today’s data-driven world , decisions are based on data all the time. Hypothesis plays a crucial role in that process, whether it may be making business decisions, in the health sector, academia, or in quality improvement. Without hypothesis & hypothesis tests, you risk drawing the wrong conclusions and making bad decisions. In this tutorial, you will look at Hypothesis Testing in Statistics.

What Is Hypothesis Testing in Statistics?

Hypothesis Testing is a type of statistical analysis in which you put your assumptions about a population parameter to the test. It is used to estimate the relationship between 2 statistical variables.

Let's discuss few examples of statistical hypothesis from real-life - 

• A teacher assumes that 60% of his college's students come from lower-middle-class families.
• A doctor believes that 3D (Diet, Dose, and Discipline) is 90% effective for diabetic patients.

Now that you know about hypothesis testing, look at the two types of hypothesis testing in statistics.

Hypothesis Testing Formula

Z = ( x̅ – μ0 ) / (σ /√n)

• Here, x̅ is the sample mean,
• μ0 is the population mean,
• σ is the standard deviation,
• n is the sample size.

How Hypothesis Testing Works?

An analyst performs hypothesis testing on a statistical sample to present evidence of the plausibility of the null hypothesis. Measurements and analyses are conducted on a random sample of the population to test a theory. Analysts use a random population sample to test two hypotheses: the null and alternative hypotheses.

The null hypothesis is typically an equality hypothesis between population parameters; for example, a null hypothesis may claim that the population means return equals zero. The alternate hypothesis is essentially the inverse of the null hypothesis (e.g., the population means the return is not equal to zero). As a result, they are mutually exclusive, and only one can be correct. One of the two possibilities, however, will always be correct.

Your Dream Career is Just Around The Corner!

Null Hypothesis and Alternate Hypothesis

The Null Hypothesis is the assumption that the event will not occur. A null hypothesis has no bearing on the study's outcome unless it is rejected.

H0 is the symbol for it, and it is pronounced H-naught.

The Alternate Hypothesis is the logical opposite of the null hypothesis. The acceptance of the alternative hypothesis follows the rejection of the null hypothesis. H1 is the symbol for it.

Let's understand this with an example.

A sanitizer manufacturer claims that its product kills 95 percent of germs on average. 

To put this company's claim to the test, create a null and alternate hypothesis.

H0 (Null Hypothesis): Average = 95%.

Alternative Hypothesis (H1): The average is less than 95%.

Another straightforward example to understand this concept is determining whether or not a coin is fair and balanced. The null hypothesis states that the probability of a show of heads is equal to the likelihood of a show of tails. In contrast, the alternate theory states that the probability of a show of heads and tails would be very different.

Become a Data Scientist with Hands-on Training!

Hypothesis Testing Calculation With Examples

Let's consider a hypothesis test for the average height of women in the United States. Suppose our null hypothesis is that the average height is 5'4". We gather a sample of 100 women and determine that their average height is 5'5". The standard deviation of population is 2.

To calculate the z-score, we would use the following formula:

z = ( x̅ – μ0 ) / (σ /√n)

z = (5'5" - 5'4") / (2" / √100)

z = 0.5 / (0.045)

 We will reject the null hypothesis as the z-score of 11.11 is very large and conclude that there is evidence to suggest that the average height of women in the US is greater than 5'4".

Steps of Hypothesis Testing

Step 1: specify your null and alternate hypotheses.

It is critical to rephrase your original research hypothesis (the prediction that you wish to study) as a null (Ho) and alternative (Ha) hypothesis so that you can test it quantitatively. Your first hypothesis, which predicts a link between variables, is generally your alternate hypothesis. The null hypothesis predicts no link between the variables of interest.

Step 2: Gather Data

For a statistical test to be legitimate, sampling and data collection must be done in a way that is meant to test your hypothesis. You cannot draw statistical conclusions about the population you are interested in if your data is not representative.

Step 3: Conduct a Statistical Test

Other statistical tests are available, but they all compare within-group variance (how to spread out the data inside a category) against between-group variance (how different the categories are from one another). If the between-group variation is big enough that there is little or no overlap between groups, your statistical test will display a low p-value to represent this. This suggests that the disparities between these groups are unlikely to have occurred by accident. Alternatively, if there is a large within-group variance and a low between-group variance, your statistical test will show a high p-value. Any difference you find across groups is most likely attributable to chance. The variety of variables and the level of measurement of your obtained data will influence your statistical test selection.

Step 4: Determine Rejection Of Your Null Hypothesis

Your statistical test results must determine whether your null hypothesis should be rejected or not. In most circumstances, you will base your judgment on the p-value provided by the statistical test. In most circumstances, your preset level of significance for rejecting the null hypothesis will be 0.05 - that is, when there is less than a 5% likelihood that these data would be seen if the null hypothesis were true. In other circumstances, researchers use a lower level of significance, such as 0.01 (1%). This reduces the possibility of wrongly rejecting the null hypothesis.

Step 5: Present Your Results 

The findings of hypothesis testing will be discussed in the results and discussion portions of your research paper, dissertation, or thesis. You should include a concise overview of the data and a summary of the findings of your statistical test in the results section. You can talk about whether your results confirmed your initial hypothesis or not in the conversation. Rejecting or failing to reject the null hypothesis is a formal term used in hypothesis testing. This is likely a must for your statistics assignments.

Types of Hypothesis Testing

To determine whether a discovery or relationship is statistically significant, hypothesis testing uses a z-test. It usually checks to see if two means are the same (the null hypothesis). Only when the population standard deviation is known and the sample size is 30 data points or more, can a z-test be applied.

A statistical test called a t-test is employed to compare the means of two groups. To determine whether two groups differ or if a procedure or treatment affects the population of interest, it is frequently used in hypothesis testing.

Chi-Square 

You utilize a Chi-square test for hypothesis testing concerning whether your data is as predicted. To determine if the expected and observed results are well-fitted, the Chi-square test analyzes the differences between categorical variables from a random sample. The test's fundamental premise is that the observed values in your data should be compared to the predicted values that would be present if the null hypothesis were true.

Hypothesis Testing and Confidence Intervals

Both confidence intervals and hypothesis tests are inferential techniques that depend on approximating the sample distribution. Data from a sample is used to estimate a population parameter using confidence intervals. Data from a sample is used in hypothesis testing to examine a given hypothesis. We must have a postulated parameter to conduct hypothesis testing.

Bootstrap distributions and randomization distributions are created using comparable simulation techniques. The observed sample statistic is the focal point of a bootstrap distribution, whereas the null hypothesis value is the focal point of a randomization distribution.

A variety of feasible population parameter estimates are included in confidence ranges. In this lesson, we created just two-tailed confidence intervals. There is a direct connection between these two-tail confidence intervals and these two-tail hypothesis tests. The results of a two-tailed hypothesis test and two-tailed confidence intervals typically provide the same results. In other words, a hypothesis test at the 0.05 level will virtually always fail to reject the null hypothesis if the 95% confidence interval contains the predicted value. A hypothesis test at the 0.05 level will nearly certainly reject the null hypothesis if the 95% confidence interval does not include the hypothesized parameter.

Simple and Composite Hypothesis Testing

Depending on the population distribution, you can classify the statistical hypothesis into two types.

Simple Hypothesis: A simple hypothesis specifies an exact value for the parameter.

Composite Hypothesis: A composite hypothesis specifies a range of values.

A company is claiming that their average sales for this quarter are 1000 units. This is an example of a simple hypothesis.

Suppose the company claims that the sales are in the range of 900 to 1000 units. Then this is a case of a composite hypothesis.

One-Tailed and Two-Tailed Hypothesis Testing

The One-Tailed test, also called a directional test, considers a critical region of data that would result in the null hypothesis being rejected if the test sample falls into it, inevitably meaning the acceptance of the alternate hypothesis.

In a one-tailed test, the critical distribution area is one-sided, meaning the test sample is either greater or lesser than a specific value.

In two tails, the test sample is checked to be greater or less than a range of values in a Two-Tailed test, implying that the critical distribution area is two-sided.

If the sample falls within this range, the alternate hypothesis will be accepted, and the null hypothesis will be rejected.

Become a Data Scientist With Real-World Experience

Right Tailed Hypothesis Testing

If the larger than (>) sign appears in your hypothesis statement, you are using a right-tailed test, also known as an upper test. Or, to put it another way, the disparity is to the right. For instance, you can contrast the battery life before and after a change in production. Your hypothesis statements can be the following if you want to know if the battery life is longer than the original (let's say 90 hours):

• The null hypothesis is (H0 <= 90) or less change.
• A possibility is that battery life has risen (H1) > 90.

The crucial point in this situation is that the alternate hypothesis (H1), not the null hypothesis, decides whether you get a right-tailed test.

Left Tailed Hypothesis Testing

Alternative hypotheses that assert the true value of a parameter is lower than the null hypothesis are tested with a left-tailed test; they are indicated by the asterisk "<".

Suppose H0: mean = 50 and H1: mean not equal to 50

According to the H1, the mean can be greater than or less than 50. This is an example of a Two-tailed test.

In a similar manner, if H0: mean >=50, then H1: mean <50

Here the mean is less than 50. It is called a One-tailed test.

Type 1 and Type 2 Error

A hypothesis test can result in two types of errors.

Type 1 Error: A Type-I error occurs when sample results reject the null hypothesis despite being true.

Type 2 Error: A Type-II error occurs when the null hypothesis is not rejected when it is false, unlike a Type-I error.

Suppose a teacher evaluates the examination paper to decide whether a student passes or fails.

H0: Student has passed

H1: Student has failed

Type I error will be the teacher failing the student [rejects H0] although the student scored the passing marks [H0 was true]. 

Type II error will be the case where the teacher passes the student [do not reject H0] although the student did not score the passing marks [H1 is true].

Level of Significance

The alpha value is a criterion for determining whether a test statistic is statistically significant. In a statistical test, Alpha represents an acceptable probability of a Type I error. Because alpha is a probability, it can be anywhere between 0 and 1. In practice, the most commonly used alpha values are 0.01, 0.05, and 0.1, which represent a 1%, 5%, and 10% chance of a Type I error, respectively (i.e. rejecting the null hypothesis when it is in fact correct).

Future-Proof Your AI/ML Career: Top Dos and Don'ts

A p-value is a metric that expresses the likelihood that an observed difference could have occurred by chance. As the p-value decreases the statistical significance of the observed difference increases. If the p-value is too low, you reject the null hypothesis.

Here you have taken an example in which you are trying to test whether the new advertising campaign has increased the product's sales. The p-value is the likelihood that the null hypothesis, which states that there is no change in the sales due to the new advertising campaign, is true. If the p-value is .30, then there is a 30% chance that there is no increase or decrease in the product's sales.  If the p-value is 0.03, then there is a 3% probability that there is no increase or decrease in the sales value due to the new advertising campaign. As you can see, the lower the p-value, the chances of the alternate hypothesis being true increases, which means that the new advertising campaign causes an increase or decrease in sales.

Why is Hypothesis Testing Important in Research Methodology?

Hypothesis testing is crucial in research methodology for several reasons:

• Provides evidence-based conclusions: It allows researchers to make objective conclusions based on empirical data, providing evidence to support or refute their research hypotheses.
• Supports decision-making: It helps make informed decisions, such as accepting or rejecting a new treatment, implementing policy changes, or adopting new practices.
• Adds rigor and validity: It adds scientific rigor to research using statistical methods to analyze data, ensuring that conclusions are based on sound statistical evidence.
• Contributes to the advancement of knowledge: By testing hypotheses, researchers contribute to the growth of knowledge in their respective fields by confirming existing theories or discovering new patterns and relationships.

Limitations of Hypothesis Testing

Hypothesis testing has some limitations that researchers should be aware of:

• It cannot prove or establish the truth: Hypothesis testing provides evidence to support or reject a hypothesis, but it cannot confirm the absolute truth of the research question.
• Results are sample-specific: Hypothesis testing is based on analyzing a sample from a population, and the conclusions drawn are specific to that particular sample.
• Possible errors: During hypothesis testing, there is a chance of committing type I error (rejecting a true null hypothesis) or type II error (failing to reject a false null hypothesis).
• Assumptions and requirements: Different tests have specific assumptions and requirements that must be met to accurately interpret results.

After reading this tutorial, you would have a much better understanding of hypothesis testing, one of the most important concepts in the field of Data Science . The majority of hypotheses are based on speculation about observed behavior, natural phenomena, or established theories.

If you are interested in statistics of data science and skills needed for such a career, you ought to explore Simplilearn’s Post Graduate Program in Data Science.

If you have any questions regarding this ‘Hypothesis Testing In Statistics’ tutorial, do share them in the comment section. Our subject matter expert will respond to your queries. Happy learning!

1. What is hypothesis testing in statistics with example?

Hypothesis testing is a statistical method used to determine if there is enough evidence in a sample data to draw conclusions about a population. It involves formulating two competing hypotheses, the null hypothesis (H0) and the alternative hypothesis (Ha), and then collecting data to assess the evidence. An example: testing if a new drug improves patient recovery (Ha) compared to the standard treatment (H0) based on collected patient data.

2. What is hypothesis testing and its types?

Hypothesis testing is a statistical method used to make inferences about a population based on sample data. It involves formulating two hypotheses: the null hypothesis (H0), which represents the default assumption, and the alternative hypothesis (Ha), which contradicts H0. The goal is to assess the evidence and determine whether there is enough statistical significance to reject the null hypothesis in favor of the alternative hypothesis.

Types of hypothesis testing:

• One-sample test: Used to compare a sample to a known value or a hypothesized value.
• Two-sample test: Compares two independent samples to assess if there is a significant difference between their means or distributions.
• Paired-sample test: Compares two related samples, such as pre-test and post-test data, to evaluate changes within the same subjects over time or under different conditions.
• Chi-square test: Used to analyze categorical data and determine if there is a significant association between variables.
• ANOVA (Analysis of Variance): Compares means across multiple groups to check if there is a significant difference between them.

3. What are the steps of hypothesis testing?

The steps of hypothesis testing are as follows:

• Formulate the hypotheses: State the null hypothesis (H0) and the alternative hypothesis (Ha) based on the research question.
• Set the significance level: Determine the acceptable level of error (alpha) for making a decision.
• Collect and analyze data: Gather and process the sample data.
• Compute test statistic: Calculate the appropriate statistical test to assess the evidence.
• Make a decision: Compare the test statistic with critical values or p-values and determine whether to reject H0 in favor of Ha or not.
• Draw conclusions: Interpret the results and communicate the findings in the context of the research question.

4. What are the 2 types of hypothesis testing?

• One-tailed (or one-sided) test: Tests for the significance of an effect in only one direction, either positive or negative.
• Two-tailed (or two-sided) test: Tests for the significance of an effect in both directions, allowing for the possibility of a positive or negative effect.

The choice between one-tailed and two-tailed tests depends on the specific research question and the directionality of the expected effect.

5. What are the 3 major types of hypothesis?

The three major types of hypotheses are:

• Null Hypothesis (H0): Represents the default assumption, stating that there is no significant effect or relationship in the data.
• Alternative Hypothesis (Ha): Contradicts the null hypothesis and proposes a specific effect or relationship that researchers want to investigate.
• Nondirectional Hypothesis: An alternative hypothesis that doesn't specify the direction of the effect, leaving it open for both positive and negative possibilities.

Find our Data Analyst Online Bootcamp in top cities:

About the author.

Avijeet is a Senior Research Analyst at Simplilearn. Passionate about Data Analytics, Machine Learning, and Deep Learning, Avijeet is also interested in politics, cricket, and football.

Recommended Resources

Free eBook: Top Programming Languages For A Data Scientist

Normality Test in Minitab: Minitab with Statistics

Machine Learning Career Guide: A Playbook to Becoming a Machine Learning Engineer

• PMP, PMI, PMBOK, CAPM, PgMP, PfMP, ACP, PBA, RMP, SP, and OPM3 are registered marks of the Project Management Institute, Inc.

Selected Works of E. L. Lehmann pp 117–138 Cite as

Most Powerful Tests of Composite Hypotheses. I. Normal Distributions

• E. L. Lehmann 2 &
• C. Stein 2  
• Open Access
• First Online: 19 November 2011

7534 Accesses

Part of the book series: Selected Works in Probability and Statistics ((SWPS))

For testing a composite hypothesis, critical regions are determined which are most powerful against a particular alternative at a given level of significance. Here a region is said to have level of significance e if the probability of the region under the hypothesis tested is bounded above by e. These problems have been considered by Neyman, Pearson and others, subject to the condition that the critical region be similar. In testing the hypothesis specify-ing the value of the variance of a normal distribution with unknown mean against an alternative with larger variance, and in some other problems, the best similar region is also most powerful in the sense of this paper. However, in the analo-gous problem when the variance under the alternative hypothesis is less than that under the hypothesis tested, in the case of Student’s hypothesis when the level of significance is less than and in some other cases, the best similar region is not most powerful in the sense of this paper. There exist most powerful tests which are quite good against certain alternatives in some cases where no proper similar region exists. These results indicate that in some practical cases the standard test is not best if the class of alternatives is sufficiently restricted.

Download to read the full chapter text

Chapter PDF

J. Neyman and E. S. Pearson, “On the problem of the most efficient tests of statistical hypotheses,” Roy. Soc. Phil. Trans ., Ser. A , Vol. 231 (1933), p. 289.

Article   Google Scholar  

A. Wald, “Test of statistical hypotheses concerning several parameters when the number of observations is large,” Am. Math. Soc. Trans Vol. 54 (1943), p. 426.

Article   MathSciNet   MATH   Google Scholar  

A. Wald, “Statistical decision functions which minimize the maximum risk,” Annals of Math., Vol. 46 (1945), p. 265.

A. Wald, “An essentially complete class of admissible decision-functions,” Annals of Math. Stat., Vol. 18 (1947), p. 549.

J. Neyman, “On a statistical problem arising in routine analysis and in sampling in- spection of mass production,” Annals of Math. Stat., Vol. 12 (1941), p. 46.

H. SCHEFFÉ, “On the theory of testing composite hypotheses with one constraint,” Annals of Math. Stat., Vol. 13 (1942), p. 280.

Article   MATH   Google Scholar  

E. Lehmann, “On optimum tests of composite hypotheses with one constraint,” Annals of Math. Stat., Vol. 18 (1947), p. 473.

E. Lehmann and H. ScheffÉ, “On the problem of similar regions,” Proc. Nat. Acad. Sci Vol. 33 (1947), p. 382.

A. Wald, “On the power function of the analysis of variance test,” Annals of Math. Slat., Vol. 13 (1942), p. 434.

P. L. Hsu, “On the power function of the E 2 -test and the T 2 -test,” Annals of Math. Stat Vol. 16 (1945), p. 278.

H. K. Nandi, “On the average power of test criteria,” Sankhyā Vol. 8 (1946), p. 67.

MathSciNet   MATH   Google Scholar  

W. Feller, “Note on regions similar to the sample sapce,” Stat. Res. Memoirs, Vol. 2 (1938), p. 117.

Google Scholar  

P. L. Hsu, “Analysis of variance from the power function standpoint,” Biometrika, Vol. 32 (1941), p. 62.

S. Kolodzieczyk, “On an important class of statistical hypotheses,” Biometrika, Vol. 27 (1935), p. 161.

P. C. Tang, “The power function of the analysis of variance tests with tables and illustrations of their use,” Stat. Res. Memoirs, Vol. 2 (1938), p. 126.

A. Wald, “Foundations of a general theory of sequential decision functions,” Eco- nometrica, Vol. 15 (1947) p. 279.

MathSciNet   Google Scholar  

Download references

Author information

Authors and affiliations.

University of California, Berkeley, USA

E. L. Lehmann & C. Stein

You can also search for this author in PubMed   Google Scholar

Editor information

Editors and affiliations.

, Department of Statistics, Rice University, 6100 Main Street, Houston, 77005, Texas, USA

Javier Rojo

Rights and permissions

This chapter is published under an open access license. Please check the 'Copyright Information' section either on this page or in the PDF for details of this license and what re-use is permitted. If your intended use exceeds what is permitted by the license or if you are unable to locate the licence and re-use information, please contact the Rights and Permissions team .

Copyright information

© 2012 Springer Science+Business Media, LLC

About this chapter

Cite this chapter.

Lehmann, E.L., Stein, C. (2012). Most Powerful Tests of Composite Hypotheses. I. Normal Distributions. In: Rojo, J. (eds) Selected Works of E. L. Lehmann. Selected Works in Probability and Statistics. Springer, Boston, MA. https://doi.org/10.1007/978-1-4614-1412-4_13

Download citation

DOI : https://doi.org/10.1007/978-1-4614-1412-4_13

Published : 19 November 2011

Publisher Name : Springer, Boston, MA

Print ISBN : 978-1-4614-1411-7

Online ISBN : 978-1-4614-1412-4

eBook Packages : Mathematics and Statistics Mathematics and Statistics (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Entropy (Basel)

The Geometry of Generalized Likelihood Ratio Test

Associated data.

Not applicable.

The generalized likelihood ratio test (GLRT) for composite hypothesis testing problems is studied from a geometric perspective. An information-geometrical interpretation of the GLRT is proposed based on the geometry of curved exponential families. Two geometric pictures of the GLRT are presented for the cases where unknown parameters are and are not the same under the null and alternative hypotheses, respectively. A demonstration of one-dimensional curved Gaussian distribution is introduced to elucidate the geometric realization of the GLRT. The asymptotic performance of the GLRT is discussed based on the proposed geometric representation of the GLRT. The study provides an alternative perspective for understanding the problems of statistical inference in the theoretical sense.

1. Introduction

The problem of hypothesis testing under statistical uncertainty arises naturally in many practical contexts. In these cases, the probability density functions (PDFs) under either or both hypotheses need not be completely specified, resulting in the inclusion of unknown parameters in the PDFs to express the statistical uncertainty in the model. The class of hypothesis testing problems with unknown parameters in the PDFs is commonly referred to as composite hypothesis testing [ 1 ]. The generalized likelihood ratio test (GLRT) is one of the most widely used approaches in composite hypothesis testing [ 2 ]. It involves estimating the unknown parameters via the maximum likelihood estimation (MLE) to implement a likelihood ratio test. In practice, the GLRT appears to be asymptotically optimal in the sense of the Neyman–Pearson criterion and usually gives satisfactory results [ 3 ]. As the GLRT combines both estimation and detection to deal with the composite hypothesis testing problem, its performance, in general, will depend on the statistical inference performance of these two aspects. However, in the literature, there is no general analytical result associated with the performance of the GLRT [ 1 ].

In recent years, the development of new theories in statistical inference has been characterized by the emerging trend of geometric approaches and their powerful capabilities, which allows one to analyze statistical problems in a unified perspective. It is important to link the GLRT to the geometrical nature of estimation and detection, which provides a new viewpoint on the GLRT. The general problem of composite hypothesis testing involves a decision between two hypotheses where the PDFs are themselves functions of unknown parameters. One approach to the understanding of performance limitations of statistical inference is via the theory of information geometry. In this context, the family of probability distributions with a natural geometrical structure is defined as a statistical manifold [ 4 ]. Information geometry studies the intrinsic properties of statistical manifolds which are endowed with a Riemannian metric and a family of affine connections derived from the log-likelihood functions of probability distributions [ 5 ]. It provides a way of analyzing the geometrical properties of statistical models by regarding them as geometric objects.

The geometric theory of statistics was firstly introduced in the 1940s by Rao [ 6 ], where the Fisher information matrix was regarded as a Riemannian metric on the manifold of probability distributions. Then, in 1972, a one-parameter family of affine connections was introduced by Chentsov in [ 7 ]. Meanwhile, Efron [ 8 ] defined the concept of statistical curvature and discussed its basic role in the high-order asymptotic theory of statistical inference. In 1982, Amari [ 5 , 9 ] developed a duality structure theory that unified all of these theories in a differential-geometric framework, leading to a large number of applications.

In the area of hypothesis testing, the geometric perspectives have acquired relevance in the analysis and development of new approaches to various testing and detection contexts. For example, Kass and Vos [ 10 ] provided a detailed introduction to the geometrical foundations of asymptotic inference of curved exponential families. Garderen [ 11 ] presented a global analysis of the effects of curvature on hypothesis testing. Dabak [ 12 ] induced a geometric structure on the manifold of probability distributions and enforced a detection theoretic specific geometry on it, while Westover [ 13 ] discussed the asymptotic limit in the problems of multiple hypothesis testing from the geometrical perspective. For the development of new approaches to hypothesis testing, Hoeffding [ 14 ] proposed an asymptotically optimal test for multinomial distributions in which the testing can be denoted in terms of the Kullback–Leibler divergence (KLD) between the empirical distribution of the measurements and the null hypothesis, where the alternate distribution is unrestricted. In the aspect of signal detection, Barbaresco et al. [ 15 , 16 , 17 ] studied the geometry of Bruhat–Tits complete metric space and upper-half Siegel space and introduced a matrix constant false alarm rate (CFAR) detector which improves the detection performance of the classical CFAR detection.

As more and more new analyses and new approaches have benefited from the geometric and information-theoretic perspectives of statistics, it appears to be important to clarify the geometry of existing problems that is promising to gain new ways to deal with the statistical problems. In this paper, a geometric interpretation of the GLRT is sought from the perspective of information geometry. Two pictures of the GLRT are presented for the cases where unknown parameters are and are not the same under each hypothesis, respectively. Under such an interpretation, both detection and estimation associated with the GLRT are regarded as geometric operations on the statistical manifold. As a general consideration, curved exponential families [ 9 ], which include a large number of the most common used distributions, are taken into account as the statistical model of hypothesis testing problems. A demonstration of one-dimensional curved Gaussian distribution is introduced to elucidate the geometric realization of the GLRT. The geometric structure of the curved exponential families developed by Efron [ 8 ] in 1975 and Amari [ 9 ] in 1982 provides a theoretical foundation for the analysis. The geometric formulation of the GLRT presented in this paper makes it possible for several advanced notions and conclusions in the information geometry theory to be transferred and applied to the performance analysis of the GLRT.

The main contributions of this paper are summarized as follows:

• A geometric interpretation of the GLRT is proposed based on the differential geometry of curved exponential families and duality structure theory developed by Amari [ 9 ]. Two geometric pictures of the GLRT are presented in the theoretical sense, which provides an alternative perspective for understanding the problems of statistical inference.
• The asymptotic performance of the GLRT is discussed based on the proposed geometric representation of the GLRT. The information loss when performingthe MLE using a finite number of samples is related to the flatness of the submanifolds determined by the GLRT model.

In the next section, alternative viewpoints on the likelihood ratio test and the maximum likelihood estimation are introduced from the perspective of information theory. The equivalences between the Kullback–Leibler divergence, likelihood ratio test, and the MLE are highlighted. The principles of information geometry are briefly introduced in Section 3 . In Section 4 , the geometric interpretation of the GLRT is presented in consideration of the geometry of curved exponential families. We present an example of the GLRT where a curved Gaussian distribution with one unknown parameter is involved, and a further discussion on the geometry of the GLRT. Finally, conclusions are obtained in Section 5 .

2. Information-Theoretic Viewpoints on Likelihood Ratio Test and Maximum Likelihood Estimation

In statistics, the likelihood ratio test and maximum likelihood estimation are two fundamental concepts related to the GLRT. The likelihood ratio test is a very general form of testing model assumptions, while the maximum likelihood estimation is one of the most common approaches to parameter estimation. Both of them are associated with the Kullback–Leibler divergence [ 18 ], which is equivalent to the relative entropy [ 19 ] in information theory.

For a sequence of observations x = ( x 1 , x 2 , … , x N ) T ∈ R N which is independently and identically distributed (i.i.d.), the binary hypothesis testing problem is used to decide whether this sequence x originates from the null hypothesis H 0 or the alternative hypothesis H 1 with probability distributions p 0 ( x ) and p 1 ( x ) , respectively. The likelihood ratio is given by

Assume q ( x ) is the empirical distribution (frequency histogram acquired via Monte Carlo tests) of observed data. For large N , in accordance with the strong law of large numbers [ 20 ], the log likelihood ratio test in the Neyman–Pearson formulation ln L ≷ H 1 H 0 γ is equivalent to

where ≷ H 1 H 0 denotes that the test is to decide H 1 if “>” is satisfied, or to decide H 0 , and vice versa. The quantity

is the KLD from q ( x ) to p ( x ) . Note that x is dropped from the notion D for simplifying the KLD expression without confusion.

Equation ( 2 ) indicates that the likelihood ratio test is equivalent to choosing the hypothesis that is “closer” to the empirical distribution in the sense of the KLD. The test can be referred to as a generalized minimum dissimilarity detector in a geometric viewpoint.

Now, consider another, slightly different problem where the observations x are from a statistical model represented by p ( x | θ ) with unknown parameters θ . The problem is to estimate the unknown parameters θ based on observations x . The likelihood function for the underlying estimation problem is

In a similar way, for large N , maximizing the likelihood ( 4 ) to find the maximum likelihood estimate of θ is equivalent to finding θ , which minimizes the KLD D ( q ∥ p θ ) , i.e.,

where p θ is used as a surrogate for p ( x | θ ) .

The above results provide an information-theoretic view to the problem of hypothesis testing and maximum likelihood estimation in statistics. From the perspective of information difference, these results have profound geometric meanings and can be geometrically analyzed and viewed in the framework of information geometry theory, from which additional insights into the analysis of these statistical problems, as well as their geometric interpretations, are obtained.

3. Principles of Information Geometry

3.1. statistical manifold.

Information geometry studies the natural geometric structure of the parameterized family of probability distributions S = { p ( x | θ ) } specifying by a parameter vector θ : = [ θ 1 , … , θ n ] , in which x is the samples of a random variable X . When the probability measure on the sample space is continuous and differentiable and the mapping θ ↦ p ( x | θ ) is injective [ 5 ], the family S is considered as a statistical manifold with θ as its coordinate system [ 4 ].

Figure 1 demonstrates the diagram of a statistical manifold. For a given parameter vector θ ∈ Θ ⊂ R n , the measurement x in the sample space X is an instantiation of a probability distribution p ( x | θ ) . Each p ( x | θ ) in the family of distributions is specified by a point s ( θ ) on the manifold S . The n -dimensional statistical manifold is composed of the parameterized family of probability distributions S = { p ( x | θ ) } with θ as a coordinate system of S .

Diagram of a statistical manifold. θ and s ( θ ′ ) denote parameters of the family of distributions from different samples X . The connection on the statistical manifold S represents a geodesic (the shortest line) between points s ( θ ) and s ( θ ′ ) . The length of the geodesic serves as a distance measure between two points on the manifold. The arrow on the geodesic starting from the point s ( θ ) denotes the tangent vector, which gives the direction of the geodesic.

Various families of probability distributions correspond to specific structures of the statistical manifold. Information geometry takes the statistical properties of samples as the geometric structure of a statistical manifold, and utilizes differential geometry methods to measure the variation of information contained in the samples.

3.2. Fisher Information Metric and Affine Connections

The metric and connections associated with a manifold are two important concepts in information geometry. For a statistical manifold consisting of a parameterized family of probability distributions, the Fisher information matrix (FIM) is usually adopted as a Riemannian metric tensor of the manifold [ 6 ], which is defined by the inner product between tangent vectors at a point on the manifold. It is denoted by G ( θ ) = [ g i j ( θ ) ] , where

where { ∂ log ( · ) / ∂ θ i } is considered as a basis for the vector space of random variable X . The tangent space of S at θ , denoted as T θ ( S ) , is identified as the vector space. Based on the above definition, the FIM metric determines how the information distance is measured on the statistical manifold.

When considering the relationships between two tangent spaces T θ and T θ + d θ at two neighboring points θ and θ + d θ ( d is the differential operator), an affine connection is defined by which the two tangent spaces become feasible for comparison. When the connection coefficients are all identically 0, then S is flat manifold that “locally looks like” a Euclidean space with zero curvatures everywhere. The most commonly used connection is called α -connections [ 9 ],

where Γ α j i m denotes the connection coefficients with i , j , m = 1 , … , n , and

In ( 7 ), α = 0 corresponds to the Levi–Civita connection, while α = 1 defines the e -connection and α = − 1 defines the m -connection. Under the e -connection and m -connection, an exponential family with natural parameter θ coordinate and a mixture family with expectation parameter η coordinate are both flat manifolds [ 9 ]. Statistical inference with respect to the exponential family greatly benefits from the geometric properties of the flat manifold. By using the methods of differential geometry, many additional insights into the intrinsic structure of probability distributions can be obtained, which opens a new perspective on the analysis of statistical problems. In the next section, a geometric interpretation of the GLRT and further discussions are sought based on the principles of information geometry.

4. Geometry of the Generalized Likelihood Ratio Test

As a general treatment, the curved exponential families, which encapsulate many important distributions for real-world problems, are considered as the statistical model for the hypothesis testing problems discussed in this paper. In this section, the MLE solution to parameter estimation for curved exponential families is derived. We then present two pictures of the GLRT, which are sketched based on the geometric structure of the curved exponential families developed by Efron [ 8 ] in 1975 and Amari [ 9 ] in 1982, to illustrate the information geometry of the GLRT. An example of the GLRT for a curved Gaussian distribution with a single unknown parameter is given, which is followed by a further discussion on the geometric formulation of the GLRT.

4.1. The MLE Solution to Statistical Estimation for Curved Exponential Families

Exponential families contain lots of the most commonly used distributions, including the normal, exponential, Gamma, Beta, Poisson, Bernoulli, and so on [ 21 ]. The curved exponential families are the distributions whose natural parameters are nonlinear functions of “local” parameters. The canonical form of a curved exponential family [ 9 ] is expressed as

where x ∈ X is a vector of samples, θ : = [ θ 1 , … , θ n ] are the natural parameters, u ∈ R m ( m < n ) are local parameters standing for the parameters of interest to be estimated, which is specified by ( 9 ), while F ( x ) : = [ F 1 ( x ) , ⋯ , F n ( x ) ] T denote sufficient statistics with respect to θ = ( θ 1 , ⋯ , θ n ) , which take values from the sample space X . φ is the potential function of the curved exponential family and it is found from the normalization condition ∫ X p ( x | θ ) d x = 1 , i.e.,

The term “curved” is due to the fact that the curved exponential family in ( 9 ) is a submanifold of the canonical exponential family p ( x | θ ) by the embedding u ⟶ θ ( u ) .

Let l ( θ , x ) = log p ( x | θ ) be the log-likelihood and ∇ u θ be the Jacobian matrix of the natural parameter θ . According to ( 9 ),

where η ( u ) is the expectation of the sufficient statistics F ( x ) , i.e.,

and is called the expectation parameter, which defines a distribution of mixture family [ 4 ]. The natural parameter θ ( u ) and expectation parameter η ( u ) are connected by the Legendre transformation [ 9 ], as

where ϕ ( η ) is defined by

Thus, the maximum likelihood estimator u ^ of the local parameter in ( 9 ) can be obtained by the following likelihood equation:

Equation ( 16 ) indicates that the solution to the MLE can be found by mapping the data F ( x ) onto F B : = { η ( u ) : u ∈ R m } orthogonally to the tangent of F A : = { θ ( u ) : u ∈ R m } . As θ ( u ) and η ( u ) live in two different spaces F A and F B , the inner product between dual spaces is defined as 〈 θ ( u ) , η ( u ) 〉 Γ : = θ ( u ) T · Γ · η ( u ) with a metric Γ . For the flat manifold, the identity matrix serves as the metric Γ . By analogy with the MLE for the universal distribution given by ( 5 ), Hoeffding [ 14 ] presented another interpretation for the MLE of the curved exponential family. In the interpretation, η ( u ^ ) represents a point in F B which is located closest to the data point in the sense of the Kullback–Leibler divergence, i.e.,

where D F ( x ) ∥ η u denotes the Kullback–Leibler divergence from the multivariate joint distributions of F ( x ) to η u .

Based on the above analysis, there are two important spaces related to a curved exponential family. One is called the natural parameter space, denoted by { θ } ⊂ A n , which denotes the enveloping space including all the distributions of exponential families, and the other is called the expectation parameter space, denoted by { η } ⊂ B n , denoting the dual space of A n . The two spaces are “dual” with each other and flat under the e -connection and m -connection, respectively. The curved exponential family ( 9 ) is regarded as submanifolds embedded in the two spaces, and the data can also be immersed in these spaces in the form of sufficient statistics F ( x ) . Consequently, the estimators, such as the MLE given by ( 16 ), associated with the curved exponential families can be geometrically performed in the two spaces.

4.2. Geometric Demonstration of the Generalized Likelihood Ratio Test

As mentioned earlier, the GLRT is one of the most widely used approaches in composite hypothesis testing problems with unknown parameters in the PDFs. The data x have the PDF p ( x | u 0 ; H 0 ) under hypothesis H 0 and p ( x | u 1 ; H 1 ) under hypothesis H 1 , where u 0 and u 1 are unknown parameters under each hypothesis. The GLRT enables a decision by means of replacement of the unknown parameters by their maximum likelihood estimates (MLEs) to implement a likelihood ratio test. The GLRT decides H 1 if

where u ^ i is the MLE of u i (by maximizing p ( x | u i ; H i ) ).

From the perspective of information geometry, the probability distribution p ( x | u i ; H i ) is an element of the parameterized family of PDFs S = { p ( x | u ) , u ∈ Ω } , where Ω ⊂ R m is the parameter set. For the curved exponential family S = { p ( x | u ) } , it can be regarded as a submanifold embedding in the natural parameter space { θ } ⊂ A n , which includes all the distributions of exponential families. The curved exponential family S can be represented by a curve (or surface) { θ = θ ( u ) } embedded in the enveloping space A n by the nonlinear mapping u ⟶ θ ( u ) . The expectation parameter space { η } ⊂ B n of S is a dual flat space to the natural parameter space { θ } , while the “realizations” of sufficient statistics F ( x ) of the distribution p ( x | θ ) can be immersed in this space. Consequently, the MLE is performed in the space B n by mapping the samples F ( x ) onto the submanifold specified by { η = η ( u ) } under the m -projection.

As the parameters u 0 and u 1 , as well as their dimensionalities, may or may not be the same under the null and alternative hypotheses, two pictures of the GLRT are presented for the two cases: one is with the same unknown parameters under each hypotheses and the other is with different parameters or different dimensionalities. The picture for the first case is illustrated in Figure 2 a. In this case, distributions under two hypotheses share the same form and the same unknown parameter u . However, the parameter takes different value sets under different hypotheses. The family of S = { p ( x | u ) } can be smoothly embedded as a surface F B specified by { η ( u ) : u ∈ R m } in the space B n . The hypotheses p ( x | u i ; H i ) with unknown u i define two “uncertainty volumes” Ω 0 and Ω 1 on the submanifold F B . These volumes are the collections of probability distributions specified by the value sets of the unknown parameter u i . The measurements x are immersed in B n in the form of sufficient statistics F ( x ) . Consequently, the MLE can be found by “mapping” the samples F ( x ) onto the uncertainty volumes Ω 0 and Ω 1 on F B . The points p 0 and p 1 in Figure 2 are the corresponding projections, i.e., the MLEs of the unknown parameter under two hypotheses. As indicated in ( 17 ), the MLEs can also be obtained by finding the points on Ω 0 and Ω 1 which are located closest to the data point in the sense of KLD, i.e.,

and the corresponding minimum KLDs can be represented by

respectively.

Geometry of the GLRT. ( a ) The case for the same unknown parameters under two hypotheses. ( b ) The case for different unknown parameters and different dimensionalities under two hypotheses.

It should be emphasized that the above “mapping” is a general concept. When the parameters to be estimated are not restricted by a given “value set”, the MLE is simply obtained by maximizing the likelihood and the projections will fall onto the submanifold F B . However, if the parameters to be estimated are restricted in a given “value set”, the MLE should be operated by maximizing the likelihood with respect to the given parameter space. In the case where the projections fall outside the “uncertainty volumes”, the MLE solutions are given by those points which are closest to the data point described by ( 19 ).

Let R ( η 0 , ρ ) be a divergence sphere centered at η 0 with radius ρ ; that is, the submanifold of the enveloping space B n consisting of points η for which the KLD D ( η 0 ∥ η ) is equal to ρ . Denote this divergence sphere by

Then, the closest point in ( 19 ) may be more easily found via the divergence sphere with center F ( x ) and radius D i tangent to Ω i at p i , as illustrated in Figure 3 . Consequently, according to ( 2 ), the GLRT can be geometrically performed by comparing the difference between the minimum KLDs D 0 and D 1 with a threshold γ ′ , i.e.,

Illustration of the mapping via divergence spheres.

In practice, the Neyman–Pearson criterion is commonly employed to determine the threshold γ ′ in ( 22 ) and the detector is of maximum probability of detection P D under a given probability of false alarm P F . As a commonly used performance index, the missingprobability P M usually decays exponentially as the sample size increases. The rate of exponential decay can be represented by [ 22 ], as

Based on Stein’s lemma, for a constant false-alarm constraint, the best error exponent is related to the Kullback–Leibler divergence D ( p 0 ∥ p 1 ) from p 0 to p 1 [ 23 ], i.e.,

where ≐ denotes the first-order equivalence in the exponent. For example,

In the above sense, the KLD from p 0 to p 1 is equivalent to the signal-to-noise ratio (SNR) of the underlying detection problem. Therefore, information geometry offers an insightful geometrical explanation for the detection performance of a Neyman–Pearson detector.

In the second case, the dimensionality of the unknown parameters u 0 and u 1 is different, while the dimensionality of the enveloping spaces is common for both hypotheses due to the same measurements x . However, the hypotheses may correspond to two separated submanifolds, Ω 0 and Ω 1 , embedded in B n caused by the different dimensionality between the unknown parameters. As illustrated in Figure 2 b, a surface and a curve are used to denote the submanifolds Ω 0 and Ω 1 , corresponding to the two hypotheses, respectively. Similar to the first case, the GLRT with different unknown parameters may also be geometrically interpreted.

4.3. A Demonstration of One-Dimensional Curved Gaussian Distribution

Consider the following detection problem:

The measurement originates from a curved Gaussian distribution

where a is a positive constant and u is an unknown parameter.

The probability density function of the measurement is

By reparameterization, the probability density function can be represented in the general form of a curved exponential family as

where C ( x ) = ln a and the potential function φ is

The above distributions with local parameter u correspond to a one-dimensional curved exponential family embedded in the natural parameter space A . The natural coordinates are

which defines a parabola (denoted by F A )

in A . The underlying distribution ( 28 ) can also be represented in the expectation parameter space B with expectation coordinates

which also defines a parabola (denoted by F B )

The sufficient statistics F ( x ) obtained from samples x can be represented by

Figure 4 shows the expectation parameter space and illustrates geometric interpretation of the underlying GLRT, where the blue parabola in the figure denotes embedding of the curved Gaussian distribution with parameter u . The submanifolds associated with two hypotheses can be geometrically represented by the blue parabolas (specified by η 1 < 0 and η 1 > 0 , respectively). Without loss of generality, assume that a = 1 , u = 2 . The blue dots signify N = 100 observations (measurements) in the expectation parameter space with the coordinates ( x , x 2 ) . The statistical mean of the measurements are used to calculate the sufficient statistics F ( x ) which are denoted by a red asterisk. The MLEs of parameter u under two hypotheses are obtained by finding the points on the two submanifolds which are closest to the data point in the sense of KLD. According to ( 22 ), the GLRT can be geometrically performed by comparing the difference between the minimum KLDs D 0 and D 1 with a threshold γ ′ .

The geometric interpretation of the GLRT for one-dimensional curved Gaussian distribution.

4.4. Discussions

The geometric formulation of the GLRT presented above provides additional insights into the GLRT. To the best of our knowledge, there is no general analytical result associated with the performance of the GLRT in the literature [ 1 ]. The asymptotic analysis is only valid under the conditions that (1) the data sample size is large; and (2) the MLE asymptotically attains the Cramér-Rao lower bound (CRLB) of the underlying estimation problems.

It is known that the MLE with natural parameters is a sufficient statistic for an exponential family, and achieves the CRLB if a suitable measurement function is chosen for the estimation [ 8 ]. For the curved exponential families the MLE is not, in general, an efficient estimator, which means that the variance of MLE may not achieve CRLB with a finite number of samples. This indicates that when using a finite number of samples there will be a deterioration in performance for both MLE and GLRT when the underlying statistical model is a curved exponential family. There will be an inherent information loss (compared with the Fisher information) when implementingan estimation process if the statistical model is of nonlinearity. Roughly speaking, if the embedded submanifold F B in Figure 2 a and Ω 0 , Ω 1 in Figure 2 b are curved, the MLEs will not achieve the CRLB due to the inherent information loss caused by the non-flatness of the statistical model. The information loss may be quantitatively calculated using the e -curvature of the statistical model [ 9 ].

Consequently, if the statistical model associated with a GLRT is not flat, i.e., the submanifolds shown in Figure 2 are curved, there will be a deterioration in performance for the GLRT using a finite number of samples. As sample size N increases, the sufficient statistics F ( x ) will be better matched to the statistical model and thus closer to the submanifolds (see Figure 2 ), and the divergence from data to the submanifold associated with the true hypothesis H i will be shorter. Asymptotically, as N → ∞ , the sufficient statistics will fall onto the submanifold associated with the true hypothesis H i , so that the corresponding divergence D i reduces to zero. By then, the GLRT achieves a perfect performance.

5. Conclusions

In this paper, the generalized likelihood ratio test is addressed from a geometric viewpoint. Two pictures of the GLRT are presented in the philosophy of the information geometry theory. Both the detection and estimation associated with a GLRT are regarded as geometric operations on the manifolds of a parameterized family of probability distributions. As demonstrated in this work, the geometric interpretation of GLRT provides additional insights in the analysis of GLRT.

Potentially, more constructive analysis can be generalized based on the information geometry of GLRT. For example, the error exponent defined by ( 24 ) and ( 25 ) provides a useful performance index for the detection process associated with GLRT. When p 0 and p 1 in ( 24 ) are the estimates of an MLE (rather than the true values) of unknown parameters under each hypothesis, there may be a deterioration in performance in the estimation process. Determining how to incorporate such an “estimation loss” into the error exponent is an issue. Another open issue is the GLRT with PDFs of different forms for each hypothesis, which leads to a different distribution embedding associated with each hypothesis.

Funding Statement

This research was funded by the National Natural Science Foundation of China under grant No. 61871472.

Author Contributions

Y.C. proposed the original ideas and performed the research. H.W. conceived and designed the demonstration of one-dimensional curved Gaussian distribution. X.L. reviewed the paper and provided useful comments. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Data availability statement, conflicts of interest.

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.