Hypothesis testing is essentially decision making. This is linked to interpreting results that were obtained in a statistical test.

Requirements

In general we need the following to conduct a hypothesis test

• The assertion that some hypothesis is true
• A numerical test that is applies to the data
• A hypothesis that is accepted or rejected depending on the outcome of the test

<aside> <img src="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/b9a3cb9d-df81-4307-bc77-35e722750158/Hypotheses.png" alt="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/b9a3cb9d-df81-4307-bc77-35e722750158/Hypotheses.png" width="40px" /> Hypotheses are statements that are either true or false. Simple hypotheses define the probability distribution function completely. Composite hypotheses combine several probability distribution functions.

</aside>

• In hypothesis tests we often compare to alternative hypotheses.

  In general, it is crucial to distinguish between one-tailed directional and two-tailed non-directional tests.

  • A two-tailed test refers to the comparison of a test outcome to a value where we don't care of whether the outcome is less than or greater than the value.
  • In the directional test the sign of the difference between test outcome and comparison value is of importance.

Type 1 / 2 errors

Significance and power

<aside> <img src="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/50460796-46f1-4f68-ad0d-7013f4931e3f/Significance.png" alt="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/50460796-46f1-4f68-ad0d-7013f4931e3f/Significance.png" width="40px" /> Significance: Type I errors are inevitable and the rate at which they occur is called significance. The significance, $\alpha$, is the integral of the probability distribution of the hypothesis over the rejection region:

$$ \alpha=\int_{Reject}P_H(x)\,\text dx $$

</aside>

<aside> <img src="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/7217bb0a-cc22-4f73-8b3c-a30c839ed741/Power.png" alt="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/7217bb0a-cc22-4f73-8b3c-a30c839ed741/Power.png" width="40px" /> Power: Considering the alternative hypothesis, we can define the integral of the probability distribution of the alternative hypothesis over the acceptance region, in other words the rate of Type II errors, as:

$$ \beta=\int_{Accept}P_A(x)\,\text dx $$

or, by integrating of the rejection region as above, we get

$$ 1-\beta=\int_{Reject}P_A(x)\,\text dx $$

where $1-\beta$ is called the power of the test

</aside>

Practical examples

🚴‍♀️ Example: Hypothesis test with a discrete distribution. In a Poisson test of the hypothesis that a counting experiment results in a count compatible with a certain $\lambda$ or smaller is:

$$ 1-\alpha\lt\int_{Accept}Poisson(x;\lambda)dx=\sum_{x=0}^{n}Poisson(x;\lambda) $$

for significance $\alpha$ and $n$ the limit of acceptance region

Null hypothesis

We can only ever reject a hypothesis with great confidence but no accept it. For any theory we want to test we have to formulate the opposite hypothesis and aim to falsify this. This hypothesis is called the null hypothesis, $H_0$.