# Statistical Significance Tests Using statistical tests, it is possible to make a statement about the significance of a set of measurements by calculating a test statistic. If it is unlikely to obtain a test statistic at least as extreme as the observed value, then the result is significant. For example, at a significance level of 5%, the probability of a false positive test result would be bounded by roughly 5%.

## Parametric vs non-parametric tests

There is a multitude of tests for determining statistical significance. These tests can be differentiated into two categories: parametric and non-parametric tests. While parametric tests make assumptions on the distribution of the data, non-parametric tests do not rely on such assumptions. For example, the parametric t-test compares the means of two groups because it assumes that the data have a normal distribution.

The non-parametric Wilcoxon rank sum test (Mann-Whitney U test), on the other hand, considers the medians of the groups instead. If the assumptions of parametric tests are met, they are generally more capable of detecting an effect than non-parametric tests. If this is not the case, however, non-parametric tests should be preferred.

## Choosing an appropriate significance test

To find an appropriate statistical test, the structure of the data should be considered. Before starting an analysis, one should ask the following questions:

• How many dependent/independent variables are there?
• What are the types of the variables?
• Are the measurements in some way associated (i.e. matched)?

## What is there besides significance?

Once you have found an appropriate test, you may want to look into topics that go beyond mere significance, such as:

• How can I use effect sizes to describe the extent of an effect?
• How can I use power analysis to identify the likelihood that a test detects an effect if it exists?
• How can I interpret measurements using other quantities such as confidence intervals?

## Posts on statistical testing

You can find answers to these questions (and more) in the following posts on statistical testing.

### Testing Symmetry on Contingency Tables from Paired Measurements: McNemar's Test

0

McNemar's test is a simple test for for checking whether pairwise measurements from two categories are independent. Here, I investigate the properties of the test and how it is used in R.