Statistical Evidence - Period 5

Hypothesis testing, for example to test whether a certain drug is better than another one, or to test whether a gene is connected to a disease, is an important and ubiquitous application of statistics. In this course we are interested in the question as to what extent statistical data provides evidence for a given hypothesis. This may seem an innocent question but this is not the case. It so happens that almost all introductory courses in statistics teach methods which are applied to situations for which they were not designed and that the results of the test are given a meaning that they do not have.

Before applying a particular test or technique in statistics, one should ask the question what the purpose of the test or procedure really is. Do you want to see to what extent the data provides evidence for a specific hypothesis? Do you want the data to force a decision between two competing hypotheses? Do you want to see what the data has to say when comparing two or more specific hypotheses? Depending on the goal, the statistical methods may differ greatly. Unfortunately, in current practice, this is very often not well understood, not only by students but also by lecturers. The result is that often statistical techniques are used for purposes for which they were not designed. Here is an example:

Suppose a researcher wants to investigate a certain hypothesis, and he designs an experiment which he repeats a number of times, say 30 times. Typically he will set up the experiment by assuming as so-called null hypothesis that his research hypothesis is not true, hoping to be able to reject that null hypothesis based on the statistical data from the 30 experiments. Suppose it so happens that he cannot reject the negation of his hypothesis with a certain significance level. He sub-sequentially decides to carry out a number of additional experiments in the hope that after, say, 50 experiments, he does reach his goal. Common practice as this may be, it is incorrect in the sense that it is impossible that he will reach his desired conclusion after he did his first analysis after 30 experiments, no matter how long he would continue, and no matter what the additional outcomes would be.

We will explain that when we want to see to what extent the data provides evidence for a certain hypothesis, the only reasonable way to do that is in comparison with another hypothesis, and in this case the so called likelihood ratio measures the relative weight of the evidence of the two hypotheses. This likelihood approach solves most of the misunderstandings and problems, although certain problems will remain.

The course is both foundational, philosophical as well as very practical in nature, with many examples.

Vrije Universiteit Amsterdam (VU Amsterdam) awards credits based on the ECTS system. Contact hours listed under a course description may vary due to the combination of lecture-based and independent work required for each course therefore, CEA?s recommended credits are based on the ECTS credits assigned by VU Amsterdam. 1 ECTS equals 28 contact hours assigned by VU Amsterdam.

This is an Honors course.