Created using Whimsical

Learning or remembering every step and requirement of the Null Hypothesis Significance Testing (NHST) framework is difficult. There are many complex concepts and choices that need to be learned and understood in order to make accurate decisions.

The NHST framework is intended to set up a logical, consistent, and accurate set of steps that help determine whether data taken from a sample supports the idea that a) there is no effect in the population, or b) the sample data is inconsistent enough compared to a null effect that it may indicate the presence of an effect in the population (Pernet, 2015).

Remember: If we are using formulas on data, it is to figure out if there is some sort of effect. The specific formulas depend on what question you are asking.

Maybe that question is about a difference in pain reduction between two groups that have been given different migraine treatments.

Maybe that question is about a difference between the proportion of product produced and the proportion of product purchased in order to determine if production lines need to be adjusted.

It is important to remember is that that randomly collected groups of people can have characteristics and scores which are very different from a simple population average. This makes the use of very small groups or sample sizes (n<30) unreliable, since the one extreme score when there are a total of 5 scores may severely sway summary statistics.

Balancing this consideration with the type of research design we are using helps ensure that our test results are as accurate and reliable as possible, and allows us to imagine what kinds of errors we might make. We can use that imagination (and some calculus from the math department) to decide on specific error probabilities based on the research design, question(s), and sample size(s) we are using.

Each type of test that uses the NHST framework was developed to help answer a specific type of question using a specific type of data.

For instance, if a researcher studying Reaction Time (in second) had an experiment where one group of 50 people received 0mg of caffeine, and another independent group of people received 500mg of caffeine, they might want to determine whether there was or was not an impact of caffeine on Reaction Time. The Student’s T-test for Two Independent Samples helps answer this question, but a component of this test requires that any treatment affecting the scores only change the mean, not the variability.

This type of requirement is integrated into the NHST framework as an Assumption, which must be confirmed before continuing with a chosen inferential statistic. Data that does not conform to assumptions may be evidence that there is a different type of effect than the test can accurately work with, so we must be able to validate our data to ensure accurate conclusions.

2024-10-15