Research Statistics Basics

Basic Concepts

• Null hypothesis: The hypothesis that the independent variable has no effect on the dependent variable. For example, "steroids do not improve outcomes in ARDS" would be a null hypothesis while the alternative hypothesis would be "steroids improve outcomes in ARDS." 

• Type 1 Error: Occurs when you reject a null hypothesis, that is in fact, true. In other words, your experiment/trial found a difference when none actually exists (ie ARDS patients receiving steroids seem to have better outcomes even though this is not true). Type 1 errors are denoted by α, or the significance level of the test, and frequently set at .05, meaning there is a 5% chance or less of committing a type 1 error and finding a difference when none exists.

• Type 2 Error: Occurs when you accept a null hypothesis, that is in fact, false. In other words, you failed to find a difference even though one actually exists (ie ARDS patients receiving steroids do better but your trial did not show this). The probability of a type 2 error is denoted by β. 

• Power: The probability of making a correct rejection, or of finding a difference when one exists. If β is the probability of not finding a difference when one exists, 1-β is the probability of finding a difference when one exists, otherwise known as power. It is the probability of correctly rejecting a false null hypothesis 

• Mean: The average value (add them all up, divide by the number of values). Can be affected by outliers. For example, average net worth in the US might look artificially high because of outliers like Bill Gates, Warren Buffet etc.

• Median: Put the numbers all in order, choose the middle number. If there are an even number of numbers (i.e. 1,2,3,4), then average the two middle numbers (in this case 2.5). The median is not affected by outliers

• Mode: Which number shows up most frequently (1,2,2,3) the mode is 2. There can be bimodal distributions as well (i.e. 1,2,2,3,3,4) you have peaks at 2 and 3 

• Positively Skewed: The tail is to the right, meaning you have outliers in the positive direction (income or net worth in the USA is an example of a positively skewed distribution)

• Negatively Skewed: The tail is on the left, meaning you have outliers in the negative direction (age at death in the USA would be negatively skewed distribution)

• Sensitivity: The true positive rate or how well a test detects the condition. For example, if a pregnancy test is 99% sensitive, it will be positive in 99/100 pregnant women.  A/(A+C). Sensitivity is an intrinsic characteristic of the test and does not depend on the prevalence of the condition.

• Specificity: The true negative rate or how well a test detects absence of the condition. For example, a pregnancy test that is 95% specific will be negative in 95 out of 100 non-pregnant women. D/(B+D). Specificity is an intrinsic characteristic of the test and does not depend on the prevalence of the condition.

• Positive Predictive Value: If a test is positive, what is the probability that it is actually true and that the person has the condition. For example, if the pregnancy test is positive, what is the likelihood that the person is actually pregnant. A/(A+B). Unlike sensitivity and specificity, PPV depends on the prevalence of the condition in the population.

• Negative Predictive Value: If a test is negative, what is the probability that it is actually true and that the person does not have the condition. For example, if the pregnancy test is negative, what is the likelihood that the person is not pregnant. D/(C+D). Unlike sensitivity and specificity, NPV depends on the prevalence of the condition in the population.

• Odds Ratio: The ratio of the odds of an event occurring in one group vs. the event occurring in another group. In the below example, the event would be lung cancer. The odds of lung cancer occurring in the smoking group is 10/10 while the odds of the event occurring in the nonsmoking group is 4/76. Hence, the odds ratio is (10/10)/(4/76) or mathematically, A/B/C/D which rearranges to AD/BC. In this case, 10(76)/10(4)= 76/4=19. Hence, the odds of developing lung cancer are 19 times higher for smokers than nonsmokers. Odds ratios are generally used in case-control or cross sectional type studies.

• Relative Risk: This is similar to the odds ratio but describes the probability of an event occurring in the exposed group vs. the nonexposed group. Using the below example, the risk of a smoker developing lung cancer is 10/20 (total of 20 smokers with 10 developing lung cancer and 10 not) while the risk of developing lung cancer as a nonsmoker is 4/80 (80 total nonsmokers). Hence, the relative risk is [10/(10+10)]/[4/(4+76)]= 10/20/4/80=10. Mathematically: [A/(A+B)]/[C/C+D)]. Smokers have a 10x greater risk than nonsmokers of developing lung cancer. Note that the same numbers give an odds ratio of 19 and a relative risk of 10. Another example may help to demonstrate the difference. If exposure A carries a risk of 99.9% and exposure B carries a risk of 99.0%, the relative risk will be just over 1 while the odds ratio will be greater than 10. Thus, for medium/high probabilities, relative risk gives a more intuitive measure of risk. Relative risk is generally used in cohort

• and randomized control trials. 

  • If RR =1, the exposure makes no difference

  • If RR >1, the exposure is associated with an increased risk of the outcome

  • If RR <1, the exposure is associated with a decreased risk of the outcome

• Attributable Risk: How much of the risk is attributable to a particular exposure. For example, using the smoking/lung cancer grid below, the risk of a smoker getting lung cancer is 50% (10/20) but how much of that is due just to smoking? You simply subtract the risk of those not exposed from the risk of those exposed (4/80 or 5%) which is 50%-5%. Hence, 50-5=45%. There is an excess risk of 45 cases per 100 people due to smoking.

• Absolute risk reduction: How much does a treatment (ie a statin) reduce your risk of an outcome (ie a heart attack). If the risk of a heart attack is 10% at baseline and 5% with a statin, the absolute risk reduction would be 10-5=5%. The relative risk would be 5%/10% = 0.5, showing statins have an association with decreased incidence of the outcome (ie they may be protective). 

• Number needed to treat: How many people have to receive the exposure (i.e. a statin) to prevent one case of the outcome (i.e. a heart attack). If statins reduce the absolute risk of a heart attack by 5%, then you have to treat 20 people to prevent one heart attack (1/0.0.05).

• Confidence Interval: A measure of the reliability of the estimate. For example, if you wanted to know the mean weight of 5 year old girls, it would be impractical to weigh all 5 year girls. You take a sample of 100 5-year old girls and get a mean of 30.5 kg. The confidence interval gives you an idea of the likely population mean in relation to your sample mean. While one wants to interpret a 95% confidence interval as having a 95% chance of containing the true population mean, this is technically not correct. A 95% confidence interval states that if the population is repeatedly sampled and confidence intervals generated, 95% of those intervals will contain the population mean. For example, if your confidence interval for the weight of 5 year old girls was 15-35 kg yet 20 published reports have found sample means of 40 kg, it would not make sense to say that there is a 95% chance your confidence interval contains the population mean.

Figure 6: Levels of Evidence

References

1) Stanford Self-Paced Online Medical Statistics Course (Free)