Hypothesis Test Calculator (z-test & t-test)

Hypothesis Testing Guide

What do I need to know about Steps in Hypothesis Testing?

1. State H₀ (null hypothesis) and H₁ (alternative hypothesis). 2. Choose significance level α (typically 5%). 3. Calculate test statistic: z = (x̄ − μ₀) / (σ/√n). 4. Find critical value from z-table (or t-table for t-test). 5. Decision rule: reject H₀ if |z| > z_critical (two-tailed) or z > z_critical (one-tailed upper). Or equivalently: reject H₀ if p-value < α. Example: H₀: μ = 50. H₁: μ > 50. x̄ = 52.3, σ = 8, n = 36. z = (52.3-50)/(8/6) = 2.3/1.333 = 1.725. One-tailed at α=0.05: critical z = 1.645. Since our calculated z (1.725) exceeds this critical value, we reject H₀ and conclude there is significant evidence the true mean exceeds 50.

What should I know about Type I and Type II Errors?

Type I error (α): rejecting H₀ when it is actually true. The significance level α is the probability of a Type I error. Choosing α = 0.05: 5% chance of incorrectly rejecting a true null hypothesis. Type II error (β): failing to reject H₀ when it is actually false. Power of test = 1 − β = probability of correctly rejecting a false H₀. There is a trade-off: reducing α (stricter) increases the chance of a Type II error. Context matters: in medical testing for a serious disease, a Type II error — missing a true diagnosis — is usually considered far worse than a Type I error, which is why screening tests are often deliberately set to be more sensitive at the cost of more false positives.

What should I know about p-value Interpretation?

The p-value is the probability of observing a test statistic as extreme as or more extreme than the one calculated, assuming H₀ is true. p-value < α: reject H₀ — statistically significant result. p-value > α: fail to reject H₀ — insufficient evidence. Common misinterpretation: the p-value is NOT the probability that H₀ is true. It is NOT the probability of getting this result by chance. It IS the probability of getting this extreme a result if H₀ were true. A p-value of 0.03 means: if H₀ were true, there's only a 3% chance of observing a result this extreme purely by random variation — evidence against the null hypothesis, but not proof it's false.

What's the difference between z-test and t-test?

Use z-test when: population standard deviation σ is known. Sample size is large (n > 30, approximately). Use t-test when: population σ is unknown and estimated from sample. Sample is small (n < 30). The t-distribution has heavier tails than the normal — it is more conservative (harder to reject H₀), reflecting the additional uncertainty from estimating σ. For large n, t approaches z — the test choice matters mainly for small samples. Degrees of freedom for t-test: df = n − 1. The critical t-value is always slightly larger than the equivalent z-value for the same confidence level, since the t-distribution accounts for the extra uncertainty of estimating the standard deviation from a small sample.

What's the difference between One-Tailed and Two-Tailed Tests?

A two-tailed test checks whether a sample result differs from the expected value in either direction — for example, testing whether a coin is biased without assuming which way. A one-tailed test checks for a difference in only one specified direction, such as testing whether a new drug increases recovery rate (not just "changes" it). The choice affects the p-value calculation: a two-tailed test splits the significance level (α) across both tails of the distribution, so each tail uses α/2, while a one-tailed test puts the entire α in a single tail, making it easier to reach significance for an effect in the predicted direction. The direction must be decided before collecting data, based on the research question — choosing a one-tailed test after seeing the data specifically because it makes a result significant is a well-known form of statistical malpractice.

What's the difference between Statistical Significance and Practical Significance?

A result can be statistically significant (p < 0.05, say) while being practically meaningless, especially with a very large sample size, because huge samples can detect tiny, real differences that don't matter in practice. For example, a weight-loss drug tested on 50,000 people might show a "statistically significant" average loss of 0.3 kg — technically real, but clinically trivial. Conversely, a genuinely important effect can fail to reach statistical significance in a small study simply because the sample wasn't large enough to detect it reliably. This is why researchers increasingly report effect sizes (like Cohen's d) alongside p-values: the p-value answers "is this likely due to chance?" while the effect size answers "how big is the difference, and does it actually matter?"

What should I know about Confidence Intervals and Hypothesis Tests: Two Views of the Same Data?

Confidence intervals and hypothesis tests are often taught as separate techniques, but they are two different ways of presenting the same underlying statistical information, and each can be derived from the other. A two-sided hypothesis test at significance level α rejects the null hypothesis H₀: μ = μ₀ exactly when μ₀ falls outside the (1−α) confidence interval calculated from the sample data; equivalently, the confidence interval contains precisely the set of hypothesised values that would not be rejected by the test at that significance level. For example, if a 95% confidence interval for a population mean is (48.2, 51.8), then a two-tailed hypothesis test at the 5% significance level would reject any null hypothesis claiming the true mean equals a value outside that range, but would fail to reject any null hypothesis claiming a value inside it. This equivalence means a confidence interval actually carries more information than a single hypothesis test's p-value: rather than answering a single yes/no question about one specific hypothesised value, it simultaneously shows which range of plausible values would survive testing, along with a visual sense of the estimate's precision — which is why many statisticians recommend reporting confidence intervals alongside, or even instead of, bare p-values.

Critical Value Table for Common Significance Levels

The critical z-value you compare your test statistic against depends on your chosen significance level (α) and whether the test is one-tailed or two-tailed:

A two-tailed test at α = 0.05 uses the same critical value (1.96) as a one-tailed test at α = 0.025, because splitting the 5% total rejection region across both tails leaves 2.5% in each tail — this relationship is a frequent source of confusion on exams, so always check which tail(s) your hypothesis actually concerns before picking a critical value from a table.

Worked Example: One-Sample z-Test for a Population Mean

A manufacturer claims a product's mean weight is 100 g (H₀: μ = 100). A quality inspector samples n = 49 units, finds a sample mean of 103.5 g, and knows from historical data that the population standard deviation is σ = 14 g. Test the claim at α = 0.05, two-tailed. Test statistic: z = (x̄ − μ₀) / (σ/√n) = (103.5 − 100) / (14/√49) = 3.5 / (14/7) = 3.5/2 = 1.75. Compare to the two-tailed critical value at α = 0.05, which is 1.96 (from the table above). Since |1.75| < 1.96, we fail to reject H₀ — there isn't enough evidence to conclude the true mean differs from 100 g. The p-value confirms this: p = 2 × P(Z > 1.75) = 2 × (1 − 0.9599) = 2 × 0.0401 = 0.0802. Since 0.0802 > 0.05, we reach the same conclusion from the p-value approach as from the critical-value approach — the two methods are always consistent because they're really asking the same question from two different angles. If the sample mean had instead been 104.5 g (test statistic z = 4.5/2 = 2.25), that would exceed 1.96 and p would drop to about 0.0244, flipping the decision to reject H₀.

Worked Example: Two-Sample t-Test for Comparing Two Means

Two fertilizer treatments are compared: Group A (n₁ = 15, mean yield x̄₁ = 48.2, s₁ = 5.1) and Group B (n₂ = 15, mean yield x̄₂ = 44.6, s₂ = 4.8). Test whether the means differ significantly at α = 0.05, using the (simplified, equal-variance-not-assumed) two-sample formula. Standard error of the difference: SE = √(s₁²/n₁ + s₂²/n₂) = √(5.1²/15 + 4.8²/15) = √(26.01/15 + 23.04/15) = √(1.734 + 1.536) = √3.270 = 1.808. Test statistic: t = (x̄₁ − x̄₂)/SE = (48.2 − 44.6)/1.808 = 3.6/1.808 = 1.99. With roughly df ≈ 27-28 for this unequal-variance case, the two-tailed critical t-value at α = 0.05 is approximately 2.05. Since |1.99| is just under 2.05, the result falls just short of statistical significance at the 5% level — a borderline case that illustrates why researchers are cautious about declaring "no effect" from a narrowly non-significant result: a slightly larger sample or a slightly larger observed difference could easily have tipped this into significant territory.