Formulating Hypotheses and Significance Levels (CFA Level 1): Defining Null and Alternative Hypotheses and Choosing a Significance Level (α). Key definitions, formulas, and exam tips.
Let’s be honest: if you’ve ever tried to make an investment decision based on a data analysis (maybe something as simple as checking whether a strategy gave you better returns than the market), you’ve basically flirted with hypothesis testing. We often want to claim that some parameter—like the expected return of a portfolio—is different from a benchmark. But to do that confidently, we need a structured way to go about it. That’s where hypotheses and significance levels come in. Think of them as the rules of engagement in the epic showdown between data and uncertainty.
This section will walk you through formulating the null hypothesis (H₀) and the alternative hypothesis (H₁ or Hₐ), choosing a significance level (α), and explaining why these decisions matter for investment and statistical analyses. Yes, we’ll do a bit of math, but we’ll also keep it practical with real-world-like examples and a friendly tone.
Most hypothesis tests revolve around two (often competing) statements about a population parameter:
Null Hypothesis (H₀): This is like the “status quo” or baseline stance. It usually says something like “no difference,” “no effect,” or “the parameter is equal to a specific value.” For investments, H₀ might say, “The true average return of this mutual fund is 5%,” or “There’s no difference in the volatility between two portfolios.”
Alternative Hypothesis (H₁ or Hₐ): This is the statement you suspect or want to show. It typically indicates a difference, a change, or an effect. For example, “The true average return is not 5%” (two-sided) or “The average return is greater than 5%” (one-sided).
Sometimes, you’ll see people using Hₐ and sometimes H₁. They mean the same thing: whichever notation you prefer is fine, as long as it’s consistent.
Let’s say you’re evaluating a new options-based trading strategy that some folks in your office claim outperforms the S&P 500. You might set up the hypotheses like this:
This way, if you find sufficiently strong evidence in your sample analysis that mean monthly returns exceed 0.5%, you can reject H₀ and conclude there’s a statistically significant reason to believe the strategy is better than the baseline.
The significance level (α) determines how aggressive or conservative we are when concluding that H₀ should be rejected. Common α levels are 0.10, 0.05, and 0.01, though many investment analyses prefer α = 0.05 to balance the risk of incorrect rejection with the desire to detect a real effect.
Here’s the critical point: if H₀ really is true but our data sampling leads us to reject it, we make a Type I error. This error happens with a probability equal to α. So, if α = 0.05, there’s a 5% chance we reject a true H₀ by mistake.
But if you lower α too much (like α = 0.001), it’s super difficult to reject H₀—even if there’s a real difference. You’ll reduce the chance of Type I errors, but ramp up the chance of missing a real effect, known as a Type II error. Striking a balance here is key, and your choice depends on the stakes: if the cost of incorrectly rejecting H₀ is huge (e.g., making an investment mistake that might cause large losses), you might choose a lower α. If the cost of missing a great opportunity is high, you might be more lenient with α.
It’s helpful to visualize the flow of hypothesis testing. Below is a simple Mermaid diagram that shows the typical steps:
flowchart LR
A["Formulate <br/> Hypotheses"] --> B["Choose Significance <br/> Level (α)"]
B --> C["Compute <br/> Test Statistic"]
C --> D["Compare with <br/> Critical Value / p-Value"]
D --> E["Decision: <br/> Reject or Fail to Reject H₀"]
Notice how we say “fail to reject H₀” rather than “accept H₀.” If we can’t reject H₀ with the data and test design, it doesn’t necessarily mean H₀ is absolutely correct. It just means the evidence at hand wasn’t strong enough to discard it.
Depending on the nature of your data and assumptions, you’ll choose a test statistic. For a test about a population mean when the population standard deviation is known, you might use a z-statistic:
where
If \(\sigma\) is unknown, you’d typically use a t-statistic:
where \(s\) is the sample standard deviation. Once you have the test statistic, you compare it to the critical value (z or t) at your chosen α. Alternatively, many software packages give you a p-value that you compare directly to α.
Years ago, I worked on a small team analyzing daily stock returns. We had a funny hypothesis: “Not discussing the market at lunch leads to higher post-lunch trading efficiency.” So, H₀ was “Talking or not talking makes no difference,” while H₁ was “There is a difference.” Our significance level was 0.05. After several weeks of data, the p-value turned out to be 0.12, which was well above 0.05. We concluded we couldn’t reject H₀—though we all felt we were trading better when we stayed silent during lunch. That taught me that you can’t always rely on a small sample, and p-values don’t always match our gut feelings.
In an investment context, rejecting H₀ means the data provides statistically significant evidence in favor of H₁. For example, if H₁ states that a particular asset has a higher expected return than some benchmark, rejecting H₀ suggests you might want to consider adding that asset to your portfolio. “Failing to reject” H₀ means you don’t have sufficient evidence to conclude the asset’s return is different. It’s not a death sentence for new strategies, but you might want to gather more data or refine your assumptions before moving forward.
It’s also important to combine statistical evidence with professional judgment, risk analysis, and your broader financial context. Hypothesis testing alone won’t give you all the answers, but it’s a powerful method to verify or challenge your assumptions.
Type I Error: Rejecting H₀ when, in fact, H₀ was true. The probability of this error is α. Think of it as a “false positive.”
Type II Error: Failing to reject H₀ when, in fact, H₁ is true. The probability of this error is β. The complement of β is called power (1 − β), which is the probability of correctly rejecting H₀ when H₁ is true. Higher sample sizes, stronger effects, and proper test design can reduce this error.
In capital markets, risk is everywhere. If you set α too high, you risk “seeing” effects that aren’t really there—like concluding your new high-frequency strategy absolutely crushes the market, only to lose money on it in the real world (been there, done that). Conversely, a super-low α might mean you never take a bold stance and may miss out on real opportunities.
Set your α by asking:
Below is a snippet illustrating how you might use Python’s SciPy library to perform a simple one-sample t-test. It’s optional, but it helps to see how a typical software approach would appear:
1import numpy as np
2from scipy import stats
3
4daily_returns = [0.3, 0.4, 0.2, 0.55, -0.1, 0.6, 0.0, 0.7, 0.45, 0.1]
5mu_0 = 0.25
6
7t_statistic, p_value = stats.ttest_1samp(daily_returns, mu_0)
8
9print(f"T-statistic: {t_statistic:.4f}")
10print(f"P-value: {p_value:.4f}")
11
12alpha = 0.05
13if p_value < alpha:
14 print("Reject H0: There's evidence the mean return is different from 0.25%.")
15else:
16 print("Fail to reject H0: Not enough evidence to say it's different.")
This snippet calculates the test statistic and the p-value on a small sample of returns. If the resulting p-value is below your chosen α, you reject H₀.
For the CFA exam, keep these points in mind when you see a question about hypothesis testing:
Important Notice: FinancialAnalystGuide.com provides supplemental CFA study materials, including mock exams, sample exam questions, and other practice resources to aid your exam preparation. These resources are not affiliated with or endorsed by the CFA Institute. CFA® and Chartered Financial Analyst® are registered trademarks owned exclusively by CFA Institute. Our content is independent, and we do not guarantee exam success. CFA Institute does not endorse, promote, or warrant the accuracy or quality of our products.