Index Funds vs. Individual Stocks: What 95 Years of Data Actually Says

Burton Malkiel, in the 1973 first edition of A Random Walk Down Wall Street, made an argument so unfashionable at the time that his publisher resisted publishing it: that a blindfolded monkey throwing darts at the stock listings of The Wall Street Journal would, on average, do as well as a professional money manager. The book has sold something north of two million copies. The argument has aged extraordinarily well.

But the question of whether you should buy individual stocks isn't really the same question as whether stock pickers, on average, beat the market. The two get conflated constantly in personal-finance writing. I want to separate them.

What we actually know from the data

The longest-running comparison of active managers to their benchmarks is the SPIVA Scorecard, published twice a year by S&P Dow Jones Indices since the early 2000s. It tracks how mutual-fund managers in various categories do against the appropriate index, after fees.

The headline finding has been consistent for more than two decades:

• Over 1 year, roughly 60-70% of large-cap U.S. fund managers underperform the S&P 500.
• Over 5 years, roughly 75-85% underperform.
• Over 15 years, roughly 85-92% underperform.
• Over 20 years, the figure is typically over 90%.

These are professionals. They have research staff, Bloomberg terminals, direct access to company management, and the most sophisticated risk-management software money can buy. They underperform.

The underlying reason is mostly arithmetic, not skill: the average dollar invested in U.S. stocks must, by definition, earn the U.S. stock market return before fees. After fees — which average somewhere around 0.6-0.8% a year for actively managed funds, versus 0.03% for a Vanguard or Fidelity index fund — the average actively managed dollar must underperform. William Sharpe laid this out, with brutal clarity, in a 1991 paper called "The Arithmetic of Active Management." It is four pages long. I'd recommend reading it.

So why do people pick stocks anyway?

Three honest reasons, plus one less honest one.

1. Some people genuinely enjoy it.

Researching companies, reading 10-Ks, building models — for the right kind of person, this is a hobby in the same way that woodworking or birding is a hobby. If you're going to do it anyway, you might as well do it with real money. (Though I'd argue: do it with 5% of your portfolio, not 50.)

2. The skill question is more nuanced than indexers admit.

The SPIVA data measures mutual fund performance, after fees and turnover-driven taxes. A patient individual investor — buying a small number of high-quality businesses and holding for decades — operates under a very different cost structure. There's a real argument that a focused, low-turnover, tax-aware private investor can do better than the average mutual fund. Whether they can do better than a low-cost index fund is the harder question, and the data on that is much thinner.

3. Some people have an actual edge.

Not many. But some. If you spent thirty years as a senior engineer at semiconductor companies, you might genuinely understand the chip industry better than most analysts on Wall Street. Peter Lynch, in One Up on Wall Street (1989), made the case that ordinary people in their professional lives often see industry shifts before the market does. The case is real. It's also less powerful than Lynch made it sound, because seeing a shift and translating that into a profitable trade are very different skills.

4. (The less honest reason.) Stock picking is more exciting than index investing. Watching VTI go up 8% a year is extremely boring. Watching a stock you picked double in six months is thrilling. A non-trivial fraction of individual stock-picking is, in my view, an entertainment expense being misclassified as an investment.

Where individual stocks actually make sense

I'll lay out the cases I think are defensible, having watched a couple thousand portfolios over the years.

Case 1: The "fun money" sleeve.

Take 5-10% of your portfolio. Pick stocks you find interesting. Accept that this portion may underperform the index. The point is that it lets you scratch the itch without putting your retirement at risk. The 90% in index funds will carry you regardless. (This is what most thoughtful financial planners recommend if a client insists on stock picking, and it's what I do myself.)

Case 2: Concentrated holdings from compensation.

If you've worked at a company that paid you in stock and the position has grown, you may have accidentally become a stock picker. You should generally diversify out of it over time, but the math on doing so is more complicated than the indexers admit (capital-gains taxes, especially in high-income states, can make rapid liquidation expensive).

Case 3: True circle-of-competence investing.

A small number of investors — Charlie Munger's word for them was "specialists" — can genuinely outperform by sticking to industries they understand at a professional level. Note the words small number, genuinely, and industries they understand at a professional level. If your circle of competence consists of "I read CNBC sometimes," this isn't you.

Where individual stocks usually don't make sense

The list is longer.

• Picking stocks based on "tips" from anyone, ever. This includes your brother-in-law, the most popular guy on Twitter, the financial advisor who calls you, and any newsletter that has a flashy logo. The hit rate on tips is appallingly low.
• Picking stocks because a company makes products you like. "I love the new iPhone" is not a thesis. The iPhone's commercial success is fully priced into AAPL within hours of any positive news.
• Picking stocks because of momentum or because everyone is talking about them. This is how meme stocks work. It is also how people lose 70% of their money in nine months.
• Picking stocks to "beat" the market. The probability that you, individually, will beat the market over thirty years through stock selection is, based on the published data, somewhere in the low single digits. Possible. Not likely.

What index investing actually means in practice

You don't need to overcomplicate this. For most U.S. investors, "index investing" means some combination of:

• A total U.S. stock market fund (VTI, ITOT, SCHB, FZROX — pick one)
• A total international stock market fund (VXUS, IXUS)
• A total U.S. bond fund (BND, AGG, FXNAX)
• Optionally, a TIPS fund for inflation protection in the bond sleeve

Vanguard's three-fund portfolio, popularized by John Bogle and the "Bogleheads" community, has been the default for serious DIY investors for over a decade for good reason: it's diversified, low-cost (total expense well under 0.10% a year), and tax-efficient. There are more sophisticated portfolios — adding small-cap value, real estate, factor tilts — and reasonable people disagree about whether they're worth the complexity. The base case isn't really in dispute.

My honest position

I think index funds should be 80-95% of almost every investor's stock allocation. I think the case for individual stocks is strongest when treated as a contained sleeve, sized so that being wrong won't change your life. I think the case for handing your money to an actively managed equity mutual fund is, with rare exceptions, weak.

I also think the indexing community can be more dogmatic about all this than the data really supports. There are smart, disciplined investors who have beaten the market for thirty years. There are industries where market efficiency genuinely breaks down for stretches at a time. The world is more interesting than the bumper-sticker version of either side.

But you, reading this article, are statistically almost certainly better off in index funds. The honest version of stock-picking advice starts with that fact, not the exception to it.