There are several compelling arguments in support of this.

First, if you look at the historical data for the US markets, very few mutual funds beat the index in the long run—especially after you account for the management fee. In fact, (and this may be technical detail), if you include volatility (a measure of ‘risk’) of returns in the equation, index fund looks even more attractive.

Second, as you mentioned, most finance theorists would tell you that you can't beat the "market” in the long run. The theory says that a “value weighted” portfolio that holds every security available in the market gives you the best return-per-unit-of-risk (also known as the Sharpe ratio)

However there are some caveats:

1. Index funds such as the S&P500 or Wilshire 1000 are not “market”; but they are a close proxy.

2. There is also the question of whether there are some exceptional managers who do add value. For example, Peter Lynch (who managed Magellan funds at Fidelity) beat the S&P 500 in 11 of the 13 years in the 1977-1989 period. Of course, one can argue that he was just lucky. For example, if we look at a set of 500 “coin-flippers,” let each of them flip 13 coins, the winner will, on average, have 11.63 heads--and, of course, that does not make him a better "coin-flipper".

However, even if Lynch was not just lucky, Lynch/Magellan results are not justification for buying mutual funds. The other question is: how do you identify the PeterLynch-es of the world before they become Peter Lynch? If you cannot identify the good fund managers “ex-ante”, then you should still stick with just the “market” portfolio—that is, invest in index funds.

- Amit