The following is a small sample from my book,which looks at the 6 steps to financial freedom.
Does investing your own portfolio work? For the majority, unfortunately it doesn’t. The S&P has historically produced about 10% as mentioned before, but the average investor has produced about 4% per annum, despite the low fees of such platforms. What could explain such a big discrepancy?
It is human nature to allow emotions such as fear, greed and egotism get in the way. Overconfidence is one of the biggest killers of portfolios. Barber and Odean in a 2000 paper show that “after accounting for trading costs, individual investors underperform relevant benchmarks. Those who trade the most realize, by far, the worst performance. This is what the models of overconfident investors predict” (https://faculty.haas.berkeley.edu/odean/papers/gender/BoysWillBeBoys.pdf)
Then with a different data set, Odean  finds that “the securities individual investors buy subsequently underperform those they sell. When he controls for liquidity demands, tax-loss selling, rebalancing, and changes in risk aversion, investors’ timing of trades is even worse. This result suggests that not only are investors too willing to act on too little information, but they are too willing to act when they are wrong.
These studies demonstrate that investors trade too much and to their detriment” (https://faculty.haas.berkeley.edu/odean/papers/gender/BoysWillBeBoys.pdf)
It is human nature for people to think they are smarter than the average, and can stock pick. Pick the winner, those individual stocks that will outperform. Shows such as CNBC and Bloomberg also encourage investors to think they cant time the markets.
Consider something for a moment. If you stock pick (individual stocks) rather than buying funds, you have an 80%+ chance of losing to the market over a 5 year period. That goes down to about 98%+ over a 40 year + investment career. There are many reasons for this. Even small costs of buying and selling build up.
Good short-term returns, moreover, increase egos, and complacency comes into play. One of the biggest reasons is that the information is all there transparently, so there is no such thing as a free lunch. Remember, all the information about companies is publicly available and there are people whose job it is to look at this information and weight the pros and cons of all that information.
Take tech as an example. It was true in the 1990s that those IT geeks who correctly predicted the future, could have made tones of money. Certainly early investors in Facebook or Amazon did.
However, it would have been madness to put a significant percentage of your wealth in Amazon in 1995. A rational investor can only make decisions based on the information he or she has available at the time.
Predicting the future is almost impossible and those who get it right once, probably won’t next time. There are so many unknown unknowns and know unknowns for all companies, and especially start-ups.
If Amazon’s CEO would have died 20 years ago, or there would have been a huge scandal or employees would have joined a competitor on mass, the company would never have succeeded. Who knows, maybe Amazon almost went bust early on before they publicly revealed information which only becomes a legal requirement once it goes public.
And more to the point, even though tech has on average done well over the last 20 years, most tech firms have gone bankrupt. Buying the market and a broad basket of companies isn’t speculating. It is just assuming that, like always, in the long-term, the biggest 100-200-300 companies in the US or elsewhere will be worth more money in 10-20-30 years than today.
So regardless of whether in 2050 most of the S&P is financial services, law, consumer goods or even bitcoin mining companies (i doubt that though!), an investor who buys the market will profit.
They just won’t profit as much as somebody who invests 100% of their wealth in the one coin that beats the market! One of the reasons the average DIY investors only makes on average 4%-5% per year when markets have gone up (historic average) 10% per year, is because they assume they are smarter than others due to `research`.
It is human nature to think you are smarter than the average, but the academic evidence is clear that enhanced knowledge won’t allow you to consistently beat average market returns. You will almost certainly beat the market some years, but on average, you will lose long-term.
Beyond that, self-motivation is an issue even for people who are hard-working. Most of the people reading this have probably gone to the gym, tried to lose weight and/or gain muscle. How many have successes? How many of people reading this have remained constantly motivated day in day out, year in year out? That is tough, but being a rational investor, requires that kind of discllipine.
It is isn’t egotism and lack of self-motivation that causes us to trade more. Panic and greed also play a part. It is a natural human emotion to want to invest more when markets are going well, or sell when markets are down or at least to stop contributing. Studies that shown that if you tell somebody that they have a 95% chance of making money, they are more likely to invest than if you tell them they have a 5% chance of losing money.
The 1990s also didn’t help. During the fantastic winning run for stocks, especially in the technology stocks on the NASDAQ, many people with little or no knowledge made a lot of money from stocks. Anybody can make money in a bull market, and this gives people overconfidence.
The human mind isn’t a rational one and an investor who wants to be logical is fighting against these urges to get better returns.