Meir Statman, a finance professor at Santa Clara University, is one of the most influential experts in behavioral finance -- the study of how your emotions and beliefs affect your decisions surrounding money.
Research in this field has led to a growing number of practices, such as automatic enrollment in 401(k) plans, intended to gently steer you toward smarter choices.
In presentations to financial advisers, and in his 2011 book What Investors Really Want, Statman explores how people try to balance the conflicting goals they have for their investments -- for example, earning top returns while reducing risk.
Statman, 65, has now started to question some of the behavioral finance dogma that has been the focus of his work. In a new paper, he argues that improving Americans' retirement security may require something stronger than a polite nudge.
His conversation with Money senior writer Kim Clark has been edited.
You've studied behavioral finance for more than 30 years, and you've seen many efforts to nudge people in the right directions. Do they work?
Nudging is very useful. Lots of people were nudged into saving for retirement by making it automatic and adding automatic escalation of savings.
Fifteen years ago, I might have said that would do the job. I doubt it now, because I see almost two populations: one that can be nudged into retirement savings, and another that is resistant. For them, we need to go beyond nudge into shove, and make retirement savings mandatory. I'm reluctant to shove people. But I think half of us, maybe more, are in a crisis.
So what's your plan?
It's similar to a 401(k) except that it is mandatory instead of voluntary. Employers would administer it. For self-employed people, there would be something like the insurance exchanges in the new health care law.
How much would people be required to save?
We should set a relatively low minimum, say 8% of one's income, satisfying pressing retirement needs. Ideally the level would be closer to 15%. That's the range in other countries that have created mandatory savings plans, such as Israel and Australia.
This is on top of Social Security?
Precisely. You might ask, "Why not just expand Social Security?" but that is not likely to fly politically, and enacting my proposal would require a new federal law.
Social Security is an insurance plan more than a retirement savings plan. It is fair for us to insure one another against dire poverty and disability. It is unfair, however, to ask us to assure others of a comfortable retirement.
Your plan seems very paternalistic.
People would resent it today, but be grateful later on. God knows, we all can tell stories about stuff our mothers forced us to do that we resented then, but for which we are grateful now.
When my dad was young and had young children, he wanted to take money from his pension fund to build another room to our house so there would be more room for the kids. He was turned down. In retirement, he was very grateful that he had been turned down, because it meant that he had more income and could live more comfortably.
We all face dilemmas between consumption now and later. A mandatory savings program prevents you from succumbing to temptation, even to one that is quite reasonable.
If I'm a person who saves already, why should I support your plan? Why should you be permitted to meddle in my life?
Savers should be indifferent to a requirement to save, because they do it anyway.
A resistance to mandates, of course, is part of American culture. I just hope that it can be overcome.
Think about spendthrift parents who arrive at retirement with nothing. And think about their adult kids who are savers. Those children might resent supporting their parents, but they are not likely to abandon them.
If you don't make the nonsavers save, one way or another they are going to fall on the shoulders of the savers.
Let's move on to other financial behavior. What are some big errors people make when investing?
One is applying wrong analogies from other parts of their lives. People think that experience will make them more competent investors, just as surgeons become more competent by performing more surgeries. Well, the analogy does not necessarily apply to investing, because the human body is not trying to fool the surgeon by moving the heart from one place to another.
In investing, the person on the other side of the trade will try to fool you. It might be an insider. It might be someone with special knowledge. People have to be disabused of the notion that investing is like surgery and realize that it's more like a tennis game where your opponent may look weak but, in truth, is much better than you.
Hindsight. If you kept a diary in 2007, it would likely say something like, "I think that the market is high. I'll wait a bit, and then decide." It would be wishy-washy. But we all look back at 2007 and we say, "Wasn't it clear that the market was going to go down?"
That kind of hindsight gives you the confidence that you can tell the future as well as you can the past.
Later on you might be tempted to sell your stocks because you are sure that their prices will fall -- only to find, three years later, that stock prices doubled while your cash was in a money-market fund.
So whenever I feel like saying, "I just knew it," I tap myself on the forehead and remind myself that I didn't. We are intelligent beings. We can identify cognitive errors and set a defense against them.
You point out that people often overlook the emotional reasons behind their investing. Can you give me an example of that?
I hope that people who trade heavily can admit to themselves that they do it not just to make more money, but because it is fun. Trading, of course, loses people money on average, but it can be fun the same way that playing videogames is fun.
I say, "Well, you know, everything in moderation." Just don't overdo it.
What's another way that emotions can affect investing?
When people are feeling poor, they are willing to take more risks.
You can have two people each earning $100,000 a year. One of them says, "This is plenty." The other feels behind. That one is more willing to risk losses in the hopes of reaching his or her aspirations.
Are there ways in which our emotions and biases actually improve our investment returns?
Whenever you trade stocks, you expose yourself to the possibility of regret. You might find out later on that you would have been better off doing something else.
I think that the anticipation of regret prevents many people from doing something stupid, such as selling all their stocks in March of 2009.
So to the extent that our aversion to regret causes us to buy and hold rather than time the market -- because any action opens a door to regret -- that is a cognitive error that helps us.
Have you seen any evidence over time that people are getting smarter about investing?
You know, I thought that I would be blue in the face before people were going to believe the logic and empirical evidence that index funds do better, on average, than actively managed funds. But it seems that people are learning and getting smarter.
There has been an increase in the proportion of investors' money that is going into index funds and exchange-traded funds. And I'm not blue in the face!