Many companies featured on Money advertise with us. Opinions are our own, but compensation and
in-depth research may determine where and how companies appear. Learn more about how we make money.

Say you want a crash course on everything you need to know about investing. If your goal is just to set up a retirement plan and move on, it's tough to improve on the basic advice University of Chicago professor Harold Pollack cleverly squeezed onto a single index card. (It boils down to save a lot and buy low-cost index funds.) For a deep dive into fundamental stock analysis, on the other hand, you'd take a crack at Benjamin Graham's dense but rewarding The Intelligent Investor.

Most of us, though, live somewhere between those poles: We're happy to outsource most stock picking to a mutual fund manager, but we feel compelled to pay attention to the markets—especially at a time like this, when equities are racking up record highs.

If that sounds like you, then the two volumes you need are A Random Walk Down Wall Street by Burton Malkiel and Irrational Exuberance by Robert Shiller. Both are classics that were republished this year in new editions addressing today's market.

Now, I should admit here that this recommendation is a little devious, because these two books disagree on a pretty fundamental point. In essence, Random Walk argues that the hive mind of the market is so good at determining the fair price for a stock that there's little point in trying to second-guess it. Irrational Exuberance, meanwhile, shows that stock prices sometimes get insanely high or low, and that you'd be equally crazy not to notice that. Frustratingly, both books make powerful cases.

But learning to wrestle with ambiguity and uncertainty is good mental training for owning stocks, which is never going to be a comfortable experience. During the 2007 to 2009 bear market, after all, even the plain vanilla Vanguard 500 Index fund dropped by more than 50%. That said, these books don't merely argue two sides of an important and interesting debate. Taken together, their insights can help you craft a smarter, safer financial plan.

Embracing Randomness

Malkiel is an emeritus professor at Princeton, but his book, first published in 1973, is not an academic tome. Instead, it's a how-to guide to building a balanced, diversified portfolio, preferably with low-cost index funds. Along the way, however, he popularizes some big, hairy ideas. Most important: the "random walk" of stock prices, and the efficient-markets hypothesis.

The idea that stock prices wander randomly doesn't mean they don't make sense, but that they're very hard to predict. For all practical purposes, tomorrow's price may as well be random. Past moves up or down won't tell you much. More controversially, neither will a company's income statement or balance sheet. Why not? Because if you could really use such information to predict where a stock was headed, other investors would've done it, and the price would be there now. In that way, today's prices aren't random at all but, in the jargon, efficient, because they already incorporate all of the information you could hope to know.

Random Walk helped make famous an old joke about the economist who wouldn't pick up a $100 bill on the street, reasoning that if it was really there someone else would have taken it. "A better version," Malkiel adds, "would be to remember that if any $100 bills are lying around, they will not be there for long." Someone else will almost always snap them up before you happen to walk by.

If Random Walk is the first book you read on finance (it was mine, 20 years ago), you'll be starting out with two giant advantages as an investor, and potentially one blind spot. The first advantage: You'll be very skeptical about paying high fees for an active mutual-fund manager. If markets are largely efficient, after all, even the very smartest pros shouldn't be able to beat the market consistently. And in fact the record shows that, after fees, most actively managed funds don't beat low-cost index funds.

The other way Malkiel is instructive is a little paradoxical: By showing how hard it is for anyone to get a trading edge, he also shows that anyone can invest and do reasonably well—just by buying an index fund.

The danger, if you aren't careful, is that efficient-markets thinking can blind you to risky market conditions. If the market is efficient, you might reason, who am I to fret when prices keep climbing higher? Fortunately, Malkiel is not a dogmatic believer that the market's price is always right: Random Walk's entertaining early chapters cover the history of speculative bubbles, from the Dutch tulip craze of the 1630s to the more recent tech and real estate booms.

Still, the inherent wisdom of financial crowds is a beguiling idea that deserves a strong counter-narrative. That's where Shiller comes in.

Deflating Exuberance

Shiller, who won a Nobel Prize for economics in 2013, devotes a chapter of Irrational Exuberance to dismantling the orthodox versions of efficient-market theory and random walks.

He reasons that if stock prices really are efficient, you ought to be able to see that in the historical record. For example, share prices would rise in anticipation of companies doing well and paying investors higher dividends. In fact, Shiller found that prices were far more volatile than future dividends would justify. What actually drives stock prices, he argues, is psychology, and it isn't always rational.

In 2000 the first edition of Irrational Exuberance noted that stock prices looked, well, exuberant, just in time for the tech stock bubble to burst. In the second edition, published in 2005, Shiller showed that housing prices were wildly above historical norms. (And we all know what happened next.) In the new edition Shiller expresses some concern about today's bond market—though he's careful to note that pessimism, not exuberance, is driving investors to seek safety in bonds. But more important than Shiller's timing are his rich descriptions of the social psychology and feedback loops that draw people into market fads.

In that way, Shiller's book is less an investing how-to guide than a how-you-should-think guide. It does, however, offer some quantitative guidance. Fundamentally, a stock is worth the profits the company will earn for investors. Earnings jump around from quarter to quarter, of course, but if you average them over a decade, you can smooth out the market cycles. When Shiller compares the price of the S&P 500 with the past decade of earnings, he finds that when the price/earnings ratio is high, low returns tend to follow over the next decade.

Note the words "tend to" and "over the next decade." Shiller's P/E is not very useful as a market-timing tool. Sure, the high P/E mark of 40 that stocks hit during the tech bubble was a great time to sell. But P/Es were high for years before that. And often these signals are more ambiguous. Shiller recently told Money that today's P/E of 27 looks relatively high—the long-run average is 16—but with safe bonds yielding just over 2%, even pricey stocks may be a reasonably appealing alternative.

A Safer Walk in an Irrational World

So what do you do with these two competing doctrines? One answer is to take them together as a warning against overconfidence.

Efficient-markets theory tells you to think twice, and then a third time, before betting that you (or your fund manager) can best an index. Indeed, over the past 10 years, 82% of blue-chip stock funds have lagged the S&P 500. Malkiel's core advice holds.

Shiller, meanwhile, is reminding you not to be too confident that the stock market will always deliver the returns you hope for. Since 1925 the average annual return of stocks has run about 10% per year, and before the twin crashes of 2000 and 2008, financial pros routinely suggested 10% could be expected over the long run. We'll know when we get there. But in the decade after 1998—a fairly long run for most people—stocks lost 1.4% of their value per year.

The moral, I think, isn't to try to time the market's twists, but rather to be a margin-of-safety planner. If you pay attention to market news and notice that prices are high, don't ask first, "Should I sell?" Ask: "Does my financial plan work with the lower returns implied by today's high prices?" Because one point Shiller and Malkiel agree on is that it's going to be hard to get 10% annual returns from here.

Their reasons? Shiller focuses on today's high P/Es. Malkiel points to dividend yields, the amount investors literally are paid for owning company stocks. Today it averages 1.9%, far less than the historical range of 4% to 5%. The dividend yield plus the long-run growth rate of earnings is a classic formula for the expected return on stocks. Since yields are low, Malkiel sees no reason to bank on today's market matching the historical average.

Planning with a margin of safety will mean different things at different times in your life. If you're relatively young and decades from retirement, take it mostly as a spur to save more to make up for the likelihood of lower returns. For you, the sharp drops that follow Shiller-esque fits of exuberance could actually be welcome news—it'll give you a chance to buy low.

Closer to retirement, as you look at how you've split your money between stocks and bonds, ask yourself if your retirement could withstand another 50% crash in the stock market like the one we experienced a few years ago. Right now the Shiller P/E is higher than it was leading up to the financial crisis.

In retirement, an expected lower return on stocks suggests you may need to be more conservative about pulling funds from your savings. A common rule of thumb suggests withdrawing 4% in year one and then increasing that amount with inflation each year. Some planners now suggest the safe number is closer to 3%.

"Safe" doesn't mean you couldn't be pleasantly surprised. By either Shiller or Malkiel metrics, the S&P 500 has been on the fair-to-expensive side since at least 2012, and has delivered a string of double-digit percentage returns since then. Nothing says 2015 and 2016 won't see even higher prices. Whether the stock market is efficient or exuberant or somewhere in between, it will never be easily tamed.

A Random Walk Down Wall Street and Irrational Exuberance are available at Amazon.com.