Many companies featured on Money advertise with us. Opinions are our own, but compensation and
in-depth research may determine where and how companies appear. Learn more about how we make money.

Money is not a client of any investment adviser featured on this page. The information provided on this page is for educational purposes only and is not intended as investment advice. Money does not offer advisory services.

512138811
Owen Smith—Getty Images

What do you tell your financial adviser when you are leaving for an algorithm?

Joe O'Connor, a 52-year-old Connecticut salesman, had to have this conversation recently. It was delicate business explaining why he was ditching the planner he had been with for over a decade, to put his money in the hands of what is known as a robo-adviser - a web-based service that automates the allocation of your investment portfolio.

There were the usual responses: "But why?" "Was it something I did?" "What can I do to make it right?"

And of course, there is the timeless relationship classic line: "It's not you, it's me."

"It wasn't fun," O'Connor said.

That kind of awkward conversation is taking place more frequently these days, thanks to the rise of robo-advisers, which have about $20.1 billion in worldwide assets under management for new entrants, according to Switzerland-based research firm MyPrivateBanking.

Of course with total U.S. investable assets at $33.5 trillion, that is barely loose change under the couch cushions, industry analyst Michael Kitces points out. But projections are for heady growth with new robo-advisers expected to grow to $42.6 billion in 2016 and $86.7 billion in 2017.

Why Leave?

Looking strictly at fees, robo-advisers offer certain advantages. Prominent site Betterment, for instance, charges .25% on accounts between $10,000-$100,000, and .15% above that. Competitor Wealthfront has a similar cost structure, charging .25% for accounts worth $10,000 or more.

Personal Capital, which Joe O'Connor uses, offers more of a blended service, combining its automated recommendations with humans (albeit primarily via video chat or email), charging .89% on portfolios up to $1 million.

That is in comparison to traditional financial planners, who charge around 1% or more of assets annually. (Fee-only planners have their own payment structure, billing per planning session instead of charging a percentage of assets.)

The low-fee logic of robo-advisers may work admirably for young savers starting out. In fact many users are converted Do-It-Yourselfers or Millennials with little investable cash, rather than mid-career professionals who have switched from existing planners, Kitces points out.

Why Stay?

You may gain something by opting for low-fee robots - but you lose the long-term financial planning aspect.

"I had a client recently leave for a robo. I told them robos are not financial planners," says Kashif Ahmed, an advisor in Woburn, Mass. "A robo will not call you when markets are going through a rough patch, and you can't call a robo to discuss your protection needs, or to ensure your estate documents are in order."

As you age, and financial responsibilities start piling up - raising kids, dealing with insurance questions, running a business, coping with elderly parents, and so on - the advantages of dealing with an actual person become more evident.

"At that point, when money has grown substantially, you may opt out of robo-investing and go find a real person," says Maggie Baker, a Philadelphia financial therapist and author of the book "Crazy About Money."

Of course, it is not always fees that cause breakups with financial planners. Far from it. In fact, the number-one reason cited by millionaires for switching advisers is due to them not returning client phone calls, according to a report from research firm Spectrem Group.

In cases like that, a robo-adviser is obviously no upgrade. After all, it is hard to get on the phone with an algorithm.

Having 'the Talk'

Baker's advice for ditching your existing planner: Just be honest. It is likely that some negative event has caused you to look elsewhere - subpar returns, maybe, or a general lack of communication - and it could be something you can talk through and resolve.

In fact, thanks to technological advances, you may not have to break up at all. Many firms, like Vanguard and Charles Schwab Corp, are gravitating towards two-tiered solutions - offering robo-allocations as a starter level, but also providing flesh-and-blood advisory services as a premium option.

With more investors considering robots to steer their finances, though, you cannot escape the regret and bitterness that linger over broken relationships.

"When a client decides to leave, I don't do anything," says Richard Colarossi, a planner in Islandia, New York. "If a client leaves to go to robo-adviser, let them go. My experience tells me that a majority of robo clients will shoot themselves in the foot."

Read next: Meet Your New Financial Adviser