Many companies featured on Money advertise with us. Opinions are our own, but compensation and
in-depth research may determine where and how companies appear. Learn more about how we make money.

Women laughing in back of car, going out at night.
Betsie Van Der Meer—Getty Images

You can probably think of a million things to do with $1 million — buy your dream home, take a trip around the world or shop till you drop. But while riches can make your life a little easier and eliminate some of the day-to-day financial worries, there’s a downside to wealth as well.

When you look at the lives of reality stars, singers and other well-known millionaires, it appears as if they have it all. But underneath some of their smiles lies a carefully guarded secret: Being a millionaire isn’t all it’s cracked up to be.

Whether it’s money, career or family, the super-rich have their own set of issues. And as Bryan Clayton, CEO of GreenPal — which is best described as Uber for lawn care — says, anyone who wants riches should be careful for what they wish for. Sometimes, you’ll deal with different problems through no fault of your own.

For people who figured out how to become a millionaire, they live a lifestyle unlike any other. But it’s not all roses and sunshine. Money might improve their quality of life, but it doesn’t bring happiness — nor does it erase all their problems. Here’s a glimpse at the harsh realities of being a millionaire.

1. You Might Be at a Greater Risk for Lawsuits

You don’t need to be rich to be sued by someone, but your odds of getting hit with lawsuits can increase once you become a millionaire. If it’s known that you’re wealthy, some vindictive people might use any and every opportunity to get a piece of the pie — even if it means filing baseless, frivolous lawsuits against you.

Clayton understands this reality firsthand. He came from a blue-collar working class background and started his first company at the age of 18.

“The one thing that really surprised me was with wealth comes lawsuits,” he said. “It was just amazing to me that because I had wealth and a prominent business that I was a magnet for lawsuits of all kinds. It’s [not fun] for a stranger to pop out of the bushes and say you have been served.”

One of the perks of being a millionaire is that you can probably afford an attorney to fight frivolous — as well as legitimate — lawsuits against you. Even if you’re an honest person, wealth makes you an easy target. Therefore, you should establish a relationship with an attorney to protect your assets.

Read More: Here’s How to Make Your First $1 Million

2. You Can’t Always Trust the People You Hire

Whether you’re a hard-worker running an empire, or you come into wealth by means of an inheritance or other windfall, you might hire a team to help manage your money. It makes sense to have a financial advisor, accountant or money manager by your side. But sometimes, the people you trust can be anything but trustworthy.

Chantay Bridges, a speaker, writer and realtor with TruLine Realty in Los Angeles, warns that if you hire a crook, you can be held responsible for their actions.

“You bring on teams of people from all walks of life, yet sometimes a bad apple slips through the cracks,” she said. “They are up to no good, [and] depending on their role in your organization, it can make it look like you were an accomplice to their evil deeds.”

When adding to your team, you should seek the most qualified candidates, as well as those who have a reputation for being trustworthy. Don’t rush the process. Do your research, which includes a background and criminal history check. You can also request references and speak to their past employers or clients.

Most importantly, follow your instinct. If you have a bad feeling about someone, don’t hire them.

3. You Attract Fake Friends

You might be the type of person who looks for the good in everyone, and you want to believe everyone in your circle is a loyal friend. Some are, but once you become a millionaire, you can’t afford to walk around with blinders on.

Being a millionaire doesn’t mean distrusting everyone who offers a smile, but you have to be on the lookout for fake friends — people who only come around when they need something.

“Money, success and riches can cause people to flock to you, yet everyone’s motives are not the same,” said Bridges. “There are individuals that will come only for what they can get from you, not with a pure or true heart to be your friend.”

These moochers can include relatives, colleagues who live paycheck to paycheck and even your friends. They’ll stick around for as long as you have your wallet open, but disappear as soon as you cut them off.

4. You Have to Deal With Unfair Haters

On the other end of the spectrum, you might also deal with people who dislike you for the simple fact that you’re a millionaire. It doesn’t matter that you’re a kind person who treats everyone with respect — some people won’t be able to get past their own insecurities and jealousy.

You have something they want. If they don’t know how to become a millionaire — or if they’re afraid to take risks — your success infuriates them. You’ll deal with their negative comments, and you might have to face negative stereotypes, such as some people perceiving you as greedy or ruthless. This could be the farthest thing from the truth, but it doesn’t matter. Some people will dislike you just because they can.

The best way to deal with haters is to ignore them and not retaliate. You don’t have to apologize for your success. You can’t please everyone, so there’s no point in trying.

Read More: How to Get a Head Start on Holiday Travel Deals

5. You Suffer From ‘Money Guilt’

Dealing with feelings of money guilt is a downside of being a millionaire that can come as a shock. Even if you sympathize with the struggles faced by others, you might have trouble empathizing with them.

Unlike some of your friends, you might no longer worry about the cost of healthcare or stress about how you’ll pay for your kid’s college education. Because of this, you could feel a twinge of guilt and make it your aim to give large sums to your favorite charities and help those with a legitimate need. There’s nothing wrong with this. But just know that it’s not your responsibility to save everyone, and you shouldn’t let others guilt you into giving away money.

“Once it becomes common knowledge that you are a person of means, everyone seems to come from out of the woodwork,” said Bridges. “From children starving in Africa, people enslaved in foreign countries, limited education for women, and the list goes on and on. You can only do so much; you cannot save or fix the entire world.”

Keep Reading: 50 Ways to Live the Big Life on a Small Budget

6. You Owe More Taxes

The more money you earn, the more you’ll owe in income taxes. Chances are you’ll have an accountant crunching the numbers and determining how much you owe the federal and state governments, but you should mentally prepare for a higher tax bill — it comes with the territory.

The 2016 tax brackets range from 10 percent up to 39.6 percent. If you’re a millionaire earning big bucks every year, you’ll most likely end up in the highest tax bracket. The amount you’ll owe in taxes depends on whether you’re an individual taxpayer, married filing jointly, married filing separately or a head of the household.

However, the financially savvy millionaires have a few secret strategies that help them save money on taxes.

7. You Feel Pressure to Maintain a Certain Lifestyle

Being wealthy can mean becoming accustomed to a certain way of life. This presents a new challenge — the pressure to maintain this lifestyle.

A 2015 survey by UBS Investor Watch found that millionaires are just as stressed as everyone else, but they have different worries. About 52 percent of those surveyed felt stuck on a treadmill and believed they couldn’t get off without sacrificing their family’s lifestyle.

Some millionaires — particularly millennial millionaires — are also more “insecure and conscious about how their wealth compares to that of peers,” according to the survey, and they often feel pressure to work long hours and keep up with the Joneses, which can mean losing quality time with their children and family.

What’s also interesting is that the survey found that some millionaires — despite having impressive net worths — are never satisfied and continually strive for more riches. For these people, it isn’t enough to have $1 million; they want $2 million, and those individuals with $2 million want $4 million and so forth. They have so much, yet it’s not enough because lifestyles expectations increase as their net worths soar.

 

This article originally appeared on GoBankingRates.