--- title: AI Tools Are Getting Better, but They Still Struggle With Money Advice description: Amid concerns about LLMs and math, one website's ChatGPT test found it could only correctly answer financial questions 65% of the time. authors: - name: Pete Grieve role: Lead News Reporter url: https://money.com/author/pete-grieve/ bio: Pete Grieve is a personal finance reporter at Money who frequently writes about news and trends related to car buying, auto insurance and gas prices. education: University of Chicago, BA Political Science at_money_since: 2022 articles: 600 covers: - Personal Finance - Banking - Business - Career Advice - Credit - Credit Cards - Debt - Economy and Politics - Education - Health and Wellness - Housing - Identity Theft - Insurance - Investing - Lifestyle - Loans - Mortgages - Professional Services - Retirement - Shopping - Taxes social: - https://twitter.com/pete_grieve - https://linkedin.com/in/pete-grieve-b5aa02138 - https://www.facebook.com/petegrieve4 published: '2025-02-13T20:25:06.000Z' modified: '2025-02-13T20:25:06.000Z' section: Personal Finance tags: - News word_count: 678 canonical: https://money.com/ai-tools-financial-advice-struggle/ type: NewsArticle source: structured-blocks --- > **Disclosure:** We research all brands listed and may earn a fee from our partners. Research and financial considerations may influence how brands are displayed. Not all brands are included. [Learn more](https://money.com/page/disclaimer/). ![Photo collage of a young woman using a smartphone with a stock chart in the background](https://img.money.com/2025/02/Investing-Dont-Trust-Chat-GPT.jpg) As ChatGPT and its competitors improve their models, a growing number of people are relying on AI tools to answer everyday financial questions. But is that a good idea? When mainstream AI tools took the world by storm two years ago, experts cautioned that ChatGPT and other services were not capable of providing trustworthy financial advice, citing the [risk of hallucinations](https://money.com/dollar-scholar-artificial-intelligence/) and their limited access to real-time information. And despite [improvements to AI chatbots](https://www.wsj.com/articles/openai-rolls-out-updated-version-of-viral-chatbot-chatgpt-f03fea27) in the time since, recent research indicates artificial intelligence tools still are not advanced enough to reliably answer many personal finance questions. ## SoFi® is for everyone, including new and seasoned investors ### With SoFi®, you can build a balanced portfolio and trade stocks, ETFs and options as frequently as you want, commission-free. Click your state to start investing today! [Start Investing](https://money.com/pr/kaca5eee7c9d?ap_referrer=%2Fai-tools-financial-advice-struggle%2F) INVESTMENTS ARE NOT FDIC INSURED • ARE NOT BANK GUARANTEED • MAY LOSE VALUE. Brokerage and Active investing products offered through SoFi Securities LLC, member FINRA / SIPC. The team at [Investing in the Web](https://investingintheweb.com/blog/chatgpt-search-fails-in-35-of-finance-related-queries/#span-classc8assessment-methodologyspan), a site that reviews financial services, conducted a test in November that looked at how well ChatGPT Search answered 100 questions about money. According to their scoring of the responses, ChatGPT was correct 65% of the time, "incomplete and/or misleading" 29% of the time and wrong 6% of the time. The questions included: - **How much should I save for my retirement?** (ChatGPT got this "correct," providing key factors that determine how much an individual should save and explaining several savings frameworks that have been endorsed by financial experts.) - **What is the retirement age in my country?** (ChatGPT gave an incomplete answer, failing to mention that the rules were changing in the user's country of Sweden.) - **How to decide when to buy or sell an investment?** (The response was graded "incorrect" because ChatGPT did not discourage the user from "timing the market," arguably the most important point to make here.) ChatGPT performed worse on questions involving shifting news and information, while performing better in areas of evergreen financial advice. Taxes and financial aid were particularly challenging subjects for ChatGPT. In a report, Investing in the Web noted that ChatGPT appeared to source some of its answers from less-reliable personal blogs, which introduced errors into its outputs. Even though researchers found that AI wasn't dependable for over a third of financial questions, the report concluded that AI may be used as "a good starting point when researching a topic." ## Financial advice, math and AI Some chatbots appear to have guardrails limiting responses to sensitive questions, including those about money. For example, Google's Gemini responds to a request for stock picks by telling the user to consult a financial professional, saying that it is unable to recommend investments. Similarly, if you ask medical questions, you'll be told to go to a doctor. Political prompts yield a message about restrictions relating to elections. But ChatGPT — owned by OpenAI — *will* produce responses to many financial questions, and surveys have shown that a [significant swath](https://www.experianplc.com/newsroom/press-releases/2024/experian--americans-are-embracing-gen-ai-to-make-smart-money-mov) of Americans are turning to AI to manage their personal finances. This is potentially concerning, in part because AI tools often struggle with basic math. In forums, internet users who've experimented with plugging financial questions into ChatGPT have pointed out that because ChatGPT is a large language model (LLM), asking it to deal with numbers and solve math problems often leads to problems. A team of Apple researchers dug into this weakness of chatbots in an October paper that concluded "current LLMs are not capable of genuine logical reasoning; instead, they attempt to replicate the reasoning steps observed in their training data." Although they have many other uses, LLMs are not necessarily reliable or advanced when it comes to actually doing math beyond the grade-school level. Instead, their attempts to solve math problems "resemble sophisticated pattern matching more than true logical reasoning," according to [Apple](https://machinelearning.apple.com/research/gsm-symbolic). The bottom line: AI tools may be able to answer some of your financial questions, but you should always tread carefully when deciding what to do with your hard-earned money or how to plan for your future. No matter where the advice you're getting comes from — even if it's a human — make sure to double check the main points with other sources. ## It's never too late to start investing ![START INVESTING](https://s3.consumersadvocate.org/prd/image/image/20107/medium_fab74697-be3e-49a4-8040-ea434cb0e5c3.png) Our Partner [START INVESTING](https://money.com/go/8fafa7ef065b?bidid=4066799050&pmeta=p129-c537-u36516&ptid=666075432&ptoken=49b83f5eefed1ba48cfd3b759a3fe3ba) ![START INVESTING](https://s3.consumersadvocate.org/prd/image/image/37704/medium_bb9499f2-9b69-44d9-930a-d80a69dc114d.webp) Our Partner [START INVESTING](https://money.com/go/aa696320bc00?bidid=4066799059&pmeta=p129-c537-u36516&ptid=666075432&ptoken=bd88884b8b15d168e4e295f0d1df6632) ![START INVESTING](https://s3.consumersadvocate.org/prd/image/image/20399/medium_4774dcec-4aba-4810-85b3-843773401646.png) Our Partner [START INVESTING](https://money.com/go/2a41c0744e50?bidid=4066799054&pmeta=p129-c537-u36516&ptid=666075432&ptoken=4841ecb83bcf34ebfa0394048f6c184a) ![START INVESTING](https://s3.consumersadvocate.org/prd/image/image/16291/medium_00c49a6e-a3eb-40b2-b805-13b4258cd431.png) Our Partner [START INVESTING](https://money.com/go/350d54a8deb3?bidid=4066799049&pmeta=p129-c537-u36516&ptid=666075432&ptoken=d39804e9ecca472173e646ad8ea890cc) ![START INVESTING](https://s3.consumersadvocate.org/prd/image/image/38704/medium_33d1fef9-951f-43b5-a923-54259b40e275.webp) Our Partner [START INVESTING](https://money.com/go/751b1e1a3625?bidid=4066799062&pmeta=p129-c537-u36516&ptid=666075432&ptoken=2ec2d129d2c2b10141a1b8fd051e2a51) ![](https://s3.consumersadvocate.org/prd/image/image/20107/medium_fab74697-be3e-49a4-8040-ea434cb0e5c3.png) Become a better investor with SoFi® [Start Investing](https://money.com/pr/v42ab7f4acf9?ap_referrer=%2Fai-tools-financial-advice-struggle%2F) ```json { "@context": "https://schema.org", "@graph": [ { "@type": "BreadcrumbList", "@id": "https://money.com/ai-tools-financial-advice-struggle/#breadcrumb", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://money.com" }, { "@type": "ListItem", "position": 2, "name": "Personal Finance", "item": "https://money.com/personal-finance/" }, { "@type": "ListItem", "position": 3, "name": "News", "item": "https://money.com/tag/news/" }, { "@type": "ListItem", "position": 4, "name": "AI Tools Are Getting Better, but They Still Struggle With Money Advice" } ] }, { "@type": "WebSite", "@id": "https://money.com/#website", "url": "https://money.com", "name": "Money", "publisher": { "@id": "https://money.com/#organization" } }, { "@type": "WebPage", "@id": "https://money.com/ai-tools-financial-advice-struggle/#webpage", "url": "https://money.com/ai-tools-financial-advice-struggle/", "name": "AI Tools Are Getting Better, but They Still Struggle With Money Advice", "isPartOf": { "@id": "https://money.com/#website" }, "datePublished": "2025-02-13T20:25:06.000Z", "mainEntity": { "@id": "https://money.com/ai-tools-financial-advice-struggle/" }, "inLanguage": "en-US", "breadcrumb": { "@id": "https://money.com/ai-tools-financial-advice-struggle/#breadcrumb" }, "dateModified": "2025-02-13T20:25:06.000Z" }, { "@type": "NewsArticle", "headline": "AI Tools Are Getting Better, but They Still Struggle With Money Advice", "articleSection": "Personal Finance", "image": [ { "@type": "ImageObject", "url": "https://img.money.com/2025/02/Investing-Dont-Trust-Chat-GPT.jpg", "width": 2880, "height": 1920, "caption": "Investing-Dont-Trust-Chat-GPT" } ], "author": [ { "@type": "Person", "name": "Pete Grieve", "url": "https://money.com/author/pete-grieve/", "@id": "https://money.com/author/pete-grieve/#author", "description": "Pete Grieve is a personal finance reporter at Money who frequently writes about news and trends related to car buying, auto insurance and gas prices.", "image": "https://img.money.com/2023/02/peter.png?w=350&h=430&crop=1", "email": "peter.grieve@money.com", "jobTitle": "Lead News Reporter", "alumniOf": "University of Chicago, BA Political Science", "sameAs": [ "https://twitter.com/pete_grieve", "https://linkedin.com/in/pete-grieve-b5aa02138", "https://www.facebook.com/petegrieve4" ], "worksFor": { "@id": "https://money.com/#organization" } } ], "publisher": "https://money.com/#organization", "datePublished": "2025-02-13T20:25:06.000Z", "description": "Amid concerns about LLMs and math, one website's ChatGPT test found it could only correctly answer financial questions 65% of the time.", "abstract": "Amid concerns about LLMs and math, one website's ChatGPT test found it could only correctly answer financial questions 65% of the time.", "@id": "https://money.com/ai-tools-financial-advice-struggle/", "inLanguage": "en-US", "dateModified": "2025-02-13T20:25:06.000Z", "wordCount": 678, "keywords": [ "News" ], "about": [ { "@type": "Thing", "name": "Personal finance", "sameAs": [ "https://en.wikipedia.org/wiki/Personal_finance" ] }, { "@type": "Thing", "name": "Artificial intelligence", "sameAs": [ "https://en.wikipedia.org/wiki/Artificial_intelligence" ] } ] } ] } ```