All Zicklin News

Should You Replace Your Accountant with ChatGPT?

November 13, 2023

You might expect ChatGPT, the generative AI tool taking the world by storm for its ability to answer questions on nearly any subject in mere seconds, to do well on accounting exams. After all, the most recent version of the chatbot scored over 1400 on the SAT.  

Brandon Lock

Brandon Lock, PhD, Stan Ross Department of Accountancy

Yet surprisingly, the chatbot scored quite poorly on accounting tests, according to a recent study in the American Accounting Association journal Issues in Accounting Education, to which Zicklin School of Business Assistant Professor Brandon Lock, PhD (Stan Ross Department of Accountancy), contributed. The study, which incorporated data from 186 institutions in 14 countries, compared the performance of students versus ChatGPT on more than 28,000 questions from accounting assessments and textbook test banks.  

Researchers found that students overall did significantly better than the bot, scoring an average of 76.7 percent on accounting assessments, compared to ChatGPT’s score of just 56.5 percent. In other words, if ChatGPT were a flesh-and-blood accounting student, it would fail the class.  





Categories: , ,