Australian Government to Receive Partial Refund for Error-Ridden Report
A large financial services company in Australia has agreed to refund a portion of the $290,000 it received for a report it delivered, which was found to be riddled with errors. These mistakes included a fake quote from a federal court judgment and references to non-existent academic research papers.
Why the Refund?
The company had been commissioned to produce a report on behalf of the Department of Employment and Workplace Relations. This report, which was first published on the department’s website in July, was found to be full of inaccurate references and footnotes. A health and welfare law researcher from Sydney University brought this to the attention of the public.
The company confirmed the inaccuracies after reviewing the 237-page report and has since agreed to repay the final installment of its contract. The exact refund amount will be revealed once it has been reimbursed.
Resolving the Issue
When asked to comment on the inaccuracies, the company simply stated that the issue had been resolved directly with the client. However, they did not respond when questioned if the errors were the result of AI generation.
AI systems are known to occasionally fabricate information, a phenomenon known as 'hallucination'. The report in question, which examined the use of automated penalties in Australia’s welfare system, was written using a generative AI language system, Azure OpenAI. The revised version of the report disclosed this fact.
Errors in the Report
The revised report removed quotes falsely attributed to a federal court judge, as well as references to non-existent reports attributed to law and software engineering experts. The Sydney University researcher who discovered the errors said he found up to 20 mistakes in the first version of the report.
One glaring error suggested that a Sydney University professor of public and constitutional law had written a non-existent book, clearly outside her field of expertise. The researcher commented, "I instantaneously knew it was either hallucinated by AI or the world’s best kept secret because I’d never heard of the book and it sounded preposterous."
Implications of the Mistakes
Not only were academic works misquoted and used as "tokens of legitimacy", but the report also misquoted a judge, which the researcher considered to be a more serious error. He stated, "They’ve totally misquoted a court case then made up a quotation from a judge and I thought, well hang on: that’s actually a bit bigger than academics’ egos. That’s about misstating the law to the Australian government in a report that they rely on. So I thought it was important to stand up for diligence."
Calls for Full Refund
A spokesperson for the Australian Greens party has insisted that the company should refund the entire $290,000. She stated, "They misused AI and used it very inappropriately: misquoted a judge, used references that are non-existent. I mean, the kinds of things that a first-year university student would be in deep trouble for."