NSW Ombudsman explores potential of AI model to improve complaint accessibility 

The NSW Ombudsman has today released groundbreaking research by the University of Newcastle on the use of Generative Artificial Intelligence (GenAI) to enhance the complaints process for consumers who speak English as a Second Language (ESL). 

The research focused on how a customised GenAI complaint portal could help address language, literacy or cultural barriers, reduce potential bias and boost the accessibility and effectiveness of complaints systems. 

Dr Christine Armstrong, who led the team of academics researching the potential benefits of the portal, said she wants it to be as easy as possible for everyone to make complaints, including to critical oversight bodies like the NSW Ombudsman.  

“The topic of how complaints are managed affects everyone – because everyone has made a complaint, or wanted to, at some point in their life. It is how we achieve justice if we are treated unfairly and I am passionate about ensuring we can all access that justice,” Dr Armstrong said. 

“Effective complaint management is something that underpins the functioning of a successful democratic society.”  

The research created and tested a prototype AI complaint portal, which is a tool that helps people compose a complaint so it’s well structured, grammatical and will be clear to complaint handlers.  

Dr Armstrong’s team conducted experiments using the prototype that yielded significant positive responses. While the tool suggested benefits for many complainants, the most significant improvements were seen among those with English as a second language. 

This function was shown to be particularly important in assisting users who may otherwise lack the confidence to complain.  

NSW Ombudsman, Paul Miller, said his office was already noticing more complaints coming to his office that appeared to have been drafted with the assistance of AI. 

“This research prototype is different, because the team built a specialised and closed tool with prompts developed specifically to improve complaints while minimising the risks of hallucination, and without complainants having to submit any information, including potentially sensitive information about their complaint, into a public, open-access platform,” Mr Miller said. “The complainant remains in control at all times about the information they lodge.” 

Mr Miller added: “There are risks involved with putting your complaint into an open AI system, including loss of privacy and the potential for AI hallucination. What this research shows, however, is that with the right design and safeguards, AI tools may have significant potential to improve complainant confidence and experience, especially for those who might, for whatever reason, feel less confident with written English.” 

The research coincides with the Ombudsman’s launch this year of an online AI assistant on its website, which provides users 24/7 support and assistance in navigating the NSW public sector complaint systems, and understanding the role of the NSW Ombudsman. The ‘chatbot’ provides responses within the boundaries of a knowledge retrieval system model, providing accurate and contextual responses.  

“We’ll continue to look for opportunities where we can leverage technology to improve the accessibility of our services and improve outcomes, especially for those who might otherwise face impediments or lack the confidence to speak up with their complaints,” concluded Mr Miller. 

The research into a GenAI complaint portal tool was conducted by Dr Christine Armstrong, Associate Professor Alicia Kulczynski, Dr Margurite Hook, Dr Moji Barari and Dr Bin Li. It was funded by the University of Newcastle and the NSW Ombudsman. 

Read the full research publication here.