Prime Minister Anthony Albanese has just announced six mandatory artificial intelligence (AI) safety guardrails.
The government will establish a new AI safety regulator to enforce the regulations, with the power to impose significant penalties.
According to the guardrails, frontier AI companies – the ones with the most powerful AI models at the cutting edge of AI development – will have to:
- implement security controls
- identify and manage serious risks
- allow independent evaluation of their models
- rapidly notify government of serious incidents
- publish safety and security frameworks to enhance governance and transparency
- respond to updated standards and rules as technologies change.
The implementation of the guardrails is one of 18 recommendations from a rapid review the government launched in September after an AI agent, deployed by OpenAI to collect data, accessed non-public Medicare stats.
In the background, the government is pursuing an extensive agenda of technology law and policy reform. There’s a new prohibition on unfair trading practices focused on online markets, a digital duty of care bill to address harms from online platforms and AI chatbots, a significant draft privacy law reform bill, and an inquiry into Australia’s sovereign AI capability.
The prime minister’s announcement focuses on “AI safety”, “AI security” and “frontier” AI models and companies. But Australians may remain unprotected from known AI harms that – though less spectacular – are already widespread and serious.
A diversified approach to AI policy and regulation
As well as the mandatory guardrails, the rapid review recommendations included criminal law reform, international cooperation, and strengthening mechanisms for government engagement with frontier AI developers.
Nearly half of the recommendations are about increasing Australia’s preparedness for AI-related cyber incidents.
The recommendations make sense. The review is right to emphasise the role of building capacity, relationships, and cooperation. Effective and responsive AI regulation must go beyond the government simply issuing and enforcing rigid rules.
Despite these strengths, the government seems to be taking a narrow view of what “AI safety” means.
What counts as AI safety?
The announcement seems to indicate that the most important risks come from frontier AI models.
But risks from AI – including serious risks to safety, public health and national security – are not confined to the frontier.
AI and algorithmic systems are increasingly integrated into important services such as healthcare, insurance, housing, finance, employment, education and social security.
For the most part, these kinds of AI systems are not likely to be frontier AI. Instead, these would be smaller, more specialised tools, such as an AI recruitment bot, an AI personal budgeting app or an educational grading algorithm.
Unaccountable, unfair, poorly managed deployments of such everyday AI systems have the potential to cause enormous harm.
We’ve seen Robodebt harm hundreds of thousands of Australians. Sexist AI-enabled job recruitment tools, AI-enabled stadium security systems that let a gun into the Melbourne Cricket Ground, racist policing algorithms, and AI meal-planners that recommend poisonous recipes are just a few among thousands of examples of everyday AI harms and risks.
As the Medicare incident showed, risks also emerge from interactions between software systems, organisations and real world contexts. It doesn’t make sense to just focus on powerful AI models out of context.
Worse, AI systems are notoriously opaque. While changes to privacy law will soon require businesses and government to state in their privacy policies if they use automated decision-making, Australians don’t have a right to an actual explanation of AI decisions in most domains.
A lack of transparency about how (non-frontier) AI systems are developed and deployed is a serious impediment to accountability.
A lighter touch
The Australian government recognised all of this in an earlier 2024 proposal for mandatory AI guardrails. However, these have largely been abandoned in favour of a lighter regulatory touch.
The 2024 proposal contemplated ten guardrails that would apply to non-frontier AI systems deployed in high-risk settings, as well as frontier systems.
These would have included requirements for transparency, accountability, testing, data governance, record keeping, risk management, human oversight of systems, and pathways for contesting AI decisions.
Most of those guardrails are missing from the new list. Now, only safety frameworks need to be transparent, for instance.
Independent oversight seems limited to evaluating AI models themselves, rather than evaluating whether the development and deployment of AI systems in various contexts is safe, fair, responsible or accountable.
There are also no general record-keeping or transparency guardrails that would create the paper trails we need for real oversight and accountability.
Some promising changes
One area that looks promising is the requirement to respond to new standards and rules. Depending on how the guardrails are implemented, standards such as Australia’s nuanced and sophisticated Voluntary AI Safety Standard could provide more detailed guidance to promote fairness, transparency and accountability.
If compliance with certain standards were mandated, then guardrails could have a deeper and wider reach.
Similarly, a proposal to introduce criminal liability for negligent AI developers could incentivise better record-keeping and more proactive risk management.
But until the government fleshes out this bullet point list of guardrails and rapid review recommendations, it will be hard to know how broad the scope of AI safety regulation will be.
For now, one thing is clear. Introducing regulation for sci-fi frontier risks such as “rogue” AI agents attacking our government or infrastructure is good politics, and sound policy. But it’s not enough to keep Australians safe from serious AI harms.
