Your provider had the breach. You have the obligation.
On 6 July 2026 the Office of the Australian Information Commissioner reported that 2025 produced the highest number of data breach notifications since the scheme commenced in 2018: 1,205 notifications, an 8 per cent increase on the 1,112 received in 2024. Malicious or criminal attack accounted for 716 of them, 59 per cent of the total. Health service providers led the sectors with 225 notifications, 19 per cent of all breaches, followed by finance with 157, the Australian Government with 118, business and professional associations with 103, and education with 81. Australian Privacy Commissioner Carly Kind said the threat posed to Australian businesses and organisations by data breaches is substantial and rising year on year.
Nothing in that release mentions artificial intelligence. That is precisely why it is worth reading closely. Over the same period, organisations have handed personal information to a new class of provider at a speed no assurance function has matched, and the breach response plans sitting in most compliance libraries do not know those providers exist.
What does the Privacy Act say about who holds the data?
The Notifiable Data Breaches scheme turns on a definition that predates every AI product in your environment. Under section 6(1) of the Privacy Act, an entity holds personal information if it has possession or control of a record that contains the information. The OAIC guidance is explicit that this extends beyond physical possession to records an entity has contractual rights to control, and that in cloud computing arrangements both the service provider possessing the records and the client controlling access hold the information.
An eligible data breach arises under section 26WE where there is unauthorised access to or unauthorised disclosure of personal information, or a loss of personal information, that an entity holds; this is likely to result in serious harm to one or more individuals; and the entity has not been able to prevent the likely risk of serious harm with remedial action.
Read those two provisions together and the position is uncomfortable but simple. Personal information that your staff paste into an AI assistant, upload into a retrieval index, or generate as an AI output about an identifiable person is information you hold. If it is exposed, the scheme is engaged for you, not only for the vendor whose infrastructure failed.
The timing provisions are where this becomes operational. Section 26WH(2) requires all reasonable steps to complete an assessment within 30 calendar days after the day the entity became aware of grounds suggesting a possible eligible data breach, and the Commissioner expects entities to treat that as a maximum timeframe and complete assessments much faster where possible. Section 26WK(2) requires the statement to the Commissioner as soon as practicable after becoming aware of the eligible data breach, and section 26WL(3) requires individuals to be notified as soon as practicable after that statement is prepared.
Why joint holding is the clause nobody drafted
Where two or more entities hold the same record, the OAIC guidance says both are generally responsible for complying with the scheme in relation to that record, but section 26WM creates an exception so that only one of the entities that jointly holds the information needs to comply with the assessment and notification requirements on behalf of the group.
That exception is a gift and a trap. It is a gift because it prevents duplicate notifications landing on the same individual from four different companies. It is a trap because an exception nobody has allocated in advance becomes an argument during the worst week of the year.
The OAIC's own recommendation is to settle it early. Its guidance says that where information is held jointly, entities should establish clear procedures for complying with the scheme when entering into service agreements or other relevant contractual arrangements, including obligations around the communication of suspected breaches, processes for conducting assessments, and responsibility for containment, remediation and notification. It also suggests the entity with the most direct relationship with the individuals at risk of serious harm may be best placed to issue a notification to them.
For almost every AI arrangement, that is you. Your customer has never heard of your model provider.
The three gaps this opens in an AI environment
The inventory gap. Most organisations can name the systems of record that hold customer personal information. Far fewer can name the AI surfaces that now hold copies of it. Prompts and uploaded documents, retrieved source material, conversation history, tool call logs, memory stores and observability traces are all candidates. The OAIC guidance on commercially available AI products states that privacy obligations apply to any personal information input into an AI system as well as to output data generated by AI where it contains personal information. If your data map stops at the application boundary, it stops short of the obligation. This is the same failure pattern we described when a customer-facing assistant runs on someone else's code, and it is why agent memory needs to be governed as a data asset rather than treated as a convenience feature.
The notice gap. Your 30 day assessment window starts when you become aware. If the contract obliges the provider to notify you within a commercially typical period, you may learn of a suspected breach with a fraction of your window remaining. The number that matters in the contract is not the provider's internal investigation timetable. It is how quickly you are told something might have happened.
The evidence gap. An assessment has to reach a conclusion about likely serious harm, which requires knowing whose information was involved and what it contained. In a conventional system that is a query. In an AI system the answer may sit in logs the provider controls, may be spread across a prompt, a retrieval call and a generated output, and may not have been retained at all. We have written before about why AI telemetry records the tokens and not the decision; at breach time that gap is not an audit inconvenience, it is the difference between an assessment and a guess.
Practical implications for GRC teams
- Extend the personal information inventory to AI surfaces. For each AI service in use, record whether personal information can enter it, by what route, whether outputs about identifiable individuals are retained, and where those records live. Treat an AI assistant with document upload as a holding until proven otherwise.
- Fix notification timing in the contract, not the SLA. Specify how quickly the provider must tell you of a suspected or confirmed security incident affecting your data, in hours rather than a vague promise of promptness, and require enough detail to start an assessment. This is a natural extension of the due diligence framework applied to AI vendors under CPS 234 for regulated entities, and it is worth doing whether or not you are prudentially regulated.
- Allocate the section 26WM position in advance. Name in the agreement which entity assesses, which entity notifies individuals, which entity notifies the Commissioner, and how the two coordinate. Record the reasoning, because a decision made calmly is defensible and a decision made in hour three is not.
- Require retention of the evidence you would need. Ask what logs exist, for how long, and how you obtain them during an incident. Where the answer is unsatisfactory, that is a finding now rather than a discovery later.
- Rehearse one AI scenario. Run a desktop exercise on a single realistic case, such as a misconfigured retrieval index exposing documents containing personal information to the wrong internal audience. The point of the exercise is to find out who in your organisation currently believes this is the vendor's problem. Pair it with the AI incident evidence pack so the exercise produces a record rather than a conversation.
Where the line sits
None of this makes the provider's failure your fault. It makes the response your responsibility, which is a different thing and a well established one in Australian privacy law. Outsourcing an activity has never outsourced the obligation attaching to the information, and the OAIC has been consistent on that point for a decade.
Nor does any of it require a new framework. The scheme is unchanged, the sections are unchanged, and the assessment methodology your privacy team already uses still works. What has changed is the population of systems the methodology has to run over, and the fact that a meaningful share of that population was procured without a privacy impact assessment because it arrived as a feature inside software the organisation already owned.
There is a second-order point worth naming for anyone preparing a board paper on this. The record notification figure is a measure of reported breaches, not of breaches, and the OAIC has never claimed otherwise. An organisation that cannot see inside an AI provider's environment, cannot obtain its logs and has no contractual right to be told about a suspected incident is not an organisation with fewer breaches. It is an organisation with less visibility, and visibility is the only thing that converts an incident into a notification. That is an assurance finding, not a compliance statistic, and it belongs in the risk report in those words.
Bottom line
The record breach year and the quiet spread of AI into ordinary workflows are the same story told twice. Personal information is in more places, held by more parties, with less clarity about who tells the individual. The Privacy Act answers that question the same way it always has, and it answers it in your name.
Do this Monday
- List every AI service in production use and mark which can receive or generate personal information
- Pull the contracts for the top three and find the incident notification clause, then write down the number of hours it actually gives you
- Draft the joint holding position for each: who assesses, who notifies individuals, who notifies the Commissioner
- Ask each provider in writing what incident logs exist, how long they are retained, and how you obtain them
- Book one desktop exercise on an AI exposure scenario before the end of the quarter
Content disclaimer: This article is for general educational and informational purposes only. It does not constitute legal advice, regulatory guidance, or a substitute for professional compliance judgement. Privacy obligations vary by entity type, circumstance and the nature of the information involved. Always refer to primary source guidance from the OAIC or the relevant regulatory authority.
Primary sources
- OAIC, Data breach notifications increase to all-time high in 2025, new NDB stats show, media release, 6 July 2026. https://www.oaic.gov.au/news/media-centre/data-breach-notifications-increase-to-all-time-high-in-2025,-new-ndb-stats-show
- OAIC, Data breach preparation and response, Part 4, Notifiable Data Breach (NDB) scheme. https://www.oaic.gov.au/privacy/privacy-guidance-for-organisations-and-government-agencies/preventing-preparing-for-and-responding-to-data-breaches/data-breach-preparation-and-response/part-4-notifiable-data-breach-ndb-scheme
- OAIC, Guidance on privacy and the use of commercially available AI products, 21 October 2024, updated 17 January 2025. https://www.oaic.gov.au/privacy/privacy-guidance-for-organisations-and-government-agencies/guidance-on-privacy-and-the-use-of-commercially-available-ai-products
- Privacy Act 1988 (Cth), sections 6(1), 26WE, 26WH, 26WK, 26WL and 26WM.
TheAICommand. Intelligence, At Your Command.
