AI Data Privacy: What You Need to Know
How the Privacy Act and Australian Privacy Principles apply to AI projects. What you need to know before going live with an AI system.
How the Privacy Act and Australian Privacy Principles apply to AI projects. What you need to know before going live with an AI system.
Australia does not have dedicated AI legislation yet. But that does not mean AI is unregulated. The Privacy Act 1988 and the Australian Privacy Principles (APPs) apply to any system that handles personal information, including AI systems.
The OAIC (Office of the Australian Information Commissioner) has made it clear: existing privacy obligations apply to AI. If your system collects, uses, or stores personal data, you need to comply.
The Privacy Act covers organisations with annual turnover above $3 million (and some smaller organisations in specific sectors). Key principles relevant to AI:
The 13 APPs that matter most for AI projects:
Cross-border trap: Using OpenAI's API, Google Gemini, or other US-hosted services means your data likely crosses borders. Under APP 8, you remain accountable for how that data is handled overseas.
Yes, the Privacy Act 1988 and the Australian Privacy Principles (APPs) apply to any system that handles personal information, including AI systems. Australia does not yet have dedicated AI legislation, but the Office of the Australian Information Commissioner (OAIC) has made it clear that existing privacy obligations apply to AI. Consent, purpose limits, data security and cross-border transfers all need attention.
Sending personal data to OpenAI, Google Gemini or other AI services hosted in the United States likely means the data crosses borders. Under Australian Privacy Principle 8, the business stays accountable for how that data is handled overseas and must ensure equivalent protections. Check where your AI provider processes and stores data, and keep data in Australia where you can.
The Australian Privacy Principles (APPs) that matter most for AI are APP 3 on collection, APP 5 on notification, APP 6 on use and disclosure, APP 8 on cross-border disclosure and APP 11 on security. In practice, only feed personal data into AI for a legitimate reason, tell people how AI will use it, stay within that purpose, and protect it.
Retrieval augmented generation (RAG) is easier to manage for privacy than fine-tuning. Personal data used to fine-tune a model becomes embedded in the model weights and is effectively irrecoverable. In a RAG system the data stays in your vector database, not in the model, which makes deletion and access requests from individuals easier to handle.
Start by mapping the personal data that flows through the AI system and running a privacy impact assessment (PIA) before deployment. Include only the personal data the AI needs, anonymising where possible, and keep data in Australia, for example in the Amazon Web Services (AWS) Sydney region. Add document-level access controls, review the vendor's data handling, and update your privacy policy.
Ask the author
Ask it here and it comes straight to the founder. No sales call, no obligation, and a real answer even if the answer is that you do not need us.
Kasun Wijayamanna
Founder, replies within one business day
Tell us what you are working on. We will come back with a practical recommendation and clear next steps.
Built here. Your data stays here.
Thanks for reaching out. We will get back to you within one business day.
See what else we do