Written by: Dr Daneél van Eck, Strategy Director, epic ERP
Here’s a question worth considering: Do you know which AI tools your people are actually using right now? Not just the ones IT has approved. Not just the ones listed in your software inventory. Which tools are your teams genuinely using – today, to do their jobs – and what information is going into them?
For most organisations, the honest answer is: probably not.
AI has moved fast – faster, in most cases, than governance. ChatGPT, Claude, Microsoft Copilot, Google Gemini, DeepSeek – these tools are brilliant, free or nearly free, and they make people meaningfully more productive. So people use them: quietly, efficiently, and often without telling anyone.
Consider the case of an accountant using ChatGPT to summarise and add insights to the board pack, a developer who pastes error logs into Claude to debug faster, or a developer who sends their whole code base to Claude Code to match standards for a new section. Your HR head could use Gemini to draft sensitive communications. They are each solving problems with the best tools available.
What worries CIOs and compliance officers: most organisations lack a clear, real-time view of where, how often, and what kind of sensitive data enters AI tools. When the data involves finance, personnel, or customer details, risk is immediate.
“The question is no longer whether your people are using AI. It’s whether you have any visibility into how – and at what risk.”
Where the risk actually lives
AI risk comes not from tools, but from the environments they operate in and assumptions about data once it leaves the organisation.
Public AI platforms are external services. Unless your organisation has a formal enterprise agreement with defined data-handling terms, you cannot assume that what your employees type into a prompt remains private.
Inputs may be logged, stored, or used to develop future model versions. Enterprise agreements with most major providers do offer real protections, but only if they’re actually in place. A free or consumer account with the same provider gives you none of that.
Two incidents illustrate this well. In 2023, Samsung engineers pasted proprietary source code into ChatGPT to solve a problem faster, triggering a company-wide ban on generative AI on internal devices. Separately, GitHub Copilot – a widely used AI coding assistant – was found to suggest passwords and API keys it had learned from public code repositories, some of which contained credentials that were never meant to be public. In both cases, the risk wasn’t the tool. It was the lack of any framework around how the tool was being used. For South African organisations, POPIA adds a further dimension. If personal information is flowing into public AI tools without appropriate safeguards, that’s a compliance exposure, not just an IT concern.
AI tools generate probable outputs, not facts; they can hallucinate or produce biased or ambiguous content. Relying on “AI-generated” is not quality control.
And then there’s accountability – the risk most frameworks miss entirely. If the financial analysis is AI-generated/assisted, who takes responsibility? Or if a message to a customer is generated, who owns that outcome? Using AI could end up greying who is ultimately accountable.
What getting it right looks like
Responsible AI governance isn’t a policy document – though a policy is part of it. It’s a layered system in which four factors tend to separate organisations that manage this well from those that don’t.
Data discipline
Clear standards for what can and cannot enter AI environments. Sensitive information should be masked or pseudonymised before it is sent to an external tool. A rule of thumb that travels well: if you wouldn’t send it in an unencrypted external email, don’t paste it into a public AI tool.
Coherent permission structures
AI inherits the environment’s access controls. If your system permissions are poorly structured or have drifted from actual job functions. AI won’t fix that; it will amplify it. Implementations consistently surface pre-existing governance weaknesses. Better to find them before deployment than to discover them after deployment through an incident.
Cultural maturity alongside technical capability
The shift that matters isn’t from “AI is exciting” to “AI is dangerous.” It’s from “AI is exciting” to “AI is powerful, which means it needs structure.” Innovation without structure doesn’t contain risk. It scales it.
The bigger picture
Make AI governance a priority. Actively commit to sustained oversight, assign clear accountability and reconsider your frameworks. Begin building your organisation’s governance capacity today! Don’t wait until a critical incident exposes the gaps.
For organisations where enterprise systems are the backbone of operations – where finance, procurement, inventory and customer data all flow through a central platform – the stakes of getting this wrong are higher than average. The more integrated your systems, the more carefully AI needs to be governed at the boundary.
Start laying the foundations for your AI governance now. Don’t wait to react. Take ownership. Evaluate your current visibility, policies and team awareness, then set the next step for governance improvement today.
About Author
Dr Daneél van Eck is Strategy Director at epic ERP, a Microsoft Dynamics & Epicor implementation partner operating across Africa. His work focuses on the intersection of enterprise systems architecture, AI governance, and digital strategy.