AI governance discussions often focus on frontier models and public chatbots. That focus is understandable, but it can create a blind spot. Many practical AI risks will not come from a single impressive model sitting in a central platform. They will come from smaller specialised models embedded inside business applications, devices, workflows, analytics tools and vendor products.
The Stanford 2026 AI Index reports continuing improvements in model capability, falling inference costs and wider diffusion of AI across sectors. Those trends make AI easier to embed. A capability that once required a specialist system can become a feature inside a productivity suite, customer platform, security tool, claims workflow, HR system or compliance product. For governance teams, the question becomes harder: how do you govern AI when AI is no longer obvious?
The shift from visible AI to embedded AI
Visible AI is easy to recognise. An employee opens a chatbot, enters a prompt and receives an answer. Embedded AI is quieter. It may classify a ticket, rank a lead, flag an anomaly, summarise a call, suggest a next best action, redact a document, predict attrition, detect fraud or prioritise a queue. The user may experience it as a normal software feature rather than a model output.
This matters because many organisations built their first AI policies around public generative AI tools. Those policies usually tell employees not to enter sensitive data, to review outputs and to avoid using AI for decisions without approval. That is useful, but it does not cover AI that arrives through procurement, software updates, vendor add-ons or device-level features.
Governance needs to follow the function, not the label. If a tool infers, classifies, recommends, generates or acts in a way that influences work, it should be in scope.
Smaller does not mean lower risk
Small models can be useful because they may be cheaper, faster, more specialised and easier to deploy in controlled environments. They can also support privacy or latency goals when run locally. However, smaller does not automatically mean safer. A small model used in a sensitive process can create more risk than a large model used for low-stakes drafting.
The OECD definition of an AI system describes a machine-based system that infers from inputs how to generate outputs such as predictions, content, recommendations or decisions that can influence environments. This definition is useful because it does not depend on model size. A compact classifier that influences a customer decision, employee assessment or risk alert is still an AI system.
This is the key governance lesson. Model capability is only one dimension of risk. Context is usually more important.
Procurement is becoming an AI control point
If AI is embedded in ordinary software, procurement becomes one of the most important governance controls. Business teams may buy a platform for case management, learning, security or customer engagement without realising that AI features are enabled by default or available through a simple toggle. A vendor may also add summarisation, recommendation or automation features after contract signature.
APRA's April 2026 letter to industry highlights third-party dependencies, operational resilience and AI-related cyber pathways such as prompt injection, data leakage and agent misuse. Although the letter is addressed to regulated financial services entities, the broader lesson applies widely. AI risk often travels through vendor systems.
A practical procurement process should ask whether the product contains AI, whether AI features are optional, what data is processed, whether customer or employee data is used for training, where data is stored, how outputs are logged, how model changes are notified, whether human review can be configured and what happens if the AI feature is disabled.
Procurement teams do not need to become model experts. They need an AI intake process that sends the right use cases to the right review pathway.
Inventories need to include embedded systems
Many organisations are building AI registers. The risk is that these registers capture only the obvious tools employees nominate. To be useful, an AI inventory should pull from multiple sources: procurement records, software asset management, browser extensions, cloud services, vendor roadmaps, data connectors, technology architecture reviews and business process maps.
The Australian voluntary AI Safety Standard includes guardrails on accountability, risk management, data governance, testing, transparency, human oversight and supply chains. These guardrails can only be applied if the organisation knows where AI is being used. An incomplete inventory undermines every later control.
The inventory should also record whether the AI is visible to users. Hidden or low-visibility AI requires stronger transparency controls, especially where employees, customers or other stakeholders may be affected.
Governance should be lightweight but persistent
The answer is not to block every embedded AI feature until a full committee review is complete. That would be unrealistic and counterproductive. The better approach is a tiered governance model. Low-risk embedded AI can proceed with standard controls. Medium-risk use needs registration, owner approval and testing. High-risk use needs formal assessment, human oversight, monitoring and assurance.
The NIST AI Risk Management Framework describes the need to govern, map, measure and manage AI risks. Embedded AI makes the mapping step especially important. Organisations need to understand context before they can decide the right control depth.
A persistent process matters more than a one-off review. Embedded AI will keep changing through software updates, vendor releases and workflow redesign. Governance should therefore include periodic review and change triggers.
The bottom line
The next AI governance blind spot is not only the newest frontier model. It is the quiet spread of smaller models and embedded AI features across everyday systems. These tools may be practical and valuable, but they can still influence decisions, expose data and reshape accountability.
The safest governance rule is simple: if a system infers, recommends, generates or acts in a way that matters, treat it as AI regardless of model size or marketing label.
References
- Stanford HAI, 2026 AI Index Report
- OECD AI system definition
- APRA letter to industry on artificial intelligence, 30 April 2026
- Australian Government Voluntary AI Safety Standard
- NIST AI Risk Management Framework
TheAICommand. Intelligence, At Your Command.
