AI Governance for Mid-Sized Organisations: Where to Start

Large enterprises are building AI governance functions with dedicated teams, committees and extensive frameworks. Mid-sized organisations often look at that and conclude that AI governance is not for them yet. Meanwhile, staff are using generative AI tools to draft emails, summarise documents and analyse data, suppliers are adding AI features to the software you already use, and someone in the business is planning a pilot.

AI governance does not have to start big. It has to start. This article sets out a practical first year for an organisation without a dedicated AI team, focused on the steps that reduce the most risk for the least effort.

Why governance matters now

AI introduces risks that existing policies may not cover:

• Data leakage. Confidential or personal information pasted into public AI tools.
• Inaccurate outputs. AI-generated content that is confidently wrong, used without checking.
• Unfair outcomes. Models that treat groups of people differently in hiring, lending, pricing or service.
• Lack of explanation. Decisions that cannot be explained to the people affected.
• Regulation. Requirements are growing. The EU AI Act, for example, introduces obligations in phases, with the strictest rules for high-risk uses. Other jurisdictions are developing their own approaches.
• Supplier risk. AI features in third-party software that process your data in ways you have not assessed.

Governance is how an organisation makes sure it gets the benefits of AI while managing these risks deliberately.

Step 1: Find out what is already in use

You cannot govern what you cannot see. Start with a simple inventory:

• Generative AI tools staff are using, approved or not.
• AI features in existing software, such as CRM, HR, finance or customer service platforms.
• Internal projects using machine learning or AI models.
• Supplier services that rely on AI to make or support decisions.

A short survey of department heads, a review of software contracts and a conversation with IT will cover most of it. Record each use with a named owner, a short description and the data involved.

Step 2: Publish a short acceptable use policy

Your first policy should be short enough that people actually read it. It should cover:

• Approved tools, and how to request new ones.
• What data must never be entered into AI tools, such as personal data, confidential client information and commercially sensitive material, unless the tool is approved for it.
• Human checking. AI outputs must be reviewed by a person before they are relied upon or sent externally.
• Transparency. When people must be told that AI was used, for example in customer communications.
• Who to ask if unsure.

A one- or two-page policy, explained in a short briefing, does more good than a long document nobody reads.

Step 3: Sort uses by risk

Not every use of AI needs the same scrutiny. Drafting an internal email is very different from screening job applicants. A simple three-tier approach works well:

• Low risk: productivity uses with human review and no significant impact on individuals, such as summarising documents or drafting text. Covered by the acceptable use policy.
• Medium risk: uses that affect customers or operations, or involve sensitive data, such as customer-facing chatbots or demand forecasting. Need a short assessment and an owner.
• High risk: uses that significantly affect people's rights, opportunities or safety, such as hiring, credit, healthcare or decisions about access to services. Need a thorough impact assessment, testing for bias and accuracy, human oversight and senior approval.

Most of your uses will be low risk. The point of tiering is to focus effort on the few that are not.

Step 4: Make someone accountable

AI governance fails when it belongs to everyone and no one. Assign:

• A senior owner for AI governance overall, often the chief operating officer, chief information officer or chief risk officer in a mid-sized organisation.
• A business owner for each medium and high-risk use, responsible for its performance, risks and compliance.
• A small working group drawing on IT, data protection, legal, risk and the business, meeting regularly to review new uses and issues.

You do not need a new department. You need clear names against clear responsibilities.

Step 5: Build oversight into the lifecycle

For medium and high-risk uses, add simple checkpoints:

• Before: an assessment of purpose, data, potential harms and how decisions will be explained.
• Before going live: testing for accuracy and, where relevant, fairness across groups.
• In use: monitoring for drift, errors and complaints, with a way for people to challenge decisions.
• Periodically: review whether the use is still appropriate and performing as expected.

Step 6: Check your suppliers

Many of your AI risks sit with suppliers. Add a few questions to procurement and contract renewals: does the product use AI, what data does it use, is your data used to train models, how are outputs tested, and what controls do you have.

Useful references

Several established frameworks can help as your governance matures, including the NIST AI Risk Management Framework, ISO/IEC 42001 for AI management systems, and guidance from data protection regulators on AI and personal data. You do not need to adopt any of them in full at the start, but they are useful for checking that you have not missed anything important.

Governance and readiness go together

Good governance is not only about avoiding harm. Organisations that govern AI well are usually better at using it, because they know what is in use, what works and where the risks lie. The AI Readiness assessment looks at strategy, data foundations, skills, technology, ethics and adoption, and pairs naturally with governance.

To see how your current approach compares, the AI Governance assessment covers policy and regulatory compliance, risk classification and impact assessment, accountability and oversight, transparency, fairness and ethical review, and lifecycle governance.

Frequently asked questions

Should we ban generative AI tools until we have a policy?

Bans are hard to enforce and usually push use underground. It is generally better to approve a small number of tools quickly, set clear rules about data, and expand from there.

Do we need an AI ethics committee?

Not necessarily at the start. A small working group with the right mix of expertise can review medium and high-risk uses. A formal committee may make sense as the number and significance of AI uses grows.

How long does it take to set up basic AI governance?

An inventory, a short policy, a risk-tiering approach and named owners can be in place within three to six months for most mid-sized organisations.