The model matters.

It is not the whole system.

A model becomes operationally important when its output enters a decision, changes a record, communicates with a person, or triggers an action.

That is where governance should begin.

Not with a generic question about whether the model is safe.

With a specific question about what the system can influence and what happens when it is wrong.

The same model can create different risks

A general-purpose model may support several uses inside one company.

It can summarize a public document.

It can draft an internal email.

It can recommend which customer case receives attention first.

It can approve a refund.

It can update a system of record.

The underlying model may be the same.

The operating risk is not.

Risk changes with:

  • The data entering the system
  • The person affected
  • The consequence of error
  • The authority granted to the output
  • The ability to detect and reverse a mistake
  • The level of human review
  • The frequency and scale of use
  • The regulatory and contractual context

A model inventory is useful.

A use-case and decision inventory is more actionable.

Name the decision first

A governable AI use case should state the decision in plain language.

Examples:

  • Recommend which service tickets require urgent review.
  • Draft a customer response for an employee to approve.
  • Extract fields from a document into a pending record.
  • Identify transactions that require investigation.
  • Propose a next action for a sales representative.
  • Execute a refund below an approved threshold.

The wording should make authority visible.

Analyze customer data is not a decision.

Recommend which customers receive retention outreach is closer.

Automatically send a retention offer and update the customer record is a materially different system.

The decision statement should identify:

  1. What the system produces
  2. Who or what receives it
  3. Whether the output is advice, approval, communication, or action
  4. Who remains accountable
  5. What can be reversed

This creates the foundation for the rest of governance.

Governance follows consequence

Not every AI use deserves the same control process.

A low-consequence drafting aid should not require the same review as a system influencing employment, credit, healthcare, safety, legal rights, or regulated records.

A useful governance model increases control as consequence and authority increase.

Assist

The system helps a person create or find information. A person remains responsible for the output before it affects another party.

Recommend

The system ranks, scores, or proposes a decision. A qualified person evaluates the recommendation.

Approve within limits

The system can complete a defined decision inside thresholds, with logging, monitoring, and exception handling.

Act across systems

The system can change records, communicate externally, or trigger downstream work. Reversal, access, monitoring, and stop authority become central.

The label is less important than the principle.

Control should follow what the system can do.

Data governance and decision governance are connected

Organizations often separate the data question from the decision question.

They approve access to a data set, then treat the use as governed.

Data access is only one part.

The same data may be acceptable for one purpose and inappropriate for another. A customer record used to summarize a service history is different from the same record used to infer vulnerability or prioritize access.

The governance record should connect:

  • Data source
  • Data classification
  • Intended purpose
  • Decision or action
  • Affected people
  • Retention
  • Access
  • Review
  • Monitoring
  • Incident response

This makes purpose part of the control system.

It also makes purpose drift visible.

A system that begins as an internal assistant can gradually become a decision engine if new integrations and permissions are added without a new review.

Human oversight needs a defined job

Human in the loop is not a complete control.

The person needs a specific responsibility.

Are they checking factual support?

Applying policy?

Reviewing an exception?

Approving a consequential action?

Monitoring the system across cases?

The reviewer needs enough context, time, authority, and training to do that work.

The system also needs to record what happened.

If people regularly revise or reject the same type of output, that pattern should reach the system owner.

Case-level review and system-level ownership are different governance functions.

Both matter.

Testing should reflect the operating context

A benchmark can show model performance under defined conditions.

Production introduces users, incentives, data quality, integrations, workload, and unexpected behavior.

Testing should therefore include the system in context.

Ask:

  • Does the system work on the actual data distribution?
  • Which groups or cases experience more errors?
  • What happens with missing or conflicting information?
  • Can users recognize uncertainty?
  • Does the interface encourage overreliance?
  • What happens when a connected system is unavailable?
  • Can a harmful action be reversed?
  • Does the fallback still work?
  • How does the system behave after a model or prompt change?

NIST’s AI Risk Management Framework organizes work around governing, mapping, measuring, and managing risk. The map comes before the metric.

A team has to understand the context, purpose, people, and consequences before it can choose useful tests.

Build one governance record per use case

A practical governance record can be concise.

It should contain:

Purpose

What business problem is being addressed?

Decision or action

What can the system recommend, approve, communicate, or change?

Owner

Who is accountable for the operating outcome?

Data

What information enters, leaves, and remains?

Affected parties

Who can experience benefit or harm?

Human role

Where does human judgment enter, and what authority does it carry?

Controls

What limits, logging, access, testing, and fallback apply?

Monitoring

What evidence will be reviewed after launch?

Stop condition

Who can pause or retire the system, and under what conditions?

The record should change when the use changes.

A new model version may require review.

A new authority level definitely does.

What this does not mean

Decision-centered governance is not a reason to ignore model quality, security, provider risk, intellectual property, or technical evaluation. It is a way to connect those controls to the business context that determines their consequence.

Governance should create a faster safe path

Poor governance is experienced as delay because the organization has not defined its lanes.

Every use case receives the same questions. Ownership is unclear. Reviews occur late. Low-risk work waits beside high-consequence systems.

A better system creates standard paths.

For example:

  • Approved drafting and research tools for public or low-sensitivity information
  • Controlled enterprise tools for internal information
  • A documented review for systems using confidential or regulated data
  • Deeper assessment for consequential decisions or autonomous action
  • Predefined technical and operating controls by authority level
  • A fast intake path for new use cases

The objective is not to remove review.

It is to make the review proportional, predictable, and early enough to shape the design.

The decision-governance test

Before approving an AI use case, ask:

  1. What decision or action can the system influence?
  2. Is the output advice, approval, communication, or execution?
  3. Who is accountable for the outcome?
  4. What data is used, and for what purpose?
  5. Who can be affected?
  6. What is the consequence of a wrong output?
  7. Where does human judgment enter?
  8. Can the action be detected, explained, and reversed?
  9. What evidence will be monitored after launch?
  10. Who can pause or retire the system?

If the team cannot answer the first question, it is not ready to govern the rest.

Conclusion

AI governance should not stop at the model boundary.

The model becomes part of an operating system that includes data, people, interfaces, workflows, integrations, decisions, and consequences.

Start with the decision.

Then govern the model and every surrounding component according to the authority the system receives and the harm it can create.

The most useful governance question is not only, Which model are we using?

It is, What are we allowing this system to decide?

Sources and notes

  • NIST AI Risk Management Framework, a voluntary framework for incorporating trustworthiness considerations into the design, development, use, and evaluation of AI systems through Govern, Map, Measure, and Manage functions.
  • NIST Generative AI Profile, on intended use, inventories, third-party risk, human oversight, testing, monitoring, incident response, change management, and decommissioning.
  • NIST AI RMF Playbook, on suggested actions for operationalizing the AI RMF across the system lifecycle.