Notes · Operations
Internal AI operations: approvals before autonomy.
An internal assistant with broad access and no audit trail is a liability. The useful version starts narrow: authoritative documents, scoped tools, and a human approval on every irreversible action.
The three failure modes
Most internal AI pilots fail in one of three ways: the assistant can read everything, it can change systems without approval, or it answers from stale documents. Each is a design failure, not a model failure.
Broad permissions make a demo easy and an audit impossible. Unapproved writes turn a helpful assistant into an incident. Stale retrieval produces confident answers from last year's process.
Start with one workflow
Pick a workflow with clear inputs, a named owner, and a measurable outcome. Vendor onboarding, procurement questions, or internal support requests work well. Avoid “help the team with everything.”
- One team, one workflow, one measurable task.
- Documents that are actually authoritative.
- Systems the workflow already uses.
- A named human owner for approvals.
Permission model
Mirror the permissions people already have. The assistant should not inherit an admin key because it was easier to connect.
| Layer | Rule |
|---|---|
| Retrieval | Only documents the user can already read. |
| Read tools | Scoped to the workflow and rate limited. |
| Write tools | Separate permission, approval required. |
| Admin tools | Not available to the assistant. |
Approvals, not autonomy
Draft, propose, and queue. A person approves any action that changes money, contracts, access, or customer records. The assistant can prepare the change and show its sources; it does not execute unsupervised.
Approval fatigue is real. Batch low-risk actions and require individual approval only where the cost of a mistake is high.
Retrieval freshness
An assistant is only as current as its index. Give every source a date and an owner. Re-index when a document changes, and mark stale sources as stale rather than letting the model guess.
- Index the authoritative version, not every copy.
- Record last-modified date and owner per document.
- Expire documents that have no owner.
- Make the assistant say when it cannot find a current source.
Audit log fields
Log enough to reconstruct a decision: user, question, retrieved sources, tool calls, approval, and result. Keep it queryable for incidents and review.
| Field | Why |
|---|---|
| User and role | Permission and accountability. |
| Prompt and sources | Reproduce the answer. |
| Tool calls | See what was read or changed. |
| Approval | Who authorized the action. |
| Outcome | Success, rollback, or incident. |
Measure adoption, not demos
Judge the pilot on weekly active use, tasks completed without intervention, approval time, and error rate. A tool nobody opens has failed, even if the demo was excellent.
The pilot succeeds when the team keeps using it after the novelty ends.
Pilot checklist
| Item | Pass condition |
|---|---|
| Scope | One workflow with a named owner. |
| Access | Permissions mirror the user, not admin. |
| Approvals | Irreversible actions require a person. |
| Freshness | Sources dated, owned, and re-indexed. |
| Audit | Every action can be reconstructed. |
| Adoption | Weekly active use, not launch-day logins. |
Next
If internal operations is the bottleneck, start with one workflow and an access map.