OPMXI Lab / 02
AI Product
Technical reference · synthetic data
Intelligence Workspace
A source-grounded knowledge experience with permissions, review, and useful product states.
What changed in the operating policy?
The revised policy introduces a designated review threshold and requires evidence to remain attached to every material decision.
Answer boundary / verified
Approved internal sources only. Restricted legal guidance excluded.
The operating problem
Designing for real consequence.
A useful knowledge assistant must do more than generate fluent text. People need source visibility, scope awareness, and a clear path when confidence is insufficient.
System response
A controlled product, not a decorative screen.
The workspace pairs every answer with evidence, permission context, tool activity, feedback, and an explicit escalation path.
Architecture
How the system separates intelligence from authority.
Identity and scope
The request begins with user identity, role, workspace, and purpose. Retrieval can only consider content the caller is already permitted to access.
Retrieval pipeline
Hybrid lexical and semantic search produces candidate passages. Metadata filters enforce tenancy, document status, and effective dates before ranking.
Answer construction
The model receives a bounded evidence packet and an explicit answer contract. Statements without support are omitted or identified as unresolved.
Citation verification
Returned citations are checked against source spans before rendering. Broken or irrelevant references fail the answer rather than appearing authoritative.
Tool boundary
External actions use typed inputs, scoped credentials, confirmation rules, and visible execution states. Tool output is added to the conversation as evidence.
Evaluation loop
Feedback is joined to retrieval traces and answer versions. Evaluation separates retrieval quality, citation quality, usefulness, and policy compliance.
Interface states
The product makes system state inspectable.
Grounded answer
Designed with explicit loading, empty, permission, error, and recovery behavior rather than a single idealized screen.
Source preview
Designed with explicit loading, empty, permission, error, and recovery behavior rather than a single idealized screen.
Tool activity
Designed with explicit loading, empty, permission, error, and recovery behavior rather than a single idealized screen.
Escalation flow
Designed with explicit loading, empty, permission, error, and recovery behavior rather than a single idealized screen.
Decision log
Tradeoffs made explicit.
Permission-filter before ranking
Filtering after retrieval can leak titles, snippets, or semantic similarity from restricted material.
Rejected alternative
Retrieve across the corpus and hide inaccessible citations only in the final interface.
Show answer boundaries
A clear statement of included sources, excluded domains, and unresolved questions helps users calibrate trust.
Rejected alternative
A single confidence score, which compresses different failure modes into an unexplained percentage.
Evaluate retrieval separately
A fluent answer can conceal a retrieval failure. Separate measures identify whether the corpus, search, prompt, or model needs work.
Rejected alternative
Using thumbs-up feedback as the only quality signal.
Failure design
The unhappy path is part of the product.
- No authorized evidence produces a bounded no-answer state with escalation options.
- A stale policy version is visibly superseded and excluded from the answer context.
- Tool calls that require confirmation remain drafts until the user approves typed parameters.
- Citation verification failure suppresses the response and records the trace for evaluation.
Measurement plan
What would be measured in production.
- Retrieval recall at a fixed candidate depth
- Citation entailment and source-span accuracy
- Task completion with and without escalation
- Permission-policy test pass rate
- No-answer precision and inappropriate-answer rate