How the AI works
Language models do the reading. Rules decide what may be said.
Mapping hundreds of differently worded questions to the right evidence is not possible with templates or keyword search. It is possible with retrieval and language models, if every output can be checked. This is the pipeline, stage by stage.
Ingest and chunk
Text is extracted from PDF, DOCX, Markdown, text, and past questionnaires. Headings are tracked as a path, and paragraphs are packed into passages of about 800 characters that never cross a section boundary.
Embed
Each passage gets a 512-dimension vector from OpenAI text-embedding-3-small, or from a deterministic local embedder when no key is configured. The model ID is stored with every vector, and search only compares vectors from the same model.
Retrieve evidence
Hybrid retrieval: BM25 over stemmed terms, with security vocabulary normalized (MFA, multi-factor, and two-factor are one concept), combined with vector similarity. The top passages per question move on.
Score coverage
Coverage measures how much of what the question asks appears in the best passages, weighted so rare terms count most. Below 0.45, the question is a gap. A named framework no source mentions (FedRAMP, HIPAA, ISO 27001) is always a gap.
Draft in your voice
The model sees only the numbered passages and your approved voice guide, and must return JSON: answer, citations with verbatim quotes, confidence, status, and a reason for any gap. The reply is validated with a schema; anything else is rejected.
Verify every quote
Each quote is matched against its passage, character for character apart from whitespace. Citations that do not match are dropped. An answer left with no verified citation is withheld and becomes Needs owner.
Score confidence
Confidence is the lower of the model's estimate and a ceiling set by evidence coverage, so a fluent answer on thin evidence cannot score high. Under 0.80 it needs a reviewer before export.
Route gaps, learn from approvals
Gaps go to an owner with a due date and a thread. Approved answers enter the reviewed answer library with their citations, and close matches are reused on the next questionnaire when their sources still exist.
What happens when evidence is missing
The same question, two outcomes.
Q-138Annual third-party penetration testing?
CitedSources include the vulnerability management policy, which names an independent firm and an annual cadence. Coverage 0.96, quote verified, confidence 0.97.
Q-138Annual third-party penetration testing?
Needs ownerSame question, with that policy removed. No passage in your sources covers penetration testing. No model is asked to fill the gap.
Model-agnostic
Your rules stay fixed. The model is replaceable.
Drafting and embeddings go through a thin layer with one function for each. Providers are tried in the order set by AI_PROVIDERS: OpenAI by default, Anthropic as the alternate. If a provider fails, the next one answers. If none is configured or all fail, Attestloom falls back to extractive mode, which assembles answers verbatim from the highest-coverage source sentences under the same gap and confidence rules. Extractive mode never guesses; it can only quote.
Requests carry only the question and the retrieved passages. Customer content is never used for training, by us or by providers, and OpenAI requests are sent with storage disabled.