---
title: Requirement gathering
description: Source-linked requirement extraction, scoring and review for a classified enquiry.
---

The worker's `pnpm --filter @flowos/worker gather:requirements` tool previews
requirements from processed enquiries in the testai mailbox. In the product,
gathering starts by itself for an enquiry whose saved reading confirms at least
one catalogue category, and an operator can process one email again from the mail
debug page, which gathers it afresh. An email without a confirmed category is not
linked as an enquiry and cannot be gathered. Classification and requirement-gathering model calls have
independent pause controls; both start paused, and a queued requirement job waits
until its calls are resumed. Results are stored against the linked enquiry and
reviewed in the inbox. An enquiry enters Active only after a job saves
successfully.

Source structuring (Phase 1) and catalogue extraction (Phase 2) use
`openai/gpt-6-luna` through the Vercel AI Gateway, configured with
`AI_GATEWAY_API_KEY`, and so does the reading of worded values in unit
standardization (Phase 3), and the naming of chemical formulas in fluid wordings
the library cannot match, which is sent only the formulas and their joins and
whose failure, a rate limit included, leaves the wording to the usual matching. Catalogue extraction uses
native structured output. Structuring and extraction are sent to Amazon Bedrock
first and, when it cannot take them, to OpenAI and then Azure, in that order,
through the gateway. Semantic verification (Phase 4) uses TypeSafe Jev,
pinned to release 1.14 (`typesafe/jev-1.14.0`); a newer Jev is taken up only when FlowOS is
updated to it. A provider error that remains once all three have been
tried, or a schema error, stops gathering. Picking an enquiry sends its text and attachments to
these models without a further confirmation.

## When gathering starts

Gathering is queued for an email as soon as it is read as an enquiry and linked,
when all of these hold:

- requirement gathering is switched on;
- gathering covers every mailbox the email was delivered to;
- the enquiry has no requirements yet.

An enquiry that has requirements keeps them. A later email of the same
conversation that is also read as an enquiry, such as a revision or an addendum,
shows in the conversation at once, and nothing is gathered from it until an
operator processes it again from the mail debug page. Requirements are never replaced
without a person. Two first emails read moments apart follow the same rule: the
one gathered first gives the enquiry its requirements, and the other shows with
nothing gathered from it.

An email read while requirement gathering is paused waits, hidden from the
inbox. Resuming queues it at once, and while gathering is on the worker looks
every five minutes for read enquiries that have no requirements and queues them,
the oldest first, behind the gathering of newly read mail. It looks only among
mail that gathering covers, so mail to other mailboxes never holds it up. Mail
read before gathering started by itself is picked up the same way. An email whose gathering failed is in Needs Review and is not queued
again by itself: gathering starts by itself once for each reading of an email,
and after a failure only **Process again** on the mail debug page tries again.
While that job waits or runs, the email is not sent back to Needs Review for the
earlier failure.

## Processing an email again from the mail debug page

**Process again** on the mail debug page processes the selected email afresh,
whatever its kind. It is classified again rather than from its saved
classification; its pictures, labels and reading are done again; and it is
linked to its enquiry and routed as usual. When it is still an enquiry, its
requirements are gathered from it and replace the enquiry's current ones. An
email no longer classified as an enquiry finishes as its new kind and keeps its
enquiry link and requirements. The action is held back while model calls are
paused. While requirement gathering is paused the email is processed and its
gathering waits, still to replace the current requirements once gathering
resumes. A gathering already running for the same reading is left to finish;
process the email again after it to replace what it gathered. An email mail is
still storing cannot be processed again.

Until a zero-retention agreement covers the gathering models, only an email
delivered solely to the test mailbox can be gathered. Any other email is not
queued by itself, an enquiry email is refused when an operator processes it
again, and the job refuses it again before anything is sent.

A job gathers each confirmed category that holds a requested item, and a category Jev is unsure of whose product the email's subject names, even when Jev is sure of no other category, such as "Metal Tube Rotameter Enquiry". A product named only in the body or an attachment is not enough. A confirmed
category with no requested item is left out, and so is an attachment with no
readable content, such as a drawing file; the job names each one it left out. The
requirements, their source excerpts and the phase results are saved
together or not at all, and a job delivered twice saves once.

Menu values are matched to the catalogue's options as the line is saved (R76): an option's
own name or another wording of it first; on a menu that only says whether a part is fitted,
such as switches Required or Not Applicable, wording that names or details the part takes the
fitted option, keeping the wording as its note, and wording that declines it the other; then
the reading model, which must be more than 80% sure. A menu value no option means is saved as Custom with the customer's wording, or, where
the menu has no Custom, with no option and any nearest options the reading model offered.
An accessory no option means is never Custom; it keeps the customer's wording.

A job that fails stays on the debug page with the step it failed at and the
reason. A failure that would repeat, such as a refused email or a model answer
that does not fit the catalogue, is not tried again. A provider rate limit pauses
gathering for a minute. Anything else, a model answer that cannot be read at all
included, is tried three times. A job cut off by the worker stopping, as when it
restarts, starts again, up to three times.

A reading whose lines come back from the email or a document without the text of any row is a failed job, not a result: a PDF sent as a file is read again from its converted text, and if that is empty too, or the empty part is not such a PDF, nothing is saved and the enquiry keeps its current requirements.

Gathering that starts by itself never replaces requirements. Processing an
email again does, review work included: lines a reviewer confirmed or excluded,
fields a person set and a client's answers filled into lines are discarded,
along with review done while its job waits. Once the current requirements hold
such work, processing again needs permission to edit the enquiry. A line
confirmed automatically (R71) is no reviewer's work and does not count, nor is
a client's change that still waits to be confirmed. A line a later email added
does count.

When a client's answer fills a line, the line's product family is looked for
again in what it now holds, if its own words named none, and the draft
quotations of the enquiry show their total and ready lines again once opened.
The same happens to the drafts when lines a client adds are gathered. None of
this gathers the enquiry again.

## A revised document

A document a client sends after the enquiry started is read against the lines the
enquiry already holds, not gathered as a second set. It is a revised document
when its name or heading carries a revision mark (R2, Rev, revised, updated);
one with no mark is read this way only before any quotation is issued.

Each line of the document is paired with a line of the requirements, by tag,
then by wording, and for a revised document by position within a category. Every
difference becomes a change on the line it is about: a new quantity, a stated
field with another value, or, for a revised document, a line it no longer holds.
A line the enquiry lacks is added as a new line before a quotation is issued, and
is listed on the revision list after. Changes wait to be confirmed before a
quotation is sent, on its revision list after, and are held after an order (R128).
No line is changed by the reading.

## Reviewing requirements

The inbox shows an enquiry's current requirements: its lines, the requirements
shared by several lines, additional information and what could not be mapped to
the catalogue, each with the excerpts that support it. A value is shown in the
unit the customer stated. An enquiry nothing has been gathered for shows as
empty.

A follow-up in the same thread joins the thread's enquiry, and so does the same
client's enquiry email about the same requirement on another conversation
([Inbox](/business/features/inbox#what-a-card-is)). Office-wide users see
ready enquiries assigned to their office; users limited to their own or their
team see only assignments in that scope. An enquiry awaiting gathering is
listed to no one in the inbox. Unassigned mail, and an enquiry with no confirmed category,
is shown only to people who read across all offices. If no eligible Office Head is available, an administrator can
assign the enquiry before review.

A reviewer who may edit corrects or adds a field; a reviewer who may approve
confirms or excludes a line. Editing a field of a confirmed line returns the line
to review, and a confirmation made on a line that has changed since it was
loaded is refused.

A menu offers only its catalogue options: those the line's product offers
first, with Custom, then the rest under "Not offered by this product".
Choosing Custom asks for the customer's wording. There is no free wording on a menu or a yes/no
field, and a yes/no field offers Required and Not required. Each value says
where it came from (see [Line items](/business/features/line-items)). An
Unresolved menu lists up to three options to pick; one click saves the pick, as
edited in review. A menu the line's product fills, as a Condition or a Default,
is saved only when a person changes it, and then as edited in review. An
accessories list is edited item by item, and cannot be saved while an item the
enquiry named has no option or an item's option is no longer on the menu.

Every line is saved awaiting review, except a line whose item Flowtech does not make, which is saved excluded with its reason; none is confirmed as it is saved (R71).
A line is confirmed without a reviewer only later, once its product suggestion
is read and nothing holds it, and then shows as auto-confirmed. A reviewer can
still open it, exclude it, or edit a value, which returns it to review; it
offers no Confirm until then (R75).

## Phase 1: Source structuring

The tool reads the email and its attachments, identifies instrument
categories, and creates a flexible source view before mapping requirements to
catalogue fields. The view retains requested items, original field labels, shared
requirements, scoped exceptions and additional information.

How an attachment is read depends on its kind:

- A PDF that has converted text, of at most 10 MB, is read from that text with
  the file beside it: every value is taken from the text, and the file decides
  which column heading each value belongs to.
- A larger PDF is read from its converted text alone.
- A scanned PDF the picture reader transcribed is read like a PDF that has
  converted text, from its transcription. A word or number the transcription
  marks `[illegible]` is read from the file instead, or left out when the file
  does not show it clearly; the marker is never taken as a value. One it has
  not transcribed, and a picture, are read from the file itself.
- Any other attachment is read from its converted text.

Of the email it reads the new text and, for a forward, the forwarded original.
Below a colleague's reply to colleagues that says nothing of its own, it reads
the client's words quoted there. Other quoted history and signatures are not
read.

Each block of the view cites its source by short verbatim excerpts rather than
copying whole passages. Blocks with the same scope are compacted.

A line that sits on a row of a table is then given that row as the document has
it: every cell under its column's heading, with the heading's unit and any
sub-heading such as Min., Nor. and Max. kept, and whatever else the reading gave
the line. A value the reading wrote differently from its cell gives way to the
cell. This is done only where the headings above the row agree with what the
reading itself called the row's cells, and the row states at least half of what
the reading gave the line. A table whose rows differ in width, a datasheet laid
out one property per row, and a document read from its file alone stay as read.

A line of a document read from its text then counts as read only when every word
and figure of it is passed on: in a line's fields or quantities, in a shared
requirement, or in the additional information. A line that falls short is kept
whole as an unresolved passage, whether or not a block cites it, so a table row,
a datasheet line or a clause the reading kept only in part still reaches the
next phase as the customer wrote it. A line item's own citations do not count as
passed on. The email body is passed on whole in any case, and a document read
from its file alone is checked against what was read from the file.

A schedule the reading left rows of without lines is asked for once more. A
schedule here is a table, in the email's own text or in a document read from its
text, with a column headed as a tag. Every unread row is named when at least two
rows state a tag and fewer than half are a line of the same source; otherwise a
row is named only when its tag is shaped like a tag a line holds, the same run
of letters and digits at the same length, as 90GCN02CF503 is like 90GTF30CF501.
A table with no tag column, such as the rest of a schedule a PDF's conversion
split from its heading at a page break, has a row named when it states such a
tag. A line's tag counts as the row's whatever its case, spacing or punctuation,
and a line that lists several tags counts for each. The
reading model is then asked a second time with the unread rows named, up to 40
of them, and its answer is kept when it leaves fewer of them unread and no
document with fewer lines. That second request is one more model call for the
enquiry, made again up to twice more when its answer cannot be read, as when the
model service answers empty. A rate limit on it makes the whole reading wait and run again, as a
rate limit on the first request does; any other failure, or an answer that
cannot be settled against the documents, leaves the first one standing. Where
rows are still not lines after that, the enquiry is saved
with what was read, and the document carries an unresolved note headed **Rows
not read as lines** that names them, beside their text among the unreferenced
passages. A cell under a tag heading unlike any tag a line holds, such as
"Documentation (2 sets)", is not named. A document the reading left out
altogether, whether read from its text or from its file, is asked for once more
by name. If the second answer leaves it out too, it is settled as one with no
lines: its unread rows are named the same way, its text is kept among the
unreferenced passages, and the gathering log says it was left out, so the
enquiry is still gathered from its other documents.

A line that is only an accessory for the source's other lines, such as an
interconnecting cable listed in a scope-of-supply table beside the instruments
it serves, is not kept as a line. It becomes a shared requirement headed
**Accessories listed as rows** on those lines, citing its row. It stays a line
when it states a quantity or spares, when it names an instrument as well, when it
is the only line of its source, or when it sits in the same table as the
instruments. See [R72](/business/business-rules/reading-lines-and-confirmation#r72--nothing-the-enquiry-states-for-a-line-is-dropped).

Shared requirements apply to requested items. Standards for unrelated instruments
and document boilerplate remain additional reference context; conditional clauses
retain their conditions rather than becoming unconditional item requirements.
The structuring instructions distinguish requirements applying to every requested
item from unresolved applicability. Uncertain passages remain additional context;
mixed instrument sections must not become one blanket shared requirement. Spare
obligations remain distinct from the specifications of installed items. These are
interpretation instructions, not a guarantee of correct model scope assignments.

A shared requirement states whether it applies to all items or to named items,
and whether it is conditional; a conditional one cites the lines that state its
condition. Additional information is marked as general context, unrelated, or
unresolved.

Each item has a stable `sourceItemRef`. Quantity is numeric: an explicitly stated
quantity takes precedence; when none is stated, the preview uses **1** and records
the quantity as defaulted in the saved source view. A default is not a customer
statement. A row containing multiple tags retains that association rather than
discarding tags. Where several lines of one document share a tag value listing
exactly as many tags, such as a datasheet column with a total quantity of 2 and
tags FT-101 / FT-102, each line takes one tag in the order listed; a value whose
tags differ in shape, or list more or fewer tags than the lines, is left as
read.

Original source text remains authoritative. The tool validates source references
and coverage of nonblank original lines. Original lines that nothing cites are
retained whole as unresolved context available to every item, with their ranges
recorded in diagnostics (`unresolvedSourceRanges`); this does not recover an
omitted item or prove the lines apply to a particular item. Invalid references
stop the run. For a document read from its file alone, the citations quote the
file and not its converted text, so a converted line the reading already holds
whole is not retained again.
After extraction, each line is compared with its own row. A line that placed
fewer than half of at least four stated values is extracted once more, ten
lines to a call. A call that answers for only some of its lines keeps those,
and the lines it left out are asked for one more time. Whatever a line has still not placed is added to its
unresolved requirements as the customer wrote it (R72). A value longer than 150
characters is added only in the parts the line holds nowhere else. A PDF read
from its text with the file beside it is read from the text alone when that
call fails. One answer from the check that cannot be
used leaves that one value unchecked; the rest of the reading is kept.

## Phase 2: Catalogue decodification

Catalogue decodification maps structured source items into the catalogue schema
derived from item groups and decodification masters. It reads the email body and
the structured source view, including every passage structuring left uncited; it
does not read the attachments again.

It runs once for each category the source view holds a requested item for. Where
that differs from the categories detected for the enquiry, the tool says so and
follows the items; a queued job gathers only the confirmed ones.

A category with more than ten requested items is read ten items to a call, in
the order the source view lists them. Each call reads the whole email and every
shared passage, and answers for its own items only. Items a call left out, or
held by a call that failed, are asked for once more; the category fails only when
an item is still missing after that, and the job then tries again as for any
other failure. The category's shared requirements are those of the first call
that answered. Each call is one more model call for the enquiry.

Each call decides for itself which field a column of the schedule fills, so two
calls can put the same column in different fields. Once every call has answered,
each column is moved into the field most calls put it in: a call counts when most
of its items place the column in one field (or none) without doubt, and four in
five of those agree; a field is chosen when more calls share it than any other. A value moves only into an empty field that holds the
same kind of value, and is never taken out of a field because the other calls left
its column unplaced.

Each category schema defines typed fields across sections (e.g. process parameters,
construction materials, connections, outputs):

- **select**: dropdown enumeration options, including a master's accessory list;
  when a customer specifies a variant outside the standard list, it is preserved
  in `customValue`.
- **field: number**: numeric values with standard engineering units.
- **field: range**: minimum, normal, and maximum operating or design intervals.
- **field: text**: descriptive or model specifications.
- **field: boolean**: binary options.

Where masters of one category describe a field differently, the field takes the
heading most of them use, is a select whenever any of them offers an option, and
otherwise takes the definition most of them share. Rows that different
families of a category hold under different keys for one library value, such as a
field in one family and a menu in another, reach the model as one field; each
line's values are then filed under its own family's keys again. An option is
carried by its library value, then by its label, and otherwise kept as the
client's wording, and a menu value moved to a free-text field becomes that
field's text. A concept one family holds twice, or whose families' menus share
no option, stays separate fields.

A category's fields are read from the products the enquiry can be offered. A
product offered only when named, such as the orifice plate, a bypass rotameter,
the ultrasonic meters or a pressure transmitter, has its own fields read only when
the enquiry names it: in the subject, the body, a document, a row of a schedule
(heading and value), or by item code or model number. A general name covering only
such products names all of them; one that also covers a product offered whenever
it fits names none. A product offered on steam is always read, because a line's
phase is known only after it is read. The products left out are listed in the
job's log. A field they share with a product that is read is still read, worded
as before, and so is any field item selection takes as an input to the line, such
as the accuracy asked for, a flow, a size or a pressure. What a saved line can hold
does not change, so a line later found to be one of those products can still be
filled in by hand.

Every schema leaf is optional and omit-if-unmentioned. An unmentioned specification
remains absent rather than guessed or filled with default values.

A requirement stated for every item of the category is returned once, as a shared
requirement, instead of being repeated on each item. Two requirements stated for
one field of one item are kept together: merged where they agree, otherwise
recorded as text that states both.

Before units are converted, each shared requirement structuring kept in the
customer's labels, and that is not conditional, fills the lines it applies to
without a model: a value whose label is the label of exactly one field of the
category goes into that field wherever a line holds nothing, cited to the
requirement's passage, and is then converted, checked and matched like any
stated value (R70).

Populated specification fields require field evidence: an email body or attachment
UUID source reference and a verbatim `exactSnippet` of at most 150 characters; a
longer excerpt is cut to 150. This limit applies only to the evidence excerpt, not
to the requirement value or the original source text supplied for scoring. Design ratings remain distinct from operating values;
evidence excerpts must contain the stated value or requirement rather than a table
header alone.

If extraction yields structured sources but no populated specification field, on
any item or among the shared requirements, the category run stops for review after
saving its response and skips scoring.

## Phase 3: Unit standardization

After catalogue extraction, a number or range field still holding the customer's
wording is read into figures and a unit by a model, using the column heading to
tell a normal figure from a maximum. A reading is kept only when every figure and
the unit appear in the customer's text, an inch fraction such as 1-1/2 or 1½
counting as its decimal, and the figures rise from minimum to maximum; otherwise
the wording stays. Fields are read up to 25 at a time, so a
reading that fails or times out keeps only its own fields' wording; while the model
provider is rate limiting, gathering waits a minute and starts the email again.

The worker then converts numeric values into the catalogue unit where the units
service can resolve the units, reading each line's fluid as the saved line will:
from its source row, else the fluid stated once for it, else a service that
names a fluid. A flow on a field held in m³/h is
standardised by the fluid's phase: a gas to Nm³/h, steam to kg/h, and a liquid,
or a fluid whose phase is unknown, to m³/h. A flow of unknown phase stated as a
normal or standard volume, such as Nm³/h, Sm³/h or SCFM, goes to Nm³/h. A fluid
named only by its steam grade, such as LPST, MPST or HP ST, is read as steam.

A flow the sheet gives in kg/cm², a pressure unit, is a slip for kg/hr. It is
converted as kg/h, with a note saying so, and the customer's unit stays beside
the figure. A flow in a unit that carries a gauge or absolute mark, such as
kg/cm2g, was meant as a pressure and is left as stated.

A conversion between mass and volume, or between cSt and cP, uses the density
the line states, else the one the enquiry states for every line, else the
specific gravity the line, or the enquiry for every line, states with no unit,
times water at 998 kg/m³ on a line that is not gas or steam, else the fluid
library's density for a liquid the line matched with certainty (R109). A gas mass flow
uses the library's normal density for a gas matched with certainty. A guessed
fluid gives no density. An absolute pressure, such as bar(a), psia, ata, atm or
torr, converts to the catalogue's gauge figure with an atmosphere of 1.01325 bar
assumed. A pressure cell that gives one figure with both an absolute and a gauge
unit, such as `1 Bar A Kg Cm2`, is a datasheet error: pressures are worked in
gauge, so the figure is read in the gauge unit (1 kg/cm²), with a note saying
so, and the line is not held for it. Each converted value records what it relied on beside the figure: a
density, as stated, from a stated specific gravity with the water it was read
against, or from the library with the fluid named, and any atmosphere,
reference conditions or reading of a unit that was assumed.

The units service performs every conversion; no model writes a conversion
formula. A conversion that cannot be resolved remains flagged for review.
If the units service is unavailable, gathering saves the stated requirements
without derived standard values.

## Phase 4: Semantic verification and scoring

Semantic verification audits populated requirements against customer source
context using TypeSafe Jev.

Jev receives one source item's populated-field questions together with the
original source lines that the item's references, and those of the shared
requirements applying to it, point at, and the item's own block: from a line
only that item's references point at, down to the line before the next item's,
at most 80 lines. A specification table or datasheet page that lists an item's
values under its title is so seen whole, while a line every item shares, such
as a table's header or a PDF's run of tags, opens no block. Nothing the
structuring model wrote is in that context. Each question names its line by the line's own row text, not by
its position in the enquiry, so a table that numbers its rows afresh does not
mislead the check. A picture, or a scanned PDF with no transcription, read from
its file alone, is the exception: it has no text to check against, so Jev is
given the
reading itself, which
is the item's row under the customer's labels, the shared requirements that
apply, and the passages quoted from the file. The shared requirements are
verified once, as their own set of questions. For each populated field, Jev assigns:

- `verificationStatus`: `MATCHED` (stated for this item), `INFERRED`
  (deduced from context or standard engineering practice), or `MISMATCHED`
  (contradicted by or absent from the source).
- `verificationConfidence`: percentage confidence in the chosen verdict.
- `supportConfidence`: 100 less the share Jev gave `MISMATCHED`, which is the
  stated and implied shares together. It decides whether the requirements
  screen marks a value as "Check this value". The screen shows the mark, not the percentage.

A `MISMATCHED` verdict won with less than half the vote is shown as "Unsure",
otherwise as "Not found in the enquiry". These percentages do not measure
quantity defaults or missing requirements.

A scoring failure preserves the unscored extraction. Provider context-limit errors
stop scoring; the tool does not truncate source context or question sets.
OCR is not integrated into this tool.

## Saved debug results

All run artifacts are saved locally to `apps/worker/.local/requirement-extraction/<email>/`:

- `structured-sources.json`: the Phase 1 structured source view and coverage diagnostics.
- `extraction-<category>.json`: the Phase 2 unscored extracted specifications.
- `scoring-input-<category>.json`: replay inputs supplied to Jev.
- `output.json`: the categories that completed, each with its items and shared
  requirements. A category that fails is reported and does not discard the others.
- `error-<category>.json` or `error-structured-sources.json`: stage and provider diagnostics on failure.