Reading documents

How attachments become text the rest of the product can work from.

What this process achieves

Turns attached datasheets, specification sheets, bills of quantity and images into text — keeping table structure where it matters — so line items and parameters can be read from them.

Who is involved

• The system — converts documents, and, once an email is typed as an enquiry, transcribes its attached pictures through the picture reader while model calls are on.

What starts it

Attachments arriving on a conversation. Documents are read when an email is processed, before it is typed; an enquiry’s pictures right after it is typed as one.

The steps

1. Each attachment is queued when the message is stored. Nested files inside a forward are lifted out as attachments in their own right. A picture shown in the email’s text, such as a signature logo or a pasted screenshot, is not a document and is not read here. A screenshot a client pastes into a later email, and a picture or a review table a client attaches to one, are read by the change reader instead; see Inbox.
2. Tables are read as tables, so a multi-instrument annexure is not flattened into prose.
3. Pictures attached to an enquiry — a photographed datasheet or bill of quantities — are transcribed by the picture reader once the email is typed as an enquiry, while model calls are on, and marked as read from a picture. The picture reader copies only what it can read, spelling and brand names as printed, and writes [illegible] where it cannot read a word or number with certainty, rather than guessing. A scanned PDF attached to an enquiry is transcribed the same way and marked as read from a scan, when every mailbox the email was delivered to is one requirement gathering covers; otherwise it is listed as needing character recognition. A typed PDF with some scanned pages, such as a typed RFQ with a scanned datasheet among its pages, keeps the text of its typed pages in page order, with a note such as “[Scanned pages 8–9 of “schedule.pdf” are not read.]” where the scanned pages stand. On the same terms as a scanned PDF, each run of those pages is then transcribed on its own and takes the note’s place; until every run is read, the PDF is not kept for later emails and is read afresh when it next arrives. A picture or scan already read for an earlier enquiry is reused. Pictures and scans in other emails are left unread by decision.
4. A conversion that fails is listed as not read, with the reason. A picture or scan is asked once more straight away only when the picture reader failed with a temporary fault, not when it came back empty, was cut off or ran out of time. After that a document is tried again only when the email is processed again, which today means a developer reprocesses it; nothing reprocesses a finished email by itself. A document still converting after 30 seconds is left unread for that run and shows as timed out; the rest of the email carries on without it.
5. A long document is never cut. Requirement gathering reads each document’s text whole, so a half-read annexure row cannot pass for a line item. A document too long for the model to take in fails that gathering, which is then sent to Needs Review as failed, instead of being trimmed. Reading the categories asked for uses a short excerpt of each document, not the whole.

Rules that apply

R22, R49, R50.

What happens if it stalls

Related: Line items · Processing a case.