← The Operations Station ~ dish 02 of 10 · the operations station

Data Extraction

Unstructured documents contain structured data buried inside. Invoices have amounts, dates, vendor names. Applications have job history, contact info, qualifications. The station reads the document and pulls the fields into the system of record. The leverage is guardrails. What confidence level triggers human review.

The job

A document arrives. It contains data you need. A human reads it. They find the vendor name, the invoice number, the amount due. They type it into the system. For fifty or one hundred documents a month, this is an admin task. For five hundred documents a month, this is a person.

The dish reads the document. It finds the fields you defined. Vendor name. Invoice date. Amount. Confidence score for each field. It returns structured data ready to drop into your system. Humans only touch documents where the station’s confidence is below your threshold. By week two, data entry goes from hours per day to spot-checking edge cases.

Plated well, this looks like: data flowing automatically from inbound documents into your systems. The data is accurate. Edge cases (unusual formats, missing information) are visible so a human can handle them fast. No data sits in a human queue waiting to be read.

The recipe

All seven ingredients still apply. The leverage is Guardrails (Ingredient #3). The station needs one hard rule: above ninety percent confidence, go to the system. Below eighty-five percent, hold for human review. Eight-five to ninety, flag for spot-check approval.

Context (Ingredient #2) is the second lever. The station needs to know what fields to extract and what format they go into. Not just “get the amount” but “get the amount in USD as a number, not a string.”

Examples (Ingredient #4) teach the station what “vendor name” looks like when the vendor is handwritten in a corner versus printed at the top. Training (Ingredient #1) is consistency. You extract amounts as numbers. You extract dates in MM-DD-YYYY format. The station learns the house standard.

How to build it

  1. List the fields you need. For invoices: vendor, date, amount, PO, payment terms. For applications: name, phone, email, current role, years of experience. Ten fields maximum or the extraction gets noisy.

  2. Define the data format for each field. Amount as number, no currency symbol. Date as MM-DD-YYYY. Phone as ten digits. These format rules go to the station as constraints, not suggestions.

  3. Pull ten examples per field from your archive. Show the station what “vendor name” looks like when it’s printed, handwritten, partially obscured, in multiple languages. Real data trains reality.

  4. Set confidence thresholds. The station returns a confidence score per field. Ninety-five percent confidence on amount goes directly to the system. Eighty percent confidence on amount gets human eyes. Below eighty, escalate to a human.

  5. Build the system output. The fields the station extracts land somewhere. A spreadsheet. A database. An API call. Define the output format. The station is not replacing human thinking. It’s replacing data entry.

  6. Test on fifty documents. Real documents from your archive. Track which fields the station gets wrong and why. Below ninety-five percent accuracy on a field means the definition is unclear. Sharpen it.

  7. Deploy to live inbound documents. The station processes new documents as they arrive. Extracted data lands in a “pending human review” state until confidence is verified. By week two, you have data on what actually happens.

  8. Set the escalation workflow. Low-confidence extractions go to a human queue. The person confirms or corrects. Every correction teaches the station. Track the most-corrected fields. Those fields need sharper definition or better examples.

What breaks it

  • Fields aren’t defined clearly. You want the station to extract “amount” but the amount on an invoice can be subtotal, tax, total, or pending adjustment. Which one do you want? If you don’t say, the station guesses wrong.

  • Confidence thresholds are misaligned with reality. You set the threshold at seventy percent because you want speed. The station starts pushing seventy-percent-confident data into your system. Errors compound. By month two, your financial reports are wrong.

  • Examples don’t represent real data. You give the station five pristine PDFs with perfect formatting. Real invoices are faxes with missing pages. Mobile photos with shadows. Data from three countries in three languages. The station trained on ideals fails on reality.

  • No feedback loop. A human corrects an extraction. The station never learns why. Six months later it’s still making the same mistake because the correction never made it back to training data.

When it’s working

By week one, seventy percent of documents are extracted with ninety-five percent confidence. The data goes straight to the system. Thirty percent land in human review because confidence is lower. The human checks, approves, or corrects. By week two, accuracy is solid on six of ten fields. The remaining four need tighter definitions or more examples. By week four, ninety percent of documents are processed automatically. The team spot-checks the automated extractions weekly and processes the low-confidence batch manually.

The signal that the recipe is sharp: a new document format arrives and the station gets eighty-five percent of extractions right on the first try because it learned the extraction patterns, not the specific templates.

Monday Move

Pick one document type. Twenty examples. Manually extract three fields from each. Show the station what you’re extracting and why. Have it extract the next batch while you watch. Document where it’s wrong. Fix one field definition. Try again. This cycle is the whole deployment.

Dish 2 of 10 on the Operations Station. Build-note leverage: Guardrails (Ingredient #3).