Extracting Client, Cover and Dates From a Policy PDF Automatically

An advisor forwarding a policy PDF to a piece of software imagines the software is reading a form. It is not. It is reading a poster.

There is no standard schema for a retail policy schedule in India. Each insurer designs its own, and the design is owned by whichever team last touched the issuance stack. The IRDAI (Protection of Policyholders' Interests, Operations and Allied Matters of Insurers) Regulations, 2024 govern what a policyholder must be told. Nothing governs where on the page it gets printed, what the field is called, or whether it is a field at all. So the sum insured is a labelled box on one insurer's health schedule, a line inside a benefits paragraph on another's, and a column header on a third's.

That variation defeats the obvious approach, which is to teach software where to look. Position-based extraction ("the policy number is in the top-right block") survives exactly one insurer and breaks the first time that insurer reissues its template, quietly and without notice to anyone downstream.

The practical count is worse than the insurer count. One insurer commonly emits three or four distinct layouts: the digital schedule from the core system, the endorsement copy, the renewal notice, and the dealership-issued motor certificate that came off a different printer entirely. An advisor placing motor, health, term and personal accident across the insurers their principal is authorised to place is not handling twelve formats. They are handling a long tail of layouts with no registry and no obligation to stay still.

Before any question about intelligence, there is a question about whether the characters exist.

A PDF generated by an insurer's core system carries a text layer: real characters with coordinates, which software reads exactly, with no interpretation and no error rate. A scan, or a photograph of a printout saved as a PDF, carries no text layer at all. It carries an image of characters, and getting text out of it requires optical character recognition, which is a guess with a good success rate rather than a read.

Both arrive in an Indian advisor's book constantly, and they look identical in a WhatsApp preview. The second category has predictable sources: the dealership that printed and scanned a motor certificate, the client who photographed his own document because the insurer email went to an address he no longer uses. A photograph adds its own failures on top of OCR: skew, glare on a laminated card, a thumb over the corner, and the compression a chat app applies to anything it thinks is a picture.

OCR error on Indian policy documents concentrates in exactly the characters an advisor cannot afford to lose. 0 and O, 1 and I and l, 5 and S, 8 and B. These are the alphabet of a policy number. A misread policy number is not a slightly wrong policy number. It is a different policy, or no policy, and every lookup against it fails in a way that looks like the insurer's fault.

The premium breakdown on a motor schedule looks like a table: columns, aligned figures, a rule under the header, a total at the bottom. To a machine it is usually none of those things. It is a set of text fragments that happen to sit at similar horizontal offsets, with nothing recording that this number belongs under that heading or in this row.

So the software reconstructs the grid from geometry, and the geometry lies in ordinary, frequent ways:

• A label wraps onto two lines and the value stays on the first, so it now looks like it belongs to the row above.
• A row has a blank cell, and the remaining values shift left into columns they do not belong to.
• Two logical columns sit close enough to read as one, so own-damage premium and third-party premium merge into a single meaningless figure.
• A schedule spans two pages and the header does not repeat, so page two is a grid of numbers with nothing naming them.
• Rupee figures use Indian digit grouping (1,50,000), which parsers written against thousands-grouping conventions truncate or reject.

Motor bites hardest, because the motor premium block is the most structurally complex thing in a retail advisor's book: own damage, third party, individually priced add-ons, discounts and tax, under headings that differ per insurer. An extractor pulling one number labelled "premium" from a motor schedule has almost certainly pulled the total the client paid, which is the right answer to the client's question and the wrong input to most other calculations.

Table reconstruction on unstructured schedules is the least reliable part of the whole exercise, and any tool presenting a clean premium breakdown without saying which figures it inferred is selling neatness.

Extraction quality is not one number. It varies by field, and the ranking is stable enough to plan around.

The fields that come out well have a distinctive shape and a nearby label. The policy number is highly patterned and sits under an unambiguous label on every schedule, so on a digital PDF it is close to a solved problem (on a scan it inherits every OCR confusion above). Premium, as a single total figure, is usually well labelled. The insurer is often identifiable from the document's own metadata without reading a field at all.

The fields that come out badly share a property: the document holds several plausible candidates and nothing in the layout says which one the reader wants.

Dates are the worst offender in the Indian document set. A schedule carries a proposal date, an issue date, a risk start date, a risk end date, sometimes a renewal-due date, and on health an original inception date that governs waiting periods and is not this year's start. Six dates, of which the renewal book needs exactly one. Labelling is inconsistent ("Period of Insurance", "Valid From/To", "Risk Commencement") and the field is often a range in one cell rather than two dates in two.

Worse, the format is ambiguous. 05/06/2027 is 5 June under the DD/MM convention every Indian insurer prints in, and 6 May to any parser defaulting to the American one. This misreads silently, produces a valid-looking date, and is only detectable on days after the 12th of a month. A tool built for Indian documents must pin DD/MM explicitly rather than infer it.

Sum insured is contested the same way. A health schedule may show a base sum insured, a cumulative bonus, a restore benefit and a total, and reasonable people disagree about which is "the" one. A motor schedule shows IDV, a different concept the extractor should not silently rename.

The name is not an identifier

Indian names carry variation that is genuine rather than erroneous. Initials expand or contract by insurer convention, so R. Krishnan, Krishnan R and Ramaswamy Krishnan are one man across three schedules. Transliteration produces spellings that are all defensible: Sanjay and Sanjai, Lakshmi and Laxmi, Chaudhary and Choudhury. A floater health policy names several people in one field with no separator the machine can rely on, and schedules carrying a regional-language block beside the English one can have the two scripts interleaved inside one extracted field.

The consequence is not that the name is wrong. It is that the name does not join. A name arriving in a different form from the one already in the book creates a second client instead of updating the first, so one household appears twice with two renewal dates and no link between them, which is worse than having extracted nothing. Join on the policy number, treat the name as an attribute to reconcile, and ask when the two differ rather than picking one.

Everything above leads to one design conclusion. Extraction that returns bare values is unusable, however accurate, because the advisor cannot tell the certainties from the guesses.

A policy number read off a text layer and one recovered by OCR from a photograph of a laminated card differ enormously in reliability, and arrive on the screen looking exactly the same. The extractor knows the difference. If it does not pass that on, it has discarded the one thing that would have let the advisor spend attention where it was needed.

So the output of a serious extractor is per field, not per document, and carries three things: the value, a confidence, and the evidence (which document, which page, which region it came from). Confidence should fall out of the mechanics rather than being decorative: whether the source was text or OCR, whether the label was unambiguous, whether competing candidates existed, whether the value passes a sanity check (does the risk end date sit one year after the start, does the premium breakdown sum to the total).

Document-level accuracy claims are close to meaningless here. "96 percent accurate" over a mixed pile of Indian schedules averages a policy number that is essentially always right with an expiry date that is sometimes confidently wrong, and hides the second inside the first. Ask instead for per-field accuracy split by digital and scanned; a vendor who cannot produce that split has not measured the thing that matters.

Confidence has to be honest downwards too. A tool that never says "I do not know" has not eliminated uncertainty. It has hidden it, and the advisor meets it later, in a worse setting.

This is the asymmetry that should govern how any advisor tool is built, and the one vendors most consistently get backwards.

A missing expiry date is a visible hole. The row is blank, the advisor sees the blank, and at some point it gets filled. It costs a minute of irritation and it announces itself.

A wrong expiry date is invisible and load-bearing. The renewal reminder fires on the wrong day. If it fires late, the policy lapsed while the advisor's own system reported it as active, and the first the advisor hears of it is the client discovering he is uncovered, possibly at a claim. On health the damage compounds past the money: a lapse running past the grace period can put waiting periods and continuity benefits back to zero, which paying the renewal premium late does not fix. The client does not experience this as a software error. He experiences it as his advisor not knowing when his policy expired.

The two failures are not comparable in cost, so they must not be comparable in the design. An extractor unsure about a date has two options and only one is defensible: report the uncertainty, or pick the likelier reading and stay quiet. The second is what maximises a demo and what destroys a book.

The answer to unreliable extraction is not to abandon it. Reading a schedule by hand is slow, and humans misread scanned policy numbers too, at rates nobody measures because nobody is scoring them. The answer is a review step, and the only one that survives a real week is short.

A review asking an advisor to check twelve fields gets skipped by the third document on a busy Saturday, and a skipped review is worse than none, because the record now carries an implied approval nobody gave. The target is a review that fits between two client calls:

1. Only surface what is uncertain. Fields off a text layer under an unambiguous label do not need a human. Showing them dilutes the ones that do.
2. Put the evidence next to the question, so the check is a glance rather than an investigation.
3. Always confirm the two fields that carry consequence: the expiry date and the policy number. One drives the calendar, the other is the join key for everything else.
4. Make correcting cheaper than accepting. If a fix takes three taps and accepting takes one, wrong values get accepted.
5. Feed corrections back. An advisor correcting the same insurer's date field every month is reporting a template the extractor has not learned.

A policy schedule is also dense with a client's personal data, and an advisor handling it does so on behalf of the insurer or intermediary that engaged them, which is the entity carrying the data-fiduciary obligations. With the Digital Personal Data Protection Act, 2023 and its 2025 Rules commencing on staggered timelines from 14 November 2025, expect your principal to push consent, purpose-limitation and retention terms down through contract.

Extraction earns its place when it turns four minutes of transcription into forty seconds of checking. It loses that place the moment it turns four minutes of transcription into a renewal date nobody verified.