Why 95% OCR Accuracy Is Not Enough for Accounting Work

The first question almost every buyer asks an OCR vendor is "how accurate is it?"

It is an understandable question, and it is one whose answer helps the decision almost not at all. In many cases it actively leads to the wrong decision. This article explains why, and offers a set of questions that work better.

95% of what

The first problem is that "95% accurate" means nothing until you know what it counts. Two vendors quoting the same number can be describing entirely different things.

Character-level accuracy measures how many characters were read correctly out of all of them. This is the figure most often quoted, because it is the prettiest. A single tax invoice holds roughly 400–600 characters and digits, so 95% at this level means about 20–30 characters misread per document.

Field-level accuracy measures how many of the fields you actually need came out correct — invoice number, date, tax ID, amount before VAT, VAT, total. A single document may have 10–15 fields that matter.

Document-level accuracy measures how many documents came out completely correct across every field. This is the only figure that matches what the people doing the work experience, because a document with one wrong field is a document that has to be opened and fixed anyway.

As an illustration, assume each of 12 required fields has the same 95% chance of being correct and field outcomes are independent. Under those assumptions, the chance that all 12 are correct is 0.95 to the power of 12, or about 54%.

This illustrates why field-level averages cannot be converted directly into document-level results. Real outcomes may differ because field errors are neither independent nor equally likely.

This is not a trick. It is ordinary arithmetic that rarely comes up in a sales meeting.

Errors do not weigh the same

Suppose a system misreads two documents in a hundred. The question is where.

If a supplier's name is off by one character, the impact is close to zero. A reviewer notices and fixes it, or nobody notices and nothing is harmed.

If the amount is wrong — 15,000 read as 1,500 — the accuracy figure is unchanged, but the financial statements are wrong, the tax filing is wrong, and somebody is tracing the cause at ten at night on the last day of the month.

An average accuracy figure hides the thing that matters most: how the damage is distributed. A system that errs often in places that do not matter is better than one that errs rarely in places that are expensive.

In practice only a few fields have to be treated as if they may never be wrong: every monetary amount, the tax ID, the date, and the document number. The rest can be wrong without causing a disaster.

The real question: does the system know when it is unsure

This is the most important point in this article, and it is the one almost nobody asks about when choosing a vendor.

Two OCR systems with identical accuracy can produce experiences that are worlds apart, depending on how each handles its own uncertainty.

The first kind reads every document and pushes all of it into the accounting system. Misread documents sit in there with no signal attached. The accounting team finds out something is wrong when the numbers do not reconcile at close, and then has to search backwards through three thousand documents.

The second kind reads every document too, but sets aside the ones it is not confident about and puts them in front of a person before anything is committed. Say it sets aside 15 out of a hundred, and 4 of those are genuinely wrong.

The second kind makes the team do more work up front — open 15 documents when only 4 are wrong. But it converts "find the needle in the haystack" into "check these 15".

The first job takes an unpredictable amount of time and is stressful. The second takes an amount of time you can estimate and plan around.

So the question to ask a vendor is not "how accurate is it" but "what does it do when it is not sure".

If the answer is "our system is accurate enough that you do not need to worry about that", the answer is telling you the system has no such mechanism.

Why Thai documents are harder than they look

Thai business documents have several characteristics that make an overseas vendor's accuracy figure useless as a reference.

Thai has no spaces between words. The system has to infer word boundaries itself, which goes wrong on unfamiliar company names.

Vowels and tone marks stack in several layers. Characters such as "เ", "ิ" and "้" sit at different heights. Scanned at low resolution, or on a copy of a copy, the upper and lower layers are the first to disappear.

Thai numerals are still in use in government documents. ๑ ๒ ๓ appear alongside 1 2 3 in the same document.

Two calendar systems. Some documents use the Buddhist era, some the Gregorian, and some write only "68", which could mean 2568 or 2068. A system not designed for Thailand converts these wrongly with no warning, and a date error decides which accounting period a transaction lands in — harder to correct later than a wrong amount.

Stamps and signatures over text. A blue or red company stamp printed across the document number is entirely normal in Thai paperwork, and it is where general-purpose systems misread most often.

This is why an accuracy figure measured against a standard international document set tells you nothing about the result you will get on the documents in your own filing cabinet.

Questions to ask instead

Rather than asking for a single number, ask this set. The answers tell you very quickly whether the vendor understands the problem.

1. Can we test with our own documents? — If not, every number they have quoted is about somebody else.
2. What does the system do when it is not confident? — A good answer is "sets it aside for a person". Not "it is accurate anyway".
3. Is the quoted accuracy per character, per field, or per whole document?
4. Are monetary fields checked further? — Amount before VAT plus VAT must equal the total. When it does not, the system should know it misread without being told.
5. What happens with a layout it has never seen? — There is always a new supplier.
6. Where is our data stored, for how long, and who can reach it? — Accounting documents are among the most sensitive business data there is.
7. If we stop using it, in what form do we get our data out? — Ask on day one, not on the day you want to leave.

A test that gives a real answer within one week

You do not need a six-month pilot to find out whether a system works. This is enough:

Pick 30–50 documents that genuinely represent your work. The important part is not to pick only the clean ones. Include photocopies, documents with a stamp over the text, thermal-printed receipts that have faded, and documents from several suppliers whose layouts differ.

Test with your prettiest documents and you will get a pretty result that means nothing.

Measure three numbers, not one

• What percentage of documents came out correct in every field, needing no touch at all
• What percentage the system set aside as uncertain
• What percentage were wrong without the system knowing — this is the most important one, because it is the damage that reaches your real records

The third number is the one to compare vendors on. A system where it is zero but which sets aside a lot for review still beats a system that sets aside little and lets bad data through.

Time it for real. Have the people who do this work every day use it, and time how long it takes to get from a file to data in the accounting system, against the current method. That figure is what a manager will ask about — not the accuracy percentage.

In short

95% accuracy does not mean 95% of the work disappears, and in some cases it means nearly half your documents still get handled by hand.

What decides whether an OCR system genuinely helps is not average accuracy. It is two things: whether the system knows when it is unsure, and what it does about it.

A system that can say "I am not confident about this one, please look" is worth more to an accounting team than a slightly more accurate system that stays silent when it is wrong.

Want to know how your documents should be assessed? Send sample documents for assessment. The team confirms the assessment method, queue, timing, fees, and report scope before work begins. The report can identify documents or fields that need further review. File handling follows the stated privacy and retention policy.

Read next: AI-OCR for accounting and finance