Which OCR Should I Use for Handwritten Math Answers?

I am building an English website for a Gaokao math paper. Gaokao is the university entrance exam in China.

Checking the objective questions is easy. Written questions give me a headache.

For a written math question, a student may fill half a page with fractions, square roots and small notes. Chinese students write very fast in the exam, so the handwriting can be squeezed together. There may be crossed-out formulas, arrows, or a new step added between two old steps. A geometry question can also have a diagram.

My first idea was to let the student take a photo and upload it. Then I thought about one simple problem. If OCR reads x + 1 as x - 1, everything after that is useless. The system may still give a very confident-looking grade.

This is the part I have not solved.

Tools I found

Mathpix looks closest to this problem. It is made for math and other STEM content. It can read images, PDFs and stroke data. I think I will test it first for paper photos.

MyScript iink is a different choice. It uses pen strokes when the user writes on a screen. This may be more accurate, but it also means I need to build a writing canvas and ask the student not to use paper.

I also checked Google Cloud Vision and Azure Document Intelligence Read. They can return handwriting and page layout. Maybe they can handle the Chinese or English notes while another service handles the formulas.

TrOCR is another direction if I want an open-source model. I have not found a clear answer about a whole page with mixed text, formulas and diagrams.

A vision model may help after OCR. I am careful about this idea. If a line is unclear, I want the system to mark it as unclear. I do not want the model to fill in the line because it knows what a normal solution should look like.

My rough plan

I will first reject photos that are blurred, cropped or badly tilted. After that, I may use one recognizer for formulas and another one for normal handwriting. I also want to keep the position and confidence of every recognized part, instead of saving only a cleaned-up answer.

The recognized work can then be compared with a solution and scoring rubric that I reviewed before the exam starts. When one important line cannot be read, the written score should stay provisional. It should not quietly become zero.

I have not tested this plan yet. I need to make a small test set with real Gaokao-style handwriting, including bad handwriting and crossed-out work. Testing only the clean examples from a service demo will tell me very little.

I would like some advice

Has anyone here used these tools for handwritten mathematics from phone photos?

I mainly want to know:

• Is Mathpix good enough for multi-line calculation?
• Is keeping the pen strokes worth the extra work?
• What open-source model should I put into the test?
• How do you decide that an answer is too unclear to grade?

The project is here: gaokaoexam.org

At present I still do not know whether photo upload is practical, or whether I should ask students to write on a screen. If you have worked on a similar problem, I would like to hear what failed in your first attempt.