How to Automate Engineering Drawing Reading: 6 YOLO Models Instead of Manual Work

Extracting data from engineering drawings by hand is tedious and error-prone. To order a custom part, someone has to type roughly 20 parameters from the drawing into a calculator: dimensions, threads, tolerance grades, surface roughness, tolerances, material, and mass. One engineering team built an automated pipeline that takes a PDF drawing, reads it the way a human would, and extracts everything it needs in structured form. The output is JSON for the calculator.

Solution architecture

The system works in three steps: data localization, text recognition, and result synthesis. A PDF drawing goes in, and JSON with the parameters comes out. The intermediate stages are:

• Normalizing resolution and contrast
• Identifying projections (front, side, and top views)
• Separating the part outline from auxiliary lines
• Locating text fields and dimension arrows
• Recognizing symbols (thread, tolerance grade, roughness)
• Linking arrows to their values through a connectivity graph

Pipeline components

The computer vision side relies on six specialized YOLO models. Each one is trained on a subset of 500+ real production drawings:

1. Projection detection — finds the front, side, and top views on the drawing. 2. Dimension localization — picks out all dimension arrows and text fields. 3. Special symbol recognition — reads thread (M10), tolerance grade (IT6), and roughness (Ra 3.2) callouts. 4. Part outline — separates the visible outline from auxiliary lines. 5. Auxiliary lines — finds centerlines and construction geometry. 6. Arrows and leaders — locates every type of arrow and the text values attached to them.

The YOLO models feed into a custom OCR. Off-the-shelf solutions handle handwritten notes and special symbols such as ∅ (diameter) and thread designations poorly. The team trained the neural network on a dataset annotated by specialists.

The arrow logic is a weighted graph: if an arrow starts at point A, passes through geometric object B, and ends near text C, then value C belongs to object A. In practice it gets messier: arrows can be dashed or S-shaped, several arrows may point to the same spot, and ambiguity creeps in.

Reality gets in the way

Testing on real production drawings exposed problems that clean datasets don't have:

• Dirty scans — 20-year-old drawings, photocopier scans, water stains, stray pencil marks.
• Typographic liberties — a thread may be written as «Ø10×1.5», «M10», or even drawn as a little spring.
• Colored annotations — someone highlights a dimension in red pen, but OCR often filters out red lines as noise.
• Crowded sheets — a single drawing carries 30+ dimensions, arrows cross each other, and things get confused.

Data augmentation solved the problem: the team generated synthetic drawings with added noise, clutter, contrast changes, and simulated old scans. After training on the expanded dataset, accuracy on dirty drawings rose from 68% to 92%.

What it means

Automated drawing reading shows how human labor gets replaced by a combo of publicly available tools (YOLO), engineering logic (the arrow graph), and specialized tuning. For manufacturing, that means a 15x speedup: 2 minutes on autopilot instead of 30 minutes of manual entry. For the business, it means sending out quotes faster without manual data entry.