🤖 Internal AI Models Guide

Complete guide to all AI models in the Internal AI Models repository for technical drawing analysis.

Last Updated: June 2026 | Total Models: 11 | Primary Framework: LitServe (9 models)

Quick Summary: This repository contains 11 specialized AI models for analyzing technical engineering drawings. Each model extracts different types of information like dimension tolerances, welding symbols, surface finish specifications, and more.

What is This Repository?

Purpose

The Internal AI Models repository is a Python monorepo that hosts specialized AI models for analyzing technical engineering drawings. These drawings contain:

Dimension specifications
Welding symbols
Material information
Surface finish specifications
Tolerance requirements
Part tables and lists

Real-World Use Case

When an engineer scans a technical drawing (blueprint), these AI models analyze it to automatically extract structured information like:

"This part should be welded with a fillet weld"
"Surface roughness should be Ra 3.2 μm"
"Maximum thickness variation: ±0.5mm"
"List of 45 parts required for assembly"

Architecture Overview

High-Level Pipeline

Technical Drawing (Image) ↓ [LitServe/FastAPI HTTP Server] ↓ [Model Inference] (GPU/CPU) ↓ [Post-processing & Format Output] ↓ JSON/HTML Response

Key Technology Stack

Deep Learning

PyTorch, YOLOX, EfficientNet-B2, BERT, Vision Encoder-Decoder

Serving

LitServe (9 models), FastAPI (2 models)

Image Processing

OpenCV, PIL/Pillow, Pydantic validation

1. canvas_ocr - Technical Drawing OCR

What it does

Extracts dimension tolerances, fit tolerances, beveling specifications, and special instructions from technical drawings.

📍 Real example: Drawing showing "TOLERANCE: ±0.05mm" → Output: {tolerance_value: 0.05, tolerance_type: "dimensional_tolerance"}

Architecture (4-Stage OCR Pipeline)

Input Image (2048×2048) ↓ [Stage 1: YOLOX Detection] ← Find tolerance symbols │ ├─ 6 classes of tolerance symbols ├─ Bounding boxes around each └─ Confidence scores ↓ [Stage 2: Bounding Box Extraction] │ ├─ Extract region for each detection ├─ Typical size: 50×50 to 200×200 pixels └─ Prepare for angle classification ↓ [Stage 3: Angle Classification] │ ├─ CNN classifier (4 classes) ├─ Determine rotation: 0°, 90°, 180°, 270° ├─ Simple MobileNet-v2 └─ Input: Cropped symbol region ↓ [Stage 4: Text Recognition (BERT)] │ ├─ Pre-trained on text OCR ├─ Reads individual characters ├─ Typical input: "±", "0.05", "mm" └─ Output: "±0.05mm" ↓ [Post-processing] │ ├─ Parse numeric value: "0.05" ├─ Extract unit: "mm", "inches" ├─ Validate tolerance type └─ Link to nearest symbol ↓ JSON Output: { "detections": [ { "class": "dimensional_tolerance", "bbox": [0.1, 0.2, 0.5, 0.6], "value": "±0.05mm", "confidence": 0.91 } ] }

Detection Classes (6 types)

c_beveling - Chamfered edge specifications
dimensional_tolerance - Size variation limits
fit_tolerance - Assembly fit specifications
r_beveling - Rounded edge specifications
special_instructions - Custom manufacturing notes
specified_tolerance - Explicitly marked tolerance values

Why 4 Pipeline Stages?

Detection: YOLOX finds where the tolerance specs are
Angle Classification: Engineers write at different angles on the drawing
Text Recognition: BERT reads the tolerance values
Post-processing: Validate format and extract numbers

How Canvas_OCR Works (Example)

Input: Drawing with "± 0.05" marking
YOLOX: Detects box at [100, 200, 300, 250] as "dimensional_tolerance"
Angle Classifier: Says text is rotated 90°
Rotation: Rotate image region to 0° (horizontal)
BERT: Reads "± 0.05"
Post-process: Parse to value=0.05, unit=unspecified, type=dimensional
Output: JSON with tolerance_value: 0.05

Model Details

YOLOX Variant: YOLOX-s (small, fast)
Angle Classifier: MobileNet-v2 (4 classes)
Text Recognition: BERT-base
Input Size: 2048×2048
Output: JSON with tolerance specs
Speed: 40ms (GPU), 400ms (CPU)

Tolerance Format Examples

Bilateral: "±0.05" (±0.05mm)
Unilateral: "+0.10/-0.00" (positive/negative)
Limits: "2.00-2.10" (max-min)
Percentage: "±5%" (percentage tolerance)

Framework: LitServe | Speed: 40ms (GPU) | Type: Multi-stage symbol + OCR extraction

2. deep_feature_v2 - Deep Feature Extraction

What it does

Converts a technical drawing image into a 1024-dimensional feature vector for similarity matching and retrieval.

📍 Real example: Input: Drawing of bracket → Output: [0.123, 0.456, -0.789, ..., 0.234] (1024 numbers)

Architecture (Detailed)

Input Image (variable size) ↓ [Resize to 224×224] ↓ [EfficientNet-B2 Backbone] ← Feature extractor │ (Image → 1280-dimensional) ↓ [GeM Pooling Head] ← Generalized Mean Pooling │ (1280-dim → 1024-dim) ↓ [Normalize] ← Make unit length ↓ 1024-dimensional Feature Vector

EfficientNet-B2 Explained

EfficientNet = Family of neural networks optimized for efficiency
B2 = Medium-sized variant (faster than B3, larger than B0)
It's a CNN (Convolutional Neural Network) that extracts visual features
Pre-trained on ImageNet + fine-tuned for engineering drawings

Key Characteristics

Input: Any size image (auto-resized to 224×224)
Output: Always 1024 numbers (suitable for vector databases)
Speed: Very fast (GPU: ~10ms, CPU: ~100ms)
Use Case: Similarity search, duplicate detection
Special Feature: Dynamic Batching (max batch size: 4)

Framework: LitServe | Speed: 10ms (GPU)

3. welding_symbol_detection - Welding Symbol Detection

What it does

Detects and classifies all welding symbols in a technical drawing. Uses YOLOX object detection to identify the location and type of welding symbols automatically.

📍 Real example: Input: Drawing with welding symbols → Output: {symbol_type: "fillet_weld", location: [x1, y1, x2, y2], confidence: 0.95}

Architecture

Input Image (2048×2048) ↓ [YOLOX Detection] ← Detects all welding symbols │ ├─ Backbone (ResNet-style) │ Extracts visual features from image │ ├─ Neck (FPN - Feature Pyramid) │ Combines features at multiple scales │ └─ Head (Dense Predictions) For each grid cell: predict • Bounding box (x, y, width, height) • Confidence score (0-1) • Class probabilities ↓ [Classes Detected] ├─ fillet_weld ├─ butt_weld ├─ groove_weld ├─ spot_weld ├─ seam_weld └─ plug_weld ↓ [NMS (Non-Maximum Suppression)] │ Remove overlapping detections │ Keep highest confidence boxes ↓ [Output Formatting] ↓ JSON: {detections: [{symbol_type: "fillet_weld", bbox: [0.1, 0.2, 0.3, 0.4], confidence: 0.95}]}

How YOLOX Works

Grid Division: Image divided into 52×52 grid cells
Per-Cell Prediction: Each cell predicts if there's a symbol inside it
Bounding Box: If symbol detected, predict exact location and size
Classification: Which type of weld symbol is it? (6 classes)
Confidence: How sure are we? (0-1 probability)
NMS: Remove duplicate detections of same symbol

Common Welding Symbol Types (6 Classes)

Fillet weld - Corner/edge join with triangular cross-section
Butt weld - Face-to-face joint
Groove weld - V-shaped or U-shaped joint
Spot weld - Localized welding points
Seam weld - Continuous line weld
Plug weld - Hole-filling weld

Model Details

Variant: YOLOX-s (small) or YOLOX-m (medium)
Input Size: 2048×2048 pixels
Output: Bounding boxes + class labels + confidence scores
Parameters: ~8-15M
Model Size: ~30-50MB

Framework: LitServe | Speed: 20ms (GPU) | Type: Single-stage object detection

4. material_shape_classification - Material Shape Classification

What it does

Classifies the shape of materials/parts shown in drawings. Given a part image or drawing, determines what geometric shape it is (cylinder, cube, plate, etc).

📍 Real example: Input: Drawing showing round rod → Output: {shape: "cylinder", confidence: 0.92}

Architecture

Input Image (variable size) ↓ [Resize to 224×224] ↓ [CNN Feature Extractor] │ ├─ Convolutional Layer 1 (32 filters) │ Detect edges, corners │ ├─ Convolutional Layer 2 (64 filters) │ Detect textures, patterns │ ├─ Convolutional Layer 3 (128 filters) │ Detect mid-level shapes │ └─ Convolutional Layer 4 (256 filters) Detect high-level features ↓ [Global Average Pooling] │ Compress spatial dimensions │ Retain feature information ↓ [Fully Connected Layer 1] (512 neurons) │ High-level reasoning ↓ [Dropout] (0.5 rate) │ Prevent overfitting during training ↓ [Fully Connected Layer 2] (256 neurons) ↓ [Output Layer - Softmax] │ 10 shape classes ↓ Shape Prediction: ├─ cylinder: 0.92 ├─ cube: 0.05 ├─ plate: 0.02 └─ others: 0.01 ↓ JSON: {shape: "cylinder", confidence: 0.92}

Possible Shape Classes

Cylinder - Round rod or tube shape
Cube/Block - Rectangular solid
Plate - Flat sheet material
Angle Stock - L-shaped material
Channel - U-shaped material
I-Beam - I-shaped material
Sphere - Round ball shape
Cone - Conical shape
Pyramid - Triangular base to point
Other - Non-standard shapes

How CNN Classification Works

Convolutional Layers: Extract visual features (edges, textures, shapes)
Pooling: Reduce spatial dimensions while keeping important features
Fully Connected Layers: High-level reasoning about shape
Softmax Output: Probability distribution over all shape classes
Argmax: Return class with highest probability

Model Details

Input Size: 224×224 pixels
Architecture: ResNet-50 or similar CNN backbone
Output Classes: 10 shape categories
Parameters: ~25M
Model Size: ~100MB

Framework: LitServe | Speed: 10ms (GPU) | Type: Image Classification

5. part_table_recognition - Part Table Extraction

What it does

Finds tables in drawings and extracts them as structured data (HTML/CSV).

📍 Real example: Input: Drawing with 10×5 parts list → Output: HTML table with all data

Complex Architecture (7-Stage Pipeline)

Input Image (2048×2048) ↓ [Stage 1: YOLOX Detection] ← Find table boundaries │ ├─ Detect table regions ├─ Get bounding box └─ Confidence score ↓ [Stage 2: Table Region Extraction] │ ├─ Crop to table boundaries ├─ Typically 500×500 to 2000×2000 pixels └─ Pass to next stages ↓ [Stage 3: Table Transformer (ONNX)] │ ├─ Special model for table structure ├─ Understands row/column layout ├─ Uses ONNX format (optimized) │ • ONNX = universal ML format │ • ~5× faster than PyTorch │ • GPU-optimized inference │ └─ Output: Cell locations and structure ↓ [Stage 4: SAHI Text Detection] │ ├─ Sliced detection (tile-based) ├─ 512×512 tiles with overlap ├─ Detect text in each cell └─ Merge tile results ↓ [Stage 5: Angle Classification] │ ├─ For each text region ├─ Classify rotation: 0°, 90°, 180°, 270° └─ Simple CNN (4 classes) ↓ [Stage 6: Text Recognition (BERT)] │ ├─ Read text from each cell ├─ Rotate to horizontal first ├─ Character-by-character recognition └─ Output: "Part No", "123-456", etc. ↓ [Stage 7: Table Assembly] │ ├─ Link cells by structure ├─ Create 2D grid ├─ Format as HTML table └─ Alternative: CSV export ↓ Output Formats: ├─ HTML:

...

├─ CSV: Part No,Quantity,Description └─ JSON: {rows: [{cells: [...]}]}

Why Table Recognition is Complex

Structure Variation: Tables have different row/column counts (3×3 to 50×50)
Merged Cells: Some cells span multiple rows/columns
Irregular Borders: Not all cells have visible borders
Mixed Content: Text, numbers, symbols, logos
Scale Variation: Different font sizes and cell dimensions

Key Component: Table Transformer (ONNX)

Purpose: Understand table structure without explicit rules
Architecture: Vision Transformer adapted for tables
Training Data: Millions of diverse tables
ONNX Format Benefits:
- Open Neural Network Exchange format
- Universal ML format (works with any framework)
- GPU-optimized inference kernels
- 5× faster than PyTorch equivalent

Model Pipeline Details

YOLOX Detection: YOLOX-s (small, fast table finding)
Table Transformer: ~86M parameters, 300MB model size
Text Detection: SAHI with 512×512 tiles
Angle Classifier: MobileNet-v2
Text Recognition: BERT-base for OCR

Framework: LitServe | Speed: 100ms (GPU) | Type: Multi-stage structured extraction

6. surface_roughness_detection - Surface Roughness Extraction

What it does

Detects surface roughness symbols and extracts the numeric roughness values. This is a complex multi-stage pipeline that finds roughness symbols, detects associated text, classifies text orientation, and recognizes the numeric values.

What is Surface Roughness?

A manufacturing specification for how smooth a surface should be. Measured in micrometers (μm):

Ra 0.8 = Very smooth (mirror-like)
Ra 3.2 = Smooth (typical for machined parts)
Ra 12.5 = Rough (unfinished casting)

📍 Real example: Input: "Ra 3.2 μm" → Output: {roughness_type: "Ra", value: 3.2, unit: "μm", confidence: 0.91}

Architecture (Most Complex Pipeline - 6 Stages)

Input Image (2048×2048) ↓ [Stage 1: YOLOX Detection] ← Find surface roughness symbols │ ├─ Detects: Ra, Rz, Rmax symbols ├─ Bounding boxes around symbols └─ Confidence scores ↓ [Stage 2: SAHI Text Detection] │ ├─ Sliced Aided Hyper Inference │ • Divides large image into tiles (512×512) │ • Detects text in each tile │ • Merges overlapping detections │ ├─ Why SAHI? Large images (2048×2048) are slow │ SAHI processes tiles → ~5× faster │ └─ Output: Text bounding boxes ↓ [Stage 3: Angle Classification] │ ├─ CNN classifier (simple) ├─ 4 classes: 0°, 90°, 180°, 270° ├─ Determines text rotation └─ Input: Cropped text image ↓ [Stage 4: Text Rotation] │ ├─ Rotate bounding box to horizontal └─ Ensures consistent text orientation ↓ [Stage 5: BERT Text Recognition] │ ├─ Pre-trained on text recognition ├─ Reads individual characters: "3", ".", "2", "μ", "m" ├─ Outputs: "3.2 μm" └─ High accuracy on clean text ↓ [Stage 6: Post-processing] │ ├─ Parse numeric value: "3.2" → 3.2 (float) ├─ Extract unit: "μm", "mm", "inches" ├─ Identify type: "Ra", "Rz", "Rmax" ├─ Validate range (typical: 0.1-100 μm) └─ Link to nearest symbol ↓ JSON Output: { "roughness_value": 3.2, "unit": "μm", "type": "Ra", "confidence": 0.91, "location": [x1, y1, x2, y2] }

Why This Pipeline is Complex

Symbol Finding: Roughness symbols are small (10-30 pixels) and visually distinct
Text Detection: Text is near symbols but not always in same region
Angle Variation: Text can be rotated (engineers write in different angles)
Multiple Units: Values in μm, mm, or inches with different formats
Precision Required: "3.2" vs "32" changes meaning 10×

Key Components Explained

SAHI (Sliced Aided Hyper Inference):

Problem: Image 2048×2048 is too large to process quickly
Solution: Divide into overlapping 512×512 tiles
Process each tile independently
Merge results from overlapping regions
Result: ~5× faster than processing whole image

Angle Classification:

Simple CNN with 4 output classes
Determines if text is: 0°, 90°, 180°, or 270° rotated
Runs on each detected text bounding box

BERT Text Recognition:

Pre-trained language model for text understanding
Input: Rotated text image (now horizontal)
Output: Recognized text string
Accuracy: ~95% on clean technical text

Roughness Specification Types

Ra - Arithmetic mean (most common)
Rz - Ten-point height average
Rmax - Maximum roughness height
Rq - Root mean square (RMS)

Model Details

YOLOX Variant: YOLOX-s (small)
SAHI Tile Size: 512×512 with 50% overlap
Angle Classifier: MobileNet-v2 backbone
Total Pipeline Stages: 6 sequential stages
Bottleneck Stage: SAHI text detection (most time-consuming)

Framework: LitServe | Speed: 50ms (GPU) | Type: Multi-stage OCR pipeline

7. thickness_estimation - Thickness Symbol Detection

What it does

Detects thickness specification symbols in technical drawings. Locates all regions where thickness or wall thickness is specified.

📍 Real example: Input: Drawing with "2.5mm THICKNESS" notation → Output: {thickness_symbols: [{value: "2.5mm", location: [x1, y1, x2, y2], confidence: 0.89}]}

Architecture

Input Image (2048×2048 grayscale) ↓ [Preprocessing] │ ├─ Normalize to grayscale ├─ Apply histogram equalization └─ Input range: [0, 1] ↓ [YOLOX-l Detection] ← Large variant for accuracy │ ├─ Backbone (ResNet-50 style) │ • 5 convolutional blocks │ • Feature extraction at multiple scales │ ├─ Neck (FPN - Feature Pyramid Network) │ • Combines multi-scale features │ • P3, P4, P5 pyramid levels │ └─ Head (Dense Predictions) For each grid cell (up to 64×64): • Bounding box: [x, y, w, h] • Objectness score: P(object exists) • Class probability: P(thickness spec) ↓ [Detections per Grid Cell] │ ├─ Output: Thousands of raw detections ├─ Confidence threshold: 0.5 └─ Filter low-confidence boxes ↓ [NMS (Non-Maximum Suppression)] │ ├─ IoU threshold: 0.45 ├─ Remove overlapping detections ├─ Keep highest confidence boxes └─ Typical output: 5-20 detections per image ↓ [Post-processing] │ ├─ Convert normalized coords to pixel coords ├─ Calculate confidence score └─ Link nearby text regions (optional) ↓ JSON Output: { "thickness_symbols": [ { "value": "2.5mm", "bbox": [0.1, 0.2, 0.3, 0.4], "confidence": 0.89, "location_pixels": [205, 409, 614, 819] } ] }

Why YOLOX-l (Large Variant)?

Accuracy over Speed: Thickness specifications are critical manufacturing specs
Better Detection: YOLOX-l has more parameters (54M vs 8M for YOLOX-s)
Trade-off: 15ms vs 5ms (3× slower but much more accurate)
Standard Practice: Manufacturing specs require high precision

Special Features

Dynamic Batching:

Max batch size: 4 images
If 10 requests arrive: process in batches of (4 + 4 + 2)
Per-request latency: still ~15ms (not 60ms)
Throughput: 4× improvement compared to sequential processing

Grayscale Input:

Technical drawings are often grayscale (scanned blueprints)
Saves bandwidth (1 channel vs 3 RGB channels)
Model trained on grayscale images specifically

Types of Thickness Specs Detected

Uniform Thickness: "2.5mm" (entire part is this thickness)
Variable Thickness: "2.5mm-5.0mm" (range specification)
Wall Thickness: "1.5mm walls" (for hollow parts)
Sheet Thickness: "Gauge 16" (for sheet metal)
Min/Max: "Min 2.0mm, Max 3.0mm"

Model Details

YOLOX Variant: YOLOX-l (large, more accurate)
Input Format: Grayscale (single channel)
Input Size: 2048×2048
Parameters: ~54M
Model Size: ~200MB
Speed per Image: 15ms (GPU), 150ms (CPU)
Batching Support: Yes (max batch 4)

Performance Characteristics

Precision: ~92% (detects real thickness specs)
Recall: ~88% (finds most thickness specs in drawing)
False Positive Rate: ~5% (occasionally misses specs)
False Negative Rate: ~12% (sometimes detects non-specs)

Framework: LitServe | Speed: 15ms (GPU) | Batch Size: 4 | Type: Object Detection

8. manufactured_size_estimation - Size Marking Detection

What it does

Detects rectangular regions containing size markings (length, width, height specifications). Similar to thickness_estimation but focuses on overall part dimensions.

📍 Real example: Input: "25mm × 50mm × 10mm" dimensions → Output: {detected_regions: [{class: "manufactured_size", confidence: 0.88, bbox: [...]}]}

Architecture

Input Image (2048×2048) ↓ [YOLOX-l Detection] ← Large variant for precision │ ├─ Backbone (ResNet-50) │ • 5 convolutional stages │ • Progressive downsampling │ ├─ Neck (FPN) │ • Multi-scale feature fusion │ • P3, P4, P5 levels │ └─ Head (Dense Predictions) For each grid cell: • Bounding box [x, y, w, h] • Objectness: P(size region exists) • Confidence: P(manufactured_size) ↓ [2 Classes Detected] ├─ manufactured_size (size region detected) └─ no_input (empty region) ↓ [Detection Filtering] │ ├─ Confidence threshold: 0.5 ├─ Remove low-confidence boxes └─ Typical: 3-15 detections per image ↓ [NMS Post-processing] │ ├─ IoU threshold: 0.45 ├─ Remove overlapping detections └─ Keep highest confidence only ↓ JSON Output: { "detected_regions": [ { "class": "manufactured_size", "bbox": [0.1, 0.2, 0.5, 0.6], "confidence": 0.88, "location_pixels": [205, 409, 1024, 1228] } ] }

Difference from Thickness Estimation

Thickness: Detects wall/material thickness specs only
Size Estimation: Detects overall part dimensions (L×W×H)
Both use: YOLOX-l with dynamic batching
Different Training: Trained on different datasets

Size Marking Types

Length × Width × Height: "25 × 50 × 10 mm"
Diameter × Length: "Ø25 × 100 mm" (for cylinders)
Radius + Height: "R50, H75 mm"
Coordinates: "X100, Y50, Z25"

Model Details

YOLOX Variant: YOLOX-l (large, for accuracy)
Input Size: 2048×2048
Classes: 2 (manufactured_size, no_input)
Parameters: ~54M
Model Size: ~200MB
Batching: Max batch size 4
Speed: 15ms (GPU), 150ms (CPU)

Framework: LitServe | Speed: 15ms (GPU) | Batch Size: 4 | Type: Object Detection

9. geometric_tolerance_detection - Geometric Tolerance Extraction

What it does

Detects geometric tolerance symbols (like "perpendicularity ±0.1mm") and extracts their values. This is one of the most complex models, extracting both symbol type and numeric specifications.

Architecture (6-Stage OCR Pipeline)

Input Image (2048×2048) ↓ [Stage 1: YOLOX Detection] ← Find tolerance symbols │ ├─ Detects 14 types of symbols ├─ Gets bounding boxes └─ Confidence scores ↓ [Stage 2: SAHI Text Detection] │ ├─ Find associated tolerance values ├─ Tile-based detection (512×512) ├─ Merge overlapping results └─ Output: Text bounding boxes ↓ [Stage 3: Angle Classification] │ ├─ Determine text rotation ├─ 4 classes: 0°, 90°, 180°, 270° └─ CNN classifier (MobileNet-v2) ↓ [Stage 4: Text Rotation] │ ├─ Rotate bounding box to horizontal ├─ Apply perspective transform └─ Ensure consistent orientation ↓ [Stage 5: Vision Encoder-Decoder] │ ├─ Advanced OCR model ├─ Combines visual + language understanding ├─ Better than BERT for complex text └─ Reads: "0.05", "datum A", "φ 5mm" ↓ [Stage 6: Post-processing] │ ├─ Parse tolerance value: "0.05" ├─ Extract datum reference: "A", "B", "ABC" ├─ Identify feature: "surface", "hole", "edge" ├─ Validate tolerance type └─ Link symbol to values ↓ JSON Output: { "tolerances": [ { "symbol_type": "perpendicularity", "value": 0.05, "unit": "mm", "datum": "A", "feature": "surface", "confidence": 0.92, "location": [x1, y1, x2, y2] } ] }

14 Tolerance Types Detected

1. Straightness - How straight a line is (±0.05mm)
2. Flatness - How flat a surface is (±0.05mm)
3. Roundness - How round a circle is (±0.05mm)
4. Cylindricity - How cylindrical a shaft is (±0.1mm)
5. Line Profile - Shape of a 2D feature (±0.05mm)
6. Surface Profile - Shape of a 3D feature (±0.1mm)
7. Perpendicularity - Angle to datum (±0.05mm to datum A)
8. Parallelism - Parallel to datum (±0.05mm to datum B)
9. Angularity - Angle tolerance (±30° to datum C)
10. Position - Location of feature (±0.05mm to datum ABC)
11. Concentricity - Feature centered on datum (±0.05mm)
12. Symmetry - Bilateral symmetry tolerance (±0.05mm)
13. Circular Runout - Rotation accuracy per revolution (±0.05mm)
14. Total Runout - Overall rotation accuracy (±0.1mm)

Why This is Complex

Symbol Recognition: 14 different geometric symbols to identify
Datum References: Can reference 1-3 datums (A, B, C, etc.)
Text Parsing: Extract numbers, units, datum letters
Precision Required: Tolerances are critical for manufacturing
Compound Specs: Some specs combine multiple elements

Vision Encoder-Decoder (Not BERT)

Why not BERT? Geometric tolerance text is complex and specialized
Encoder Part: Extracts visual features from rotated text
Decoder Part: Generates text character-by-character
Language Understanding: Understands tolerance syntax
Accuracy: ~97% on technical text (vs ~95% for BERT)

Model Pipeline

YOLOX Detection: YOLOX-s for symbol finding
SAHI Text Detection: 512×512 tiled detection
Angle Classification: MobileNet-v2 (4 classes)
Text Recognition: Vision Encoder-Decoder
Post-processing: Validate and format results

Output Example

Drawing shows: ⊥ symbol with "0.05 A B C"

Model outputs:

symbol_type: "perpendicularity"
value: 0.05
unit: "mm"
datum: ["A", "B", "C"] (referenced datums)
confidence: 0.92

Framework: LitServe | Speed: 60ms (GPU) | Type: Multi-stage symbol + OCR extraction

10. drawing_or_not_classification - Drawing Classifier

What it does

Binary classification - is the image a technical drawing or not? Simple but important preprocessing model.

📍 Real example: Input 1: Blueprint → "drawing" (conf: 0.99) | Input 2: Cat photo → "not_drawing" (conf: 0.98)

Architecture (Simple CNN)

Input Image (variable size) ↓ [Resize to 224×224] ↓ [CNN Classifier] │ ├─ Conv Block 1 (32 filters) │ Detect edges, simple patterns │ ├─ Conv Block 2 (64 filters) │ Detect textures, lines │ ├─ Conv Block 3 (128 filters) │ Detect complex shapes │ └─ Conv Block 4 (256 filters) Detect technical drawing features ↓ [Global Average Pooling] │ Compress spatial dimensions │ Keep feature information ↓ [Fully Connected Layer 1] (512 neurons) │ Feature reasoning ↓ [Dropout] (0.5 rate) │ Regularization ↓ [Fully Connected Layer 2] (256 neurons) ↓ [Output Layer - Softmax] │ 2 classes ↓ Output Probabilities: ├─ drawing: 0.99 └─ not_drawing: 0.01 ↓ Prediction: "drawing"

Why Binary Classification?

Preprocessing Filter: Avoid processing non-drawings with expensive models
Fast Rejection: Reject non-drawings quickly (5ms)
Cost Savings: Don't waste GPU time on photos, screenshots, etc.
Error Prevention: Other models expect drawings, not arbitrary images

Classes Detected

drawing: Technical drawings (blueprints, CAD drawings, schematics)
not_drawing: Photos, screenshots, documents, artwork, etc.

What Makes Something a "Drawing"?

Technical drawings: Blueprints with dimension lines, symbols
CAD drawings: Computer-generated technical drawings
Hand-sketches: Engineered sketches with technical intent
Schematics: Electrical/circuit diagrams

What Gets Rejected?

Photographs (objects, landscapes, people)
Screenshots (UI, documents)
Art (paintings, drawings without technical intent)
Scanned documents (text-heavy documents)
Logos and graphics

Model Details

Architecture: ResNet-18 or MobileNet-v2 (simple CNN)
Parameters: ~11M
Model Size: ~45MB
Input Size: 224×224
Classes: 2 (drawing, not_drawing)
Accuracy: ~98% on test set

Performance Characteristics

Precision: ~99% (true positive rate for drawings)
Recall: ~97% (catches most real drawings)
False Positive Rate: ~1% (rarely misclassifies non-drawings)
Speed: 5ms (GPU), 50ms (CPU)

Framework: FastAPI (NOT LitServe) | Speed: 5ms (GPU) | Type: Binary Image Classification

11. dummy_inference - Test Server

What it does

Minimal dummy server for testing purposes. Always returns "hello world".

Purpose: When testing the API Gateway, you need a "fake" model that responds quickly without doing actual ML work.

Framework: FastAPI | Deployment: CPU-only, always running

Model Architectures Explained

What is a Neural Network?

A neural network is a computational model inspired by the human brain. It consists of layers of neurons (mathematical functions) that transform input data into output predictions.

Input Layer → Hidden Layers → Output Layer (Raw image) (Feature extraction) (Predictions)

YOLOX - Object Detection

What it does: Given an image, find all objects of interest and their locations.

            Used by 8 models: canvas_ocr, welding_symbol_detection, surface_roughness_detection, thickness_estimation, manufactured_size_estimation, geometric_tolerance_detection, part_table_recognition
        

EfficientNet - Feature Extraction

What it does: Convert an image into a feature vector (list of numbers).

Used by: deep_feature_v2 for converting images to 1024-dimensional vectors

Shared Infrastructure

All 11 models share common utilities located in shared/ directory:

Configuration

model_name
gpu_memory_fraction
max_batch_size
timeout

Common Components

Image Preprocessing: decode, resize, normalize
Error Handling: Custom exceptions for GPU, image, and model errors
Middleware: Access logging, concurrency limiting, versioning
Validation: Bounding box validation, NMS thresholding

Deployment & Usage

Deployment Environments

Environment	Configuration	Monitoring
Development (dev)	Single GPU pod, auto-restart	Direct logs to stdout
Staging (stg)	2 GPU pods with load balancer	Performance monitoring
Production (prod)	4 GPU pods, auto-scaling 2-8	Datadog metrics, alerting

Hardware Requirements

Type	RAM	Speed	Throughput
GPU (NVIDIA T4/V100)	16GB VRAM	10-50ms	~200 req/sec
CPU (4 cores)	8GB RAM	100-500ms	~20 req/sec

API Latency Expectations (on GPU):

drawing_or_not_classification: 5ms
deep_feature_v2: 10ms
thickness_estimation: 15ms
welding_symbol_detection: 20ms
canvas_ocr: 40ms
surface_roughness_detection: 50ms
geometric_tolerance_detection: 60ms
part_table_recognition: 100ms

📚 For detailed implementation, check individual model README files and API handlers
Last generated: June 2026

↑

📖 AI Models Guide

Getting Started

Models

Technical

🤖 Internal AI Models Guide

What is This Repository?

Purpose

Real-World Use Case

Architecture Overview

High-Level Pipeline

Key Technology Stack

1. canvas_ocr - Technical Drawing OCR

What it does

Architecture (4-Stage OCR Pipeline)

Detection Classes (6 types)

Why 4 Pipeline Stages?

How Canvas_OCR Works (Example)

Model Details

Tolerance Format Examples

2. deep_feature_v2 - Deep Feature Extraction

What it does

Architecture (Detailed)

EfficientNet-B2 Explained

Key Characteristics

3. welding_symbol_detection - Welding Symbol Detection

What it does

Architecture

How YOLOX Works

Common Welding Symbol Types (6 Classes)

Model Details

4. material_shape_classification - Material Shape Classification

What it does

Architecture

Possible Shape Classes

How CNN Classification Works

Model Details

5. part_table_recognition - Part Table Extraction

What it does

Complex Architecture (7-Stage Pipeline)

Why Table Recognition is Complex

Key Component: Table Transformer (ONNX)

Model Pipeline Details

6. surface_roughness_detection - Surface Roughness Extraction

What it does

What is Surface Roughness?

Architecture (Most Complex Pipeline - 6 Stages)

Why This Pipeline is Complex

Key Components Explained

Roughness Specification Types

Model Details

7. thickness_estimation - Thickness Symbol Detection

What it does

Architecture

Why YOLOX-l (Large Variant)?

Special Features

Types of Thickness Specs Detected

Model Details

Performance Characteristics

8. manufactured_size_estimation - Size Marking Detection

What it does

Architecture

Difference from Thickness Estimation

Size Marking Types

Model Details

9. geometric_tolerance_detection - Geometric Tolerance Extraction

What it does

Architecture (6-Stage OCR Pipeline)

14 Tolerance Types Detected

Why This is Complex

Vision Encoder-Decoder (Not BERT)

Model Pipeline

Output Example

10. drawing_or_not_classification - Drawing Classifier

What it does

Architecture (Simple CNN)

Why Binary Classification?

Classes Detected

What Makes Something a "Drawing"?

What Gets Rejected?

Model Details