Shapes: On-device Single-Stroke Shape Recognition
On-device single-stroke shape recognition that turns one hand-drawn stroke into a clean rectangle, ellipse, triangle, star, or line.
Shapes takes one hand-drawn stroke, a list of [x, y] points, and returns clean geometry: rectangle, ellipse, triangle, star, or line. It fits precise vector shapes, snaps to axes, circles, squares, and 15-degree rotations, and rejects scribbles that are not shapes.
On Apple it adds Notes-style smart-shape snapping to PencilKit with a single call, live preview intact. Apple's own "Snap to Shape" is private API third-party apps cannot call, so Shapes fills that gap. It is small enough to ship inside the app with no model download, and the web build runs pure-JS inference with no ONNX or WASM runtime.
Demo
Performance
Under 10 ms per stroke, from a 0.2 MB model with no inference runtime on the web.
Internal measurements. Pure-JS inference on the web, Core ML on Apple.
Use cases
Notes-style smart shapes
Add snap-to-shape to a PencilKit canvas with canvasView.enableShapeSnapping(). Live preview, undo and redo preserved, filling the gap left by Apple's private Snap to Shape API.
Diagram and whiteboard tools
Clean freehand strokes into precise geometry. Works with tldraw and Excalidraw stroke data.
Cross-platform drawing
The same recognizer runs on iOS, Android, and the web, so a drawing feature behaves identically everywhere.
No model download
At about 0.2 MB it ships inside the app. Unlike Google ML Kit Digital Ink there is no ~20 MB model to fetch at runtime.
What it does
- Recognizes rectangle, ellipse, triangle, star, and line, and rejects non-shapes
- Snaps to axes, circles, squares, and 15-degree rotations
- PencilKit live snapping via one call:
canvasView.enableShapeSnapping() - Pure-JS web inference: no ONNX or WASM runtime
Platforms and install
// Swift Package Manager .package(url: "https://github.com/Desert-Ant-Labs/shapes-swift", from: "0.1.0")
import Shapes let shape = try ShapeRecognizer().recognize(points: stroke) // Shape? -> rectangle, ellipse, triangle, star, line
// build.gradle.kts (via JitPack)
implementation("com.github.Desert-Ant-Labs:shapes-kotlin:0.1.0")
import ai.desertant.shapes.Shapes val shape = Shapes.recognize(points) // Shape? -> rectangle, ellipse, triangle, star, line
npm i @desert-ant-labs/shapes
import { recognize } from "@desert-ant-labs/shapes";
const shape = await recognize(points);
// { type: "ellipse", center: [120, 90], ... } or null
Specs
- Shapes
- Rectangle, ellipse, triangle, star, line
- On-device size
- 0.2 MB (4-bit Core ML on Apple)
- Latency
- <10 ms
- Web runtime
- None; pure-JS inference