Vectorization — converting a raster image into vector paths — sounds simple in theory. In practice, it's one of the harder problems in computer graphics. A photograph contains millions of pixels, each a slightly different color. A useful SVG contains dozens to hundreds of clean, smooth curves. Bridging that gap requires several distinct algorithmic steps.
Color Quantization
The first step is reducing the color space. A typical photograph uses thousands of unique colors. Vectorization works by grouping pixels into regions of similar color, so we need to collapse those thousands into a manageable palette — usually 2 to 256 colors, depending on the use case.
The algorithm clusters colors in perceptual color space (CIELAB rather than RGB) so that the reduced palette looks natural to the human eye. Two colors that are numerically far apart in RGB might be perceptually similar, and vice versa. Getting this right is what makes the difference between a clean result and a muddy one.
Edge Detection and Region Segmentation
Once the image is quantized, each pixel belongs to exactly one color region. The next step is finding the boundaries between those regions. This produces a set of pixel-level contours — jagged, staircase-shaped outlines that follow the edges of each color area.
These raw contours are topologically complex. Regions can have holes, islands within holes, and deeply nested hierarchies. The algorithm builds a containment tree that captures which regions are inside which others, preserving the layering order that SVG will need.
Path Tracing and Curve Fitting
The jagged pixel contours need to become smooth curves. This is where the Potrace-family algorithms come in. The pixel boundary is first simplified into a polygon, then each segment of the polygon is approximated by a Bezier curve.
The fitting process balances two competing goals: accuracy (the curve should closely follow the original pixel boundary) and simplicity (fewer control points means a smaller, cleaner SVG). An error threshold controls this tradeoff. Too tight, and you get noisy paths that faithfully reproduce JPEG artifacts. Too loose, and fine details disappear.
Optimization
The raw SVG output from path tracing is functional but verbose. A final optimization pass cleans up the result:
- Merging adjacent paths that share the same fill color
- Removing redundant path nodes that don't affect the visual output
- Simplifying path commands — converting curves to lines where curvature is negligible
- Reordering elements to minimize SVG file size
A well-optimized SVG can be 50–70% smaller than the naive output with no visible difference.
Why Speed Matters
SVG.new's vectorizer is written in Rust, which gives us predictable, low-latency performance. A typical image vectorizes in under two seconds. This matters because the primary consumers of the API are automated workflows and AI agents that process images as part of a larger pipeline. A ten-second vectorization step would be a bottleneck. A sub-second one is invisible.
The combination of perceptual color science, topologically-aware segmentation, and aggressive path optimization is what produces SVGs that are genuinely useful — clean enough to edit, small enough to embed, and accurate enough to print.