ABC Tools

Seam Carving, Explained

After reading this you will understand how content-aware resizing finds and removes the least interesting pixels, how to read the energy map, and when carving beats a plain resize.

What seam carving does

Ordinary resizing scales every pixel by the same factor. Shrink a 1200-pixel-wide photo to 800 and every column loses a third of its width, subject and background alike. A person becomes 33% narrower. Seam carving takes a different route: it removes whole vertical paths of pixels that run through the flattest parts of the image, leaving the busy regions untouched.

Picture a beach photo, 1200 pixels wide, with a surfer on the left third and open water filling the rest. To reach 800 pixels wide, seam carving deletes 400 vertical seams. Almost all of them thread through the water, because water is smooth and cheap to remove. The surfer keeps her exact proportions. A plain resize would have squashed her by a third.

Each seam is a connected top-to-bottom path, one pixel per row, where each step moves down by one row and left, straight, or right by at most one column. Remove it and every pixel to its right shifts left by one. Do this 400 times and the image is 400 pixels narrower.

When to use it, and when not to

Carving wins when the image has large low-detail regions: sky, water, grass, a plain studio backdrop, an empty wall. Those regions donate pixels without visible damage.

It fails on images with detail everywhere. Faces, text, brick walls, tiled floors, and repeating patterns have no smooth path to spare. Every possible seam crosses something the eye is watching, so the algorithm has to cut through it. Distortions usually appear once you carve past roughly a third of the width. Carve 400 seams from that 1200-pixel beach and it holds; carve 600 and the surfer starts to warp.

If you only need exact dimensions and the image is busy everywhere, a plain Image Resizer or an Aspect-Ratio Crop will look better than an over-carved result. Carving is worth it when you want to change the aspect ratio without cropping and without squashing the subject.

The energy map

Before it can pick a cheap seam, the algorithm needs a number for every pixel that says how interesting it is. That number is the energy. A common choice is the gradient magnitude: how much the pixel differs from its neighbours.

e(x, y) = \left| \frac{\partial I}{\partial x} \right| + \left| \frac{\partial I}{\partial y} \right|

Here I is the pixel brightness, \partial I / \partial x is the change from the pixel on the left to the pixel on the right, and \partial I / \partial y is the change from the pixel above to the pixel below. Flat regions give small differences and low energy. Edges give large differences and high energy.

Work an example by hand. Take a pixel whose left neighbour has brightness 120 and right neighbour 128, so the horizontal difference is |128 - 120| = 8. The pixel above is 119 and below is 121, a vertical difference of |121 - 119| = 2. The energy is 8 + 2 = 10, a low value. That pixel sits in a smooth patch and is a good candidate for removal. A pixel on a hard edge might have differences of 150 and 90, energy 240, and the algorithm will route around it.

Finding the cheapest seam

A seam is the connected top-to-bottom path with the smallest total energy. Checking every possible path is hopeless: on an image 500 rows tall there are roughly 3^{499} of them. Dynamic programming finds the best one in a single pass.

Build a cost table M the same size as the image. The first row equals the energy. Each pixel below records the cheapest way to reach it from the row above.

M(x, y) = e(x, y) + \min\big( M(x-1, y-1),\ M(x, y-1),\ M(x+1, y-1) \big)

The three terms inside the minimum are the pixel's three possible parents in the row above: up-left, straight up, and up-right. After you fill the table, the smallest value in the bottom row is the total cost of the cheapest seam. Walk back up, always stepping to the parent that supplied the minimum, and you have the seam. The whole pass is linear in the number of pixels.

A worked example on the demo image

Carving a small grid

The tool's demo uses its field defaults. To see the arithmetic, shrink it to a 4-column energy grid and carve one vertical seam. Suppose the energy values are these:

Energy e(x, y) for a 4 by 4 patch
rowcol 0col 1col 2col 3
09268
17319
25824
36372
  1. Row 0 of M equals the energy: 9, 2, 6, 8.
  2. Row 1, col 1: energy 3 plus \min(9, 2, 6) = 2, giving 5. Col 2: 1 plus \min(2, 6, 8) = 2, giving 3. Full row: 16, 5, 3, 14.
  3. Row 2, col 2: energy 2 plus \min(5, 3, 14) = 3, giving 5. Full row: 10, 13, 5, 7.
  4. Row 3, col 3: energy 2 plus \min(13, 5, 7) = 5, giving 7. Full row: 15, 8, 12, 7.
  5. The smallest bottom-row value is 7 at col 3. Trace back: col 3, then col 2 (source of the 5), then col 2 (source of the 3), then col 1 (source of the 2).

The seam is columns 1, 2, 2, 3 from top to bottom, total energy 2 + 1 + 2 + 2 = 7. That is the cheapest of all paths. Remove those four pixels and each row is now three wide, with the pixels to the right of the seam shifted left by one.

Reading the results

The tool shows three things worth watching. The energy map is a greyscale picture: bright means high energy (edges, the subject), dark means low energy (smooth regions). The seams appear as thin lines that hug the dark valleys. Watch where they cluster. If they crowd into one background region, carving is safe. If they start crossing the subject, you are near the distortion limit.

The comparison against a plain resize is the honest test. On a good candidate the carved subject stays the same shape while the plain resize is visibly narrower. On a bad candidate both look wrong, and the plain resize often looks less strange because its distortion is uniform rather than localised.

Energy removed per seam stays low at first, then climbs steeply. The shaded region past 400 seams (a third of 1200) is where cheap paths run out and the algorithm starts cutting the subject.

As you carve more seams, the energy of each new seam rises. On a 1200-pixel image with a plain background the first 400 seams each cost under 20 energy units, while the 600th seam costs over 200 because every remaining path crosses the subject.

Protecting and removing regions

You can override the energy map by painting. Paint a region to protect and its energy is set very high, so no seam will cross it: use this on a face the gradient alone rated as removable. Paint a region to erase and its energy is set very low or negative, so seams are pulled through it first, deleting an unwanted object in a few dozen seams before normal carving resumes.

Object removal is the most useful trick here. Mark a stray sign in a landscape, carve until the marked pixels are gone, then optionally insert seams to restore the original width. The sign disappears and the background closes over the gap.

Inserting seams to stretch

The same machinery enlarges an image. Instead of removing the cheapest seam, find it, then insert a new column beside it by averaging its neighbours. To add 200 columns, find the 200 cheapest seams first (not one at a time, or you would keep re-finding the same path and stretch one strip), then duplicate them together. The subject keeps its proportions while the smooth regions gain width.

The download is the reduced working size used for the animation, not your original resolution. If the result looks soft, pick a larger working size before carving. For genuine enlargement beyond the original pixels, an Image Upscaler reconstructs detail; seam insertion only spreads existing pixels apart.

Common mistakes

Three problems account for most disappointing results.

Carving too far
Past about a third of the dimension the cheap seams are exhausted. On the 1200-pixel example, seams 1 to 400 average under 20 energy each; seam 550 costs over 150. Watch the seam lines: when they stop hugging the background, stop carving.
Trusting gradient energy on faces
Skin is smooth, so a cheek can score lower energy than the sky behind it. The algorithm will happily carve a nose. Paint the face as protected before you start.
Using it as a file shrinker
Carving changes dimensions and content, not file efficiency. To make a file smaller at the same content, use the Image Compressor instead.

Related tools

Carving changes shape by removing content. When you need a different result, reach for a neighbour: the Favicon Generator builds every icon size from one square source, which is a fixed-size job rather than a content-aware one.

Frequently asked questions

Why does the subject keep its shape but the background shrinks?

Every removed seam runs through low-energy pixels, which live in the background. The subject has high energy, so no cheap seam crosses it and its pixels are never removed. It keeps its exact width while the background loses columns.

How many seams can I safely remove?

Roughly a third of the width or height on images with a plain background. On a 1200-pixel-wide photo that is about 400 seams. Beyond that the per-seam energy climbs steeply and the subject starts to warp.

Why is my downloaded image smaller than the original?

Large photos are carved at a reduced working size so the animation stays smooth, and the download is that working size. Choose a larger working size for a sharper result.

Can seam carving remove an object from a photo?

Yes. Paint the object as a region to erase, which sets its energy very low, and carve until the marked pixels are gone. Insert seams afterwards to restore the original width if you need it.

Is my image uploaded anywhere?

No. Everything runs locally in your browser and no image is sent to any server.

Try it yourself in the tool

Related tools

Image Compressor

Shrink JPEG, WebP and PNG files with a live size preview.

Image Resizer

Resize one image or a whole batch to exact dimensions.

Aspect-Ratio Crop

Crop images freely or locked to presets like 1:1 and 16:9.

Favicon Generator

Every favicon size plus manifest from one image.

Image Upscaler

Enlarge images 2-4x with Lanczos or pixel-art algorithms.

EXIF Viewer & Remover

See and strip photo metadata: camera, GPS, dates.