Edge Masking

Mix.install([
  {:image, "~> 0.42"},
  {:req, "> 0.0.0"},
  {:kino, "> 0.0.0"}
])

Introduction

Image edge masking is used to create an alpha mask from the edges of an image. The mask can then be used to add transparency to a base image or to perform transformations to a region of an image.

Open the base image

Our base image is a jigsaw piece. It has a dark border around an orange fill on a white background. We’ll source it from github.

{:ok, response} =
  Req.get(
    "https://raw.githubusercontent.com/elixir-image/image/main/test/support/images/jigsaw.png"
  )

jigsaw = Image.open!(response.body)

Edge Detection

There are several ways to do edge detection. Vix, which powers the Image library and which is itself based upon the amazing libvips, provides both canny and sobel edge detection functions. For this example, we can use the uniformity of the image colors to simplify the edge detection by converting the image to greyscale and multipling all the pixels in the image by 3. This will result in all but the dark edge pixels becoming white. Why 3? Its just a number found by empirical means to deliver the required result.

Image provides several basic math functions in the Image.Math module. For clarity and in alignment with the goals of Elixir to be explicit, those functions are to be preferred. However there are cases where being explicit also introduces confusion. In these cases we can use Image.Math to override the basic arithmetic operators. We’ll use Image.Math for some of this tutorial.

# `using` Image.Math means the `*` operator is overriden and will call 
# `Image.Math.multiply(jigsaw, 3)`.
use Image.Math
{ok, grey_jigsaw} = Image.to_colorspace(jigsaw, :bw)
edges = grey_jigsaw * 3

Refining the selection

Now we have the edge (that was easy!) on a white background. But the edge is quite faint, in the final image, a slightly thicker edge would be easier to visualise. We use the concept of dilation and erosion to increase the thickness of the edge. Image.dilate/2 and Image.erode/2 support those functions.

Note that dilation is the process of increasing the volume of light areas and erosion is the idea of increasing the volume of dark areas. Therefore Image.erode/2 is the tool we want.

thick_edges = Image.erode!(edges, 3)[0]

Notice that we used Image.erode!(edges, 3)[0]. The [0] uses the Access Behaviour to return just the 0th band of the image (band is the term used in libvips, you might know it as a channel in OpenCV).

Now we have strong edges, but we can now see there are varying greyscale values and we would much prefer uniform black for the edge mask. Here we can use the handy Image.if_then_else/3 function. In this example, if the pixel value is > 200 in the thick_edges image, then set the corresponding pixel to 255 (white) and if not, set it to 0 (black).

mask = Image.if_then_else!(thick_edges > 200, 255, 0)

Thats it for creating our mask. We’ll use this image as the alpha mask on our final image to define transparency.

Creating an image outline

In our example we want to produce a coloured outline of the jigsaw set on a transparent background. We’ve now created the mask that will manage the transparency part. How do we create the green outline?

We can use our old friend Image.if_then_else/3 for that.

green_outline = Image.if_then_else!(mask < 100, :green, :white)

Compositing the outline and the mask

Now we have our mask image in mask and our green outline in green_outline. How do we apply the mask to the green outline? We can use the Image.add_alpha/2 function to use mask as our alpha layer for the green_outline image.

You’ll see that we are using only the 0th band of the mask image since the we just need one band for the alpha - and in our mask image all three bands are the same since its a black and white image.

Image.add_alpha!(green_outline, mask[0])

Oh no - what happened! The image has become all white! That because our mask image is actually the inverse of how masking works. For an alpha mask, black is transparent and white is opaque. We can invert the mask with Image.invert!/1.

inverted_mask = Image.invert!(mask[0])
final_image = Image.add_alpha!(green_outline, inverted_mask)

And there we have it. A green outline on a transparent background. Liveview and Kino don’t yet have a way to show the transparency for visual checking but you can use the code block below to same the image somewhere and check in your favourite imaging app. The file preview function on MacOS works well for that.

Make sure you save the image to a .png file, or other image format that supports transparency. Note that JPEG does not support transparency.

# Image.write(final_image, "some/path/to/final_image.png")

{:ok, %Vix.Vips.Image{ref: #Reference<0.4180048996.1535508509.233750>}}

Summary

If you got this far then well done and congratulations on your patience. You’ll have worked out that most of the process described here can be collapsed into a couple of short pipelines - thats certainly what I would do.