In the first part of this series, we looked at how to open images and perform some basic manipulations using Scikit Image. This second part will introduce you to digital image filters!

## Digital filters for image analysis

Often we need to perform some operation on our images. For example, we may want to reduce noise, find object edges in the image. Digital filters can help with that!

In image analysis, a digital filter is a mathematical operation that is applied to the image to perform specific operation on it. The output image results from the application of said operation in a pixel-wise manner on the input image.

Usually we apply the operation to a region around each pixel of interest, most often square or rectangular; we then slide this region throughout the image to apply the operation to each pixel. This sliding window is called the filter **kernel**.

## Simple filters: min, max, mean, median

These are probably the simplest examples of filters. They consist of a $n \times m $ kernel that “moves” through the image and applies a specific operation (e.g. the minimum) to the neighbourhood of each pixel.

For instance, imagine that our image contains the following values

$$\begin{bmatrix}10 & 15 & 25 & 38\\115 & 22 & 28 & 125\\ 31 & 47 & 34 & 32\\220 & 216 & 83 & 92\end{bmatrix}$$

We want to apply a $3 \times 3$ maximum filter to the image. Let’s consider the value 47 in the original image. The filter will take a $3 \times 3$ neighbourhood centred around it and find the maximum value. So the neighbourhood is

$$\begin{bmatrix}115 & 22 & 28\\ 31 & \mathbf{47} & 34\\220 & 216 & 83\end{bmatrix}$$

We apply our operation, in this case $max(X)$ to this sub-matrix. The maximum value is 220, which will become the value of the pixel at that position in the output image. For the next pixel (original value 34) the same filter will return 216.

### Edge pixels

You may be wondering what to do with pixels at the edge. After all, the top left pixel in the example above does not have a 3×3 neighborhood. There are various ways around this.

- You can
**crop the kernel**, simply ignoring any pixel that is not there. So, you will just use smaller neighbourhoods for edge pixels. - You can
**extend the image**by repeating the border pixels values. - Alternatively you can
**wrap**the image around. When doing this, the left neighbours of the pixels in the first column will be the pixels in the last column, the top neighbours of the first row of pixels will be those in the last row and so on. - A different solution is to
**mirror**the image at the edges. - Finally you can
**crop the image**, and not apply the filter to the edge pixels. This will, however, result in a smaller output image.

The schematic below summarises these options for the top-left pixel of an image

### Basic filters in Scikit Image

The Scikit Image `filters`

submodule contains a lot of different functions to apply filters to images. The simple filters described above are found within the `filters.rank`

submodule.

Let’s start by loading an image. This shows some cell nuclei, and is quite noisy.

```
from skimage.io import imread, imshow
nuclei = imread("nuclei_noisy.tif")
imshow(nuclei)
```

We can now apply the minimum, maximum, mean and median filters to the image

```
import numpy as np
from skimage.filters.rank import minimum, maximum, mean, median
img_min = minimum(nuclei, selem=np.ones((5, 5)))
img_max = maximum(nuclei, selem=np.ones((5, 5)))
img_med = median(nuclei, selem=np.ones((5, 5)))
img_mean = mean(nuclei, selem=np.ones((5, 5)))
```

The `selem`

parameter defines the kernel as a matrix of 0 and 1. We use `np.ones`

to define a $5 \times 5$ matrix of 1. We can use this to use kernels that are not square for example `selem = np.array[[0,1,0], [1,1,1], [0,1,0]]`

(or you can use Numpy functions such as `np.diamond`

, or, `np.disk`

).

Here is what we get

In general, the minimum filter acts as a “shrinking” filter, while the maximum filter expands object borders. This are useful, for example, when detecting objects in an image.

The mean and median filter are good at removing noise, by eliminating the effect of very bright or very dark pixels; usually the median filter works better, and is often used at the beginning of many image analysis pipelines.

## Convolutional filters

Similar to what discussed above, convolutional filters can be used to process images.

Convolution takes each pixel of the image, together with its neighbours, and adds them together, weighting the sum by the value of a kernel of the same size of the neighbourhood. The kernel can be of arbitrary size, and its values should generally sum to 1. So, for instance, having the following image and kernel

The output value for the pixel marked in red will be

$$255 * 0 + 50 * (-1) + 80 * 0 + 80 * (-1) + 80 * 5 + 0 * (-1) +$$

$$255 * 0 + 50 * (-1) + 80 * 0 = \mathbf{220}$$

A very commonly used convolutional filter is the Gaussian filter. Some examples of Gaussian kernels are shown here

The Gaussian filter has the effect of blurring the image. For example

```
from skimage.filters import gaussian
# 5x5 kernel
imshow(gaussian(nuclei, sigma=5))
```

Subtracting the blurred version of an image from the original one, and adding back to the original has the effect of actually sharpening it! This filter is called unsharp mask, and is available in `skimage.filters`

. The `radius`

parameter is the kernel size for the Gaussian smoothing, and amount is a multiplier. The output of the filter is

$\text{output} = \text{original} + \text{amount} * (\text{original} – \text{blurred})$

```
from skimage.filters import unsharp_mask
nuclei_sharp = unsharp_mask(nuclei, radius=10, amount=1)
```

Another commonly used convolutional filter is the Sobel filter, used for edge detection. It uses two kernels, one for determining horizontal edges and one for vertical edges

```
from skimage.filters import sobel
nuclei_edges = sobel(nuclei)
```

There are many other filters you can apply, I would suggest having a look at the Explained Visually where you can play with convolutional filters! I hope this post gave you a brief introduction to filters and keep an eye for new parts of this series!