preface
Pixel style first appeared in 8bit video games. Restricted by the size of computer memory and single display color, it can only use a small number of pixels to present content, but it has achieved many classic pixel games. With the improvement of memory capacity and screen resolution, the limitation of memory and display media is no longer a problem, and pixel style has slowly evolved into a unique creative style.
The general rendering process of pixel painting includes line drawing, color filling, etc., and pixel by pixel rendering requires a lot of time. Some popular art methods, such as line drawing and painting, have gradually appeared automatic or semiautomatic generation methods. This article will be implemented from scratchSLIC
[1] Algorithm, and implement a tool to generate pixel painting.
What is SLIC algorithm
The reason why pixel painting is not simple is that direct down sampling can not accurately capture key pixels, and it is easy to lose edge information. The generated pixel painting is often unsatisfactory. The purpose of manual line drawing and color filling is to select appropriate pixels. Therefore, our problem becomes how to select the appropriate pixels for color filling.
First, we introduce a concept – super pixel. Super pixel is an image segmentation technology proposed and developed by Xiaofeng Ren in 2003. It refers to irregular pixel blocks with certain visual significance composed of adjacent pixels with similar texture, color, brightness and other characteristics [1].
By dividing the picture into super pixels, similar pixel clusters can be obtained. Similar pixels are filled with the same color, and the pixel picture will be more reasonable.
The methods of superpixel segmentation include contour extraction, clustering, gradient rise and so on. paper[1]
ProposedSLIC
Super pixel segmentation algorithm (simple linear iterative clustering,simple linear iterative clustering
）Is one of them, which is based onKmeans
Clustering algorithm, clustering according to the color and distance characteristics of pixels to achieve good segmentation results, compared with several superpixel segmentation algorithms,SLIC
It has the advantages of simplicity, flexibility, good effect and fast processing speed.
How to implement SLIC algorithm
SLIC
The basic process is as follows:

Image preprocessing.
Image from
RGB
Color space conversion toCIELab
Color space,Lab
Color space is more in line with human visual perception of color. The distance in this space can reflect the color difference people feel, and the relevant calculation is more accurate.Lab
The color space also has three channels, which arel
，a
，b
Wherel
Represents brightness, and the value range is[0,100]
，a
Indicates the component from green to red, and the value range is[128,127]
，b
Represents the blue to yellow component, and the value range is[128,127]
。RGB
andLAB
There is no direct conversion formula between. You need toRGB
Convert toXYZ
Color space toLAB
, see the complete code at the end of the text for the code. 
Initialize the cluster center.
The number of super pixels, that is, the number of areas to be divided, is determined according to the parameters. Suppose the picture has
N
Pixels, estimated to be divided intoK
Super pixels, each with a size ofN/K
, the distance between adjacent centers isS=Sqr(N/K)
, getK
Cluster coordinates. 
Optimize the initial cluster center. In the cluster center
3*3
In the neighborhood, the pixel with the smallest gradient is selected as the new cluster center.The image is regarded as a twodimensional discrete function, and the gradient is the derivation of this function. When the values of adjacent pixels change, there will be a gradient, and the gradient of pixels on the edge is the largest. Moving the cluster center to the place with the minimum gradient can prevent it from falling on the edge contour and affect the clustering effect.
The gradient calculation of discrete gradient will not be derived in detail here. Since it contains several * squares and square root, it will be simplified to the operation of * like * square and * square root with absolute value. The simplified calculation coordinates are
(i,j)
The gradient formula of the pixel of is:among
（i+1,j）
And（i,j+1）
Is the coordinates of the point on the right side of the pixel and the point below the pixel.l(a,b)
by（a，b）
Luminance channel value of pixels on coordinatesl
。 
Calculate the distance between the pixel and the cluster center.
In the area with cluster center distance s
2S*2S
The distance between pixels and each cluster center is calculated in the neighborhood of.The distance here is the European distance, the total distance
D
fromdc
Color distance andds
Space distance is composed of two parts. The formula is as follows:If you directly
l
，a
，b
，x
，y
Spliced into a vector to calculate the distance. When the size of the super pixel changes,x
，y
For example, if a graph1000*1000
, the space distance can reach1000*Sqr(2)
, and the maximum color distance is only10*Sqr(2)
, resulting in the space distanceds
The weight proportion is too large.Therefore, it is necessary to normalize and divide by the maximum value, that is, the initial width of the super pixel
S
, map values to[0,1]
。And the color space distance will also be given a fixed value
m
To adjust the influence weight of color distance and space distance,m
The value range is[1,40]
。The distance formula becomes
When
m
Larger, color space divided bym
The smaller the value after, that is, the greater the weight of the spatial distance, the more regular the shape of the generated pixels will bem
The smaller the size, the greater the color distance weight, the more compact the super pixels will be at the edge, and the shape size will be more irregular. 
Pixel classification.
The category that marks each pixel is the category that is the smallest from its cluster center.

Recalculate cluster centers.
Calculate the * mean vector values of all pixels belonging to the same cluster, and get the cluster center again.

iteration
4～6
Process.Until the distance between the old cluster center and the new cluster center is less than a certain threshold or reaches a certain number of iterations, generally speaking, when the number of iterations reaches
10
, the algorithm can reach convergence. 
Clustering optimization.
At the end of the iteration, there may be isolated pixels that do not belong to the same connectivity domain as the cluster center, which can be assigned to the most * cluster label using the connectivity algorithm.
The paper does not give a specific implementation algorithm. The application scenario of this paper is to generate pixel paintings, which will be down sampled and not refined to each pixel. Therefore, this paper does not do clustering optimization.
To sum up, the SLIC algorithm process is roughly the same asKmeans
It is consistent. The cluster with the smallest distance is calculated iteratively. The difference is that only theS
The calculation of pixels within the distance reduces a lot of calculation.
Generate pixel painting
be based onSLIC
Algorithm, we can already divide a graph intoN
Super pixels. The pixels in each super pixel are relative. That is, each pixel is classified as a super pixel with a cluster center. Then assign the color of the pixel to the color of its cluster center to get the desired effect.
Set a certain stepstride
, usingCanvas
, everystride
Pixels, and assign the pixel to the color of its cluster center, that is, the final pixelation result is obtained.
Everyone’s subjective feelings about pixel painting are inconsistent. In order to let users have more choices and get their own satisfactory results. More manual intervention parameters can be exposed, such as canceling the termination condition of clustering optimization, and setting the number of iterations and the step size of the final pixel value by the user. Manually set parameters include
 Super pixel size
blocksize
；blocksize
The smaller the size, the finer the super pixel segmentation.  Iterations
iters
；iters
The larger the size, the more accurate the segmentation result and the longer the calculation time.  Color space weights
weight
；weight
The larger the, the greater the influence of color on the segmentation results.  Take pixel step
stride
；stride
The smaller, the more * super pixels are connected to the generated pixel map, and the more delicate it will be.
Implement user interface
As a tool, user interaction interface is naturally required. The frontend interface is based onHTML/Javascript/CSS
Build, useCanvas API
Draw image content, and the user interaction panel selectsdat.gui
[3] Library.dat.gui
Is a lightweight graphical interface library, which is very suitable for parameter modification. It is often used as a demonstration of visual demo. Supported parameter types includeNumber
、String
、Boolean
, custom functions, etc. Corresponding response events can be bound for different attributes, and the event will be triggered automatically when the attribute value changes.
Add the following attributes and events to the build pixelation tool:
 When
iters、stride、blockSize、weight
(color space weight m) restart when the parameter changesSLIC
Calculate the algorithm and redraw the calculation results;  add to
Upload image
AndExport image
Button, which supports users to upload pictures and download pixelated pictures;
When drawing an imageCanvas
Overlay one layer on the canvas layerCanvas
Canvas to visualize the results of the algorithm and add the following functions
grid
Switch to control whether to draw pixel grid;Centers
Switch to control whether the cluster center is displayed;Contours
Switch to control whether the cluster edge contour is displayed;
Cluster centerCenters
Direct use ofctx.fillRect
Just pass in the coordinates of the center point.
Superpixel outlineContours
The contour points need to be calculated first.
Each pixel and the surrounding8
Compare pixels. If the number of pixels with different cluster centers is greater than2
, it means that there are more than two points of different categories around this pixel, and this point is the outline. The effect is as follows:
Finally, we get a simple tool to generate pixel drawings.
Full version code address (JS version)
reference
[1] Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Su ̈sstrunk S. SLIC superpixels. Technical Report. IVRG CVLAB; 2010.
[2] Gerstner T , Decarlo D , Alexa M , et al. Pixelated image abstraction with integrated user constraints[J]. Computers & graphics, 2013.
[3] https://github.com/dataarts/dat.gui
Welcome to bump lab blog:aotu.io
Or follow aotulabs official account and push articles from time to time: