Pixel style first appeared in 8bit video games. Restricted by the size of computer memory and single display color, it can only use a small number of pixels to present content, but it has achieved many classic pixel games. With the improvement of memory capacity and screen resolution, the limitation of memory and display media is no longer a problem, and pixel style has slowly evolved into a unique creative style.
The general rendering process of pixel painting includes line drawing, color filling, etc., and pixel by pixel rendering requires a lot of time. Some popular art methods, such as line drawing and painting, have gradually appeared automatic or semi-automatic generation methods. This article will be implemented from scratch
SLIC Algorithm, and implement a tool to generate pixel painting.
What is SLIC algorithm
The reason why pixel painting is not simple is that direct down sampling can not accurately capture key pixels, and it is easy to lose edge information. The generated pixel painting is often unsatisfactory. The purpose of manual line drawing and color filling is to select appropriate pixels. Therefore, our problem becomes how to select the appropriate pixels for color filling.
First, we introduce a concept – super pixel. Super pixel is an image segmentation technology proposed and developed by Xiaofeng Ren in 2003. It refers to irregular pixel blocks with certain visual significance composed of adjacent pixels with similar texture, color, brightness and other characteristics .
By dividing the picture into super pixels, similar pixel clusters can be obtained. Similar pixels are filled with the same color, and the pixel picture will be more reasonable.
The methods of super-pixel segmentation include contour extraction, clustering, gradient rise and so on. paper
SLICSuper pixel segmentation algorithm (simple linear iterative clustering,
simple linear iterative clustering）Is one of them, which is based on
K-meansClustering algorithm, clustering according to the color and distance characteristics of pixels to achieve good segmentation results, compared with several super-pixel segmentation algorithms,
SLICIt has the advantages of simplicity, flexibility, good effect and fast processing speed.
How to implement SLIC algorithm
SLICThe basic process is as follows:
RGBColor space conversion to
LabColor space is more in line with human visual perception of color. The distance in this space can reflect the color difference people feel, and the relevant calculation is more accurate.
LabThe color space also has three channels, which are
lRepresents brightness, and the value range is
aIndicates the component from green to red, and the value range is
bRepresents the blue to yellow component, and the value range is
LABThere is no direct conversion formula between. You need to
XYZColor space to
LAB, see the complete code at the end of the text for the code.
Initialize the cluster center.
The number of super pixels, that is, the number of areas to be divided, is determined according to the parameters. Suppose the picture has
NPixels, estimated to be divided into
KSuper pixels, each with a size of
N/K, the distance between adjacent centers is
Optimize the initial cluster center. In the cluster center
3*3In the neighborhood, the pixel with the smallest gradient is selected as the new cluster center.
The image is regarded as a two-dimensional discrete function, and the gradient is the derivation of this function. When the values of adjacent pixels change, there will be a gradient, and the gradient of pixels on the edge is the largest. Moving the cluster center to the place with the minimum gradient can prevent it from falling on the edge contour and affect the clustering effect.
The gradient calculation of discrete gradient will not be derived in detail here. Since it contains several * squares and square root, it will be simplified to the operation of * like * square and * square root with absolute value. The simplified calculation coordinates are
(i,j)The gradient formula of the pixel of is:
（i,j+1）Is the coordinates of the point on the right side of the pixel and the point below the pixel.
（a，b）Luminance channel value of pixels on coordinates
Calculate the distance between the pixel and the cluster center.
In the area with cluster center distance s
2S*2SThe distance between pixels and each cluster center is calculated in the neighborhood of.
The distance here is the European distance, the total distance
dcColor distance and
dsSpace distance is composed of two parts. The formula is as follows:
If you directly
ySpliced into a vector to calculate the distance. When the size of the super pixel changes,
yFor example, if a graph
1000*1000, the space distance can reach
1000*Sqr(2), and the maximum color distance is only
10*Sqr(2), resulting in the space distance
dsThe weight proportion is too large.
Therefore, it is necessary to normalize and divide by the maximum value, that is, the initial width of the super pixel
S, map values to
And the color space distance will also be given a fixed value
mTo adjust the influence weight of color distance and space distance,
mThe value range is
The distance formula becomes
mLarger, color space divided by
mThe smaller the value after, that is, the greater the weight of the spatial distance, the more regular the shape of the generated pixels will be
mThe smaller the size, the greater the color distance weight, the more compact the super pixels will be at the edge, and the shape size will be more irregular.
The category that marks each pixel is the category that is the smallest from its cluster center.
Recalculate cluster centers.
Calculate the * mean vector values of all pixels belonging to the same cluster, and get the cluster center again.
Until the distance between the old cluster center and the new cluster center is less than a certain threshold or reaches a certain number of iterations, generally speaking, when the number of iterations reaches
10, the algorithm can reach convergence.
At the end of the iteration, there may be isolated pixels that do not belong to the same connectivity domain as the cluster center, which can be assigned to the most * cluster label using the connectivity algorithm.
The paper does not give a specific implementation algorithm. The application scenario of this paper is to generate pixel paintings, which will be down sampled and not refined to each pixel. Therefore, this paper does not do clustering optimization.
To sum up, the SLIC algorithm process is roughly the same as
K-meansIt is consistent. The cluster with the smallest distance is calculated iteratively. The difference is that only the
SThe calculation of pixels within the distance reduces a lot of calculation.
Generate pixel painting
be based on
SLICAlgorithm, we can already divide a graph into
NSuper pixels. The pixels in each super pixel are relative. That is, each pixel is classified as a super pixel with a cluster center. Then assign the color of the pixel to the color of its cluster center to get the desired effect.
Set a certain step
stridePixels, and assign the pixel to the color of its cluster center, that is, the final pixelation result is obtained.
Everyone’s subjective feelings about pixel painting are inconsistent. In order to let users have more choices and get their own satisfactory results. More manual intervention parameters can be exposed, such as canceling the termination condition of clustering optimization, and setting the number of iterations and the step size of the final pixel value by the user. Manually set parameters include
- Super pixel size
blocksizeThe smaller the size, the finer the super pixel segmentation.
itersThe larger the size, the more accurate the segmentation result and the longer the calculation time.
- Color space weights
weightThe larger the, the greater the influence of color on the segmentation results.
- Take pixel step
strideThe smaller, the more * super pixels are connected to the generated pixel map, and the more delicate it will be.
Implement user interface
As a tool, user interaction interface is naturally required. The front-end interface is based on
Canvas APIDraw image content, and the user interaction panel selects
dat.guiIs a lightweight graphical interface library, which is very suitable for parameter modification. It is often used as a demonstration of visual demo. Supported parameter types include
Boolean, custom functions, etc. Corresponding response events can be bound for different attributes, and the event will be triggered automatically when the attribute value changes.
Add the following attributes and events to the build pixelation tool:
iters、stride、blockSize、weight(color space weight m) restart when the parameter changes
SLICCalculate the algorithm and redraw the calculation results;
- add to
Export imageButton, which supports users to upload pictures and download pixelated pictures;
When drawing an image
CanvasOverlay one layer on the canvas layer
CanvasCanvas to visualize the results of the algorithm and add the following functions
gridSwitch to control whether to draw pixel grid;
CentersSwitch to control whether the cluster center is displayed;
ContoursSwitch to control whether the cluster edge contour is displayed;
CentersDirect use of
ctx.fillRectJust pass in the coordinates of the center point.
ContoursThe contour points need to be calculated first.
Each pixel and the surrounding
8Compare pixels. If the number of pixels with different cluster centers is greater than
2, it means that there are more than two points of different categories around this pixel, and this point is the outline. The effect is as follows:
Finally, we get a simple tool to generate pixel drawings.
 Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Su ̈sstrunk S. SLIC superpixels. Technical Report. IVRG CVLAB; 2010.
 Gerstner T , Decarlo D , Alexa M , et al. Pixelated image abstraction with integrated user constraints[J]. Computers & graphics, 2013.
Welcome to bump lab blog:aotu.io
Or follow aotulabs official account and push articles from time to time: