Tool library for image processing

Time:2022-5-9

The image processing libraries listed below have the skills of blur, compression, clipping, rotation, synthesis and comparison. It can meet our basic operation of using pictures.

You will learn:

  • How to distinguish the types of pictures (non file suffix);
  • How to get the size of the picture (not right-click to view the picture information);
  • How to preview local pictures (not picture Readers);
  • How to realize image compression (non image compression tool);
  • How to operate bitmap pixel data (non PS and other image processing software);
  • How to realize picture steganography (not visible to the naked eye).

1、 Basic knowledge

1.1 bitmap

“Bitmap images, also known as dot matrix images or raster images, are composed of a single point called a pixel (picture element).”These points can be arranged and dyed differently to form a pattern. When you zoom in on the bitmap, you can see the countless individual blocks that make up the whole image. The effect of increasing the bitmap size is to increase a single pixel, making the lines and shapes appear uneven.

“Photos taken with a digital camera, pictures scanned by a scanner and computer screenshots are bitmap.”The characteristic of bitmap is that it can show the change of color and the subtle transition of color, and produce realistic effect. The disadvantage is that it needs to record the position and color value of each pixel when saving, which takes up a large storage space. Commonly used bitmap processing software includes Photoshop, painter and drawing tools of windows system.

Resolution is the insurmountable barrier of bitmap. When scaling and rotating the bitmap, new pixels cannot be produced. Therefore, the original pixels will be enlarged to fill the blank, which will make the picture appear unclear.

(image source:https://zh.wikipedia.org/wiki…

The small squares in the figure are called pixels. These small squares have a clear position and assigned color value. The color and position of the small square determine the appearance of the image.

Pixels can be regarded as indivisible units or elements in the whole image.“Indivisibility means that it cannot be cut into smaller units or elements. It exists in a small cell of a single color.”Each dot matrix image contains a certain number of pixels, which determine the size of the image on the screen.

1.2 vector diagram

The so-called vector graph is a graph described by lines and curves. The elements of these graphs are some points, lines, rectangles, polygons, circles and arcs,*「They are all calculated by mathematical formulas, and have the characteristics of no distortion after editing.*For example, the vector graphics of a painting actually form the outline of the outer frame by line segments, and the color displayed by the outer frame and the color enclosed by the outer frame determine the color of the painting.

“Vector graphics are mostly geometric graphics, which can be infinitely enlarged without color change and blur.”It is often used in the design of patterns, signs, VI, text, etc. Common software include CorelDRAW, illustrator, freehand, Xara, CAD, etc.

Here we take SVG, which is familiar to web developers(“Scalable vector graphics – scalable vector graphics”)As an example, let’s understand the structure of SVG:

Scalable vector graphics (SVG) is a graphic format based on extensible markup language (XML) to describe two-dimensional vector graphics. SVG is an open standard developed by W3C.

Svg mainly supports the following display objects:

  • Vector display objects, basic vector display objects include rectangle, circle, ellipse, polygon, straight line, arbitrary curve, etc;
  • Embedded external images, including PNG, JPEG, SVG, etc;
  • Text object.

After understanding the difference between bitmap and vector graph, let’s introduce the mathematical representation of bitmap.

1.3 mathematical representation of bitmap

The pixels of a bitmap are assigned specific position and color values. The color information of each pixel is represented by RGB combination or gray value.

According to the bit depth, bitmaps can be divided into 1, 4, 8, 16, 24 and 32-bit images. The more bits of information used by each pixel, the more colors available, the more realistic the color performance, and the larger the corresponding amount of data.

“1.3.1 binary image”

A pixel bitmap with a bit depth of 1 has only two possible values (black and white), so it is also called a binary image. The pixels of binary image are only black and white, so each pixel can be represented by 0 and 1.

For example, a 4 * 4 binary image:

1 1 0 1
1 1 0 1
1 0 0 0
1 0 1 0

“1.3.2 RGB image”

RGB image consists of three color channels, in which RGB represents the colors of red, green and blue channels. Each channel in an 8-bit / channel RGB image has 256 possible values, which means that the image has more than 16 million possible color values.

RGB images with 8 bits / channel (BPC) are sometimes referred to as 24 bit images (8 bits x 3 channels = 24 bit data / pixel). Generally, the bitmap represented by 24 bit RGB combined data bits is called true color bitmap.

RGB color images can be represented by three matrices: one represents the intensity of red in pixels, one represents green and the other represents blue.

(image source:https://freecontent.manning.c…

“The essence of image processing is actually to calculate these pixel matrices.”In fact, the image types in bitmap include gray image, index image and YUV image in addition to binary image and RGB image. We won’t introduce too much here. Interested partners, please check the relevant materials by yourself.

2、 Image processing library

2.1 AlloyImage

Professional image processing open source engine based on HTML 5.

https://github.com/AlloyTeam/…

Alloyimage is a professional image processing library based on HTML5 technology, from Tencent alloyteam team. It has the following features:

  • Based on multi-layer operation – the processing of one layer does not affect other layers;
  • 17 layer blending modes corresponding to PS – facilitate the seamless migration of PS processing tutorials;
  • A variety of basic filter processing effects – the basic filter is constantly enriched and expandable;
  • Basic image adjustment functions – hue, saturation, contrast, brightness, curve, etc;
  • Simple and fast API – chain processing, simple and easy to use API and flexible parameter transmission;
  • Multiple combination effect encapsulation – one sentence of code can easily realize one style;
  • Single and multi threading support with consistent interface – single and multi threading switching does not need to change one line of code, and multi threading maintains the fast API feature.

For this library, the use scenarios suggested by the alloyteam team are as follows:

  • Desktop software client embedded web page operation mode > > > packaged WebKit kernel: user large avatar upload style processing, user album style processing (average processing time < 1s);
  • Win8 Metro Application > > > the user uploads the avatar, and uploads the smaller picture style after processing (ie 10 under win8 supports multithreading);
  • Mobile app > > > Android platform and IOS platform need small picture style web processing, such as phonegap application, style processing when uploading online avatars, style processing when sharing pictures on mobile web, etc.

“Use example”

//$AI or alloyimage initializes an alloyimage object
var ps = $AI(img, 600).save('jpg', 0.6);
//Save saves the composite image as a Base64 format string
var string = AlloyImage(img).save('jpg', 0.8);
//Savefile downloads the composite image locally
img.onclick = function(){
  AlloyImage(this). Savefile ('processed image. JPG ', 0.8);
}

“Online example”

http://alloyteam.github.io/Al…

(image source:http://alloyteam.github.io/Al…

2.2 blurify

blurify.js is a tiny(~2kb) library to blurred pictures, support graceful downgrade from css mode to canvas mode.

https://github.com/JustClear/…

blurify. JS is a small JavaScript Library (about 2 KB) for image blur, and supports elegant degradation from CSS mode to canvas mode. The plug-in supports three modes:

  • CSS mode: UsingfilterAttribute, default mode;
  • Canvas mode: UsingcanvasExport Base64;
  • Auto mode: CSS} mode is preferred, otherwise it will automatically switch to canvas mode.

“Use example”

import blurify from 'blurify';
new blurify({
    images: document.querySelectorAll('.blurify'),
    blur: 6,
    mode: 'css',
});
// or in shorthand
blurify(6, document.querySelectorAll('.blurify'));

“Online example”

https://justclear.github.io/b…

(image source:https://justclear.github.io/b…

See here is not some small partners think it’s just fuzzy processing, feel not enjoyable, can you have something cooler. Hey, hey, whatever you ask! Po, come here right away“Cool”Midori, which is used to create animation for background images, uses three JS to write and use webgl. Originally, I wanted to give a demonstration diagram, but a single GIF file is too large, so I can only put an experience address. Interested partners can experience it by themselves.

Midori sample address:https://aeroheim.github.io/mi…

2.3 cropperjs

JavaScript image cropper.

https://github.com/fengyuanch…

Cropper. JS is a very powerful but simple image cutting tool. It can be configured very flexibly. It supports the use of mobile phones and modern browsers above IE9. It can be used to meet the needs such as cutting avatar upload, commodity picture editing and so on.

Cropper. JS supports the following features:

  • Support 39 configuration options;
  • Support 27 methods;
  • Support 6 events;
  • Support touch (mobile terminal);
  • Support zoom, rotate and flip;
  • Support clipping on canvas;
  • Support cutting images through canvas on the browser side;
  • Support processing EXIF direction information;
  • Cross browser support.

Exchangeable image file format (English: Exchangeable Image file format, officially referred to as EXIF) is a file format specially set for photos of digital cameras, which can record attribute information and shooting data of digital photos. EXIF can be attached to JPEG, tiff, riff and other files to add the content of shooting information of digital camera and the version information of index map or image processing software.

EXIF information starts with 0xffe1, and the last two bytes represent the length of EXIF information. Therefore, the maximum EXIF information is 64 kb, and TIFF format is used internally.

“Use example”

// import 'cropperjs/dist/cropper.css';
import Cropper from 'cropperjs';
const image = document.getElementById('image');
const cropper = new Cropper(image, {
  aspectRatio: 16 / 9,
  crop(event) {
    console.log(event.detail.x);
    console.log(event.detail.y);
    console.log(event.detail.width);
    console.log(event.detail.height);
    console.log(event.detail.rotate);
    console.log(event.detail.scaleX);
    console.log(event.detail.scaleY);
  },
});

“Online example”

https://fengyuanchen.github.i…

2.4 compressorjs

JavaScript image compressor.

https://github.com/fengyuanch…

Compressorjs is a JavaScript image compressor. Use browser nativecanvas.toBlobThe API does compression work, which means it is lossy compression. The common usage scenario is to pre compress the image on the browser before uploading it.

To realize image compression on the browser side, in addition to usingcanvas.toBlobIn addition to the API, you can also use another API provided by canvas, namelytoDataURLAPI, which receivestypeandencoderOptionsTwo optional parameters.

amongtypeIndicates the picture format. The default isimage/png。 andencoderOptionsUsed to represent the quality of the picture. When the specified picture format isimage/jpegorimage/webpIn this case, the quality of the picture can be selected from the range of 0 to 1. If the value range is exceeded, the default value will be used0.92, other parameters are ignored.

comparisoncanvas.toDataURLFor API,canvas.toBlobThe API is asynchronous, so there are more than onecallbackParameter, thiscallbackThe default first parameter of the callback method is the converted oneblobFile information.canvas.toBlobYour signature is as follows:

canvas.toBlob(callback, mimeType, qualityArgument)

“Use example”

import axios from 'axios';
import Compressor from 'compressorjs';
// <input type="file" id="file" accept="image/*">
document.getElementById('file').addEventListener('change', (e) => {
  const file = e.target.files[0];
  if (!file) {
    return;
  }
  new Compressor(file, {
    quality: 0.6,
    success(result) {
      const formData = new FormData();
      // The third parameter is required for server
      formData.append('file', result, result.name);
      // Send the compressed image file to server with XMLHttpRequest.
      axios.post('/path/to/upload', formData).then(() => {
        console.log('Upload success');
      });
    },
    error(err) {
      console.log(err.message);
    },
  });
});

“Online example”

https://fengyuanchen.github.i…

2.5 fabric.js

Javascript Canvas Library, SVG-to-Canvas (& canvas-to-SVG) Parser.

https://github.com/fabricjs/f…

Fabric. JS is a framework that allows you to easily use HTML5 canvas elements. It is an interactive object model located on the canvas element, and it is also a「SVG-to-canvas」Parser for.

Use fabric JS, you can create and fill objects on the canvas. The so-called object can be a simple geometry, such as rectangle, circle, ellipse, polygon, or more complex shape, containing hundreds or thousands of simple paths. Then, you can use the mouse to zoom, move and rotate these objects. And modify their properties – color, transparency, Z-index, etc. In addition, you can manipulate these objects together, that is, grouping them through simple mouse selection.

Fabric. JS supports all mainstream browsers. The specific compatibility is as follows:

  • Firefox 2+
  • Safari 3+
  • Opera 9.64+
  • Chrome (all versions)
  • IE10,IE11,Edge

“Use example”

<!DOCTYPE html>
<html>
<head></head>
<body>
    <canvas id="canvas" width="300" height="300"></canvas>
    <script src="lib/fabric.js"></script>
    <script> var canvas = new fabric.Canvas('canvas');
        var rect = new fabric.Rect({
            top : 100,
            left : 100,
            width : 60,
            height : 70,
            fill : 'red'
        });
        canvas.add(rect); </script>
</body>
</html>

“Online example”

http://fabricjs.com/kitchensink

(image source:https://github.com/fabricjs/f…

2.6 Resemble.js

Image analysis and comparison

https://github.com/rsmbl/Rese…

Resemble. JS uses HTML canvas and JavaScript to analyze and compare pictures. Compatible with nodes greater than 8.0 JS version.

“Use example”

//Compare two pictures
var diff = resemble(file)
    .compareTo(file2)
    .ignoreColors()
    .onComplete(function(data) {
        console.log(data);
     /*
     {
        misMatchPercentage : 100, // %
        isSameDimensions: true, // or false
        dimensionDifference: { width: 0, height: -1 }, 
        getImageDataUrl: function(){}
     }
    */
});

“Online example”

http://rsmbl.github.io/Resemb…

2.7 Pica

Resize image in browser with high quality and high speed

https://github.com/nodeca/pica

Pica can be used to resize an image in a browser without pixelation and is fairly fast. It automatically selects the best available technologies: webworkers, webassembly, createimagebitmap, pure JS.

With pica, you can:

  • Reduce the upload size of large images and save upload time;
  • Save server resources in image processing;
  • Generate thumbnails in the browser.

“Use example”

const pica = require('pica')();
//Resize Canvas / picture
pica.resize(from, to, {
  unsharpAmount: 80,
  unsharpRadius: 0.6,
  unsharpThreshold: 2
})
.then(result => console.log('resize done!'));
//Resize and convert to blob
pica.resize(from, to)
  .then(result => pica.toBlob(result, 'image/jpeg', 0.90))
  .then(blob => console.log('resized to canvas & created blob!'));

“Online example”

http://nodeca.github.io/pica/…

2.8 tui.image-editor

🍞🎨 Full-featured photo image editor using canvas. It is really easy, and it comes with great filters.

https://github.com/nhn/tui.im…

tui. Image editor is a full-featured image editor using HTML5 canvas. It is easy to use and provides powerful filters. At the same time, it supports image clipping, flipping, rotation, drawing, shape, text, mask and image filtering.

tui. The browser compatibility of image editor is as follows:

  • Chrome
  • Edge
  • Safari
  • Firefox
  • IE 10+

“Use example”

// Image editor
var imageEditor = new tui.ImageEditor("#tui-image-editor-container", {
     includeUI: {
       loadImage: {
         path: "img/sampleImage2.png",
         name: "SampleImage",
       },
       theme: blackTheme, // or whiteTheme
         initMenu: "filter",
         menuBarPosition: "bottom",
       },
       cssMaxWidth: 700,
       cssMaxHeight: 500,
       usageStatistics: false,
});
window.onresize = function () {
  imageEditor.ui.resizeEditor();
};

Online example

https://ui.toast.com/tui-imag…

2.9 gif.js

JavaScript GIF encoding library

https://github.com/jnordberg/…

gif. JS is a JavaScript GIF encoder running on the browser side. It uses typed arrays and web workers to render every frame in the background, which is really fast. The library can work in browsers that support: web workers, file API and typed arrays.

gif. JS browser compatibility is as follows:

  • Google Chrome
  • Firefox 17
  • Safari 6
  • Internet Explorer 10
  • Mobile Safari iOS 6

“Use example”

var gif = new GIF({
  workers: 2,
  quality: 10
});
// add an image element
gif.addFrame(imageElement);
// or a canvas element
gif.addFrame(canvasElement, {delay: 200});
// or copy the pixels from a canvas context
gif.addFrame(ctx, {copy: true});
gif.on('finished', function(blob) {
  window.open(URL.createObjectURL(blob));
});
gif.render();

“Online example”

http://jnordberg.github.io/gi…

2.10 Sharp

High performance Node.js image processing, the fastest module to resize JPEG, PNG, WebP and TIFF images. Uses the libvips library.

https://github.com/lovell/sharp

The typical application scenario of sharp is to convert large images in common formats into images in JPEG, PNG and webp formats with small size and network friendliness. Due to its internal use of libvips, image resizing is usually 4-5 times faster than setting with ImageMagick and graphicsmagick. In addition to supporting image resizing, sharp also supports functions such as rotation, extraction, synthesis and gamma correction.

Sharp supports reading JPEG, PNG, webp, tiff, GIF and SVG images. The output image can be JPEG, PNG, webp and TIFF format, or uncompressed original pixel data.

“Use example”

//Change image size
sharp(inputBuffer)
  .resize(320, 240)
  .toFile('output.webp', (err, info) => { ... });
       
//Rotate the input image and change the image size 
sharp('input.jpg')
  .rotate()
  .resize(200)
  .toBuffer()
  .then( data => { ... })
  .catch( err => { ... });

“Online example”

https://segmentfault.com/a/11…

This example is from the article “generating exclusive shared pictures by sharp” written by Po Ge in 18 years. It mainly uses the image synthesis function provided by sharp to generate exclusive shared posters for each user. Interested partners can read the original text.

const sharp = require("sharp");
const TextToSVG = require("text-to-svg");
const path = require("path");
//Load font file
const textToSVG = TextToSVG.loadSync(path.join(__dirname, "./simhei.ttf"));
//Create a circular SVG for Avatar clipping
const roundedCorners = new Buffer(
  '<svg><circle r="90" cx="90" cy="90"/></svg>'
);
//Set parameters related to SVG text elements
const attributes = { fill: "white" };
const svgOptions = {
  x: 0,
  y: 0,
  fontSize: 32,
  anchor: "top",
  attributes: attributes
};
/**
 *Generate SVG using text
 * @param {*} text 
 * @param {*} options 
 */
function textToSVGFn(text, options = svgOptions) {
  return textToSVG.getSVG(text, options);
}
/**
 *Layer overlay to generate shared pictures
 * @param {*} options 
 * 
 */
async function genShareImage(options) {
  const { backgroudPath, avatarPath, qrcodePath, 
    userName, words, likes, outFilePath
  } = options;
  //Background picture
  const backgroudBuffer = sharp(path.join(__dirname, backgroudPath)).toBuffer({
    resolveWithObject: true
  });
  const backgroundImageInfo = await backgroudBuffer;
  //Avatar picture
  const avatarBuffer = await genCircleAvatar(path.join(__dirname, avatarPath));
  //QR code picture
  const qrCodeBuffer = await sharp(path.join(__dirname, qrcodePath))
    .resize(180)
    .toBuffer({
      resolveWithObject: true
    });
  //User name
  const userNameSVG = textToSVGFn(userName);
  //User data
  Const; userdatasvg = texttosvgfn (` wrote ${words} words and got ${likes} likes';
  const userNameBuffer = await sharp(new Buffer(userNameSVG)).toBuffer({
    resolveWithObject: true
  });
  const userDataBuffer = await sharp(new Buffer(userDataSVG)).toBuffer({
    resolveWithObject: true
  });
  const buffers = [avatarBuffer, qrCodeBuffer, userNameBuffer, userDataBuffer];
  //Layer overlay parameter list
  const overlayOptions = [
    { top: 150, left: 230 },
    { top: 861, left: 227 },
    {
      top: 365,
      left: (backgroundImageInfo.info.width - userNameBuffer.info.width) / 2
    },
    {
      top: 435,
      left: (backgroundImageInfo.info.width - userDataBuffer.info.width) / 2
    }
  ];
  //Combine multiple layers: picture + text layer
  return buffers
    .reduce((input, overlay, index) => {
      return input.then(result => {
        console.dir(overlay.info);
        return sharp(result.data)
          .overlayWith(overlay.data, overlayOptions[index])
          .toBuffer({ resolveWithObject: true });
      });
    }, backgroudBuffer)
    .then((data) => {
      return sharp(data.data).toFile(outFilePath);
    }).catch(error => {
      throw new Error('Generate Share Image Failed.');
    });
}
/**
 *Generate a round head
 *@ param {*} avatarpath} avatarpath
 */
function genCircleAvatar(avatarPath) {
  return sharp(avatarPath)
    .resize(180, 180)
    .overlayWith(roundedCorners, { cutout: true })
    .png()
    .toBuffer({
      resolveWithObject: true
    });
}
module.exports = {
  genShareImage
};

3、 Practical examples

3.1 how to distinguish the types of pictures

“Computers don’t distinguish different picture types by the suffix of the picture, but by the magic number.”For some types of files, the contents of the first few bytes are fixed, and the file type can be determined according to the contents of these bytes.

The magic numbers corresponding to common picture types are shown in the following table:

file type file extension magic number
JPEG jpg/jpeg 0xFFD8FF
PNG png 0x89504E47
GIF gif 0x47494638(GIF8)
BMP bmp 0x424D

Here we take the avatar of Po Ge (Abao. PNG) as an example to verify whether the type of the picture is correct:

In the daily development process, if we encounter the scene of detecting image types, we can directly use some ready-made third-party libraries. For example, if you want to judge whether a picture is PNG type, you can use is PNG library, which supports both browser and node JS, the use example is as follows:

「Node.js」

// npm install read-chunk
const readChunk = require('read-chunk'); 
const isPng = require('is-png');
const buffer = readChunk.sync('unicorn.png', 0, 8);
isPng(buffer);
//=> true

「Browser」

(async () => {
 const response = await fetch('unicorn.png');
 const buffer = await response.arrayBuffer();
 isPng(new Uint8Array(buffer));
 //=> true
})();

3.2 how to obtain the size of the picture

The size, bit depth, color type and compression algorithm of the picture will be stored in the binary data of the file. Let’s continue to take Abao’s Avatar (Abao. PNG) as an example to understand the actual situation:

528 (decimal) = > 0x0210

560 (decimal) = > 0x0230

Therefore, if we want to obtain the size of the picture, we need to analyze the binary data of the picture according to different picture formats. Fortunately, we don’t need to do this by ourselves. Image size is the node JS library has helped us achieve the function of obtaining the file size of mainstream image types:

“Synchronization mode”

var sizeOf = require('image-size');
var dimensions = sizeOf('images/abao.png');
console.log(dimensions.width, dimensions.height);

“Asynchronous mode”

var sizeOf = require('image-size');
sizeOf('images/abao.png', function (err, dimensions) {
  console.log(dimensions.width, dimensions.height);
});

The image size library is quite powerful. In addition to supporting PNG format, it also supports BMP, GIF, ICO, JPEG, SVG, webp and other formats.

3.3 how to preview local pictures

Using HTML FileReader API, we can also easily realize the local preview function of pictures. The specific code is as follows:

<input type="file" accept="image/*" onchange="loadFile(event)">
<img id="output"/>
<script> const loadFile = function(event) {
    const reader = new FileReader();
    reader.onload = function(){
      const output = document.querySelector('output');
      output.src = reader.result;
    };
    reader.readAsDataURL(event.target.files[0]);
  };
</script>

After the local picture preview is completed, the data URLs corresponding to the picture can be directly submitted to the server. In this case, the server needs to do some related processing to save the uploaded pictures normally. Here, take express as an example, and the specific processing code is as follows:

const app = require('express')();
app.post('/upload', function(req, res){
    let imgData = req. body. imgData; // get the base64 image data in the post request
    let base64Data = imgData.replace(/^data:image/w+;base64,/, "");
    let dataBuffer = Buffer.from(base64Data, 'base64');
    fs.writeFile("image.png", dataBuffer, function(err) {
        if(err){
          res.send(err);
        }else{
          Res.send ("picture uploaded successfully!");
        }
    });
});

3.4 how to realize image compression

In some cases, when uploading local pictures, we hope to compress the pictures first and then submit them to the server, so as to reduce the amount of data transmitted. In the front end, we can use the image compression provided by canvas objecttoDataURL()Method, which receivestypeandencoderOptionsTwo optional parameters.

amongtypeIndicates the picture format. The default isimage/png。 andencoderOptionsUsed to represent the quality of the picture. When the specified picture format isimage/jpegorimage/webpIn this case, the quality of the picture can be selected from the range of 0 to 1. If the value range is exceeded, the default value will be used0.92, other parameters are ignored.

Let’s take a look at how to realize image compression:

function compress(base64, quality, mimeType) {
  let canvas = document.createElement("canvas");
  let img = document.createElement("img");
  img.crossOrigin = "anonymous";
  return new Promise((resolve, reject) => {
    img.src = base64;
    img.onload = () => {
      let targetWidth, targetHeight;
      if (img.width > MAX_WIDTH) {
        targetWidth = MAX_WIDTH;
        targetHeight = (img.height * MAX_WIDTH) / img.width;
      } else {
        targetWidth = img.width;
        targetHeight = img.height;
      }
      canvas.width = targetWidth;
      canvas.height = targetHeight;
      let ctx = canvas.getContext("2d");
      ctx. clearRect(0, 0, targetWidth, targetHeight); // clear canvas
      ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
      let imageData = canvas.toDataURL(mimeType, quality / 100);
      resolve(imageData);
    };
  });
}

For the returned image data in data URL format, in order to further reduce the amount of data transmitted, we can convert it into blob object:

function dataUrlToBlob(base64, mimeType) {
  let bytes = window.atob(base64.split(",")[1]);
  let ab = new ArrayBuffer(bytes.length);
  let ia = new Uint8Array(ab);
  for (let i = 0; i < bytes.length; i++) {
    ia[i] = bytes.charCodeAt(i);
  }
  return new Blob([ab], { type: mimeType });
}

After the conversion, we can encapsulate the blob object corresponding to the compressed image in the formdata object, and then submit it to the server through Ajax:

function uploadFile(url, blob) {
  let formData = new FormData();
  let request = new XMLHttpRequest();
  formData.append("image", blob);
  request.open("POST", url, true);
  request.send(formData);
}

3.5 how to operate bitmap pixel data

If you want to manipulate the image pixel data, we can use the data provided by canvasrenderingcontext2dgetImageDataTo obtain picture pixel data, where getimagedata () returns an imagedata object to describe the pixel data implied in the canvas area. This area is represented by a rectangle, with a starting point of (SX, SY), a width of SW and a height of SH. amonggetImageDataThe syntax of the method is as follows:

ctx.getImageData(sx, sy, sw, sh);

The corresponding parameters are described as follows:

  • SX: the X coordinate of the upper left corner of the rectangular area of the image data to be extracted.
  • SY: the Y coordinate of the upper left corner of the rectangular area of the image data to be extracted.
  • Sw: width of rectangular area of image data to be extracted.
  • SH: the height of the rectangular area of the image data to be extracted.

After obtaining the pixel data of the picture, we can process the obtained pixel data, such as graying or inverse color processing. After processing, to display the processing effect on the page, we need to use another API provided by canvasrenderingcontext2d——putImageData

This API is the canvas 2D API’s method of drawing data from an existing imagedata object to a bitmap. If a drawn rectangle is provided, only the pixels of the rectangle are drawn. This method is not affected by the canvas transformation matrix. The syntax of the putimagedata method is as follows:

void ctx.putImageData(imagedata, dx, dy);
void ctx.putImageData(imagedata, dx, dy, dirtyX, dirtyY, dirtyWidth, dirtyHeight);

The corresponding parameters are described as follows:

  • imageData: ImageData, an array object containing pixel values.
  • DX: the position offset of the source image data in the target canvas (the offset in the x-axis direction).
  • Dy: the position offset of the source image data in the target canvas (the offset in the y-axis direction).
  • Dirtyx (optional): the position of the upper left corner of the rectangular area in the source image data. The default is the upper left corner (x coordinate) of the entire image data.
  • Dirtyy (optional): the position of the upper left corner of the rectangular area in the source image data. The default is the upper left corner (Y coordinate) of the entire image data.
  • Dirtywidth (optional): the width of the rectangular area in the source image data. The default is the width of the image data.
  • Dirtyheight (optional): the height of the rectangular area in the source image data. The default is the height of the image data.

After introducing the relevant APIs, let’s take a practical example:

<!DOCTYPE html>
<html lang="zh-CN">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    < title > Image anti color and grayscale processing < / Title >
  </head>
  <body onload="loadImage()">
    <div>
      < button id = "invertbtn" > reverse color < / button >
      < button id = "grayscalebtn" > grayscale < / button >
    </div>
    <canvas id="canvas" width="800" height="600"></canvas>
    <script> function loadImage() {
        var img = new Image();
        img.crossOrigin = "";
        img.onload = function () {
          draw(this);
        };
        //This is brother Po's head
        img.src = "https://avatars3.githubusercontent.com/u/4220799";
      }
      function draw(img) {
        var canvas = document.getElementById("canvas");
        var ctx = canvas.getContext("2d");
        ctx.drawImage(img, 0, 0);
        img.style.display = "none";
        var imageData = ctx.getImageData(0, 0, canvas.width, canvas.height);
        var data = imageData.data;
        var invert = function () {
          for (var i = 0; i < data.length; i += 4) {
            data[i] = 255 - data[i]; // red
            data[i + 1] = 255 - data[i + 1]; // green
            data[i + 2] = 255 - data[i + 2]; // blue
          }
          ctx.putImageData(imageData, 0, 0);
        };
        var grayscale = function () {
          for (var i = 0; i < data.length; i += 4) {
            var avg = (data[i] + data[i + 1] + data[i + 2]) / 3;
            data[i] = avg; // red
            data[i + 1] = avg; // green
            data[i + 2] = avg; // blue
          }
          ctx.putImageData(imageData, 0, 0);
        };
        var invertbtn = document.getElementById("invertbtn");
        invertbtn.addEventListener("click", invert);
        var grayscalebtn = document.getElementById("grayscalebtn");
        grayscalebtn.addEventListener("click", grayscale);
      } </script>
  </body>
</html>

Note that when callinggetImageDataWhen using the method to obtain image pixel data, you may encounter cross domain problems, such as:

Uncaught DOMException: Failed to execute 'getImageData' on 'CanvasRenderingContext2D': The canvas has been tainted by cross-origin data.

For this question, you can read“Zhang Xinxu”Great God’s article “solving the cross domain problem of canvas picture getimagedata and todataurl”.

3.6 how to realize picture steganography

“Steganography is a skill and science about information hiding. The so-called information hiding refers to not letting anyone except the intended recipient know the transmission event or content of the information.”Steganography is called steganography in English. It comes from steganography, a book about cryptography and steganography by tritmius. The title of the book comes from Greek, which means “secret writing”.

The following figure is an online image steganography tool“The way of cultivating immortals in the whole stack”These six words are hidden in the original picture, and then the corresponding decryption tool is used to decrypt the results of the hidden information:

(online picture steganography experience address:https://c.p2hp.com/yinxietu/

At present, there are many schemes to realize picture steganography. The following are several common schemes:

  • Additional picture steganography;
  • Image steganography based on file structure;
  • Picture steganography based on LSB principle;
  • Jpg image steganography based on DCT domain;
  • Steganography of digital watermark;
  • Steganography of image tolerance.

Due to the limited space, we will not continue here. We will introduce each scheme separately. Interested partners can read the article “picture steganography of steganography (I)”.