Opencv: a practical case of easy to use digital recognition

Time:2021-4-20

Abstract:The case of credit card recognition uses some basic operations of image processing, which is relatively friendly for those who have just started cv.

This article is shared from Huawei cloud community《Python opencv case: credit card number recognition》The echo of dark blue.

preface

Practice is the only criterion for testing truth.

Because it’s too boring to learn opencv in a straight line, I found a project-oriented tutorial on the Internet. Don’t say much, just do it.

1、 Case introduction

Provide digital template on credit card:

Opencv: a practical case of easy to use digital recognition

Requirements: identify the number on the credit card and print it directly on the original picture. Although it looks stupid, since the number can be printed on the picture, it indicates that the number has been successfully recognized, so it can also be converted into digital text for storage. The idea of license plate recognition is similar to this case.

Example:

Opencv: a practical case of easy to use digital recognition

Original picture

Opencv: a practical case of easy to use digital recognition

Processed graph

2、 Steps

It can be roughly divided into the following steps:
1. Template reading
2. Template preprocessing, separate the template numbers and sort them
3. Input image preprocessing to extract the digital part of the image
four . The highest matching rate is the corresponding number.

1. Template read in, as well as some package import, function definition, etc

import cv2 as cv
import numpy as np
import myutils
def cv_ show ( name ,  img ):        #  Custom display function
    cv.imshow(name, img)
    cv.waitKey(0)
#Read in template map
n = 'text'
img = cv.imread("images/ocr_a_reference.png")
#  cv_ show ( n ,  template )        #  Custom display function, easy to display pictures

2. Template preprocessing, separate the template numbers and sort them

Opencv: a practical case of easy to use digital recognition

The preprocessing sequence of template: gray image, binarization, and then contour detection. It should be noted that the white border is detected when opencv detects the contour, so the digital binarization of the template graph should be changed to white.

Opencv: a practical case of easy to use digital recognition

#Template conversion to grayscale image
ref = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
# cv_show(n, ref)

#Convert to binary graph, turn the digital part into white
ref =  cv.threshold (ref, 10, 255,  cv.THRESH_ BINARY_ Inv [1] #, function multiple return values are tuples, here take the second return value
cv_show(n, ref)

#The template is detected to get the contour information
refCnts, hierarchy = cv.findContours(ref.copy(), cv.RETR_EXTERNAL, cv.CHAIN_APPROX_NONE)
cv.drawContours (IMG, refcnts, - 1, (0, 0, 255), 2) # the first parameter is the target image
# cv_show(n, img)

Opencv: a practical case of easy to use digital recognition

The red part is the detected contour.

Next, the contours are sorted. Because the detected contours are unordered, they are sorted according to the X coordinates of the upper left corner of the contours. After the contours are sorted and put into the dictionary in order, the key value pairs in the dictionary are correctly matched. For example, ‘0’ corresponds to contour 0, and ‘1’ corresponds to contour 1.

#Outline sort
refCnts = myutils.sort_contours(refCnts)[0]
digits = {}

#A single contour is extracted into a dictionary
for (i, c) in enumerate(refCnts):
    (x, y, w, h) = cv.boundingRect(c)
    roi  =  ref [ y : y  +  h ,  x : x  +  w ]  #  Copy the outline in the template
    roi =  cv.resize (ROI, (57, 88)) # changed to contour of the same size
    Digits [i] = ROI # at this time, the outline corresponding to the dictionary key is the corresponding number. For example, key '1' corresponds to contour '1'

So far, the processing of the template diagram is completed.

3. The input image is preprocessed to extract the digital part of the image

Opencv: a practical case of easy to use digital recognition

In this step, each number on the credit card needs to be extracted and matched one by one with the template obtained in the previous step. Firstly, the convolution kernel is initialized to facilitate tophat operation and closed operation.

#Initialize convolution kernel
rectKernel = cv.getStructuringElement(cv.MORPH_RECT, (9, 3))
sqKernel = cv.getStructuringElement(cv.MORPH_RECT, (5, 5))

Next, read in the picture, adjust the size of the picture, and convert it to grayscale.

#  Read in the image to be analyzed , Pretreatment
card_image = cv.imread("images/credit_card_01.png")
# cv_show('a', card_image)
card_ image =  myutils.resize (card_ Image, width = 300) # change image size
gray = cv.cvtColor(card_image, cv.COLOR_BGR2GRAY)
# cv_show('gray', gray)

Opencv: a practical case of easy to use digital recognition

Then, tophat can highlight the bright areas in the image and filter out the darker parts

tophat = cv.morphologyEx(gray, cv.MORPH_TOPHAT, rectKernel)
# cv_show('tophat', tophat)

Opencv: a practical case of easy to use digital recognition

Then, Sobel operator is used to detect the edge, and a close operation is performed to fill the hole.

#Sobel operators in X direction
gradX = cv.Sobel(tophat, cv.CV_32F, 1, 0, ksize=3) 

gradX =  np.absolute (gradx) # absolute: calculate the absolute value
min_Val, max_val = np.min(gradX), np.max(gradX)
gradX = (255 * (gradX - min_Val) / (max_val - min_Val))
gradX = gradX.astype("uint8")

#  Connect numbers by closing operation (first expansion, then corrosion) .   Expand 4 boxes which are originally 4 numbers into 1 box , It won't corrode
gradX = cv.morphologyEx(gradX, cv.MORPH_CLOSE, rectKernel)
# cv_show('close1', gradX)

#Binarization
thresh = cv.threshold(gradX, 0, 255, cv.THRESH_BINARY | cv.THRESH_OTSU)[1]

#  Closed operation , Fill in the void
thresh = cv.morphologyEx(thresh, cv.MORPH_CLOSE, sqKernel)
# cv_show('close2', thresh)

Opencv: a practical case of easy to use digital recognition

Then you can look for the outline.

threshCnts = cv.findContours(thresh.copy(), cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)[0]
card_copy = card_image.copy()
cv.drawContours(card_copy, threshCnts, -1, (0, 0, 255), 2)
cv_show('Input_Contours', card_copy)

Opencv: a practical case of easy to use digital recognition

4. Template matching

After the template number and the image to be recognized are processed, the matching can be carried out.

locs  = []  #  Save a qualified profile
for i, c in enumerate(threshCnts):
    #Calculate rectangle
    x, y, w, h = cv.boundingRect(c)

    ar = w / float(h)
    #Choose the right area, according to the actual task, here are basically a group of four numbers
    if 2.5 < ar < 4.0:
        if (40 < w < 55) and (10 < h < 20):
            #The right ones stay
            locs.append((x, y, w, h))

#Sort the matched profiles from left to right
locs = sorted(locs, key=lambda x: x[0])

Next, traverse each large contour, each large contour has four numbers, corresponding to four small contours. Match the small outline to the template.

Output = [] store the correct number
For (I, (GX, Gy, GW, GH)) in enumerate (locs): # traverse each set of large contours (including 4 numbers)
    groupOutput = []

    #Extract each group according to coordinates (4 values)
    Group = gray [Gy - 5: Gy + GH + 5, GX - 5: GX + GW + 5] #
    # cv_show('group_' + str(i), group)
    #  Pretreatment
    group =  cv.threshold (group, 0, 255,  cv.THRESH_ BINARY |  cv.THRESH_ Otsu [1] # binary Group
    # cv_show('group_'+str(i),group)
    #Calculate the contour of each group so that it is divided into four small profiles
    digitCnts = cv.findContours(group.copy(), cv.RETR_EXTERNAL, cv.CHAIN_APPROX_SIMPLE)[0]
    #Sort
    digitCnts = myutils.sort_contours(digitCnts, method="left-to-right")[0]

#Calculate and match each value in each group
    For C in digitcnts: # C represents the end coordinates of each small contour
        z = 0
        #Find the contour of the current value and reset to the appropriate size
        (x, y, w, h) =  cv.boundingRect (c) Circumscribed rectangle
        roi  =  group [ y : y  +  h ,  x : x  +  w ]  #  Take out the small contour coverage area in the original image , It's numbers
        roi = cv.resize(roi, (57, 88))
        # cv_show("roi_"+str(z),roi)

        #  Calculate match score :  What's the score of 0 , What's the score of 1 ...
        Scores = [] # in a single cycle, scores stores the maximum score of one value matching 10 template values

        #Calculate each score in the template
        #The digit of digits is exactly the value 0,1,..., 9; digitroi is the characteristic representation of each value
        for (digit, digitROI) in digits.items():
            #For template matching, res is the result matrix
            res  =  cv . matchTemplate ( roi ,  digitROI ,  cv . TM_ CCOEFF )  #  In this case, ROI is x, digitroi is 0 and then 1 , two ..  Match 10 times , What's the highest score of the template
            Max_ score  =  cv . minMaxLoc ( res )[ one ]  #  Return 4 , Take the second maximum maxscore
            scores . append ( Max_ score )  #  10 maximum values
        # print("scores:",scores)
        #Get the most appropriate number
        groupOutput . append ( str ( np . argmax ( scores )))  #  Returns the position of the maximum value in the input list
        z = z + 1
#Draw it
    cv.rectangle (card_ Image, (GX - 5, Gy - 5), (GX + GW + 5, Gy + GH + 5), (0, 0, 255), 1) # upper left corner, lower right corner
#Puttext parameters: picture, added text, upper left corner coordinate, font, font size, color, font thickness
    cv.putText(card_image, "".join(groupOutput), (gx, gy - 15), cv.FONT_HERSHEY_SIMPLEX, 0.65, (0, 0, 255), 2)

Finally, print it out and the task is finished.

cv.imshow("Output_image_"+str(i), card_image)
cv.waitKey(0)

Opencv: a practical case of easy to use digital recognition

summary

The case of credit card recognition uses some basic operations of image processing, which is relatively friendly for those who have just started cv.

Click follow to learn about Huawei’s new cloud technology for the first time~