Fast and Optimal Binary Template Matching Application to Manga Copyright Protection Mathieu Delalandre Laboratory of Computer Science University of Tours, Tours city, France
[email protected]
Introduction
Motoi Iwata and Koichi Kise Graduate School of Engineering Osaka Prefecture University, Osaka, Japan
[email protected]
Optimal template matching
Definition. Template matching can be defined as a method of parameter estimation, to choose the template position that minimizes a similarly measure between a discrete function Tx,y (taking values in a window W) and an image I e.g. with sum of squared differences
min e =
∑ ∑ (I
( i , j )∈I ( x , y )∈W
Definition. Optimal or Full-Search (FS) template matching scans the entire image and evaluates the similarity between the pattern and an area.
− Tx , y )
2
x+i , y + j
Method
Complexity
Coding
Brute-force FFT
O(MNmn) integer K×O(M*2log2M*) float
Bitwise operator
K×O(MNr)
K=1
θk Sk p
the size of the template the computation cost of the similarity measure the image size the orientation search parameter the scale search parameter the template number
boolean
with MN and mn the image/RoI and template size respectively, M* is M padded of the template size and round to a power of two, r = min(m, n)
Application to Manga Copyright Protection Database (2) Preprocessing Illegal copy database
Legal copy database (4) Template matching
(1) Web crawler
We consider X,Y two n-dimensional binary vectors
X = (x1 ,..., xm ,..., xn ) is one of the templates X 1 ,..., X C
copyright verification
Y = ( y1 ,..., ym ,..., yn ) is the image to compare
m =1
Based on n00, n11, n10, n01, 76 binary similarity measures S(X,Y) are defined in the literature with close relationships. In DIA, common measures includes those of Table I
Template database 128×256
legal illegal 128 dpi 10 10 3844 3844 7688
Pre-processing include canny edge detection and morphological dilatation operator Template selection ensures a minimum ratio n1x n0 x + n1 x Binary template matching implements optimal approaches and binary measures
Binary similarity measure characterization can be expressed as an inter-class intra-class discrimination problem, Illegal copy database Random patches selection
Templates with localizations
Patch comparison
frequency
n
v 0 1 0 n00 n01 negative matches missmatches u 1 n10 n11 missmatches positive matches
resolution magazines pages total pages
inter-class intra-class
Max
380 000 comparisons
distance
The Bhattacharyya coefficient measures the amount of overlap between two probability distributions p(x), q(x)
frequency
inter-class intra-class
∀x
Localization error
distance
Recall
Bitwise FFT faster faster
localization difference (in pixels)
Precision/Recall Yule Jaccard
illegal copy database
BC (q, p ) = ∑ p( x )q( x )
Yule class distribution
Precision
BC(p,q) β = n11 n00 β =1 Jaccard 0,01614 Dice 0,01857 Russ 0,25975 Yule 0,00926 0,00224 RT 0,16604 0,01641 SM 0,16898 0,25461
Matching Template #k Illegal page #k
3844 comparisons
frequency
Binary similarity (or dissimilarity) measures can be applied to binary images by requiring significantly less resource comparing to the ones working in the gray domain.
(3) Template selection
Processing time (K=1) Processing time (ms)
Binary Similarity Measures
nuv = ∑ δ m (u, v)
Yule, CORR Yule, CORR
K depends of the distance computation
The combination of the parameters m, θk, Sk, p is the search space, the computation cost depends on the search space dimension and the used similarity measure O(f(n))
1 if xm = u y m = v δ m (u , v ) = otherwise 0
K≥3
IP, RUSS, Ham, Jaccard, Dice, SM KUL, RT IP, RUSS, Ham, Jaccard, Dice, SM, RT,
The template matching problem is concerned with different parameters n O(f(n)) m
K=2
FFT 128×256 bitwise 32×256 bitwise
RoI size
Conclusion and perspectives Weighting is presented as very effective for boosting classification performance with negative matches i.e. ‘0’ provides less information of separability than ‘1’. We can define a weighted version for each dissimilarity measure by weighting n00 with β (0 ≤ β ≤ 1) . The IP, Jaccard, DICE, RUSS, and KUL are independent of n00.
Template matching is a good candidate for copyright verification supporting skew distortion, image shifting, scale approximation, compression & digitalization artifacts, scalable recognition. With a suitable selection of the distance and the template size, copyright verification problem is almost separable (near to a 1/1 Precision/Recall) on medium resolution images (128 dpi). Fast and optimal template matching can be achieved in some ten ms from regions of interest of 64×64 to 128×128 sizes, fitting with the localization errors, using a priori positions of templates. Major time improvement could be obtained with non-optimal approaches and template pruning strategies.
