Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Graph Matching Problems Romain Raveaux LI lab (EA 6300) – Universit´ e de Tours
October 13, 2016 International Master of Research in Computer Science: Computer Aided Decision Support
1 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
About Me I
Romain Raveaux Teacher at the university of Tours. Polytech’Tours Researcher at the Computer Science Laboratory Researcher Activity: Machine Learning Pattern Recognition Computer Vision Machine Learning with graphs Graph Matching Graph-Based representation Image analysis
2 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
About Me I
Romain Raveaux Teacher at the university of Tours. Polytech’Tours Researcher at the Computer Science Laboratory Researcher Activity: Machine Learning Pattern Recognition Computer Vision Machine Learning with graphs Graph Matching Graph-Based representation Image analysis
2 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
About Me I
Romain Raveaux Teacher at the university of Tours. Polytech’Tours Researcher at the Computer Science Laboratory Researcher Activity: Machine Learning Pattern Recognition Computer Vision Machine Learning with graphs Graph Matching Graph-Based representation Image analysis
2 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
About Me I
Romain Raveaux Teacher at the university of Tours. Polytech’Tours Researcher at the Computer Science Laboratory Researcher Activity: Machine Learning Pattern Recognition Computer Vision Machine Learning with graphs Graph Matching Graph-Based representation Image analysis
2 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
About Me I
Romain Raveaux Teacher at the university of Tours. Polytech’Tours Researcher at the Computer Science Laboratory Researcher Activity: Machine Learning Pattern Recognition Computer Vision Machine Learning with graphs Graph Matching Graph-Based representation Image analysis
2 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
About Me I
Romain Raveaux Teacher at the university of Tours. Polytech’Tours Researcher at the Computer Science Laboratory Researcher Activity: Machine Learning Pattern Recognition Computer Vision Machine Learning with graphs Graph Matching Graph-Based representation Image analysis
2 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
About Me I
Romain Raveaux Teacher at the university of Tours. Polytech’Tours Researcher at the Computer Science Laboratory Researcher Activity: Machine Learning Pattern Recognition Computer Vision Machine Learning with graphs Graph Matching Graph-Based representation Image analysis
2 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
About Me I
Romain Raveaux Teacher at the university of Tours. Polytech’Tours Researcher at the Computer Science Laboratory Researcher Activity: Machine Learning Pattern Recognition Computer Vision Machine Learning with graphs Graph Matching Graph-Based representation Image analysis
2 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
About Me I
Romain Raveaux Teacher at the university of Tours. Polytech’Tours Researcher at the Computer Science Laboratory Researcher Activity: Machine Learning Pattern Recognition Computer Vision Machine Learning with graphs Graph Matching Graph-Based representation Image analysis
2 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
About Me I
Romain Raveaux Teacher at the university of Tours. Polytech’Tours Researcher at the Computer Science Laboratory Researcher Activity: Machine Learning Pattern Recognition Computer Vision Machine Learning with graphs Graph Matching Graph-Based representation Image analysis
2 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
About Me I
Romain Raveaux Teacher at the university of Tours. Polytech’Tours Researcher at the Computer Science Laboratory Researcher Activity: Machine Learning Pattern Recognition Computer Vision Machine Learning with graphs Graph Matching Graph-Based representation Image analysis
2 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
About Me II Main publications : Referenced International Journal Romain Raveaux et al. Structured representations in a content based image retrieval context. Journal of Visual Communication and Image Representation, Volume 24, Issue 8, November 2013, Pages 1252-1268. Zeina Abu-Aisheh et al. Anytime graph matching. Pattern Recognition Letters. In press (2016). Romain Raveaux et al. Learning graph prototypes for shape recognition. Computer Vision and Image Understanding 115(7): 905-918 (2011) Romain Raveaux et al. A graph matching method and a graph matching distance based on subgraph assignments. Pattern Recognition Letters 31(5): 394-406 (2010)
3 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
About Me II Main publications : Referenced International Journal Romain Raveaux et al. Structured representations in a content based image retrieval context. Journal of Visual Communication and Image Representation, Volume 24, Issue 8, November 2013, Pages 1252-1268. Zeina Abu-Aisheh et al. Anytime graph matching. Pattern Recognition Letters. In press (2016). Romain Raveaux et al. Learning graph prototypes for shape recognition. Computer Vision and Image Understanding 115(7): 905-918 (2011) Romain Raveaux et al. A graph matching method and a graph matching distance based on subgraph assignments. Pattern Recognition Letters 31(5): 394-406 (2010)
3 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
About Me II Main publications : Referenced International Journal Romain Raveaux et al. Structured representations in a content based image retrieval context. Journal of Visual Communication and Image Representation, Volume 24, Issue 8, November 2013, Pages 1252-1268. Zeina Abu-Aisheh et al. Anytime graph matching. Pattern Recognition Letters. In press (2016). Romain Raveaux et al. Learning graph prototypes for shape recognition. Computer Vision and Image Understanding 115(7): 905-918 (2011) Romain Raveaux et al. A graph matching method and a graph matching distance based on subgraph assignments. Pattern Recognition Letters 31(5): 394-406 (2010)
3 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
About Me II Main publications : Referenced International Journal Romain Raveaux et al. Structured representations in a content based image retrieval context. Journal of Visual Communication and Image Representation, Volume 24, Issue 8, November 2013, Pages 1252-1268. Zeina Abu-Aisheh et al. Anytime graph matching. Pattern Recognition Letters. In press (2016). Romain Raveaux et al. Learning graph prototypes for shape recognition. Computer Vision and Image Understanding 115(7): 905-918 (2011) Romain Raveaux et al. A graph matching method and a graph matching distance based on subgraph assignments. Pattern Recognition Letters 31(5): 394-406 (2010)
3 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
About Me II Main publications : Referenced International Journal Romain Raveaux et al. Structured representations in a content based image retrieval context. Journal of Visual Communication and Image Representation, Volume 24, Issue 8, November 2013, Pages 1252-1268. Zeina Abu-Aisheh et al. Anytime graph matching. Pattern Recognition Letters. In press (2016). Romain Raveaux et al. Learning graph prototypes for shape recognition. Computer Vision and Image Understanding 115(7): 905-918 (2011) Romain Raveaux et al. A graph matching method and a graph matching distance based on subgraph assignments. Pattern Recognition Letters 31(5): 394-406 (2010)
3 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Problem statement Graph Definition
Graphs are everywhere... Graphs in Reality Graphs model objects and their relationships. Also referred to as networks. All common data structures can be modeled as graphs. How similar are two graphs? Graph similarity is the central problem for all learning tasks such as clustering and classification on graphs. 4 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Problem statement Graph Definition
Graphs are everywhere... Graphs in Reality Graphs model objects and their relationships. Also referred to as networks. All common data structures can be modeled as graphs. How similar are two graphs? Graph similarity is the central problem for all learning tasks such as clustering and classification on graphs. 4 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Problem statement Graph Definition
Graphs are everywhere... Graphs in Reality Graphs model objects and their relationships. Also referred to as networks. All common data structures can be modeled as graphs. How similar are two graphs? Graph similarity is the central problem for all learning tasks such as clustering and classification on graphs. 4 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Problem statement Graph Definition
Graphs are everywhere... Graphs in Reality Graphs model objects and their relationships. Also referred to as networks. All common data structures can be modeled as graphs. How similar are two graphs? Graph similarity is the central problem for all learning tasks such as clustering and classification on graphs. 4 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Problem statement Graph Definition
Graph
Definition Graph G = (V , E ) V is a set of vertices E is a set of edges such as E ⊆ VXV The graph size |G | refers to the number of nodes of the given graph G .
5 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Problem statement Graph Definition
Subgraph ¯ is a graph whose set of vertices E¯ and set of edges A subgraph G ¯ form subsets of the sets V and E of the graph G . A subgraph V ¯ G of a graph G is said to be induced (or full) if, for any pair of ¯ , e(vi , vj ) is an edge of G ¯ if and only if vertices vi and vj of G ¯ e(vi , vj ) is an edge of G. In other words, G is an induced subgraph of G if it has exactly the edges that appear in G over the same vertex set. Definition Subgraph ¯ ) ∪ V (G ) V (G ¯ E (G ) ∪ E (G )
6 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Problem statement Graph Definition
Directed and Undirected Graphs
A graph G is said to be undirected when all the edges ei ’s of the set E have no direction. Consequently, the edges e1 =(u, v ) and e2 =(v , u) are identical as the graph G is undirected. In contrast to the directed graphs which respect the direction that is assigned to each edge ei . Thus, for the directed graphs e1 6= e2 .
7 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Problem statement Graph Definition
Attributed and Non-attributed Graphs
Non-attributed graphs are only based on their topological structures, e.g. molecular graphs where the structural formula is considered as the representation of a chemical substance. Thus, no attributes can be found on neither the edges nor the nodes of the graph. Whereas in the attributed graphs (AG), significant attributes can be found on edges, nodes or both of them which efficiently describe objects (in terms of shape, color, coordinate, size, etc.) and their relations.
8 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Problem statement Graph Definition
Attributed Graph definition Definition Let LV and LE denote the set of node and edge labels, respectively. A labeled graph G is a 4-tuple G = (V , E , µ, ζ) , where V is the set of nodes, E ⊆ V × V is the set of edges µ : V → LV is a function assigning labels to the nodes, and ζ : E → LE is a function assigning labels to the edges. Let G1 = (V1 , E1 , µ1 , ζ1 ) be the source graph And G2 = (V2 , E2 , µ2 , ζ2 ) the target graph With V1 = (v11 , ..., v1n ) and V2 = (v21 , ..., v2m ) respectively 9 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Problem statement Graph Definition
Attributed Graph definition
Labels for both nodes and edges can be given by : Labelling functions a set of integers L = {1, 2, 3, . . .} a vector space L = Rn a finite set of symbolic labels L = {x,y, z, . . .} ... This definition allows us to handle arbitrarily structured graphs with unconstrained labelling functions.
10 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Problem statement Graph Definition
Graph Examples
11 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Problem statement Graph Definition
Attributed Subgraph Now that we have defined the notions of subgraph 2 and attributed graph 3, we can define the notion of attributed subgraph. It relies on the definition of subgraph 2 with some extends to consider constraints due to attributes. Definition Attributed Subgraph Given a graph G = (V,E,µ,ζ), a subgraph of G is a graph S = (VS ,ES ,µS , ζS ) such that VS ⊆ V , ES ⊆ E , ∀e = (v 1, v 2) ∈ ES , v 1 ∈ VS , v 2 ∈ VS and µS and ζS are the restrictions of µ and ζ to VS and ES , i.e. µS (v ) = µ(v ) and ζS (e) = ζ(e).
12 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Graph Matching presentation → Matching of vertices Graph matching A matching of GA = (VA , EA , µA , ζA ) and GB = (VB , EB , µB , ζB ) = A relation m ⊆ VA xVB → (vA,1 , vB,1 ) ∈ m ⇒ vertex vA,1 is mapped to vertex vB,1 Type of matching Bijectif matching : cardinality = (1,1) Injectif matching : cardinality = (1,0..1) Univalent matching : cardinality = (0..1, 0..1) ... Multivalent matching : cardinality = (0..|VA | ,0.. |VB |) 13 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Constraints on Graph Matching [1] Matching must satisfy (eventually) some constraints : to satisfy exactly hard constraints to satisfy at best soft constraints Edge constraints hard : mandatory soft : wanted (to be optimized) Label constraints hard : labels must be identical soft : labels must be similar (To be maximised).
14 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Two categories of graph matching
Exact graph matching : No topological changes and labels must be identical. Error-tolerant graph matching : Topological changes and labels must be similar (also called inexact graph matching). Warning : Nothing in common with exact or approximate methods
15 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Exact Matching
In this type of problems and at the aim of matching two graphs, significant part of the topology together with the corresponding node and edge labels in g1 and g2 have to identical. Exact matching methods can only be applied on labelled graphs or non-attributed graph.
16 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Graph isomorphism
The mapping between the nodes of the two graphs must be edge-preserving in the sense that if two nodes in the first graph are linked by an edge, they are mapped to two nodes in the second graph that are linked by an edge as well. This condition must hold in both directions, and the mapping must be bijective. That is, a one-to-one correspondence must be found between each node of the first graph and each node of the second graph.
17 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Graph isomorphism G1 = (V1 ,E1 ,LV1 ,LE1 ,µ1 ,ζ1 ) and G2 = (V2 ,E2 ,LV2 ,LE2 ,µ2 ,ζ2 ) we are looking for a bijective function f : V1 → V2 which maps each vertex ui ∈ V1 onto a vertex vk ∈ V2 such that certain conditions are fulfilled: Definition Graph isomorphism A bijective function f : V1 → V2 is a graph isomorphism from G1 to G2 if: 1
∀ui ∈ V1 , µ1 (ui ) = µ2 (f (ui ))
2
∀(ui , uj ) ∈ V1 × V1 , e(ui , uj ) ∈ E1 ⇔ e(f (ui ), f (uj )) ∈ E2
3
∀e(ui , uj ) ∈ E1 , ζ1 (e(ui , uj )) = ζ2 (e(f (ui ), f (uj )))
18 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Graph isomorphism Figure 2 depicts the graph isomorphism problem.
V E
V’ E’
f
f-‐1(v’) f(v)
Figure: Graph isomorphism problem
19 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Graph isomorphism Figure 2 depicts the graph isomorphism problem.
Figure: Graph isomorphism problem
To decide if two graphs are identical Bijectif matching + hard constraints on edges.
Complexity ? Still an open question. 20 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Induced Subgraph Isomorphism (SGI)
It requires that an isomorphism holds between one of the two graphs and a node-induced subgraph of the other.
21 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Induced Subgraph Isomorphism (SGI) More formally, when comparing two graphs G1 = (V1 ,E1 ,LV1 ,LE1 ,µ1 ,ζ1 ) and G2 = (V2 ,E2 ,LV2 ,LE2 ,µ2 ,ζ2 ) we are looking for a function f : V1 → V2 which maps each vertex ui ∈ V1 onto a vertex uj ∈ V2 such that certain conditions are fulfilled : Definition Induced subgraph isomorphism An injective function f : V1 → V2 is a subgraph isomorphism from G1 to G2 if: 1
∀ui ∈ V1 , µ1 (v ) = µ2 (f (ui ))
2
∀(ui , uj ) ∈ V1 × V1 , e(ui , uj ) ∈ E1 ⇔ e(f (ui ), f (uj )) ∈ E2
3
∀e(ui , uj ) ∈ E1 , ζ1 (e(ui , uj )) = ζ2 (e(f (ui ), f (uj )))
22 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Induced Subgraph Isomorphism (SGI)
In its exact formulation, the subgraph isomorphism must preserve the labeling, i.e., µ1 (ui ) = µ2 (vk ) and ζ1 (e(ui , uj )) = ζ2 (e(vk , vz )) where ui , uj ∈ V1 , vk , vz ∈ V2 , e(ui , uj ) ∈ E1 and e(vk , vz ) ∈ E2 .
23 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Monomorphism
Monomorphism is a light form of subgraph isomorphism. It drops also the condition that the mapping should be edge-preserving in both directions. It requires that each node of the first graph is mapped to a distinct node of the second one, and each edge of the first graph has a corresponding edge in the second one; the second graph, however, may have both extra nodes and extra edges.
24 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Monomorphism Definition Monomorphism An injective function f : V1 → V2 is a subgraph isomorphism from G1 to G2 if: 1
∀ui ∈ V1 , µ1 (ui ) = µ2 (f (ui ))
2
∀e(ui , uj ) ∈ E1 , (f (ui ), f (uj )) ∈ E2
3
∀e(ui , uj ) ∈ E1 , ζ1 (e(ui , uj )) = ζ2 (e(f (ui ), f (uj )))
As in SGI, in the exact formulation of subgraph monomorphism problem, the subgraph isomorphism must preserve the labeling, i.e., µ1 (ui ) = µ2 (vk ) and ζ1 (e(ui , uj )) = ζ2 (e(vk , vz )) where ui , uj ∈ V1 , vk , vz ∈ V2 , e(ui , uj ) ∈ E1 and e(vk , vz ) ∈ E2 . 25 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Subgraph isomorphism Figure 3 depicts the graph isomorphism problem.
Figure: Subgraph isomorphism problem
To decide if one graph is inside another Injectif matching + constraints on edges. hard constraints if induced subgraph isomorphism hard+soft constraints if partial subgraph isomorphism (monomorphism)
NP-complete problem
26 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Type of morphisms
Figure: Three different types of morphisms 27 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Maximum Common Subgraph (MCS)
It maps a subgraph of the first graph to an isomorphic subgraph of the second one; since such a mapping is not uniquely defined, usually the goal of the algorithm is to find the largest subgraph for which such a mapping exists. Actually, there are two possible definitions of the problem, depending on whether node-induced subgraphs or plain subgraphs are used. In the first case, the maximality of the common subgraph is referred to the number of nodes, while in the second it is the number of edges that is maximized.
28 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Maximum Common Subgraph (MCS) Figure 5 depicts the graph isomorphism problem.
Figure: Maximum Common Subgraph
Maximum common subgraph (partial or induced) between G1 and G2 Univalent matching + constraints on edges. hard constraints if induced subgraph isomorphism hard+soft constraints if partial subgraph isomorphism
NP-hard problem 29 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Maximum Common Subgraph (MCS)
Definition Maximum Common Subgraph (MCS) Let G1 = (V1 , E1 ) and G2 = (V2 , E2 ) be two graphs. A graph Gs = (Vs , Es ) is said to be a common subgraph of G1 and G2 if there exists subgraph isomorphism from Gs to G1 and from Gs to G2 . The largest common subgraph is called the maximum common subgraph, or MCS, of G1 and G2 .
30 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Complexity of exact matching problems The matching problems mentioned above are all NP-complete except for graph isomorphism, for which it has not yet been demonstrated if it belongs to NP or not. However, there is still no algorithm that can solve the problem in polynomial time. Yet, readers who are aware of the recent rise of graph isomorphism might have heard about the claim of Babai in [?] of solving graph isomorphism in quasipolynomial time. The NP-completeness proof of subgraph isomorphism can be found in [?]. Polynomial isomorphism algorithms have been developed for special kinds of graphs (e.g. for trees by Aho et al .1 in 1974, for planar graphs by Hopcroft and Wong68 in 1974, for bounded valence graphs by Luks97 in 1982) but no polynomial algorithms are known for the general case. 31 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Inexact Graph Matching
The stringent constraints imposed by exact matching are in some circumstances too rigid for the comparison of two graphs. So the matching process must be tolerant: it must accommodate the differences by relaxing, to some extent, the constraints that define the matching type.
32 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Inexact Graph Matching
Usually, in these algorithms the matching between two nodes that do not satisfy the edge-preservation requirements of the matching type is not forbidden. Instead, it is penalized by assigning to it a cost that may take into account other differences (e.g. among the corresponding node/edge attributes).
33 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Problem transformation : from inexact to exact
An inexact matching is generally needed when no exact mapping between vertex and/or edge labels can be found, but when the mapping can be associated to a cost. For example, this case occurs when vertex and edge labels are numerical values (scalar or vectorial). The cost for the mapping can then be defined as the sum of the distances between label values.
34 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Problem transformation : from inexact to exact
A first solution to tackle such problems relies on a discretization or a classification procedure to transform the numerical values into nominal/symbolic labels. The main drawback of such approaches is their sensitivity to frontier effects of the discretization or misclassification. A subsequent exact matching algorithm would then be unsuccessful.
35 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Problem transformation : from inexact to exact
A second solution consists in using exact matching algorithms and to customize the compatibility function for pairing vertices and edges. The main drawback of such approaches is the need to define thresholds for these compatibilities.
36 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Problem transformation : from inexact to exact
A last way consists in using an inexact matching procedure that overcomes this drawback by integrating the numerical values during the mapping search. In this case, the matching problem turns from a decision one to an optimization one.
37 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Substitution-Tolerant Subgraph Isomorphism
Definition Substitution-Tolerant Subgraph Isomorphism An injective function f : V1 → V2 is a subgraph isomorphism of G1 = (V1 ,E1 ,LV1 ,LE1 ,µ1 ,ζ1 ) and G2 = (V2 ,E2 ,LV2 ,LE2 ,µ2 ,ζ2 ) if the following conditions are satisfied: 1
∀ui ∈ V1 , µ1 (ui ) ≈ µ2 (f (ui ))
2
∀(ui , uj ) ∈ V1 × V1 , e(ui , uj ) ∈ E1 ⇔ e(f (ui ), f (uj )) ∈ E2
3
∀e(ui , uj ) ∈ E1 , ζ1 (e(ui , uj )) ≈ ζ2 (e(f (ui ), f (uj )))
38 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Substitution-Tolerant Subgraph Isomorphism In PR applications, where vertices and edges are labeled with measures which may be affected by noise, a substitution-tolerant formulation which allows differences between attributes of mapped vertices and edges is mandatory. However, these differences are associated to costs where the objective is to find the mapping corresponding to the minimal global cost, if one exists. i.e., µ1 (ui ) ≈ µ2 (vk ) and ζ1 (e(ui , uj )) ≈ ζ2 (e(vk , vz )). Figure 6 depicts the substitution-tolerant subgraph isomorphism problem. V E 0.5 0.15
V’ E’
f-‐1(v’) f(v)
f 0.3
0.13 39 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Inexact Subgraph Isomorphism
It requires that each node/edge of the first graph is mapped to a distinct node/edge of the second graph or to a dummy node/edge. This dummy elements can absorb structural modifications between the two graphs. It drops also the condition that the penalty costs should take into account extra nodes and extra edges of the second graph.
40 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Inexact Subgraph Isomorphism Definition Error-Tolerant Subgraph Isomorphism An injective function f : V1 → V2 is an error-tolerant subgraph isomorphism from G1 = (V1 ,E1 ,LV1 ,LE1 , µ1 ,ζ1 ) to G2 = (V2 ,E2 ,LV2 ,LE2 ,µ2 ,ζ2 ) if the following conditions are satisfied: 1
∆V2 is a set of dummy vertices
2
∆E2 is a set of dummy edges
3
∀ui ∈ V1 , f (ui ) ∈ V2 ∪ ∆V2
4
∀e(ui , uj ) ∈ E1 , e(f (ui ), f (uj )) ∈ E2 ∪ ∆E2
5
∀ui ∈ V1 , µ1 (ui ) ≈ µ2 (f (ui )) and ∀e(ui , uj ) ∈ E1 , ξ1 (e(ui , uj )) ≈ ξ2 (e(f (ui ), f (uj )))
41 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Inexact Subgraph Isomorphism Figure 7 depicts the inexact subgraph isomorphism problem.
V E 0.5 0.16
V’ E’
f Δe’
Matched
Unmatched
0.3
f(e)
0.13
Figure: Inexact Subgraph Isomorphism problem
42 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Inexact Graph Isomorphism
A significant number of inexact graph matching algorithms base the definition of the matching cost on an explicit model of the errors (deformations) that may occur (i.e. missing nodes, etc.), assigning a possibly different cost to each kind of error. These algorithms are often denoted as error-correcting or error-tolerant.
43 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Inexact Graph Isomorphism Definition Inexact Graph Isomorphism A bijective function f : V → V 0 is an inexact graph isomorphism from a graph G = (V , E , µ, ζ) to a graph G 0 = (V 0 , E 0 , µ0 , ζ 0 ) such that certain conditions are fulfilled : 1
∆v and ∆0v are two sets of dummy vertices
2
∆e and ∆0e are two sets of dummy edges
3
∀v ∈ {V ∪ ∆v }, f (v ) = v 0 ∈ {V 0 ∪ ∆0v }, f −1 (v 0 ) = v
4
5
6
∀e = (vi , vj ) ∈ {E ∪ ∆e }, there exists a distinct edge e 0 = (f (vi ), f (vj )) ∈ {E 0 ∪ ∆0e } ∀e 0 = (vi0 , vj0 ) ∈ {E 0 ∪ ∆0e }, there exists a distinct edge e = (f −1 (vi0 ), f −1 (vj0 )) ∈ {E ∪ ∆e } µ(v ) ≈ µ0 (v 0 ) and ζ(e) ≈ ζ 0 (e 0 ) 44 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Inexact Graph Isomorphism Figure 8 depicts the inexact graph isomorphism problem.
V E
V’ E’ 0.5 0.16
Δe Δe’
Δv f 0.3
0.13
Figure: Inexact graph isomorphism problem
45 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Inexact Graph Isomorphism Figure 9 depicts the inexact graph isomorphism problem.
Figure: Inexact Graph Isomorphism
Generalization of maximum common subgraph between G1 and G2 to labelled graphs. Univalent matching + soft constraints on edges+ soft constraints on labels.
NP-hard problem 46 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Inexact Graph Isomorphism Graph Edit Distance The graph edit distance [?] between G1 and G2 is defined by: k X GED(G1 , G2 ) = min c(ei ) EP={e1 ,··· ,ek }∈Γ(G1 ,G2 )
i=1
Allowed Edit Operation: ei Substitution, deletion and insertion of nodes and/or edges.
Γ(G1 , G2 ) is the set of all edit sequences allowing to transform G1 into G2 .
Figure: 47 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Inexact Graph Isomorphism Graph Edit Distance The graph edit distance [?] between G1 and G2 is defined by: k X GED(G1 , G2 ) = min c(ei ) EP={e1 ,··· ,ek }∈Γ(G1 ,G2 )
i=1
Allowed Edit Operation: ei Substitution, deletion and insertion of nodes and/or edges.
Γ(G1 , G2 ) is the set of all edit sequences allowing to transform G1 into G2 .
Figure: 47 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Inexact Graph Isomorphism Graph Edit Distance The graph edit distance [?] between G1 and G2 is defined by: k X GED(G1 , G2 ) = min c(ei ) EP={e1 ,··· ,ek }∈Γ(G1 ,G2 )
i=1
Allowed Edit Operation: ei Substitution, deletion and insertion of nodes and/or edges.
Γ(G1 , G2 ) is the set of all edit sequences allowing to transform G1 into G2 .
Figure: 47 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Inexact Graph Isomorphism Graph Edit Distance The graph edit distance [?] between G1 and G2 is defined by: k X GED(G1 , G2 ) = min c(ei ) EP={e1 ,··· ,ek }∈Γ(G1 ,G2 )
i=1
Allowed Edit Operation: ei Substitution, deletion and insertion of nodes and/or edges.
Γ(G1 , G2 ) is the set of all edit sequences allowing to transform G1 into G2 .
Figure: 47 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Inexact Graph Isomorphism Graph Edit Distance The graph edit distance [?] between G1 and G2 is defined by: k X GED(G1 , G2 ) = min c(ei ) EP={e1 ,··· ,ek }∈Γ(G1 ,G2 )
i=1
Allowed Edit Operation: ei Substitution, deletion and insertion of nodes and/or edges.
Γ(G1 , G2 ) is the set of all edit sequences allowing to transform G1 into G2 .
Figure: 47 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Inexact Graph Isomorphism Graph Edit Distance The graph edit distance [?] between G1 and G2 is defined by: k X GED(G1 , G2 ) = min c(ei ) EP={e1 ,··· ,ek }∈Γ(G1 ,G2 )
i=1
Allowed Edit Operation: ei Substitution, deletion and insertion of nodes and/or edges.
Γ(G1 , G2 ) is the set of all edit sequences allowing to transform G1 into G2 .
Figure: 47 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Inexact Graph Isomorphism Graph Edit Distance The graph edit distance [?] between G1 and G2 is defined by: k X GED(G1 , G2 ) = min c(ei ) EP={e1 ,··· ,ek }∈Γ(G1 ,G2 )
i=1
Allowed Edit Operation: ei Substitution, deletion and insertion of nodes and/or edges.
Γ(G1 , G2 ) is the set of all edit sequences allowing to transform G1 into G2 .
Figure: 47 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Multivalent Matching
Multivalent matching drops the condition that nodes in the first graph are to be mapped to distinct nodes of the other; hence, the correspondence can be many-to-one. Note multivalent matching can be whether exact or inexact. The possible kind of multivalent mapping are the following : 1
One to many
2
Many to one
3
Many to many
48 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Multivalent Matching Figure 11 depicts the level of constraint of each graph isomorphism problem.
Figure: Graph isomorphism constraints 49 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Different kind of problems Exact Matching Inexact Graph Matching (IGM) Multivalent Matching
Graph Distance
f (G1 ∩ G2 ) − g (splits(m)) f (V1 ∪ E1 ∪ V2 ∪ E2 ) A generic graph distance based on multivalent matching
simm (G1 , G2 ) =
S. Sorlin, C. Solnon, J-M. Jolion [1] 2007
50 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
How Graph Matching can be used in Pattern Recognition Problems
Given a graph database consisting of n graphs, D = g 1, g 2, ..., gn, and a query graph q, almost all existing algorithms of processing graph search can be classified into the following four categories: Full graph search, Subgraph search, Similarity search and Graph mining.
51 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Full graph search
Find all graphs gi in D s.t. gi is the same as q.
52 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Subgraph search
Find all graphs gi in D containing q or contained by q.
53 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Subgraph spotting
Find all occurrences of q in D.
54 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Similarity search
Find all graphs gi in D s.t. gi is similar to q within a user-specified threshold based on some similarity measures.
55 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Graph mining
Graph mining problem gathers similar graph or subgraph of D in order to find clusters or prototypes. No query is provided by the user.
56 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Conclusion
In this chapter, we have defined the two main families of graph matching: exact and inexact graph matching families. We have also given an overview about three types of graphs: non-attributed, labeled and continuous attributed graphs.
57 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Conclusion
GI, SGI, monomorphism and MCS problems are restricted to non-attributed and labeled graphs.
58 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Conclusion
The scope of inexact morphisms is less limited and so graphs labelled with continuous attributes can be involved into the matching procedure. Cost penalties are assigned to approximately matched nodes and edges in order to consider structure and attribute differences.
59 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
Conclusion
There are many other matching problems (hyper graph matching problems, multi graph matching problems, ...).
60 / 60
Graph Graph Matching Problems Graph Matching in Pattern Recognition Problems Conclusion
S´ebastien Sorlin, Christine Solnon, and Jean-Michel Jolion. A generic graph distance measure based on multivalent matchings. In Abraham Kandel, Horst Bunke, and Mark Last, editors, Applied Graph Theory in Computer Vision and Pattern Recognition, volume 52 of Studies in Computational Intelligence, pages 151–181. Springer, 2007.
60 / 60
