Formalizing Interoperability for Test Case Generation Purpose: Plan Generality on interoperability testing and test architectures Interoperability formal definitions and comparison Interoperability test case generation :
n
Formalizing Interoperability for Test Case Generation Purpose
n n
Alexandra DESMOULIN & César VIHO IRISA / Université de Rennes 1
[email protected], http://www.irisa.fr
Classical approach Our approach Comparison
¾ ¾ ¾
Conclusion and future work
n
1
2
Interoperability testing in a context one-to-one
Formal definitions of interoperability testing and test generation n
Specification S1
Specification S2
Conformance testing I1-S1
Conformance testing I2-S2
Implementation Under Test (IUT)
n n
I2 Interoperability Testing I1-I2
I1
n
Interoperability : verification of the communication between the IUTs and of the provided services Formal definitions improves conformance testing Some existing attempts to formalize interoperability but no precise characterization. Interoperability test generation : no method based on formal definitions
3
4
Interoperability testing architecture
Model of IOLTS and notations
Test System (TS)
Tester (T1)
Specifications S1 and S2
Tester (T2)
UT1
0
UT2
UP1
LT1
UP2
LT2
LP1
LP2 UI2
IUT1
U!B
5
l?a
l!b
System S 2
l?b
LI2 SUT(System Under Test)
U!C
l!a
IUT2 Lower Interface
B C
l!c
Upper Interface U
1
UI1
LI1
A
0
U?A
3
1
l?c
a b c Lower Interface l
4 6
1
Global interoperability criteria iopG:
Interoperability definitions nBased on conformance notions: Conformance relation ioco I ioco S =∀σ∈Traces(Δ(S)) ⇒ Out(Δ(I),σ)⊆ Out(Δ(S),σ) nDifferent definitions of interoperability based on the different architectures are possible
iopG = two implementations are considered interoperable iff, after a trace of the asynchronous interaction of the specifications (and in I1||AI2), all outputs and quiescence observed during the (asynchronous) interaction of the implementations must be foreseen in the asynchronous interaction of the specifications. z Formally : iopG(I1, I2)= ∀σ∈Traces(S1||AS2) ⇒Out(I1||AI2, σ)⊆Out (S1||AS2,σ) z
IUT I
S σ
Out(I,σ)
σ
Out(S,σ)
7
8
Bilateral interoperability criteria iopB:
Example
iopB = after a trace of S1 observed during the asynchronous interaction of the implementations, all outputs and quiescence observed in I1 must be foreseen in S1, and the same in the point of view of I2 implementing the specification S2. n Formally : iopB(I1, I2)= ∀σ1∈Traces(Δ(S1)), σ∈Traces(I1||AI2), σ/ΣI1 = σ1 ⇒
0
n
Out
((I1||AI2)/ΣI1, σ)
∀σ2∈Traces(Δ(S2)), σ∈Traces(I1||AI2),
= σ1 ⇒
1
U!B
l?a
U!C
l?b
2 l?c
3
I1
1
U!C
l!a l?b 3
S1
2 l?c
iopB ≡ iopG (same power of noninteroperability detection)
n
S1||AS2 calculated in iopG, not in iopB Interoperability test generation based on iopB whithout the calculation of the interaction
⇒
0
0
U?A 1 U!B l!a
4 l?b
9
n
1
S2
4
I2
U?A U!C
l!c
l!b
Implementations
0
Out ((I1||A I2)/ΣI1, σ) ⊆ Out (Δ(S1),σ1).
Comparison of iopB and iopG
0
l!a
⊆ Out (Δ(S1),σ1) and σ/ΣI1
Specifications
U?A
3
0
l?a
l?a
l!c
l!b 1
1
I4
I3
2 10
Interoperability test generation based on iopG TP
S1||AS2
Interoperability test generation algorithm
SUT(I1||AI2)
TC Test execution
Verdict V 11
12
2
Interoperability test generation based on iopB (S1, S2) S1
TP
n
S2
Test purpose derivation
TPS2
TPS1
n
Conformance test generation algorithm
Conformance test generation algorithm
SUT(I1||AI2)
TC1
Comparison of the two methods
Test execution
n
TC2
n
Test execution
Verdict V1
Test Purpose Derivation : separating in TP events from S1 and S2 to obtain “unilateral” test purposes. No calculation of S1||AS2 with method based on iopB ⇒ no state-space explosion problem. Same verdicts with the two methods Test cases obtained with our method are unbiaised
Verdict V2 Verdict V’ = V 13
14
Exemple of test cases : TP=l1!a.U1!B 0
Conclusion and future work
Specifications
0
n
U?A 1
U!B
l?a
U!C
l!b
l!a l?b
2 l?c
3 0
U1!A
1
TC1 0
TC
S1 4
TPS1=l1!a.U1!B
1
S2
1
n
TPS2=l2?a.l2!b ?(l1?b)
?(l1!a)
3
2
U1?B Pass
0 ?(l2?a) 1 ?(l2!b) Pass
?(l2?a)
?(l1!a) 2
3
?(l2!c)
TC2
?(l1?c) Inc U1!A
l!c
?(l2!b)
?(l2!c) 15
4
?(l1?b)
5
U1?B
Inc
n
Proof of equivalence between global interoperability criterion iopG and bilateral interoperability criterion iopB Proposition of a method to generate interoperability test cases from iopB avoiding state-space explosion problem Future work : ¾ ¾
P
Experimenting the method on real protocols Studying the case of (N>2) implementations
I 16
3
