Combining Static and Dynamic Approaches to Software Verification in Frama-C S´ebastien Bardin, Nikolai Kosmatov joint work with O.Chebaro, M.Delahaye, A.Giorgetti, A.Jakobsson, J.Julliand, G.Petiot, J.Signoles, . . .

Imperial College, London, July 1st , 2015

S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

1 / 55

Static vs. Dynamic analysis techniques I

for a long time, seen as orthogonal and used separately

I

more recently, realization of potential synergy and complementarity

Static analysis

Dynamic analysis

Analyzes the source code without executing it

Executes the program on some test data

I

Instructions reported as safe are safe (complete)

I

Detected errors are really errors (precise)

I

Detected potential errors can be safe (imprecise)

I

Cannot cover all executions (incomplete)

This talk presents some combinations of both approaches in Frama-C S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

2 / 55

Outline Frama-C, a platform for analysis of C code Detecting runtime errors by static analysis and testing (SANTE) Deductive verification assisted by testing (STADY) Optimizing testing by value analysis and weakest precondition (LTest) Accelerating runtime assertion checking by static analysis (E-ACSL) Conclusion

S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

3 / 55

Frama-C, a platform for analysis of C code

Outline Frama-C, a platform for analysis of C code Detecting runtime errors by static analysis and testing (SANTE) Deductive verification assisted by testing (STADY) Optimizing testing by value analysis and weakest precondition (LTest) Accelerating runtime assertion checking by static analysis (E-ACSL) Conclusion

S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

4 / 55

Frama-C, a platform for analysis of C code

A brief history

I

90’s: CAVEAT, Hoare logic-based tool for C code at CEA

I

2000’s: CAVEAT used by Airbus during certification process of the A380 (DO-178 level A qualification)

I

2008: First public release of Frama-C (Hydrogen) Today: Frama-C Sodium (v.11)

I

I I I

Multiple projects around the platform A growing community of users. . . and of developers

S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

5 / 55

Frama-C, a platform for analysis of C code

Frama-C at a glance

I

A Framework for Modular Analysis of C code

I

Developed at CEA LIST and INRIA Saclay

I

Released under LGPL license

I

ACSL annotation language Extensible plugin oriented platform

I

I I I

I

Collaboration of analyses over same code Inter plugin communication through ACSL formulas Adding specialized plugins is easy

http://frama-c.com/ [Cuoq et al. SEFM 2012, FAC 2015]

S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

6 / 55

Frama-C, a platform for analysis of C code

ACSL: ANSI/ISO C Specification Language I

Based on the notion of contract, like in Eiffel, JML

I

Allows users to specify functional properties of programs

I

Allows communication between various plugins

I

Independent from a particular analysis

I

Manual at http://frama-c.com/acsl

Basic Components I

First-order logic

I

Pure C expressions

I

C types + Z (integer) and R (real)

I

Built-in predicates and logic functions particularly over pointers: \valid(p) \valid(p+0..2), \separated(p+0..2,q+0..5), \block_length(p)

S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

7 / 55

Frama-C, a platform for analysis of C code

Example: a C program annotated in ACSL /∗@ r e q u i r e s n>=0 && \ v a l i d ( t + ( 0 . . n − 1 ) ) ; a s s i g n s \nothing ; e n s u r e s \ r e s u l t != 0 ( \ f o r a l l i n t e g e r j ; 0 t [ j ] == 0 ) ; ∗/ int a l l z e r o s ( int t [] , int n) { int k ; /∗@ l o o p i n v a r i a n t 0 0); a inv = e a c s l m a l l o c ( s i z e o f ( i n t )∗ l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i −−) { /∗@ a s s e r t \ v a l i d ( a + i ) ; ∗/ int e acsl valid = valid (a + i , sizeof ( int )); e acsl assert ( e acsl valid ); a inv [ len − i − 1] = a [ i ] ; } e acsl free ( a inv ); d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e b l o c k (& l e n ) ; delete block (a ); Memory model: {(a, 16)} •

S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

49 / 55

Accelerating runtime verification by static analysis (E-ACSL)

Instrumented program with full memory monitoring int a [ ] = {1 ,2 ,3 ,4} , len = 4 , i , ∗ a inv ; store block (a ,16); • s t o r e b l o c k (& l e n , 4 ) ; s t o r e b l o c k (& i , 4 ) ; s t o r e b l o c k (& a i n v , 4 ) ; /∗@ a s s e r t l e n > 0 ; ∗/ e acsl assert ( len > 0); a inv = e a c s l m a l l o c ( s i z e o f ( i n t )∗ l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i −−) { /∗@ a s s e r t \ v a l i d ( a + i ) ; ∗/ int e acsl valid = valid (a + i , sizeof ( int )); e acsl assert ( e acsl valid ); a inv [ len − i − 1] = a [ i ] ; } e acsl free ( a inv ); d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e b l o c k (& l e n ) ; delete block (a ); Memory model: {(a, 16), (&len, 4)} S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

49 / 55

Accelerating runtime verification by static analysis (E-ACSL)

Instrumented program with full memory monitoring int a [ ] = {1 ,2 ,3 ,4} , len = 4 , i , ∗ a inv ; store block (a ,16); s t o r e b l o c k (& l e n , 4 ) ; • s t o r e b l o c k (& i , 4 ) ; s t o r e b l o c k (& a i n v , 4 ) ; /∗@ a s s e r t l e n > 0 ; ∗/ e acsl assert ( len > 0); a inv = e a c s l m a l l o c ( s i z e o f ( i n t )∗ l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i −−) { /∗@ a s s e r t \ v a l i d ( a + i ) ; ∗/ int e acsl valid = valid (a + i , sizeof ( int )); e acsl assert ( e acsl valid ); a inv [ len − i − 1] = a [ i ] ; } e acsl free ( a inv ); d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e b l o c k (& l e n ) ; delete block (a ); Memory model: {(a, 16), (&len, 4), (&i, 4)} S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

49 / 55

Accelerating runtime verification by static analysis (E-ACSL)

Instrumented program with full memory monitoring int a [ ] = {1 ,2 ,3 ,4} , len = 4 , i , ∗ a inv ; store block (a ,16); s t o r e b l o c k (& l e n , 4 ) ; s t o r e b l o c k (& i , 4 ) ; • s t o r e b l o c k (& a i n v , 4 ) ; /∗@ a s s e r t l e n > 0 ; ∗/ e acsl assert ( len > 0); a inv = e a c s l m a l l o c ( s i z e o f ( i n t )∗ l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i −−) { /∗@ a s s e r t \ v a l i d ( a + i ) ; ∗/ int e acsl valid = valid (a + i , sizeof ( int )); e acsl assert ( e acsl valid ); a inv [ len − i − 1] = a [ i ] ; } e acsl free ( a inv ); d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e b l o c k (& l e n ) ; delete block (a ); Memory model: {(a, 16), (&len, 4), (&i, 4), (&a inv, 4)} S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

49 / 55

Accelerating runtime verification by static analysis (E-ACSL)

Instrumented program with full memory monitoring int a [ ] = {1 ,2 ,3 ,4} , len = 4 , i , ∗ a inv ; store block (a ,16); s t o r e b l o c k (& l e n , 4 ) ; s t o r e b l o c k (& i , 4 ) ; s t o r e b l o c k (& a i n v , 4 ) ; /∗@ a s s e r t l e n > 0 ; ∗/ e acsl assert ( len > 0); • a inv = e a c s l m a l l o c ( s i z e o f ( i n t )∗ l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i −−) { /∗@ a s s e r t \ v a l i d ( a + i ) ; ∗/ int e acsl valid = valid (a + i , sizeof ( int )); e acsl assert ( e acsl valid ); a inv [ len − i − 1] = a [ i ] ; } e acsl free ( a inv ); d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e b l o c k (& l e n ) ; delete block (a ); Memory model: {(a, 16), (&len, 4), (&i, 4), (&a inv, 4), (a inv, 16)} S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

49 / 55

Accelerating runtime verification by static analysis (E-ACSL)

Instrumented program with full memory monitoring int a [ ] = {1 ,2 ,3 ,4} , len = 4 , i , ∗ a inv ; store block (a ,16); s t o r e b l o c k (& l e n , 4 ) ; s t o r e b l o c k (& i , 4 ) ; s t o r e b l o c k (& a i n v , 4 ) ; /∗@ a s s e r t l e n > 0 ; ∗/ e acsl assert ( len > 0); a inv = e a c s l m a l l o c ( s i z e o f ( i n t )∗ l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i −−) { /∗@ a s s e r t \ v a l i d ( a + i ) ; ∗/ int e acsl valid = valid (a + i , sizeof ( int )); e acsl assert ( e acsl valid ); a inv [ len − i − 1] = a [ i ] ; } • e acsl free ( a inv ); d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e b l o c k (& l e n ) ; delete block (a ); Memory model: {(a, 16), (&len, 4), (&i, 4), (&a inv, 4), (a inv, 16)} S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

49 / 55

Accelerating runtime verification by static analysis (E-ACSL)

Instrumented program with full memory monitoring int a [ ] = {1 ,2 ,3 ,4} , len = 4 , i , ∗ a inv ; store block (a ,16); s t o r e b l o c k (& l e n , 4 ) ; s t o r e b l o c k (& i , 4 ) ; s t o r e b l o c k (& a i n v , 4 ) ; /∗@ a s s e r t l e n > 0 ; ∗/ e acsl assert ( len > 0); a inv = e a c s l m a l l o c ( s i z e o f ( i n t )∗ l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i −−) { /∗@ a s s e r t \ v a l i d ( a + i ) ; ∗/ int e acsl valid = valid (a + i , sizeof ( int )); e acsl assert ( e acsl valid ); a inv [ len − i − 1] = a [ i ] ; } e acsl free ( a inv ); • d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e b l o c k (& l e n ) ; delete block (a ); Memory model: {(a, 16), (&len, 4), (&i, 4), (&a inv, 4), (a inv, 16)} S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

49 / 55

Accelerating runtime verification by static analysis (E-ACSL)

Instrumented program with full memory monitoring int a [ ] = {1 ,2 ,3 ,4} , len = 4 , i , ∗ a inv ; store block (a ,16); s t o r e b l o c k (& l e n , 4 ) ; s t o r e b l o c k (& i , 4 ) ; s t o r e b l o c k (& a i n v , 4 ) ; /∗@ a s s e r t l e n > 0 ; ∗/ e acsl assert ( len > 0); a inv = e a c s l m a l l o c ( s i z e o f ( i n t )∗ l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i −−) { /∗@ a s s e r t \ v a l i d ( a + i ) ; ∗/ int e acsl valid = valid (a + i , sizeof ( int )); e acsl assert ( e acsl valid ); a inv [ len − i − 1] = a [ i ] ; } e acsl free ( a inv ); d e l e t e b l o c k (& a i n v ) ; • d e l e t e b l o c k (& i ) ; d e l e t e b l o c k (& l e n ) ; delete block (a ); Memory model: {(a, 16), (&len, 4), (&i, 4), (&a inv, 4), (a inv, 16)} S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

49 / 55

Accelerating runtime verification by static analysis (E-ACSL)

Instrumented program with full memory monitoring int a [ ] = {1 ,2 ,3 ,4} , len = 4 , i , ∗ a inv ; store block (a ,16); s t o r e b l o c k (& l e n , 4 ) ; s t o r e b l o c k (& i , 4 ) ; s t o r e b l o c k (& a i n v , 4 ) ; /∗@ a s s e r t l e n > 0 ; ∗/ e acsl assert ( len > 0); a inv = e a c s l m a l l o c ( s i z e o f ( i n t )∗ l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i −−) { /∗@ a s s e r t \ v a l i d ( a + i ) ; ∗/ int e acsl valid = valid (a + i , sizeof ( int )); e acsl assert ( e acsl valid ); a inv [ len − i − 1] = a [ i ] ; } e acsl free ( a inv ); d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; • d e l e t e b l o c k (& l e n ) ; delete block (a ); Memory model: {(a, 16), (&len, 4), (&i, 4), (&a inv, 4), (a inv, 16)} S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

49 / 55

Accelerating runtime verification by static analysis (E-ACSL)

Instrumented program with full memory monitoring int a [ ] = {1 ,2 ,3 ,4} , len = 4 , i , ∗ a inv ; store block (a ,16); s t o r e b l o c k (& l e n , 4 ) ; s t o r e b l o c k (& i , 4 ) ; s t o r e b l o c k (& a i n v , 4 ) ; /∗@ a s s e r t l e n > 0 ; ∗/ e acsl assert ( len > 0); a inv = e a c s l m a l l o c ( s i z e o f ( i n t )∗ l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i −−) { /∗@ a s s e r t \ v a l i d ( a + i ) ; ∗/ int e acsl valid = valid (a + i , sizeof ( int )); e acsl assert ( e acsl valid ); a inv [ len − i − 1] = a [ i ] ; } e acsl free ( a inv ); d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e b l o c k (& l e n ) ; • delete block (a ); Memory model: {(a, 16), (&len, 4), (&i, 4), (&a inv, 4), (a inv, 16)} S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

49 / 55

Accelerating runtime verification by static analysis (E-ACSL)

E-ACSL: Goal and Solution Goal: avoid the monitoring of irrelevant statements I

Annotations do not necessarily evaluate memory-related properties for all memory locations

I

Full memory monitoring is costly and seldom necessary

Solution: E-ACSL performs a pre-analysis of the input program, which: I

Consists in a backward data-flow analysis

I

Over-approximates the set of variables that must be monitored to verify memory related annotations

I

Identified irrelevant memory locations are not monitored

S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

50 / 55

Accelerating runtime verification by static analysis (E-ACSL)

Instrumented program with full memory monitoring int a [ ] = {1 ,2 ,3 ,4} , len = 4 , i , ∗ a inv ; store block (a ,16); s t o r e b l o c k (& l e n , 4 ) ; s t o r e b l o c k (& i , 4 ) ; s t o r e b l o c k (& a i n v , 4 ) ; /∗@ a s s e r t l e n > 0 ; ∗/ e acsl assert ( len > 0); a inv = e a c s l m a l l o c ( s i z e o f ( i n t )∗ l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i −−) { /∗@ a s s e r t \ v a l i d ( a + i ) ; ∗/ int e acsl valid = valid (a + i , sizeof ( int )); e acsl assert ( e acsl valid ); a inv [ len − i − 1] = a [ i ] ; } e acsl free ( a inv ); d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e b l o c k (& l e n ) ; delete block (a ); S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

51 / 55

Accelerating runtime verification by static analysis (E-ACSL)

Instrumented program after pre-analysis int a [ ] = {1 ,2 ,3 ,4} , len = 4 , i , ∗ a inv ; store block (a ,16); store block(& len,4); store block(& i,4); store block(& a inv,4); /∗@ a s s e r t l e n > 0 ; ∗/ e acsl assert ( len > 0); a inv = e a c s l m a l l o c ( s i z e o f ( i n t )∗ l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i −−) { /∗@ a s s e r t \ v a l i d ( a + i ) ; ∗/ int e acsl valid = valid (a + i , sizeof ( int )); e acsl assert ( e acsl valid ); a inv [ len − i − 1] = a [ i ] ; } e acsl free ( a inv ); delete block(& a inv); delete block(& i); delete block(& len); delete block (a ); S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

52 / 55

Accelerating runtime verification by static analysis (E-ACSL)

E-ACSL: Experiments and Results I

In E-ACSL plugin, static analysis helps to avoid I I

irrelevant memory monitoring systematic usage of an unbounded integer library (GMP)

I

Experiments on a set of 9 annotated programs

I

Static analysis provides a significant speedup (55% in average, going up to 98% in some examples)

[Delahaye et al. SAC 2013; Kosmatov et al. RV 2013; Jakobsson et al. SAC 2015, JFLA 2015]

S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

53 / 55

Conclusion

Outline Frama-C, a platform for analysis of C code Detecting runtime errors by static analysis and testing (SANTE) Deductive verification assisted by testing (STADY) Optimizing testing by value analysis and weakest precondition (LTest) Accelerating runtime assertion checking by static analysis (E-ACSL) Conclusion

S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

54 / 55

Conclusion

Conclusion

I

Combining Static and Dynamic analyses can be beneficial for various domains of software verification: I I I I

detection of runtime errors and security vulnerabilities, deductive verification, runtime assertion checking, test generation, . . .

I

Both ways: static helps dynamic and dynamic helps static

I

Frama-C provides a rich and extensible framework for combined analyses

S.Bardin, N. Kosmatov (CEA LIST)

Combinations of Static and Dynamic Analyses

2015-06-01

55 / 55