7th International Conference on Hydroinformatics HIC 2006, Nice, FRANCE

FUZZY KNOWLEDGE-BASED CURVE EVALUATION FOR 1-D RIVER MODEL CALIBRATION JEAN-PHILIPPE VIDAL Hydrology-Hydraulics Research Unit, Cemagref, 3 bis quai Chauveau BP 220 69 336 Lyon Cedex 09, France (now at HR Wallingford, Howbery Park, Oxfordshire, OX10 8BA, UK) SABINE MOISAN Orion project, INRIA Sophia-Antipolis, 2004 route des Lucioles BP 93 06902 Sophia-Antipolis, France Model calibration requires an evaluation of the agreement between model outputs and reference data. This article presents an automatic fuzzy knowledge-based approach to identify the relevant discrepancies. This evaluation module is intended to be integrated within an existing knowledge-based calibration support system. INTRODUCTION Model calibration remains a complex and critical task involving many different steps. The structure of the task – as undertaken by experienced modellers – has been identified by Vidal et al. [8]. Among the different calibration steps, this article focuses on the comparison of outputs from the numerical model and reference data, hereafter referred to as the output comparison task. An expert usually achieves this task by visually inspecting the data in order to deliver a qualitative interpretation of the agreement between measurements and computed results. This process is unfortunately largely irreproducible because of its heuristic nature, and corresponding judgements mainly rely on expertise. Less experienced modellers thus often struggle to identify the relevant discrepancies which may be reduced by adjusting the model parameters. This article thus proposes a way of automating the output comparison task by using a knowledge-based curve evaluation module and mimicking the way an expert addresses this issue. The next section aims at introducing the technical aspects of the curve evaluation module and exemplifying its use in 1-D river hydraulics. Finally, an integration of the module within an existing knowledge-based calibration support system is proposed and discussed. CURVE EVALUATION MODULE Experts, in hydraulics as in many other domains, make an extensive use of 2D curves to display, analyze and compare sets of numerical values [4]. Some attempts have been 1

made to automate this kind of qualitative analysis in the past, but it has been restricted to specific domains like well testing [3], rheology [5], or wave impacts monitoring [6]. We therefore decided to develop a generic curve evaluation module, that computes a symbolic description of a sampled curve representing a Cartesian function. This curve evaluation module provides not only symbolic description but also comparison facilities; both aspects are detailed below. Symbolic descriptions The symbolic description of a curve is obtained by first filtering and segmenting the curve and then describing the different elements of this simplified curve using expertdefined symbolic qualifiers for their features. A similar curve segmentation has been used by Syed Mustapha et al. to interpret viscometric flow curves [5]. The first step of the curve description algorithm takes a set of numerical coordinates (x, y) describing a curve, which can optionally be smoothed using a filter. The second step samples and segments the curve in order to obtain a simplified view of the curve as a set of segments that match the initial curve with an acceptable error. Coordinates are then normalized. Curve elements considered for description are segments, peaks, and slope breaks. A segment is described through its slope and width. A peak, composed of two segments forming a small angle, is characterized by it position on x axis, width, height and shape. Slope breaks correspond to a significant change in slope between two segments without sign change. Interesting features are their magnitude and position on x axis. Numerical values of each of these features are translated into symbolic qualifiers thanks to a dictionary. Such a dictionary defines the correspondence between the vocabulary used by experts and fuzzy sets as defined by Zadeh [13]. This approach allows the imprecision of natural language to be taken into account when characterizing curve features. For a given curve element feature, the dictionary may define as many qualifiers as needed. Figure 1 illustrates the “peak shape” entry of a dictionary and corresponding definition of three expert-defined qualifiers.

1 0.5

very sharp

sharp

rounded peak definition threshold

0

0

20

40

60

80 100 peak angle (°)

120

140

160

180

Figure 1: Example of membership functions for peak shape qualifiers. A peak showing a 65° angle is here qualified as both “sharp” with grade 0.2 and “rounded” with grade 0.8.

2

7th International Conference on Hydroinformatics – HIC 2006

Such symbolic qualifiers depend obviously on the target domain, but also on the type of data represented by the curve. Indeed, the same “strong decrease” qualifier for a segment slope will for example be used for quite different numerical values when a water surface profile or a discharge hydrograph is considered. A dictionary has thus to be provided for each curve type. At the end of the three-steps algorithm, the curve is translated into a set of symbolic descriptions of its interesting features, based on the vocabulary given by the expert in the dictionary. Symbolic comparisons In addition to curve descriptions, the module comes with facilities to compare (1) two curves and (2) a set of points with a curve. These facilities are thus of particular interest in the calibration context. Comparing two curves Curves can be compared in two ways. First, features of specific curve elements, such as a segment width or peak shape, can be individually tested. This kind of comparison allows users to take account of one specific curve characteristic which may be of particular interest for their current purpose. Second, a global fit may be computed in order to assess the correspondence between two curves. The symbolic properties of the segments (width and slope) of each segmented curve are compared thanks to a string replacement algorithm [12], in order to get a numerical similarity coefficient. This comparison facility may be useful for example when it comes to assess the differences between the results from two model runs, or between the results obtained at different time steps. This module can thus for example evaluate the qualitative changes in water surface profiles occurring during a flood event. Comparing a curve and a set of points As a complement to curve objects which are bound to represent (nearly) continuous data, we introduced the sets of points in order to represent discrete data. Indeed, computed results may be produced more or less easily in a nearly continuous form, but measurements taken as a reference during model calibration are often few and far between, and thus cannot be taken as continuous. Comparisons between a curve and a set of points are made in terms of relative distance (e.g., “under” or “above”) and relative position (e.g., “close” or “far”). Symbols describing relative position and the distance of a set of points with regards to a curve are defined by the expert in the dictionary associated with the curve type. Two ways of comparing a curve to a set of points are provided: (1) the curve can be referenced with respect to each point in terms of relative distance and position, and (2) considering the set of points as a whole, statistical measures can be used to combine all individual symbolic comparisons.

7th International Conference on Hydroinformatics – HIC 2006

3

Using the curve evaluation module allows to generate automatically qualitative descriptions of discrepancies between model outputs and reference data. Next section shows how these descriptions can be used within the calibration process, and how the curve evaluation module is an appropriate complement of an existing knowledge-based calibration support system. INTEGRATION WITHIN A CALIBRATION SUPPORT SYSTEM Presentation of CaRMA-1 CaRMA-1 (Calibration of River Model Assistant) is a knowledge-based calibration support system designed to help end-users calibrate 1-D river hydraulics models in accordance with good modelling practice. The system includes both numeric and symbolic tools to go through all calibration steps defined in Figure 2. In particular, it makes use of a code called Mage developed at Cemagref to perform the simulations. The calibration process and the knowledge base constituting the core of CaRMA-1 has been built upon a review of guidelines extracted from both the literature and interviews with practitioners [7]. Performing a calibration with the help of this decision support system consequently assures that the process is both reliable and reproducible. A detailed description of the knowledge base and the prototype CaRMA-1 may be found elsewhere [11]. The output comparison task For a given simulated event, the output comparison task aims at producing an assessment of the reproduction of each available reference data by model predictions, as shown in Figure 2. The current version of CaRMA-1 performs the output comparison task in an interactive way. To achieve this task, the system displays a graph showing both the selected reference data (as a set of points) and corresponding model prediction (as a curve). Throughout a series of prompted closed questions, the user is requested to provide qualitative assessment on the agreement. These questions depend on the nature of the reference data and may concern different types of discrepancies, listed in Table 1. Table 1. Basic typology of identifiable discrepancies between reference data and model outputs in 1D river hydraulics.

4

Reference data

Model prediction

Water levels

Water surface profile

Floodmarks

Envelop water surface profile

Discharge hydrograph Stage hydrograph

Discharge hydrograph Stage hydrograph

Type of discrepancy – Alternate level deviations – Consistent level deviations – Alternate level deviations – Consistent level deviations Temporal shift Temporal shift

7th International Conference on Hydroinformatics – HIC 2006

:DomainOfIntendedApplication

:NumericalModel

:EventData :EventData

[uncalibrated]

ModelCalibration

ParameterDefinition

:NumericalModel

[with defined parameters]

DataAssignment

ParameterInitialisation

ParameterAdjustment :EventInputDataSet

:NumericalModel

[with attributed parameter values]

:NumericalModel

SimulationRun

:EventReferenceData :EventReferenceData

[tested]

:EventPrediction :EventPrediction

:EventReferenceDataSet [perfectible agreement] [parameter definition ok]

OutputComparison [perfectible agreement] [else]

[else]

:EventReferenceDataSet [best possible agreement]

:EventReferenceDataSet [perfectible agreement]

ModelPerformanceDescription

:PerformanceCriteria

:NumericalModel

[calibrated]

:DomainOfApplicability

Figure 2: Calibration process formalized through an UML activity diagram –. An object is represented by a rectangle, whereas a calibration step is shown as a rounded rectangle.

7th International Conference on Hydroinformatics – HIC 2006

5

Working example: Calibration of a model of the River Hogneau The calibration of a model of the River Hogneau served as a case study for the prototype CaRMA-1 [9]. Figure 3 shows available reference data for the calibration event – floodmarks collected along the reach – and corresponding model predictions – envelop water surface profiles – from different simulation runs. After each simulation, the user is asked by the system about a consistent deviation along the reach between the curve and the set of points. A positive answer leads then to questions about the global relative position and distance of the curve with respect to the whole set of points. A negative answer leads to the same questions for specific parts of the modelled reach corresponding to zones with homogeneous roughness. CaRMA-1 uses these answers to determine which parameters to be adjusted to reduce the identified discrepancies. For example, if the computed envelop water surface profile is qualified by the user as consistently “under” the floodmarks (cf. Run #1 in Figure 3), the system will increase all roughness parameters and perform a new simulation (cf. Run #2 in Figure 3). The curve evaluation module has especially been designed to automatically provide these answers from the expert knowledge implemented in the dictionary, and thus without relying on the user judgement. 24 Run #1 Run #2 Run #n

23 22

) m ( n oi t a v el E

21 20 19 18 17 16 3.1

3.2

3.3 3.4 3.5 Distance along the river (m)

3.6

3.7 x 10

4

Figure 3. Comparison of measured floodmarks (diamonds) and computed envelop water surface profiles for different simulation runs (solid, dashed and dotted lines).

6

7th International Conference on Hydroinformatics – HIC 2006

Technical integration Within the CaRMA-1 knowledge-base, measurements characterizing a flood event are all derived from the generic set of points concept. In the same way, model predictions about an event are derived from the generic curve concept through an inheritance property [10]. The curve evaluation module has been implemented in the same language as the CaRMA-1 knowledge base, so that the curve symbolic description is transparently called as soon as the knowledge base introduces a curve object. The symbolic characteristics extracted from the original numerical curve may thus directly be used in the reasoning process, and especially when compared with a set of points. A defuzzyfication procedure similar to the one described by Amdisen [1] allows it to automatically associate a qualitative comparison assessment to each reference data processed. DISCUSSION AND CONCLUSIONS The curve evaluation module described in this article provides a generic way to derive qualitative comparisons between a curve and a set of points. Dictionaries define the correspondence between curve symbolic descriptions – as well as comparisons with corresponding sets of points – and numerical values. The current version of the curve evaluation module includes two basic dictionaries relative to discharge hydrographs and water surface profiles. These dictionaries constitute a “capitalization” of the expertise used during the output comparison task and may easily be refined by adding new entries for the description of each feature and each type of discrepancy. These entries should come from a consensus on the vocabulary used by experts, which is not always easy to reach. Validity testing of the implemented dictionary entries should be done on an extensive set of data to cover the shape variability of hydrograph and water profiles. The symbolic and qualitative approach used in the calibration context can be a relevant alternative to the commonly used numerical goodness-of-fit criteria. As emphasised by Green and Stephenson [2], no single criterion is sufficient to provide an overall measure of fit between model prediction and reference data. Visual inspection often remains the most effective way to identify relevant discrepancies, and the curve evaluation module provides a way to automate this task by formalizing expert assessment. Together with the other features of CaRMA-1, it assures that the calibration process is both reliable and reproducible, and thus contributes to good calibration practice. ACKNOWLEDGEMENTS This work was partially supported by the RIO2 (“Risques InOndations”) program of the French Ministry in charge of the environment (grant n° 01008), and by the ECCO (“ECosphère COntinentale : processus et modélisation”) program of the French Department in charge of Research. The authors are grateful to Julien Canet for his work on the implementation of the curve evaluation module.

7th International Conference on Hydroinformatics – HIC 2006

7

REFERENCES [1] Amdisen, L. K., “A fuzzy set model for the interpretation of discrepancies between observed and expected situations”, Proceedings of the Hydroinformatics 1994 Conference, Delft (Verwey A., Minns A. W., Baboviü V. and Maksimoviü ý., eds.), Vol. 1, (1994), pp 43-48. [2] Green, I. R. A. and Stephenson, D., “Criteria for comparison of single event models”, Hydrological Sciences Journal, Vol. 31, (1986), No 3, pp 395-411. [3] McIlraith S., “Qualitative data modelling: Application of a mechanism for interpreting graphical data”, Computational Intelligence, Vol. 5, (1989), No 2, pp 111-120. [4] Sargent R. S., Some subjective validation methods using graphical displays of data, Proceedings of the 1996 Winter Simulation Conference (J. M. Charnes, D. J. Morrice, D. T. Brunner and J. J. Swain, eds.), (1996), pp 345-351. [5] Syed Mustapha S. M. F. D., Phillips T. N., Price C. J., Moseley L. G. and Jones T. E. R., “Viscometric flow interpretation using qualitative and quantitative techniques”, Engineering Applications of Artificial Intelligence, Vol. 12, (1999), No 3, pp 255-272. [6] Vann, A. M. and Davis, J. P., “A shape recognition technique for hydraulic applications”, Proceedings of the Hydroinformatics 1994 Conference, Delft (Verwey A., Minns A. W., Baboviü V. and Maksimoviü ý., eds.), Vol. 2, (1994), pp 699-708. [7] Vidal J.-P., Assistance au calage de modèles numériques en hydraulique fluviale – Apports de l’intelligence artificielle (Computer-aided model calibration in river hydraulics – Contributions from artificial intelligence, in French), (2005), PhD Thesis, Institut National Polytechnique de Toulouse, 228 pp + annexes. [8] Vidal J.-P., Moisan S. and Faure J.-B., “Knowledge-based hydraulic model calibration”, Lecture Notes in Artificial Intelligence, Vol. 2773 (Knowledge-Based Intelligent Information and Engineering Sytems: 7th International Conference KES'2003 ; V. Palade, R. J. Howlett et L. C. Lain, eds.), (2003), pp 676-683. [9] Vidal J.-P., Faure J.-B., Dartus D. et Moisan S., “Decision support system for calibration of 1-D river models”, Proceedings of the Hydroinformatics 2004 Conference, Singapour (S. Y. Liong, K. K. Phoon et V. M. Babovic, eds.), Vol. 2, (2004), pp 1067-1094. [10] Vidal J.-P., Moisan S., Faure J.-B. and Dartus D., “Towards a reasoned 1-D river model calibration”, Journal of Hydroinformatics, Vol. 7, No 2, (2005), pp 91-104. [11] Vidal J.-P., Moisan S., Faure J.-B. and Dartus D., “River model calibration, from guidelines to operational support tools”, Environmental Modelling and Software, (submitted, January 2006). [12] Wagner R. A. and Fischer, M. J., “The string-to-string correction problem”, Journal of the Association for Computing Machinery, Vol. 21, No 1, pp 168-173. [13] Zadeh, L. A., “Fuzzy sets”, Information and Control, Vol. 8, No 3, pp 338-353.

8

7th International Conference on Hydroinformatics – HIC 2006