Annex G: An Example of Evaluation Plan Contents Site input needed on: What will be measured, How will it be measured, who will be responsible, what existing data, when will it be measured?
Evaluation Plan 1.
INTRODUCTION 1.1 Background 1.2 The Consortium 1.3 Project Objectives 1.4 Relationship between project measures and evaluation 1.5 Report Objectives
2.
PROJECT CITY BACKGROUNDS 2.1 La Rochelle 2.2 Ploeiti 2.3 Preston
Commentaire [TP1] : TTR has done this already.
3.
PROJECT CITY TARGETS 3.1 Introduction 3.2 City 1 3.3 City 2 3.4 etc APPROACH TO EVALUATION 4.1 Introduction 4.2 Impact Evaluation 4.2.1 Levels of Evaluation 4.3 Process Evaluation
Commentaire [TP2] : TTR has done this by copy/paste from IR. LR to review and suggest track-changes if required.
4.
5.
6.
CITY LEVEL AND MEASURE LEVEL EVALUATION 5.1 Introduction 5.2 City 1 (5 pages max) 5.2.1 Local Evaluation Plan (overview) – 17 juillet (HLB) 5.2.2 Roles and Responsibilities – 17 juillet (HLB) 5.2.3 Selection of Indicators – send 5.3 City 2 etc 5.3.1 Local Evaluation Plan (overview)
PROCESS FOR ACHIEVING THE EVALUATION RESULTS
Commentaire [TP3] : This is OK.
6.1
6.2 6.3 6.4 6.5 6.6 6.7 6.8 7.
Evaluation Tools 6.1.1 General 6.1.2 City 1 (5 pages max) –send (will be completed for 12th July) 6.1.3 City 2 6.1.4 City 3 Ex-Ante evaluation (before) Ex-Poste Evaluation Samples sizes BaU-send Upscaling – completed for 12th Gender issues-send Process evaluation
WAY FORWARD 7.1 Local Conclusions – in progress (for 17 July) 7.2 Overall Conclusions
Commentaire [TP4] : OK, think I have this from the “success_rapport4juillet.doc”
Commentaire [TP5] : Good. Will use. Commentaire [TP6] : Did not receive. You can miss this if you want. We can discuss at next meeting. Commentaire [TP7] : Good, will use.
CITY LOCAL ANNEX A1
Introduction and Overview (10 pages max) – in progress for 17 July
A2
Measure evaluation Plans – 17 July
A2.1 Work Package 5: Clean Vehicles and Alternative Fuels- the technical works are in progress; so, it will be a simple presentation of objectives, targhets, indicators -17 July A2.1 Description of Measure A2.2 Timetable for Measure A2.3 Measure Evaluation Indicators A2.4 Timetable of Evaluation A2.2
…etc A4 A5 8.2 8.3 8.4 8.5
Work Package 6: Access Management-12th July A3.1 Description of Measure A3.2 Evaluation Indicators A3.4 Evaluation Timeframe Work Package 7: Integrated Pricing Strategies- 17 july Work Package 8: Stimulation of Public Transport Modes –send
Commentaire [TP8] : Did not receive.
Commentaire [TP9] : OK Commentaire [TP10] : OK Commentaire [TP11] : MAK E a sub-section where is outlined again the timetable/timing for evaluation (this can be text only if you want for this measure) Commentaire [TP12] : MISSI NG information/heading on timetable for evaluation.
A6
Work Package 9: New forms of Vehicle Use and Ownership-12th july
Commentaire [TP13] : Did not receive.
A7
Work Package 10: New Concepts for Goods Distribution 12th july
Commentaire [TP14] : Did not receive.
A8
Work Package 11: Innovative Soft Measures –17 july
A9:
Work Package 12: Telematics-17 july
A3
Methodological approaches- 12 th july
Questionnaire(s) - Appendix.
Commentaire [TP15] : Did not receive. Commentaire [TP16] : OK, can include.
5.2
City level and measure level evaluation: La Rochelle 5.2.1
Local Evaluation Plan
The Local Evaluation Plan is defined by all strategies, methods and exploitation tools which were chosen for the data collection (models, measurements, surveys, questionnaires). This work was done in collaboration with each Measure Leader, in particular for the selection of pertinent indicators. The measures to be implemented to estimate are presented bellow.
Work Package 5: Clean vehicles and alternative fuels 5.1
Measure Development of clean collective transport
Hybrid buses, bio-diesel
Keyword
5.2
Development of clean light vehicle fleet
Hybrid vehicles
5.3
Implementation of diester fuelling station
Build fuelling station
5.4
Development of hybrid taxi fleet
Hybrid taxi's,
5.5
Cooking oil recycling pilot project
Cooked oil recycling plant
Work Package 6 : Access Management 6.1
Measure Develop and extend access control zones
Access control zones
Keyword
6.2
Design access control scheme for tourist buses
Access control zones
Work Package 7 : Integrated pricing strategies 7.1
Measure Implement further integration of ticketing system
PT ticketing system
Keyword
7.2
Strategies for integrated pricing system
Parking pricing system
Work Package 8: Stimulation of PT modes Measure
Keyword
Implementation of a second P&R
Parking management P&R
8.2
Implementation of dedicated bus lanes
High mobility corridor
8.3
Reorganisation of the bus network
Optimising bus network
8.4
Night taxi services
Night taxi
8.5
Extension of bike-bus scheme
Bikes on buses
8.6
Infrastructure improvement for collective transport
PT infrastructure improvement
8.1
Work Package 9: New forms of vehicle use and ownership Measure
Keyword
9.1
Deployment of new car sharing fleet
Car sharing
9.2
Bike sharing
Bike sharing
Work Package 10: New concepts for goods distribution Measure 10.1
City logistics strategic extension
City logistics strategy
10.2
Customers services associated to goods distribution
Customer goods distribution
10.3
Development of partnership with logistic operators
Stakeholder group
Work Package 11: Innovative soft measures Measure
Keyword
11.1
Implementation of new structure for alternative modes
Access restrictions
11.2
Business travel plan
Business travel plan
11.3
Students travel plan
School travel plan
Work Package 12: Telematics Measure
Keyword
12.1
Development of integrated transport management systems
Stakeholder participation
12.2
Implementation of a common transport information data base
Transport information database
12.3
Real-time informations systems
Real-time information system
12.4
Smart card system
Access control zones
12.5
Development of an integrated pricing system
Integrated pricing system
5.2.2
Roles and responsibilities
We distinguished several type of contribution in the local evaluation plan: Roles in Evaluation Hervé LE BERRE Luminita ION-BOUSSIER Hichem OMRANI Mélanie RAIMBAULT Sylvie ORSINI Anjali AWASTHI
Local Evaluation Manager – CdA La Rochelle Local Evaluation Manager Support – EIGSI – School of industrial Systems Engineering Evaluation team Member - EIGSI Evaluation team Member – CdA La Rochelle Evaluation team support – EIGSI Evaluation team support – EIGSI
Matthieu GRAINDORGE Dominique BREUIL Matthieu AUBINEAU N
Site Manager – CdA La Rochelle Technical Coordinator Manager– EIGSI Technical Coordinator Manager Support – CdA La Rochelle Dissemination Manager– CdA La Rochelle
Sub contractors evaluation support ACOUSTEX ATMO Poitou-Charentes STERELLA
Acoustic Measurements Environnemental Measurements Traffic flows counting
Also contacts for evaluation LEPTAB – University Of La Rochelle Environnement Department – City Of La Rochelle Public Lightning Department – City Of La Rochelle On-street Equipment Department – City Of La Rochelle Environnement Department – CdA La Rochelle JC DECAUX VIACOM
WP Leaders WP1 WP2 WP3 WP4 WP5 WP6 WP7 WP8 WP9 WP10 WP11 WP12
PROJECT MANAGEMENT TECHNICAL COORDINATION EVALUATION DISSEMINATION AND EXPLOITATION CLEAN AND ENERGY EFFICIENT VEHICLES ACCESS MANAGEMENT INTEGRATED PRICING STRATEGIES STIMULATION OF COLLECTIVE TRANSPORT MODES NEW FORMS OF VEHICLE USE AND OWNERSHIP NEW CONCEPTS FOR THE DISTRIBUTION OF GOODS INNOVATIVE SOFT MEASURES TELEMATICS
Matthieu GRAINDORGE Dominique BREUIL Hervé LE BERRE Sandrine RAYMOND Nicolas CHAUVINEAU Thierry REVEILLERE Laure MARTIN Thierry REVEILLERE Thierry REVEILLERE Thierry REVEILLERE Nicolas CHAUVINEAU Laure MARTIN
Also involve in Work Packages WP1 WP4
WP5
WP6 WP7 WP8
WP9
WP10 WP11
WP12
5.2.3
Marie LAUNAY – TTR France Jean-Marie GRELLIER – CdA La Rochelle Valérie SIGOGNEAU – CdA La Rochelle Nathalie DEBORD – CdA La Rochelle Carine PAILLE – CdA La Rochelle Caroline HUBAULT – CdA La Rochelle Milène MAZZOCCO – CdA La Rochelle Armèle PASQUIER – CdA La Rochelle Joel BREHERET – EIGSI François DUCLAUT – EIGSI Thierry REVEILLERE – CdA La Rochelle Matthieu AUBINEAU – CdA La Rochelle Sophie BON – CdA La Rochelle Sébastien LAMPIN – RTCR Monsieur Thierry MAYOLLE – OCECARS Monsieur Laurent COCHAIN – KEOLIS LITORAL Jean-Michel LUCAS – JC DECAUX Philippe MORISSET – City Of La Rochelle Alain LUCAZEAU – CdA La Rochelle Anne CHANE – CdA LA Rochelle Sylvie ORSINI – EIGSI Jacques MOLLARD – Ville de La Rochelle Dominique BREUIL – EIGSI Sylvie ORSINI – EIGSI Sophie BON – CdA La Rochelle Guy PEUDUPIN – Les Nouveaux Armateurs Louis MARROU – Université Of La Rochelle Dominique BREUIL – EIGSI Xavier ROCQ – CdA La Rochelle Jean-Philippe MEHEUST – CdA La Rochelle
Selection of indicators
The evaluation of indicators is done in agreement with the GUARD’s indications. For several measures we proposed, in addition, specific indicators associated to a measure. • Example: for the new Park&Ride implementation, we proposed a list containing several common indicators suggested by Guard, but we will also evaluate: the rate of the occupancy of the vacant places, the number of customers, the type of car,… First of all, a general analysis was done in La Rochelle for each measure; a list of indicators were firstly chosen for five categories (economic, energy, environment, society and transport) without to taking into account the “intensity” of the measure. Afterwards, the managers of the measures selected the pertinent indicators in agreement with their potential impacts and suggested the level city of each measure. The doubt concerning the pertinence of some indicators (in particular in pollution and transport fields) was analysed using the principle of the “worst case”. • Example: the bike sharing system, not very spread (50 bikes actually) seems to have not a significant effect in the consumption of energy, quality of the air, gas emission in the atmosphere or in terms of noise pollution. However, even if it seems difficult to estimate the environmental impacts of using bike sharing, it can be interesting to approximate some indicators. From the average emission of a CO2 motorcar and the distance crossed, the implementation of a simplistic mathematical model is possible. This model will allow
quantifying approximately the gain in emission of CO2 in the atmosphere due to the use of the bike sharing (the occupancy rate of a car will be estimated by survey). This approach also allows for giving information concerning the “up scaling” case. It is possible to evaluate, for the “worst case”, which is the “level” for what the measure will be superior at a fixed value (example for CO2>1%). The indicators chosen will be listed in the presentation of each measure as well as the planning for their evaluation.
6.1
Evaluation tools
A part of the evaluation team work was focused this year on the choice of methods to evaluate the impacts for each category and the characterisation of the “before” states of the urban areas affected by the impacts of the measures SUCCESS. The various methods to evaluate an indicator can thus be: -
A measurement: to take into account some given, it is necessary to measure various factors having to be taken into account for the study. It is for example the case for a number of kilometres, or a time of course. These measurements will be used then for the construction of a model usable in various cases and adapting to the various situations and the various needs for the cities in which reorganization is wished.
-
A model: we can build a mathematical model to anticipate the modifications to be carried out by forecast and study of the various desired cases of figures. Thus, once this model established, we will be able to study the various specific data. For example, the costs of operation are calculated in a equivalent way of one line to the other and carried out calculations take into account the same factors; we can thus create a model to evaluate this indicator
-
An inquire: to understand practices and waitings of the users, and to answer their needs as well as possible, they can answer to an enquire. The treatment of their answers will be able to make it possible to target the points to be re-examined and to make sure that the modifications carried out is well appropriate to them.
-
A counting: certain indicators require just a counting such as for example the number of users per day, by bus...
For a study of an indicator even more precise, it is possible to use several methods. We present some elements for each category:
6.1.1
Economy category
COSTS For each measure, a model for estimate the costs will take into account investments implemented for the measure (M. Preston suggestions), repair and maintenance, administration costs and traffic management costs. BENEFITS The model to estimate the benefits takes into account: -
Direct benefits such as the price of the service: tickets, subscriptions (counting,…)
-
Indirect benefices for the city: reduction of fuel (measurements), reduction of congestion road (model and survey), decrease of the risk of accident (survey)
-
Indirect benefits for user: optimisation of Value of Time (questionnaire).
6.1.2
Energy category
The consumption of energy could be evaluated in two ways: • quantity of fuel paid at pumps, • simplified model to estimate the fuel reduction after modal change
6.1.3
Environment category
AIR QUALITY AND EMISSIONS Guard suggested that the level of the atmospheric pollution in SUCCESS project should use the indicators presented below: - Air quality: NOx, CO, PM - Emissions: CO2, CO, NOx, PM10 …) The indicators usually used are based on legal ones (like the mean value of the concentrations, the maximal value). The work will be done in collaboration with ATMO Poitou-Charentes, company agreed ISO for the evaluation of air quality and atmospheric emissions for all Poitou-Charentes Region. In this context, only the methodology to select an experimental strategy is possible; the models used are proposed by ATMO. 1)
Implemented stations
There are six permanent stations implemented in La Rochelle area which stoke daily information of the concentrations of CO, NOx and PM. The most part of them give information for the global evolution of the city: the results are representative for an evolution at long term of the concentrations as the result of various factors (global evolution of the traffic, the development of industrial sites, or the evolution of human activities). After the study of the configuration of the area, only one seems to be able to give rigorous information in relationship with the traffic evolution (figure 6.1). The station called “La Grille” is implemented in a street with high buildings (to neglect the diffusion phenomenon) and placed at an important distance on the activities areas (producing supplementary pollutants).
Figure 6.1
2)
Fixed stations for pollution measurements in La Rochelle
Mobile stations
Mobile stations will stock the measurements of concentrations of CO, NOx and micro particles will be placed in strategic points in the areas concerned by changes (in particular for measures like access controlled areas, park&ride, carsharing. To study the impact of actions done in SUCCESS project on the pollutant, these mobile stations will be positioned in strategic places, where the modifications of traffic will be most important. Data collection would be carried out before the application of a new measure (the initial state), then the evaluation of impacts will be repeated to evaluate the impacts during and after the implementation of the measure.
Three zones were defined (see paragraph A.1). For each area, a mobile cabin for NOx, CO, and PM will be installed (figure 6.2). The choice of the site to place these cabins is very important; we tried to find canyon streets, with high buildings, it is possible to detect even the low evolutions of the indicators (in practice, it was not always possible). The environmental measurement will be done both in summer season and at the beginning of autumn during the implementation period (see the 2006 reference calendar in table 6.1).
Table 6.1
2006 calendar for environmental measurement
The data collected with the fixed station of Verdun (figure 6.1) will complete the necessary information for the zone B. Tubes for benzene will be placed in the vicinity
3 1
Zone B
Zone C
Zone A
2
Figure 6.2 spatial locations in three zones of the city
3)
Diffusion tubes
The mobile cabin allows to obtain the measurements in a given place, but to have spatial information we will use the passive samplers. They are tubes of 7 cm distributed on the 3 studied zones. Each area is covered with about 15 points of measurements and each point is equipped with 2 tubes. (see figure n° 6.3 above).
Zone A
Figure 6.3. Cartography of diffusion tubes Zone A (working paper) The results will be presented in a cartography format (concentrations during the period of measurement-figure 6.4).
Figure 6.4. Cartography obtained using the diffusion tubes The extent of the zone of study can go from the scale of road to that of an agglomeration.
4)
Modelling CO2, CO, NOX, PM
The model ADMS Urban will be used to estimate by hour the concentrations in the air: starting from the data of emissions (which integrates weather data, data on the configuration of the zone of study) this model allows estimate the concentrations in several points. ADMS-Urban is distinctive in its ability to describe in detail what happens on a range of scales, from the street scale to the city-wide scale, taking into account the whole range of relevant emission sources: traffic, industrial, commercial, domestic and other less well-defined sources
The science of ADMS-Urban is significantly more advanced than that of most other air dispersion models (such as CALINE, ISC and R91) in that it incorporates the latest understanding of the boundary layer structure, and goes beyond the simplistic Pasquill-Gifford stability categories method with explicit calculation of important parameters. The model uses advanced algorithms for the heightdependence of wind speed, turbulence and stability to produce improved predictions. It is necessary to have the results of a series of measurements “to calibrate” the model (figure 6.5), i.e. to make sure that the concentrations estimated by the model are in agreement with the reality. Fine estimates on the scale of the agglomeration, in particular for the emissions related to the traffic, can be carried out within the framework of an assessment CO2 of the La Rochelle. Campagne d'octobre NO2 µg/m3
ADMS
90.00
Mesure ATMO
80.00 70.00 60.00 50.00 40.00 30.00 20.00 10.00 1
13
1
13
1
13
1
13
1
13
1
13
1
13
1
13
1
13
1
13
1
13
1
13
1
Heure
Figure 6.5 Comparison between experimental measurements and model
For a complete exploitation of the model, traffic data will be obtained using automatic counting of the traffic level in the streets affected by SUCCESS changes. 5)
Modelling of emissions CO2, CO, NOX, P
COPERT estimates emissions of all regulated air pollutants (CO, NOx, VOC, PM) produced by different vehicle categories (passenger cars, light duty vehicles, heavy duty vehicles, and motorcycles) as well as CO2 emissions on the basis of fuel consumption.. However, it can been shown that the methodology can also be used with a sufficient degree of certainty at a higher resolution too, i.e. for the compilation of urban emission inventories with a spatial resolution of 1x1 km2 and a temporal resolution of 1 hour. The emissions associated to traffic are estimated using the methodology COPERT III, and it is based in traffic data. The estimations are done for each main road of the network (figure 6.6). The impact of the measures SUCCESS is easier to quantify for the emissions than the concentrations. It will also be possible to adapt some particular indicators: for actions like the car-sharing or the bikesharing, it could be interesting to evaluate the “benefit in emissions”/vehicle. ACOUSTIC ANALYSIS The evaluation of the acoustic intensity is suggested by questionnaire (perception of people). We propose to complete our analysis by measurements and model. The explanation of this extension is that in the zone C (where a dedicated bus lane and an access controlled zone will be implemented), it is not possible to use a questionnaire (away for users or commercials, a transit zone). For the three areas, in the proximity of the mobile cabins, a numeric microphone will be placed to evaluate the noise (dBA). In the same time, using the counting of the flows, it will be possible to evaluate an approach value of the acoustic intensity with a mathematical model using the last version of the software CadnaA 3.5. The new version CadnaA 3.5 comes with a completely new concept for defining evaluation
parameters. The new concept offers a wide variety of selectable evaluation levels (for individual or combined time periods), and additional evaluation parameters (e.g. uncertainty). If highways or railway routes are planned or would be able to be modified during the SUCCESS project, the noise levels in the surrounding residential areas are calculated. The result of such a study comprises a list of these measures, colored noise charts for presentation, and tables with the noise levels for any number of emission points.
Figure 6.6 Comparison between NO2 collected during several years At the end of the project, we envisage to be possible to give a global evolution of the noise level in La Rochelle by fusion of information (database with experimental measurements in three zones, the results of acoustic model using the traffic database): the cartography of the noise (2009) will be compared with the cartography obtained in 1999 (figure 6.7 for an example).
Figure 6.7 Cartography for the noise level in La Rochelle
For the zone B (where two measures are conjointly implemented 8.2, 8.3), the noise of the flows corresponding to the highway risks to perturb the measurements; this noise will be simultaneously estimated with a model based on the counting of the number of vehicles. So it will be possible to find the really evolution of the acoustic intensity associated to the implementation of these measures. 6.1.4
Society category
This category is generally evaluated using the questionnaire methods. There are two types of questionnaires concerned by the evaluation: one concerning the declared preferences (before) and another one concerning the observed behaviours (after). Statistical treatment of data will be collected by questionnaires which deal with behavioural information by social demographic category and by gender. An important work was done to elaborate questionnaires for different indicators (acceptance, satisfaction level, perception of the quality of a service,..).
6.1.5
Transport category
EXPERIMENTAL MEASUREMENTS The Urban Community of La Rochelle is going to make counting of vehicles in those three zones. Different elements will be measured: -
number of vehicle/hour,
-
type of vehicles (lorries, cars, busses),
-
speed of the traffic,
The counters will strategically be placed to apprehend the differences between the situation before and after the implementation of the measures. The idea is to note the global figure evolution and the measure impact, linked with environmental measurement, after their implementation. Counting will also be done two times in the year: the summer season and a autumnal period (between middle September and the end of October). An example is done in figure 6.8.
Figure 6.8
Zone B
Moreover the Urban Community of La Rochelle has an automatic system in order to evaluate the traffic level in strategic points of the city (series of measurements are implemented each year, during two weeks since 2000) (figure 6.9). We can notice the low number of measurement points, which didn’t allows a global analysis of the traffic management for all network in La Rochelle.
Figure 6.9 The red and white points on the map refer to the emplacement of counters. Graphical examples are presented in figure 6.10. for the hourly analysis for incoming and out coming flow in the three zones previously defined. The amplitude is resumed on a 0 – to – 100 scales. Tr a f f i c f or M i ni me s
90 80 70 60 50
Input
40
Output
30 20 10 0
Figure 6.10 Flows in Zone A This database will be use to give a global comparison of the traffic level in La Rochelle before and after the implementation of all measures. MODELLING We have not a traffic model to evaluate the changes of flows distribution of cars according with the measures in SUCCESS. In fact, only a global evolution should be interesting because of the low intensity of the measures: example: 50 bikes for the bike-sharing (for instant). We try to establish a model based on the attractiveness areas (evaluation using the density of activity points of the city) and a matrix OD (origin-destination) element (evaluated by surveys of car users, public transport user
7.1
Local Conclusions 7.1.1
Main works
Most part of the measures are analysed but the implementation phase of each one is at different phases at the moment; that’s why the evaluation plan for each one is therefore at different levels. The main works in the evaluation plan in La Rochelle were focused on: • to establish the list of the pertinent indicators • to define specific indicators for several measures (rotation rate for car-sharing, number of subscribers for several services,…) • to find methods and tools for the evaluation of each indicator • to design questionnaires for each measure in society and transport categories • to define the clustering procedures • take contacts and establish contracts with companies for the experimental measurements (counting of traffic flows, pollution measurements – acoustic and atmospheric domains) • to organise, with care for the spatial and temporal synchronisation, the measurements in environment, transport and society analysis • to develop a methodology for a global evaluation of the efficiency of a measure • to design a decision-making aid tool
7.1.2.
Particularities of La Rochelle evaluation
We insist in several particularities of the work done this year: • because the most part of the measures are implemented in this period, the evaluation concerned essentially the “before scenario” with a particular attention for the gender issue • if a measure substituted another one similar, a detailed analysis of the old system was done in order to benefit from the anterior experience (park&ride, reorganisation of the network buses, car-sharing) • because of activity related to tourism in La Rochelle, the experimental measurements were organised during two seasons (with and without tourists) • because of the low “intensity” of some measures, a spatial and temporal clustering was proposed • complementary analysis are done in order to validate the results (example: acoustic measurements for the noise level coupled with questionnaire for the perception of the sound, experimental measurements coupled with models for the emissions concentrations)
7.1.3.
Dissemination
An important work was done to present SUCCESS and the methodology to the scientist community. Some works are presented below: [1]
L. Ion, H. Omrani, P. Trigano, C Popescu, M. Augustin, D. Vlad, « evaluation des actions pour l’amelioration de la
mobilite urbaine », to appear in the International revue of UPG, Romania [2] L. Ion, H. Omrani, P. Trigano, C Oanaitescu, M Popa, «Decision making aid tool to manage urban projects», Quality in Services 2006, September 7-9th 2006, Paisley, Scotland (accepted) [3] H. Omrani, L. Ion, P. Trigano, «An Approach for Environmental Impacts Assessment based on Multi Criteria Analysis and Belief Theory», MCDM 2006 (MCDM’06), June 19-23 2006, Chania, Greece. [4] A. Awasthi, H. Omrani, L. Ion, D. Breuil, P. Trigano, «An AHP/DS based hybrid approach for evaluating environmentally sustainable transport solutions», MCDM 2006 (MCDM’06), June 19-23 2006, Chania, Greece. [5] H. Omrani, L. Ion, P. Trigano, «An Approach for Environmental Impacts Assessment using Belief Theory», IEEE IS’06, 3rd IEEE Conference On Intelligent Systems, 4-6 September 2006, UK. [6] H. Omrani, L. Ion, P. Trigano, «An approach to decision-making for assessment of environmental impacts», IPMU’06, The 11th Information Processing and Management of Uncertainty International Conference, July 2-7 2006, Paris, France.
[7] H. Omrani, L. Ion, P. Trigano, « Une Approche hybride basée sur la logique floue et la théorie des croyances pour l’analyse des impacts liés à la mobilité urbaine», LFA’06, Rencontres francophones sur la Logique Floue et ses Applications, LFA 2006, Toulouse, les 19-20 Octobre, France, accepted. [8] H. Omrani, L. Ion, P. Trigano, «Système Expert pour l’Amélioration Environnementale de la mobilité urbaine : S.E.A.E», Conférence TAIMA-05 : Traitement et Analyse de l’Information, Méthodes et Applications, 26 septembre 2005, Hammamet-Tunisie.
7.1.4.
Collaboration in SUCCESS framework
Please remark that the two last papers are conjointly done by the research teams of Universitatea Ploiesti and EIGSI La Rochelle. In the same time we proposed a CNRS project (EGIDE, Brancusi) in order to ensure exchanges between these two institutions (pedagogic and scientist exchanges). La Rochelle also proposed to organize experimental measurements in NO2 concentration during the next spring in Ploiesti. This project is in study.
7.1.5.
Work in progress or for short time
• we began (July, for two weeks), one of the experimental series in the zones A,B,C • the procedure of the clustering of the questionnaire is in work (first application in September and October) • the methodology to evaluate the efficiency of a measure is at the implementation phase for the decision-making aid tool design. • a model based in the neuronal network is at the moment in the validation step • a simplified model to link the acoustic intensity and the traffic flows will be develop • the cost-benefits analysis will be tested for some measures
1
LOCATION OF MEASURES
The measures will be implemented in different points of the city but the main impacts are essentially focused in three zones which will be defined latter. The map below (figure 1.1) shows their locations.
Figure 1.1:
Map with the main locations of the implementation points in La Rochelle
Because of this focalisation, as well as the low “intensity” of several measures, a local evaluation will be conjointly done for several indicators (especially in the environment and transport categories). The boundary limits (areas) of these zones were chosen because several measures targeted behavioural changes for the users; an example could be the introduction of the access controlled zones or the dedicated bus lanes. The reaction of the network users and car drivers, are not always the same as expected (modal change or departure hour variation); motorists can use their car on an alternative road, in the vicinity of the road concerned by the measure. So, for the three zones, the counting of the traffic flows as well as the experimental measurements of NO2 concentration (diffusion tubes) will be done in a larger zone around the concerned way. Even if we are focused in a local evaluation, the models will cover a larger area, coupling the experimental results done in SUCCESS framework with a lot of information done by counting machines in several points of the city and which have no particular link with the SUCCESS measures (fixes stations for the measurements, counting of the flows during 2 weeks in September). In this case, a global view of the state of the art of the city (transport, environment) will be obtained before and after SUCCESS project. Even if, each measure independently will have not significant impacts, the cumulative effect could be pointed out.
Project Evaluation Plan Report
June 06
Page 1
2
CLUSTERING PROCEDURE
For the evaluation of the indicators, we sometimes operated three types of clustering:
2.1
Spatial and temporal clustering
Because of the “low impact” of several measures, we considered that no considerable changes will be remarked concerning the transport and environment impacts. After preliminary analyses of the potential effects of the SUCCESS measures and the choice of the pertinent indicators, we carried out the grouping of measurements concerned in the three areas (figure 2.1). • zone A: bike-sharing, car-sharing* and access controlled area • zone B: park&ride, dedicated bus line to park&ride, secure cycle ways for bikes • zone C: access controlled areas, bus line. • * (car-sharing stations are not definitively specified, so we considered that the almost part will be implemented in the sites of the actual car-sharing system) Experimental measurements, surveys and questionnaires will be conjointly done in the same period for each zone for the “before” scenario. After the implementation of all measures/zone, an intermediary evaluation will be done. At the end of the project a global view will be proposed for all zones in order to estimate the global effect of all measures at the city level.
Figure 2.1:
Project Evaluation Plan Report
Proposal for spatial clustering of the measures
June 06
Page 2
2.2
Thematic clustering
Several measures treat a common problematic (even if the measures are included in different Work Packages). This aspect will be exploited specially (for questionnaires) as follow: • for each measure, a preliminary questionnaire is established, • common questions are selected for all concerned measures, • common questions and specific ones are included in a final questionnaire; it will be proposed for measures which are concerned by the same problematic Example: a common questionnaire could be proposed for the buses use. Specific questions (information system, dedicated bus lane, pass tickets…) will be added and the users will give their responses if they are concerned about. On the other hand, some indicators in society and transport categories (like the acceptance and satisfaction level, perception of the accessibility, etc.…) will be studied with the same questionnaire.
2.3
Correlated measures clustering
We applied several measures which are strongly link and a common questionnaire will be established for it. Example: a common questionnaire will be proposed for the access control zone and the dedicated bus lane (zone B). These clustered measures will need as a result a great synchronisation of periods of experimental measurements and questionnaires. We consider that the travels of the users (drivers, pedestrians or bus users) are in agreement with the activities performed by each one. One of the characteristic of our city is the activity related to tourism; so the experimental measurements are divided in two distinct periods: season with tourists and season without tourists (summer and autumn). Measures such as the park&ride implementation, bikebus, will be strongly concerned by this difference. This effect is not so strong for all three zones, so it will be studied essentially in zones A and C. In the Local Evaluation annex, the timetables will be detailed for each measure. The methods used for the evaluation of the indicators and the procedure to select the indicators were presented in the next paragraphs.
Project Evaluation Plan Report
June 06
Page 3
3. GENDER ISSUE A particular attention is focused on the gender issue for several measures. Such as for example, the database of the actual car-sharing system which was exploited and the results give interesting information: even if the number of female customers is lower than of male ones, we could notice that the use of this service is particularly appreciated by females (figure 3.1). Customers per age / gender 250
200
150 Females Males
100
50
0 36 / 45
26 / 35
18 / 25
46 / 55
> 55
companies
Uses per Age/Sexe
18 / 25
26 / 35
36 / 45
46 / 55
Females
124
82
47
41
Males
217
157
82
78
companies
> 55 6000
28
17
76 5000
4000
3000
2000
1000
0 18 / 25
> 55
46 / 55
36 / 45
26 / 35
companies
18 / 25
26 / 35
36 / 45
46 / 55
> 55
Females
3597
2396
2833
874
466
Males
5517
2377
2883
3102
2236
companies 292
Figure 3.1 Subscribers of car-sharing per age and gender This tendency is also remarked for the average value of loans per year/age/gender (table 3.1):
18/25
26/35
36/45
46/55
> 55
Companies
Females
29
29.2
60.2
21.3
16.6
17.2
Males
25.4
15.1
35.2
39.8
Table 3.1
29.4
Evolution of the subscriber characteristics
A main conclusion of this work is that the women use more frequently the LISELEC car-sharing service than the men (31.5 average values for women). Differences are more remarkable especially for young people.
Project Evaluation Plan Report
June 06
Page 4
The software car-sharing of the new system will give similar information; the dissemination could be specially targeted in this way. Another example which was not yet quantified is the use of buses by gender (measures 8.2, 8.3, 8.5); we will identify the frequency rate per gender in public transports services with data processing ticketing system implemented in La Rochelle or with coloured tickets. Contrarily to the actual service, the new Park&Ride (measure 8.1) will also have a software system able to give information on people identity and the database will be used for a statistical analysis/per gender. The overall questionnaires contain questions concerning the gender of the respondent; a statistical analysis per gender will be performed.
4
BUSINESS AS USUAL
For the experimental measurements (traffic, pollution, number of users of a service), this scenario will be evaluated by extrapolation (when a sufficient database is available). An example is the number of users of the park&ride (measure 9.1). It is easy to notice that the frequentation rate increased the last years (a value can be obtained by extrapolation using the occupancy rate of the places). In the same time, a strong increase corresponds to the tourist season, when the capacity of the park&ride becomes insufficient (figure 4.1).
2001
2002
2003
2004
2005
Figure 4.1 Evolution of number of users for the actual Park&Ride
Concerning the indicators to be evaluated by questionnaire, we will use essentially the needs of the potential users; for the par&ride use, 87% of users pointed out the necessity to create another park&ride in the north and/or east of La Rochelle. This remark is one of the reasons which are at the origin of the measure 9.1 (implementation of a park&ride in the north of La Rochelle).
5
UP-SCALING
For the up-scaling scenario will be done essentially for the indicators which are measured or obtained by model; generally we will search the limit value of an indicator for which the measure could have a sensible impact. For that, we will use the principle of the “worst” case.
Project Evaluation Plan Report
June 06
Page 5
6
CONTAINS OF THIS ANNEX
In the next part we will present the main works done in the evaluation plan in La Rochelle. Some measures are not yet completely defined. Even if the list of the pertinent indicators was established, we will not detail this part because some indicators could be changed. Example: the vehicle used for the car-sharing could have a thermal or hybrid motor. On the other hand, the hybrid motors can have a different configuration: electric motor for start (so the measurements of the noise level are interesting, electric motor coupled with thermal motor for the acceleration and the deceleration phases, so this indicator is not available. For each measure which was technically defined we will present the main work in progress. For several measures, some questionnaires will be proposed in corresponding appendix; the clustering
Project Evaluation Plan Report
June 06
Page 6
A3
Methodological approaches
La Rochelle
METHODOLOGY TO ESTIMATE THE GLOBAL EFFICIENCY OF A MEASURE
After the implementation of a measure it is necessary to estimate the global efficiency of it. Indeed, some indicators can increase, while other ones could decrease. In agreement with the main objects of the project, the weight of the indicators can be different (example: for the access controlled zones, the safety level of the pedestrians could be more important than the satisfaction level of drivers who must change their itineraries). The problem is to determine how a global impact of a measure could be quantified. The METEOR guidelines and the workshop in June organised by Guard give several ways to estimate the global efficiency of a measure. We focused our analysis in two approaches: - costs-benefits analysis, - multi-criteria analysis.
1.1
Costs-benefits analysis (CBA)
This method deals with of a detailed state of the art and a very interesting work presented by John Preston during the Evaluation Workshop organised by Guard in June. The basic idea is that the costs and benefits of the measure are expressed in monetary terms and are systematically presented in a balance sheet. Costs can be subtracted from the benefits, the result of which will indicate the efficiency of a measure.
1.1.1
Main costs and benefits of a transport project
The main terms to be estimated in order to evaluate the efficiency of a transport project are: MAIN COSTS: INTERNAL COST : user pays (caused and borne by users) A) Monetary costs (fuel charges, registration fees, insurance premium, vehicle purchase, private garages, tools and parking charges) B) Non-monetary (travel time including delays, excess fuel in congested roads, risk of accidents) EXTERNAL COSTS: user causes but does not fully pay (non-users also pay) A) Environmental impacts (health, damage to plants and materials) B) Noise impacts C) Depletion of non-renewable resources D) Climate change E) Deaths, injuries, stress, trauma, loss of non-user property MIXED COSTS: government pays and partly recovers by users A) Provision and maintenance of infrastructure (noise barriers, safety provision) B) Administration coast C) Emergency services (police, ambulance) D) R/D in alternative fuels and other road transportation research SOCIAL COSTS: consequence of the nature of road transportation A) Disruption in social life fabric and loss of community coherence B) Loss of the bio-diversity and other ecosystem impacts C) Reduction in aesthetics D) Ugliness of ribbon development along corridors E) Loss of productive land F) Reduced access to disadvantaged group in the society G) Loss of mobility independence and need for chauffeuring children H) Loss of inner city vitality
MAIN BENEFITS 1. accessibility and mobility 2. industry – reduction in production cost 3. employment and economic growth 4. increase in land values For SUCCESS measures we detected several terms which seems us important for the studied categories.
1.1.2
Estimation of external and social costs for SUCCESS
1) In Energy category Fuel costs include: special tax for domestic oil industry, security costs Estimation: the cost of depleting non-renewable reserves is obtained by subtracting the cost of fuel paid at the pumps from the cost of recovering fuel for motor use from alternative renewable sources) 2) In Society category Accidents costs Estimation 1. Loss of productivity approach that considers the purely economic value of the output foregone 2. Society’s willingness to pay to save lives or reducing the risk of accidents Infrastructure costs include: cost of construction, repair and maintenance, administrative costs, cost of traffic management Estimation: users pay part of infrastructure (vehicle registration, fuel taxes, tolls, parking) Social costs: property values, relocation costs, mitigation / control costs, structural damage, lost utility of cultural and recreational facilities Estimation: two methodologies: Contingent Valuation (CV)- people are directly asked to state how much they are willing to pay for a particular good or benefit, such as an improving the in air quality Stated Choices (SC)- each scenarios, made up of a combination of attributes, can be considered as an alternative, since it represents a product or a service which may not be observed in a real market (taking into account every attribute, people are asked to choose between two scenarios the alternative which suits the best with their preference) 3) In environment category Air pollution costs: Estimation 1. extremely difficult to estimate directly (by measure) because several other sectors of economy also generate pollution (the environmental costs should be apportioned according to emissions for each sector) 2. extremely difficult to estimate indirectly 9using the consequences) because the impacts affect differently the other costs (SO2- bronchitis, CO – fertility, particulates – cardiovascular diseases but lead – loss of productivity) Noise pollution costs include: property values, relocation costs, mitigation / control costs, structural damage, lost utility of cultural and recreational facilities
Estimation: by revealed preference approach and by measuring the reduction in market value of housing exposed to noise compared with similar housing in quieter areas 4) In Transport category Congestion costs include: the delay or lost time, stress to road users, additional wear of vehicle, excess fuel used, increase in accident risk, additional emissions Estimation: with the exception of emissions, the costs are paid by user
1.1.3
Advantages and inconvenient of CBA
Interests of this approach the evaluation of impacts in monetary terms is a easily estimation of the global impacts at long term; the decision-makers are more sensible for an evaluation in monetary terms in project efficiency. Inconvenient of this approach: it is not easy to find an equivalent in monetary terms for all consequences of a measure (in particular concerning the impacts at long term); the weight of indicators is difficult to establish; the decision makers may have a range of other objectives without link with the costs/benefits analysis. We concluded in the complexity of this approach for some measures because of the necessity to select and to evaluate many terms. This method will be used for some measures which will be specified later.
1.2
Multi-criteria analysis (MCA)
This is a flexible method taking into account quantitative and qualitative multidimensional effects of transport projects. Because this approach represented our choice for the development of a decision making aid tool, we will make a more detailed description. Generally, the decision analysis looks at the paradigm in which an individual decision maker (or decision group) contemplates a choice of action in an uncertain environment. In multi-criteria decisionmaking context, the selection is facilitated by evaluating each choice on the set of criteria. The criteria must be measurable; even if the measurement is performed only at the nominal scale (yes/no; present/absent) and their outcomes must be measured for every decision alternative. Criterion outcomes provide the basis for comparison of choices and consequently facilitate the selection of one, satisfactory choice. Criterion outcomes of decision alternatives can be collected in a table (called decision matrix or decision table) comprised of a set of columns and rows. The table rows represent decision alternatives, with table columns representing criteria. A value found at the intersection of row and column in the table represents a criterion outcome a measured or predicted performance of an alternative (Ap) on a criterion (Cq). In SUCCESS project, the studies will concern the relationship between indicators, parameters and transport design variables, analysis of the crossed influences of these different factors through the various experiments and results. Some of the particularities of this project are: • the evaluation of a measure is based on several indicators which will have a different weight in the evaluation process and which may be illustrated by numerical values while others by linguistic variables. • several actors (experts) may give their evaluations; they can be political specialists, town planning decision makers, traffic engineers, citizens, etc. Each one has a degree of
knowledge in a particular category, whose opinions are based on several sources having varying reliability. • the information sources of each expert may be different for the initial and ex-post evaluations. In conclusion, the problem treated is complex with insufficient, fuzzy and uncertain data. The evaluation process could be schematically presented as follow (figure 1.1). measured Set of indicators
Mesure(s)
estimated
Values with interval
Sensors Models, Questionnaire
quantitative values and / or
Experts
qualitative evaluated
Figure 1.1
The evaluation process
In this case, the matrix of criteria using information for multisources will be presented as illustrated in figure 1.2. Criteria
M
M
E, S, Q, M
M E, Q
M S, M
[a11]
K K
Cq
K ] K
[a1q
Cn [a1n]
M K K K K [ap1]
[as1]
K
[apq]
K
[apn]
KK KK K [a ] K [a ] M
sq
sn
Vector of Values / Scores
At
Experts, Sensors, Questionnaires, Model Sources
C1
M.C.A & M.S
Figure 1.2
Matrix of opinions (with several alternatives, multi-experts and multi-sources of information)
This approach is the result of a coupling of MCA with the fuzzy logic and the belief theory. The three main steps are: weight of indicators - evaluation of indicators - evaluation of the efficiency of a measure 1) Weight of indicators: The Repertory Grid theory (Kelly method) allows to find dependencies between indicators: The AHP (Analytic Hierarchy Process) is then used in order to establish the weight of the indicators (the scale is 1-9). The process is schematically presented bellow (figure 1.3)
Weight estimation: AHP Indicators
Iq=1,…,n
Evaluation with linguistic terms Example:
Fuzzy AHP (Analytic Hierarchy Process)
Weights wq
n
(Small, Medium, Large, etc.)
With: å wq = 1 q= 1
Figure 1.3. Process for evaluating the weights of indicators
Impacts are weighted depending on their relative importance or priority in terms of meeting the objectives of the measures being considered. The choice is done by the manager of the measure and experts in different fields (economy, transport, environment,…). 2) Evaluation of indicators All information concerning an indicator is transformed using the belief theory (an assignment of masses is done). The interest of this approach is the fusion information collected using different sources. This problem do not concerns the “after” scenario, but the “before” scenario and the “business as usual”. In La Rochelle, like in a great part of French cities, there are many organisms which had evaluated the same impacts with various means (questionnaire, measurements). Sometimes the information is in conflict and the belief theory allows us to take into account the doubt, the ignorance or the imprecise information. An example for a pollutant concentration using experimental data collected in La Rochelle (figure 1.4). Crisp reasoning
Mass assignment (crisp)
Pollutant “NO2”
C O
M
N C
E A
of
E N T R
N
A T I O
Day N Hour Mean concentration of NO2, hour by hour, year: 2004 (left) and mean daily concentration (right) via the sensor of “La Grille, La Rochelle”
Figure 1.4. Mean concentration of NO2
The estimation of the density of probability of NO2 (Gaussian estimator Kerr) (left) and superposition with its histogram (right) allows to obtain classes and to attribute a mass distribution (figure 1.5).
Density of NO2
Histogram of NO2
- If (NO2 in 9-35) then the level is «Good» - If (NO2 in 35-69) then the level is «Acceptable» - If (NO2 in 69-135) then the level is «Poor» Mass distribution (m)
m(Good)= 0.232 m(Acceptable)=0.428 m(Poor)= 0.339 NO2
Figure 1.5. Mass assignment The greatest advantage of Dempster-Schafer (DS) theory is the robustness of its way of combining information coming from various sources with the DS orthogonal rule. Just consider that the studied measure is the Park&Ride implementation. Concerning the number of visitors we had two sources of information: a survey with a semantic evaluation and a counting system. For example, let us denote two mass distributions m1 and m2 from two sources (figure 1.6).
Example1: experts opinions Frame of discernment (level of evaluation): Ω={V.S, S, M, L, V.L} = {Very Small, Small, Medium, Large, Very Large}
Fuzzy sets corresponding to the frame of discernment
The set of mass {m(TF), m(F),…, m(TE)} correspond to functions drain aux fonctions tracées en bleu, et m(Hi, Hj) correspondent aux fonctions sont tracées en noir.
Example2: Sensors
Numbers of visitors in 2004, day by day Nb of V I S I T O R S
Day
All the Year 2004
- If (users numbers150 & users numbers250) then Frequency of appearance
Class= Large
Mass distribution (m)
Crisp mass
m(S)= 0.293 m(M)= 0.469 m(L)= 0.241
Figure 1.6. Mass assignment using two sources of information
The technique of discounting allows taking in consideration the reliability of the information source. In order to give a “score” for an indicator (evaluated with a single source or by a multi-sources approach), we have realised a pignistic transformation and have used the utility theory. 3) Evaluation of the efficiency of a measure The global evolution of the efficiency of a measure can be made in two different ways:
for each indicator, a score is done after and before the implementation of the measure. It is easy to conclude if the impact was benefic or not, with a global assessment (it can crisp or fuzzy) for a given measure At according to all criteria (criteria in all, Ck , k = 1, 2,K , n , having each one respectively as pertinence degree wk , k = 1, 2,K , n ) by taking into account the weight of each indicator. A global utility is computed for “before” and “after” scenarios; if we want to compare the « before » (ph1) and « after » (ph2) scenarios we can use
E .r M t
( )ph1® ph2
=
(Ch(u% )-
( )) Ch (u% tph )
t ph2
Ch u% tph1 1
The efficiency of the measure (E .r ) is the relative difference between the global scores before and after the implementation of the measure.
The main steps of the evaluation of a measure can be schematically be presented as follow (figure 1.7):
Figure 1.7
Steps of the evaluation of the efficiency of a measure
We are currently developing a data-processing prototype allowing the visualization of knowledge resulting from the sources of information (figure 1.8).
Figure 1.8
Architecture of our evaluation tool (Decision Support System for Environmental Impacts Assessment of the Urban Mobility
Remarks concerning the MCA: This method does not take into account the effectiveness of the measures in comparison to their costs, although they can include costs as one of the criteria by which the measure is chosen.
1.3
Choice of an approach to evaluate the global efficiency of a measure
Even if we have worked for a decision-making aid tool based on Multi-criteria Analysis, we consider that no individual method can give a complete, global or rigorous description of the impacts of all measures. The risk for each measure will also be evaluated. The solution could be to couple or to adapt different approaches for various measures: Examples: MCA: park&ride, car-sharing, bike-sharing, access controlled zone,… CBA: telematics measures, fleet of hybrid taxis
