HaCDAIS 2010: Combined proposal for half‐day tutorial and half‐day workshop at ECML/PKDD 2010
Tutorial and workshop title: Handling Concept Drift and Reoccurring Contexts in Adaptive Information Systems: Importance, Challenges and Solutions. Tutorial instructors and workshop chairs: Mykola Pechenizkiy, Information Systems Group, Department of Computer Science, Eindhoven University of Technology, P.O. Box 513, 5600 MB Eindhoven, The Netherlands, Tel.: +31 40 2474977, Fax: +31 40 246 3992, E‐mail:
[email protected], http://www.win.tue.nl/~mpechen/ Indrė Žliobaitė, Faculty of Mathematics and Informatics, Vilnius University, Naugarduko 24, Vilnius LT‐03225, Lithuania, E‐mail:
[email protected], http://zliobaite.googlepages.com/ The topic: In the real world data is often non stationary. In predictive analytics and machine learning, the concept drift means that the statistical properties of the target variable, which the model is trying to predict, change over time unexpectedly. This causes problems because the predictions might become less accurate as the time passes or opportunities to improve the accuracy might be missed. The problem of concept drift is of increasing importance to machine learning and data mining as more and more data is organized in the form of data streams rather than static databases, and it is rather unusual that concepts and data distributions stay stable over a long period of time. It is not surprising that the problem of concept drift has been studied in several research communities including but not limited to machine learning and data mining, data streams, information retrieval, and recommender systems. Different approaches for detecting and handling concept drift have been proposed in the literature, and many of them have already proved their potential in a wide range of application domains, e.g. fraud detection, adaptive system control, user modeling, information retrieval, text mining, biomedicine, etc. With the proposed tutorial we would like to reach the following goals: 1) Introduce the importance of concept drift handling mechanisms in various adaptive information systems. 2) Provide an overview of existing approaches for handling different types of concept drift, emphasizing the underlying assumptions that these approaches make (whether explicitly or implicitly) about the nature and causes of change. 3) Discuss practical aspects of applying concept drift handling mechanisms to different applications, present examples of success stories showing that concept drift handling does matter (as for the model accuracy as for business decisions), and consider the ongoing and foreseeing development in this field. With the proposed focused workshop which will follow the tutorial we aim at bringing together 1/11
HaCDAIS 2010: Combined proposal for half‐day tutorial and half‐day workshop at ECML/PKDD 2010
concept drift researchers to discuss recent advances in this area, consider both basic research and application issues, and directions of further development in the field. Motivation for organizing the proposed tutorial and workshop (and having them combined) Despite of the fact that the concept drift problem has been actively studied in the past decade, this research area is still emerging, and currently there is no ‘mainstream’. We think that this is partly due to the diversity of application domains and somewhat isolated studies in different research communities. We could see a few recent success stories illustrating the importance of addressing the problem of concept drift in practice. For instance, the winning team of the Netflix completion, and follow up KDD’09 paper by Yehuda Koren on Collaborative Filtering with Temporal Dynamics is likely the most well‐known case. Therefore, we believe that it is a good time to organize a dedicated workshop on this topic. We hope that bringing together the researchers working on detection and handling of concept drift will help to advance this area by discussing this problem from different perspectives. It will also contribute for getting a better understanding of what have been addressed already and what was not across the community. It will also help to identify the gaps as prospects for future research. Having a tutorial on the same topic will help attracting attention of more researchers to this interesting problem. The tutorial‐workshop is particularly relevant due to the wide nature of the topic. We believe that the tutorial will be valuable not only for a wide data mining/machine learning audience, but for the participants of the workshop as well. What we have been noticing is a twofold tendency. On one hand there are many purely application oriented studies where the problem of concept drift is address explicitly or implicitly, but usually with ad‐hoc solutions. On the other hand there exist (almost purely) theoretical studies proposing new approaches which are claimed to be generic, but tested on a few (often artificially generated) benchmarks. We hope that our tutorial will provide a helicopter view on this problem and this way help to better position and defend the further developments in this field. The tutorial should serve for the participants to be on the same page with respect to the framework and terminology when proceeding to the workshop. The workshop is intended not only present recent advances in concept drift handling but also outline the directions and desiderata for the future research. Thus the tutorial will picture where we are now and will serve as a starting platform for the workshop discussions. Format: We will start in the morning with the half‐day tutorial aimed to give an introduction to the current state of the art in both basic and applied concept drift research. 2/11
HaCDAIS 2010: Combined proposal for half‐day tutorial and half‐day workshop at ECML/PKDD 2010
The tutorial will consist of three parts: Part 1: Overview of application areas within user modeling, recommenders, classification and prediction and application domains within (bio‐)medicine, industry and commerce, in which concept drift occurs and has to be handled. We will give illustrative examples of the importance of and challenges related to handling concept drift. We will consider different drift types and application setting with respect to the availability of the data and labels. Part 2: Overview of existing approaches to handle concept drift and reoccurring context proposed in the literature during the last decade. Our focus will be on the categorization of these approaches, discussion of their applicability, underlying assumptions, advantages and drawbacks. Part 3: Reflecting on the past, presence and future of concept drift research and the utility of its results in practice. The afternoon (workshop) sessions will consist of presentations of the selected peer‐reviewed papers by the researchers in the field. In the final session, we will lead an open discussion aimed to foresee the future of concept drift research and to identify immediate opportunities for collaboration. This discussion will also address the point of more frequent community interaction, covering topics such as sharing data and applications as well as the prospects of organizing a future meeting. A more detailed outline of the tutorial: Part 1: Introduction to the area and overview of typical applications.
What is concept drift? (different terminology in different areas)
Types of changes (gradual vs. sudden, global vs. local) and intuition for the base adaptation strategies (training windows, instance selection)
Examples from machine learning and data mining, data streams, information retrieval, and recommender systems. Different approaches for detecting and handling concept drift have been proposed in the literature, and many of them have already proved their potential in a wide range of application domains, including e.g. fraud detection, adaptive system control, user modeling, information retrieval, text mining, biomedicine
Part 2: Solutions
Categorization of different concept drift handling approaches with respect to 3/11
HaCDAIS 2010: Combined proposal for half‐day tutorial and half‐day workshop at ECML/PKDD 2010
Task – prediction vs. classification vs. recommendation vs. clustering
Handling mechanisms ‐ with/without explicit detection
Availability of training data and labels
The “landscape” of change detection methods
Offline detection vs. online detection
Statistical approaches in different application contexts
Streaming error analysis
Integration of background knowledge
Windowing and instance selection and weighing techniques
Moving window, Adaptive window sizes
Online learning under concept drift
Combining time and space similarity
Ensemble approaches for
Detect changes Approximating probability distributions. Threshold on the likelihood of the new data with respect to the assumed distributions. Detecting parameter shift. Evaluating geometrical differences in the class parameters, e.g., cluster centres. Examining Feature relevance. Looking for differences in the importance of features or combinations thereof. Monitoring model complexity. Change in the number of rules in rule‐based classifiers or the number of support vectors in SVM classifiers may signify concept drift. Running error rate. Monitoring the running error rate of the online classifier/ensemble.
Constant update
Use dynamic combiners (horse racing): experts are trained in advance and only the combiner changes Re‐train the individual classifiers online Change the ensemble structure (e.g. replace the loser)
Evaluation of concept drift handling approaches 4/11
HaCDAIS 2010: Combined proposal for half‐day tutorial and half‐day workshop at ECML/PKDD 2010
Problem with ground truth and background knowledge in general
Benchmarking and success criteria
Visualizing Concept Drift
Part 3: Summary, conclusions and future directions
Summary about the problem, approaches, applications and open questions.
Where concept drift research is heading?
Summarizing the commonalities and key differences of prediction vs. classification vs. recommendation tasks in the context of the concept drift problem.
Evaluation framework. Do we need real benchmark data and benchmark experimental protocol for online classification. Or is there a need for formulating more specific problems and trying to solve them. Where do we need to look on the problem from a ‘global’ vs. a ‘local’ perspective.
Different performance tradeoffs: accuracy vs. speed/computational complexity vs. cost sensitive approaches.
Miscellaneous issues.
Prospective participants (tutorial):
Data mining and machine learning researchers interested in getting an introduction into (or knowing more about) this exciting topic.
Data mining practitioners and developers of adaptive information systems in various domains.
Recommender systems researcher (RecSys 2010 is collocated with ECML/PKDD 2010; in recommender systems temporal dynamics and context‐awareness are two important topics and we hope that our tutorial and workshop will attract participants from this collocated event).
No prior knowledge of concept drift problem is required. It is assumed that the participants have basic knowledge of machine learning and data mining techniques. Prospective participants (workshop):
There are few dozens of researchers active in concept drift area. We hope to attract them to contribute and/or attend this workshop. 5/11
HaCDAIS 2010: Combined proposal for half‐day tutorial and half‐day workshop at ECML/PKDD 2010
Researcher in neighboring research areas, including e.g. transfer learning, online learning, context‐aware systems to name a few.
Prospective participants of the tutorial.
Prospective participants of RecSys 2010 conference
(Partially) related special tracks/sessions, workshops and tutorials organized in the past:
Concept Drift, Nonstationary Environments, Mining Data Streams, special session at IJCNN / WCCI 2010
Workshop on Transfer Mining at IEEE ICDM’09
Series of SensorKDD Workshops (at KDD’07, KDD’08, KDD’09) and series of Mining Data Streams Workshops (at ACM SAC and ECML/PKDD)
Mining Ubiquitous Data Streams: From Theory to Applications, tutorial at ICDM’08
Sample Selection Bias – Covariate Shift: Problems, Solutions, and Applications, tutorial at ICDM’08
Knowledge Discovery from Evolving Data, tutorial at ECML/PKDD 2008
Classifier ensembles for handling concept change in streaming data, invited talk at SWIFT 2008
Relevant material/bibliography: Some bibliography is available from http://en.wikipedia.org/wiki/Concept_drift, there are a few published reviews and recent phd thesis which allow to analyze the current state of the art. A technical report by one of the organizers “Learning under Concept Drift: an Overview” http://sites.google.com/site/zliobaite/Zliobaite_CDoverview.pdf A preprint of the review of applications and case studies focused on handling concept drift will be made available before the start of the conference. Practicalities:
Our preference is to have haft‐a‐day workshop to fit both the tutorial and the workshop in a
6/11
HaCDAIS 2010: Combined proposal for half‐day tutorial and half‐day workshop at ECML/PKDD 2010
one day event. Nevertheless, we think that we will be able to attract enough high quality submissions to fill the program of a full day workshop as well if this becomes necessary.
Tutorial and Workshop will have a website at http://wwwis.win.tue.nl/hacdais2010/, tutorial handouts and workshop proceedings will be published online.
We would like to have printed proceedings for our workshop.
The tutorial handouts for audience will be prepared by the organizers several weeks before the start of the conference in accordance with the important dates.
Special requirements: no special requirements; only a beamer will be needed.
Estimated number of attendees: besides the organizers and presenters we expect about 25 people from ML/DM, an d 10 people from RecSys.
Publicizing: besides sending announcements to the main active lists, and social networks, we intend to send personal invitation to the researchers currently active in concept drift, data streams, recommender systems, user modeling, and information retrieval.
Scientific qualifications of the organizers: Mykola Pechenizkiy, Ph.D., Assistant Professor, Department of Computer Science, Eindhoven University of Technology (TU/e), The Netherland. Mykola Pechenizkiy joined TU/e as an assistant professor in 2006. He holds a doctorate in computer science from the University of Jyväskylä, Finland. His main research interests have been in data‐driven intelligence, including selected topics in data mining, machine learning, information retrieval, recommender systems and adaptive hypermedia. Currently he focuses on the development of generic frameworks and techniques for adaptation within and Handling Concept Drift in Adaptive Information Systems and Generic Adaptation Framework projects funded by the Netherlands Organization for Scientific Research. Mykola has experience in organizing workshops, special tracks, and conferences. He also has five years teaching experience including intensive tutorial like courses for postgraduates. For further information see http://www.win.tue.nl/~mpechen/ Indrė Žliobaitė, Faculty of Mathematics and Informatics, Vilnius University, Naugarduko 24, Vilnius LT‐03225, Lithuania, http://zliobaite.googlepages.com/ Indrė Žliobaitė is a final year PhD student at Vilnius University (PhD defense scheduled for April 1, 2010). Her basic and applied research interests concentrate on data mining under concept drift, focusing on adaptive training set formation and ensemble techniques. Papers relevant for this tutorial and (co‐)authored by the organizers:
Bakker, J., Pechenizkiy, M., Žliobaite, I., Ivannikov, A. & Kärkkäinen, T. (2009) Handling Outliers and Concept Drift in Online Mass Flow Prediction in CFB boilers, In Proceedings of the Third International Workshop on Knowledge
7/11
HaCDAIS 2010: Combined proposal for half‐day tutorial and half‐day workshop at ECML/PKDD 2010
Discovery from Sensor Data (SensorKDD'09), ACM Press, pp. 13‐22. (Best paper award) Pechenizkiy, M., Bakker, J., Žliobaite, I., Ivannikov, A. & Kärkkäinen, T. (2009) Online Mass Flow Prediction in CFB Boilers with Explicit Detection of Sudden Concept Drift, SIGKDD Explorations. Tsymbal, A., Pechenizkiy, M., Cunningham, P. & Puuronen, S. (2008)Dynamic Integration of Classifiers for Handling Concept Drift, Information Fusion, Special Issue on Applications of Ensemble Methods, 9(1), pp. 56‐68. Puuronen, S., Pechenizkiy, M. & Tsymbal, A. (2008) Effectiveness of Local Feature Selection in Ensemble Learning for Prediction of Antimicrobial Resistance, In Proceedings of 21st IEEE International Symposium on Computer‐ Based Medical Systems (CBMS'08), IEEE Computer Society, pp. 632‐637. Žliobaite, I., Bakker, J. & Pechenizkiy, M. (2009) OMFP: an Approach for Online Mass Flow Prediction in CFB Boilers, In Proceedings of 12th International Conference on Discovery Science (DS'09), Lecture Notes in Computer Science 5808, Berlin: Springer, pp. 272‐286. Žliobaite, I., Bakker, J. & Pechenizkiy, M. (2009) Towards Context Aware Food Sales Prediction, In Proceedings of IEEE International Conference on Data Mining (ICDM'09) Workshops, IEEE Computer Society, pp. 94‐99. Žliobaitė, I., Kuncheva, L. (2009). Determining the Training Window for Small Sample Size Classification with Concept Drift. Proc. of 2009 IEEE int. conf. on Data Mining Workshops, the 1st int. workshop on Transfer Mining (TM‐09), p. 447‐452. Žliobaite, I. (2009). Combining Time and Space Similarity for Small Size Learning under Concept Drift. Proc. of the 18th Int. Symposium on Methodologies for Intell. Systems (LNCS 5722), p. 412‐421. Kuncheva, L.I. and Žliobaitė, I. (2009). On the Window Size for Classification in Changing Environments. Intelligent Data Analysis 13(6), p. 861‐872. Kuncheva, L., Žliobaitė, I. (2008). Linear Discriminant Classifier (LDC) for Streaming Data with Concept Drift. SSPR/SPR 2008 Žliobaitė, I. (2008). Expected Classification Error of the Euclidean Linear Classifier under Sudden Concept Drift. Proc. of the 5th Int. Conf. on Fuzzy Systems and Knowledge Discovery (FSKD 2008). IEEE Computer Society: vol 2, p. 29‐33. Žliobaitė, I. (2007). Introduction of New Expert and Old Expert Retirement under Concept Drift. Progress in Pattern Recognition, series: Advances in Pattern Recognition. S. Singh, M. Singh (Eds.) 2007, p.64‐74. Raudys, Š., Žliobaitė, I. (2005). Prediction of Commodity Prices in Rapidly Changing Environments. Pattern Recognition and Data Mining, proc. of the 3rd int. conf. on Advances in Pattern Recognition, ICAPR 2005 (LNCS 3686), p. 154‐163.
Recent talks relevant for this tutorial and given by the organizers: by Mykola Pechenizkiy
Context‐aware prediction in the wild: lessons learnt from case studies. Information Systems colloquium (IE&IS/IS ‐ W&I/IS), TU/e, Eindhoven (October 2009) OMFP: An Approach for Online Mass Flow Prediction in CFB Boilers. The 12th International Conference on Discovery Science (DS’09), Porto, Portugal (October 2009) Online Mass Flow Prediction in CFB Boilers. The 9th Industrial Conference on Data Mining (ICDM’09), Leipzig, Germany (July 2009) Handling Outliers and Concept Drift in Online Mass Flow Prediction in CFB Boilers. The 3rd International Workshop on Knowledge Discovery from Sensor Data (SensorKDD‐2009), Paris, France (June 2009) Food Sales Prediction: “If Only It Knew What We Know”. The 2nd International Workshop on Domain Driven Data Mining (DDDM’08 at IEEE ICDM 2008), Pisa, Italy (December 2009).
8/11
HaCDAIS 2010: Combined proposal for half‐day tutorial and half‐day workshop at ECML/PKDD 2010
Facets of Concept Drift Problem in Adaptive Information Systems. LIAAD/LIACC Seminar, University of Porto, Portugal (October 2007) Dynamic Integration of Classifiers for Handling Concept Drift. The 19th IEEE Symposium on Computer Based Medial Systems (June 2006)
by Indre Zliobaite
Learning in changing environment. Vytautas Magnus University, Lithuania (2010); Vilnius University, Lithuania (2010); Eindhoven University of Technology, the Netherlands (2009) Training Set Selection in the Presence of Concept Drift. Helsinki University of Technology (2009), Helsinki; and Eindhoven University of Technology, the Netherlands (2009) Determining the Training Window for Small Sample Size Classification with Concept Drift. IEEE int. conf. on Data Mining Workshops, the 1st Int. workshop on Transfer Mining (TM‐09), Miami Florida, USA (2009). Combining Time and Space Similarity for Small Size Learning under Concept Drift. The 18th Int. Symposium on Methodologies for Intell. Systems, Prague, Czech Republic. (2009) Pattern Recognition in the Presence of Concept Drift. Bangor University, UK (2008) Expected Classification Error of the Euclidean Linear Classifier under Sudden Concept Drift. The 5th Int. Conf. on Fuzzy Systems and Knowledge Discovery (FSKD 2008), Jinan, China. (2008) Pattern Recognition under Concept Drift. Summer School on Modern Data Mining Technologies. Druskininkai (2007) Introduction of New Expert and Old Expert Retirement under Concept Drift. International Workshop on Advances in Pattern Recognition (IWAPR 2007), Plymouth, UK.
Organized workshops, special tracks, conferences:
Architectures and building blocks of web‐based user‐adaptive systems Workshop Co‐Chair at UMAP'2010; LOIS Workshop Process Mining meets Data Mining (PMDM'09) Co‐organizer Dynamic and Adaptive Hypertext: Generic Frameworks, Approaches and Techniques (DAH'09) Workshop Co‐Chair at HT'2009; The 22nd IEEE International Symposium on Computer‐Based Medical Systems (IEEE CBMS 2009) Special Tracks Chair; Knowledge Discovery and Decision Support in Biomedicine (KDDSB'09) Special Track (Co‐)Chair since 2005; The 21st Benelux Conference on Artificial Intelligence (BNAIC'09) Industry Track Chair & Local co‐organizer Induction of Process Models (IPM'08) Workshop Co‐Chair at ECML/PKDD'2008; The 21st IEEE International Symposium on Computer‐Based Medical Systems (IEEE CBMS 2008) Programme Chair Dutch‐Belgian Database Day (DBDBD'07) Local co‐organizer Educational Data Mining (EDM@ICALT07) Workshop Co‐Chair at IEEE ICALT'07; Applying Data Mining in e‐Learning (ADML'07) Workshop Co‐Chair at EC‐TEL'2007; PMKD'06 Workshop Co‐chair.
Related teaching skills (besides the recent talks on the topic): intensive courses
Educational Data Mining (Fall 2008, Open University), 1 full day course to researchers (handout accessible from http://www.win.tue.nl/~mpechen/talks/edm08_ou.pdf)
SIKS/LOIS Course on Process Mining and Data Mining (Fall 2009, SIKS@TU/e), two day course for PhD students.
9/11
HaCDAIS 2010: Combined proposal for half‐day tutorial and half‐day workshop at ECML/PKDD 2010
================================================================================== Preliminary Call for Papers: ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐ HaCDAIS 2010: First International Workshop on Handling Concept Drift in Adaptive Information Systems: Importance, Challenges and Solutions (http://wwwis.win.tue.nl/hacdais2010/) at ECML/PKDD 2010, Barcelona, Spain, September 20‐24, 2010 ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐ In the real world data is often non stationary. In predictive analytics and machine learning, the concept drift means that the statistical properties of the target variable, which the model is trying to predict, change over time unexpectedly. This causes problems because the predictions might become less accurate as the time passes or opportunities to improve the accuracy might be missed. The problem of concept drift is of increasing importance to machine learning and data mining as more and more data is organized in the form of data streams rather than static databases, and it is rather unusual that concepts and data distributions stay stable over a long period of time. It is not surprising that the problem of concept drift has been studied in several research communities including but not limited to machine learning and data mining, data streams, information retrieval, and recommender systems. Different approaches for detecting and handling concept drift have been proposed in the literature, and many of them have already proved their potential in a wide range of application domains, e.g. fraud detection, adaptive system control, user modeling, information retrieval, text mining, biomedicine, etc. We invite papers in different categories including but not limited to:
Generic frameworks for handing concept drift and reoccurring contexts
New approaches advancing the current state of the art
Case studies illustrating importance, challenges and solutions
Data generators, evaluation frameworks, and software reports
Discussion/position papers of the more provocative nature
Format The workshop sessions will consist of presentations of selected peer‐reviewed papers by the researcher in the field. In the final session, we will lead an open discussion aimed to foresee the future of concept drift research and to identify immediate opportunities for collaboration. This discussion will also address the point of more frequent community interaction, covering topics such as sharing data and applications as well as the prospects of organizing a future meeting.
10/11
HaCDAIS 2010: Combined proposal for half‐day tutorial and half‐day workshop at ECML/PKDD 2010
The workshop will be preceded by the half‐day tutorial aimed to give an introduction to the current state of the art in both basic and applied concept drift research. See further information at http://wwwis.win.tue.nl/hacdais2010/tutorial.html (currently not active) Important dates
Submissions due June 10, 2010
Author Notification on July 12, 2010
Final Papers due July 21, 2010
Workshop September XX, 2010
Paper submission Papers (up to twelve pages in Springer format) or extended June 10, 2010. Final versions of accepted papers will appear in the informal ECML/PKDD workshop proceedings and will be made available on the workshop website before the workshop takes place. Submission implies the willingness of at least one of the authors to register and present the paper. Authors of accepted extended abstracts will be asked to submit a short 4 to 8 page paper in PDF format (following the Springer LNCS guidelines for preparing manuscripts) that describes their research in more detail. Workshop organizers Mykola Pechenizkiy Eindhoven University of Technology, The Netherlands Indrė Žliobaitė
Vilnius University, Lithuania
Intended Program committee (tbc) Albert Bifet
University of Waikato, New Zealand
Gladys Castillo
University of Aveiro, Portugal
Sarah Jane Delany
Dublin Institute of Technology, Ireland
Anton Dries
Katholieke Universiteit Leuven, Belgium
Bogdan Gabrys
Bournemouth University, UK
Joao Gama
University of Porto, Portugal
Ricard Gavalda
Universitat Politecnica de Catalunya, Spain
Ludmila Kuncheva Bangor University, UK Leandro Minku
University of Birmingham, UK
Robi Polikar
Rowan University, USA
Alexey Tsymbal
Siemens AG, Germany
11/11
