Enseignement Interactif à l'UPMC Final Report
16 juin 2012 Alexander L. Rudolph*, Michael Joyce, Brahim Lamine, et Patrick Boissé** Université Pierre et Marie Curie *Professeur Invité **Directeur de la faculté de physique
Abstract Interactive Learning (IL), a set of pedagogical strategies developed, researched, and tested for more than 30 years, has now been effectively implemented at UPMC. In the 2011-12 academic year, IL has been used at UPMC in over 20 classrooms enrolling almost 1200 students in three departments: Physics, Mathematics, and Engineering. IL has been used in multiple settings (Amphi, TD), in a wide variety of different modes (Think-Pair-Share questions and interactive demonstrations in Amphi, Tutorials in TD), and in different classes of various levels (L1 through M1) and subjects (Physics, Mathematics, and Engineering). Tests of the effectiveness of IL have been conducted in three major areas: 1) promoting student learning, 2) improved classroom environment (e.g., increased student engagement and motivation); and 3) instructor and students’ attitudes about IL and its impact on student learning and motivation relative to traditional instruction. Analysis of the results of these tests is underway; preliminary results suggest that IL is very effective in promoting student learning, when compared to traditional teaching methods, and that instructor and student attitudes towards IL and the classroom environment it enables is very positive. A séminaire bilan was held among instructors who used IL during the past academic year, to discuss their experiences and make plans for the future. As a consequence of this séminaire bilan a number of actions are being taken to promote the further development of IL at UPMC and sharing of this experience with other colleagues. The main barrier identified to promoting wider use of such strategies at UPMC is the lack of knowledge about IL and the evidence of its effectiveness in other departments. Support and dissemination of this knowledge, from levels at the university above those of individual departments, is therefore essential for the future development of strategies of this type at UPMC. I. Introduction and background Although the French educational system is different from the American in many ways, the two systems share the goal of helping science students to develop a deep conceptual understanding of their disciplines. Extensive research has shown that many university science students do not truly understand the basic concepts of their field when they are taught in the traditional lecture style (Hake 1998, Prather, Rudolph, and Brissenden 2009, McDermott and Redish 1999, Crouch and Mazur 2001): for example, one assessment showed that students in a traditional lecture class learned only 25% of the material that they did not already know (Crouch and Mazur 2001). For the past 30 years, education researchers in physics, astronomy, and other scientific fields (e.g., biology, chemistry, and mathematics), have been working to understand students’ misconceptions about basic concepts in these fields, and to develop the means to improve that understanding (McDermott and Redish 1999, Bailey and Slater 2005). Research shows that pedagogical strategies that actively engage students in the material can be much more effective than traditional lecture alone (National Research Council 1999). This work has led to the development of pedagogical materials, commonly referred to collectively as interactive learning (IL) strategies, designed to be used in university science classes, as well as in classes for younger students. As a consequence of discussions beginning in June 2010, Dr. Alexander Rudolph, Professor of Physics and Astronomy at California State Polytechnic University, and an expert in interactive pedagogy, was invited to UPMC as Professeur Invité for Spring semester 2012, to assist in the implementation of IL in science classrooms at UPMC. A number of instructors at UPMC had already begun using such strategies in the prior year, and inviting Dr. Rudolph was considered a way to promote further
experimentation with these innovative pedagogies. While at UPMC, Dr. Rudolph has had four main tasks: 1) conducting ateliers to train faculty interested in using IL in their classrooms on the best practices for their effective implementation (see Figure 1); 2) visiting classrooms on the invitation of instructors to observe their classrooms and provide additional feedback on effective implementation of IL; 3) collecting data on the effectiveness of IL in some of the classrooms implementing IL to assess if Figure 1. Faculty at UPMC participating in an atelier on interactive learning. these strategies are effective in the French university classroom, and to consider how to tailor their implementation for France; and 4) teaching classes in LP226, Physics in English, a second year class designed for students who wish to improve their scientific English. Dr. Rudolph taught 4 cours of 2 hours each, using interactive learning strategies throughout each class. II. Interactive Learning at UPMC A. Overview Table 1 lists the classrooms and instructors implementing interactive learning at UPMC during the 2011-12 academic year. Over 20 classrooms enrolling almost 1200 students implemented some form of IL during this past year, the majority during the second semester while Dr. Rudolph was visiting. Although most of the classes using IL were in Physics, a number of classes in Engineering and Mathematics also successfully implemented IL in their classrooms, emphasizing that IL techniques can apply to any subject. Table 1. Classrooms implementing interactive learning at UPMC UE LP104 LP111B
LP112
LP203 LP205 LE207 LP211 LE356 LP342 LP344 LP347 LP348 LC324 MP059 LM121 LM121 LM121 LM110 LX-ONC LG304
Utilisation Amphi TD Benoit Semelin X Laurence Rézeau X Michael Joyce X Pascal Viot X Michael Joyce X (tutorials) Benoit Semelin X (tutorials) Romain Bernard X (tutorials) Helene Roussel X (tutorials) William Sacks X (tutorials) Helene Vignolles X (tutorials) Philippe Thomen X Brahim Lamine X Yannis Zouganelis X (tutorials) Florence Ossart X Sébastien Payan X (F) X Hélène Roussel X Maria Barbi X Edouard Kierlik X Patrick Boissé X Jean-Hugues Fillion X Tristan Briant X X Tristan Briant X Ramona Anton X Claire David X (F) Andreas Hoering X (F) Claire Cazes/Pierre Jaraud X Thierry Marchand X Thierry Marchand X Enseignant
Etudiants Période parcours effectifs L1 BGPC 110 P1 L1 PCME/MIME 120 P2 L1 PCME/MIME 50 P2 L1 PCME/MIME 60 P2 L1 PCME/MIME P2 L1 PCME/MIME P2 L1 PCME/MIME P2 110 L1 PCME/MIME P2 L1 PCME/MIME P2 L1 PCME/MIME P2 L2 FP/PC/PM 100 P2 L2 PI/L2 3S 50 P2 L2 3S 20 P2 L2 Elec 90 P2 L2 FP/PC/PI 55 P2 L3 Elec 60 P2 L3 PI 50 P1 L3 PI 50 P1 L3 PI 50 P1 L3 PI 70 P2 L3 PC 40 P1 M1 25 P2 L1 MIME 70 P2 L1 70 P2 L1 PCME 21-24 120 P2 L1 20 P2 L1 20 P1&P2 L3 20 P2
Implementation of IL in UPMC classrooms was different in every class, but fell into two main categories. The first is Think-Pair-Share (TPS) questions, typically used in Amphi (see Figure 2), whereby students are asked to answer a QCM designed to test their knowledge of a science concept being presented in the class, first thinking by themselves and choosing an answer (Think), then discussing their answers with their neighbors (Pair), and finally choosing their answer a second time (Share), possibly revising their answer in response to the discussion with their peers (Peer Instruction). This IL strategy is one of the simplest and yet most effective ways for Figure 2. Students in LP205 discussing their answers to a TPS question. students to actively engage in the classroom (Mazur 1997, Bruff 2009). The students choices can be collected in various ways: at UPMC the most common was to use clickers (boîtiers réponses), small remote devices that allows an instructor to record the students’ answers on their computer, and display them in real time as a histogram. Clickers were installed in Amphi A2 on small chains (see Figure 3), allowing multiple instructors to use clickers with minimum effort. Other instructors carried small boxes of clickers to class and handed them out and collected them each class. Another solution that has been used is to distribute the clickers to students at the beginning of the semester, and to collect them at the end of the semester. The best method of making clickers available to Figure 3. A clicker attached to the desk in Amphi A2. students at UPMC is still being discussed.
A
B
C
D
Figure 4. A colored flashcard.
An alternative method of collecting student responses for TPS is the use of colored flash cards (see Figure 4). Students simply fold the card to reveal the desired answer and display them to the instructor simultaneously and anonymously (by holding them to their chests), when instructed. (Those instructors who used flash cards in the Amphi have an “F” shown in Table 1.) This method is very inexpensive and easy to implement, but does not allow some of the features that clickers do.
A related, and very powerful method of IL used in the Amphi is the predictive experiment or demonstration. In this case, the instructor shows the students an experiment with two or more possible outcomes and then asks them to predict the results before conducting the experiment. Students select their answer, then talk to their neighbors to discuss their choices, select their answer again (as in TPS), after which the instructor conducts the experiment to see the outcome. Students who have participated in this form of instruction are very interested to see the outcome of the experiment and research has shown that they have a much better understanding of the reason the experiment turns out the way it does, and have better recall of the results as a consequence.
The other major form of interactive learning implemented at UPMC is the tutorial, used in the Travaux Dirigés (TD). Tutorials, which are done in small groups of 3-4 students (see Figure 5), consist of a worksheet of questions designed to elicit students’ naïve ideas about a particular topic, and to lead them to a better understanding of those concepts by confronting their naïve ideas by showing them contradictions with the relevant scientific understanding of the topic, and resolving those conflicts in favor of the scientific understanding. The tutorials being Figure 5. Student in an LP112 TD working on a used (McDermott and Shaffer 2002) are tutorial. designed to take about one hour. A number of instructors at UPMC implemented tutorials in their TD, as shown in Table 1. B. Motivation The faculty who implemented IL in their classrooms were surveyed as to their motivation for implementing interactive learning in their classrooms. Responses tended to focus on the difficulties students have in staying engaged in a 2-hour long class, and the advantages of having students become active in the classroom, both for reasons of attention and understanding. With regard to the latter, instructors mentioned the concern they had that students did not really understand the material based on lecture alone (a concern borne out for many instructors when they saw the results of the TPS voting in their classes). Some instructors were also concerned that students in TD tended to be very passive, not engaging in the exercises, but simply waiting for the instructor to provide solutions. It was their hope (subsequently found to be true) that students using tutorials in TD would become more active and engaged in their work. C. Instructor reaction Reaction of the faculty who used IL in their classrooms at UPMC has been uniformly positive. In a seminaire bilan held in May, ten faculty gave presentations on their experiences of using IL this past semester. All felt that IL had improved the atmosphere in their classroom, and were particularly impressed at the way IL got the students to talk about science in the classroom. One instructor noted that one of the most positive outcomes of using IL was, “Hearing students spontaneously explain their reasoning in front of their fellows during lectures! In a French university, this is a near-miracle.” Another noted, “I found teaching much more satisfactory and enjoyable in Amphi - simply I had a real sense I felt of what is really going on for the students, what they are understanding or not. In TD I had a similar sense, and also I could actually see students working more autonomously, managing to work out problems for themselves (with help from me and their classmates).” D. Student reaction In a number of classes, students were surveyed about their reactions to IL, both at a midpoint on the semester, and at the end. The end-of-semester results are still being evaluated, but the mid-term results are very positive. For example, in LP205, a second year class on Electricity and Magnetism, almost 90% of students stated they had a positive reaction to IL in the classroom, and almost 70% stated that it had a
positive impact on their learning in the class. Similar results were found in a survey of two Amphi in LP112, the first year mechanics course. III. Evaluation of Efficacy of Interactive Learning at UPMC In two of the largest classes using IL in their classes, LP112, the second semester of first year mechanics, and LP203/LP205/LE207, the second year Electricity and Magnetism classes, data is being collected to study the efficacy of IL in the French university classroom. These data will be analyzed and will form the basis for one or more peer-reviewed publications in international journals of science education (e.g., the International Journal of Science Education or IJSE). The data being collected falls into three broad categories: 1) Measures of student learning. To test whether IL is effective in helping students learn the material in the class, students in both classes are being given a research-validated learning assessment twice: once at the beginning of the semester, before instruction begins (pre-instruction), and once at the end, at the end of instruction (post-instruction). For the mechanics class, the assessment being used is the Force Concept Inventory (FCI; Hestenes, Wells, & Swackhamer 1992). For the E&M classes, we are using the Conceptual Survey of Electricity and Magnetism (CSEM; Maloney et al. 2001). By comparing students’ scores before and after instruction, it is possible to measure the gain in learning due to the classroom instruction. In addition to these research-validated assessments, we are collecting the final exam scores for both the LP112 and the L2 E&M classes. In LP112, the final exam is common to all sections of the class. In LP205, there was a common exercise used on the final exam for all three classes, allowing comparison between sections of the class. Both assessments (FCI and CSEM) and all of the common exam questions were vetted and approved by the instructors in each course, who all agreed that they were reasonable measures of student learning, consistent with the learning goals all the instructors shared for the classes. In both LP112 and the L2 E&M classes, some sections of the class are using interactive learning (both in Amphi and TD), and others aren’t, the latter acting as natural control groups. Though the evaluation of such data is complex, due to effects such as systematic differences in the abilities of students enrolled in different sections of a course, we hope to test the impact of interactive learning strategies on student understanding of the basic physics concepts in these two fundamental classes. To aid in the understanding of the systematic effects mentioned above, we are also asking each student to answer a series of questions about themselves (gender, academic background, etc.). Preliminary results from these two classes is very positive. Table 2 shows the results for the FCI, a mechanics assessment, shown by rigorous education research to provide a reliable measure of students’ learning of basic Newtonian mechanics (Hestenes et al. 1992 ; Hake 1998). The FCI was given to students in all sections of LP112 both before instruction (pre) and after instruction (post). Though the FCI is actually better suited to assess students’ learning in LP111, it was used in LP112 because of its reliability, and because there are some questions on the FCI well-matched to topics taught in LP112. The results of the FCI in Table 2, though preliminary, show quite clearly that the students in the interactive sections learned some basic concepts significantly better than those in the non-interative sections. The first two rows of the table (FCI – entire) show results for the entire 30-question FCI. The columns AvgPre and AvgPost show the average percentage of questions correctly answered (out
of 30) before and after instruction, and the column Gain shows the difference (Post-Pre). The students in the interactive classes showed an average gain of 7.65%, 2.6 times higher than the non-interactive classes which showed an average gain of 0.88 questions. The column shows the normalized gain ( = Post-Pre/30-Pre), a measure of the fraction of available knowledge the students acquired in the class. The normalized gain of 0.15 shown by the interactive classes is still lower than desirable but that may be partly due to the fact that the FCI focuses more on topics from LP111. Table 2. FCI results for LP112
Group
Npre
FCI
INTERACTIVE
104
74
50.2
57.9
7.7
0.15
Entire
NON-‐INTERACTIVE
237
186
49.8
52.8
2.9
0.06
FCI 4 questions
INTERACTIVE
96
71
56.5
71.1
14.6
0.34
NON-‐INTERACTIVE
56.7
58.3
1.6
0.04
Npost
222
AvgPre (%) AvgPost (%) Gain (%)
172
To test this latter hypothesis, we separately scored the four questions of the FCI most closely tied to what is taught in LP112, and these results are shown in the bottom two rows of the table (FCI – 4 questions). On these four questions, the interactive classes showed an average gain of 14.6% for a normalized gain of = 0.34, almost 10 times higher than the non-interactive classes ( = 0.036). These preliminary results suggest that interactive learning has significantly improved students’ learning of these basic concepts of Physics, relative to traditional teaching methods. Similar results are found for LP203/LP205/LE207, the second year classes in Electricity and Magnetism. Table 3 shows results of the Conceptual Survey of Electricity and Magnetism (CSEM; Maloney et al. 2001), given pre- and postinstruction to students in these classes. There were a total of 4 Amphi in these three classes: one was highly interactive (a large number of TPS questions every meeting of the Cours magistral and tutorials used in one of the three TD); two classes were somewhat interactive (a few TPS questions used every Cours magistral with no tutorials in TD); and one class was non-interactive (traditional). The normalized gain in the highly interactive class was clearly the highest among the four Amphi, = 0.34, almost 2.5 times higher than the normalized gain in the non-interactive (traditional) class ( = 0.14). The somewhat interactive classes also had higher gains than the traditional class, confirming that interactive learning can have a positive impact on students’ learning of basic science concepts. Table 3. CSEM results for LP203/LP205/LE207
Class Highly interactive Somewhat interactive 1 Somewhat interactive 2 Non-‐interactive
Npre
Npost 49 97 63 93
42 87 47 88
AvgPre (%) AvgPost (%) Gain (%) 30.0 53.9 23.9 30.6 47.2 16.6 26.5 44.5 17.9 28.3 38.3 10.0
0.34 0.24 0.24 0.14
2) Instructor feedback Each instructor who has implemented IL in their classroom was asked to fill out a detailed questionnaire on their experiences. Questions included asking instructors whether they believe that IL has helped students learn the concepts more effectively that traditional teaching methods (lecture only), whether students are motivated to be more active and diligent in class as a result of the use of IL in the class, and what the instructor found effective or ineffective about IL strategies in their own experiences in their classroom. Though direct measures of students’ learning are the most important measure of success in the use of IL, the opinions of experienced instructors is another important measure of the success of any pedagogy, and is the most usual way that pedagogies are judged. Initial results from these surveys are quite positive. On a scale from 1-5, where 1 is “Not at all” and 5 is “A great deal”, the instructors selected an average score of 3.9 to indicate that IL improved student learning during the class, and selected an average score of 3.6 to indicate that IL helped motivate students to be active and diligent in class. 3) Student feedback To understand students’ reactions to IL in their classrooms, we are administering a student feedback survey in conjunction with the FCI and CSEM assessments. These surveys are being given to students in both traditional as well as interactive classrooms, allowing us to compare students evaluation of their class between the two types of instruction, as well as to ask the students in the interactive classes to compare their experiences with other, traditional classes they have taken. The questions in these surveys are designed to elicit students’ opinions about how IL (or traditional lecture) has helped them learn the concepts in the course; to what extent the two teaching methods help them develop interest in the material of the course; to what extent the two methods help them be diligent in class; as well as their overall opinion of the two teaching methods (traditional and IL). These results are being collected now and have not yet been evaluated. IV. The future A séminaire bilan was organized at the physics faculty on May 21, 2012. It brought together many of the instructors who used IL during the academic year, as well as others who had expressed some interest in this approach. In the first part of the session, about a dozen colleagues presented their own experiences in using IL. This was followed by a discussion of possible future developments of these activites, given the positive feedback from teachers and students to these first experiments. The short presentations showed that IL has been used in very diverse ways at UPMC. Some instructors have done a lot of work to adapt their course and used IL extensively while others have just made a few trials, e.g., as part of their use of experimental demonstrations in their classes. Most instructors have found that students are happy to actively participate in class: they answer questions, actively discuss together and become more attentive during the course. While one might have expected to encounter some difficulties in large classes to regain silence after discussions, classes considered “difficult” turned out to be more manageable, in general, than when taught in a traditional style. Generally, instructors find that the IL approach requires more time to cover the same
material, though experiences elsewhere suggest this problems improve with time. The impact of IL seems to be very positive in TD, where it is often very difficult to have students actively engaged using conventional teaching methods. Some instructors find IL convenient to review the content of the previous session at the beginning of a course or, on the other hand, to review the main concepts at the end of the semester, before the exam. Overall, IL appears to be a flexible method that allows instructors to track, in real time, what students have understood or not, a very powerful pedagogical tool in its own right. The general discussion of the séminaire bilan focused on how we should now proceed to continue and build on our generally very positive experience. Questions were raised concerning the interest expressed by students: will it decrease as IL is no longer new for them? The US example points to the opposite. Instructors agree that a lot of work is required to set up good questions/answers: the conclusion is that it is crucial to identify already existing material and to encourage collective work within the faculty. Concerning equipment issues, some wondered whether smartphones will be usable instead clickers within a few years, but this seems unrealistic. In any case, several instructors emphasized that “clickers” are just a tool and that what is really important is the “spirit” of the method: paying attention to what students understand or not, and creating the conditions for them to express their ideas, instruct each other, and become engaged in class. All the instructors at the séminaire bilan said they expect to continue using IL next year. As Brahim Lamine is moving to Toulouse, Tristan Briant has been designated to manage all the activities related to IL in general. In order to further develop the use of IL at UPMC, the following actions have been decided on: i) The L1 department, represented by Sami Mustapha at the séminaire bilan, accepted to fully equip the amphi A2 with clickers. This will require the purchase of approximately 150 clickers, and an estimated investment of 6 k€. ii) An explicit mention of the use of IL strategies will appear on some of the appel d'offre from the tableau de service. This is, for example, already the case for LP205 in the licence department. iii) Instruction needs to be put in place to help teachers use IL efficiently. Some of the instructors at UPMC probably now already have sufficient experience to provide such instruction given their own experience and their participation in the workshops given by Alexander Rudolph. Reflection is underway on the specific forms to be given to such instruction (workshops, seminars, co-teaching etc.). iv) Communication among the instructional staff in all faculties at UPMC is important in order to disseminate the positive results of experiments so far and thus encourage their use by others. For this purpose a short video presenting the use of IL at UPMC this year is currently being edited (with a release expected at end of June). Support from groups at UPMC above the level of individual departments is critical to this dissemination of information to other departments. Lastly it is to be noted that a major reorganisation of the undergraduate programs currently underway provides a potentially very positive framework for the development of pedagogical innovation such as IL.
References Bailey, J. M., and Slater, T. F. 2005, Resource Letter AER-1: Astronomy Education Research, American Journal of Physics, 73(8), 677. Bruff, D. 2009, Teaching with Classroom Response Systems: Creating Active Learning Environments, Jossey-Bass, San Francisco. Crouch, C. H. and Mazur, E. 2001. Peer Instruction: Ten Years of Experience and Results, American Journal of Physics, 69, 970. Hake, R. 1998, Interactive-Engagement versus Traditional Methods: A SixThousand-Student Survey of Mechanics Test Data for Introductory Physics Courses, American Journal of Physics, 66, 64. Hestenes, D., Wells, M., and Swackhamer, G. 1992, Force Concept Inventory, Physics Teacher, 30, 141. Maloney, D.P., O’Kuma, T.L., Hieggelke, C.J., and Van Heuvelen, A. 2001, Surveying students’ conceptual knowledge of electricity and magnetism, Am. J. Phys. Suppl., 69 (7), S12. Mazur, E. 1997, Peer Instruction: A User’s Manual, Prentice Hall, NJ McDermott, L. C., and Redish, E. F. 1999, Resource Letter on Physics Education Research, American Journal of Physics, 67, 755. McDermott, L. C., and Shaffer, P. S. 2002, Tutorials in Introductory Physics, Prentice Hall, Upper Saddle River, NJ National Research Council 1999, How People Learn, National Academy Press, Washington, DC Prather, E. E., Rudolph, A. L., and Brissenden, G. 2009, Teaching and Learning Astronomy in the 21st Century, Physics Today, 62(10), 41.
