2.1

Course structure

The course on which we perform this study was a first year undergraduate Physics course which was taught over a 13-week semester. The course was split into two modules, namely Optics and Electromagnetism. In this study, only the Optics part of the course was flipped. The Optics module consisted of 4 weeks of lectures (12 allocated lecture hours), and covered standard materials involving geometric optics and wave optics.

The students were split into a flipped class and a control class which was taught traditionally. The two classes were taught by different instructors as the two classes were arranged to take place simultaneously by senior management. Both instructors were experienced lecturers and had taught this course or similar materials for many years, namely 12 years for the flipped class instructor and over 20 years for the control class instructor. The two classes were taught in English.

The flipping of the flipped class was conducted in a conventional flipped classroom style, where students were asked to learn the course content for the particular lecture in advance using the interactive videos developed for this study (out-of-class activity), while interactive in-class activities, such as clicker questions and student discussions, were conducted during the lecture hours. The details of the in-class and out-of-class activities are described in Sec. 2.4 and 2.5. The control class was taught traditionally where no preparation was required for the students before a class, and the course contents were covered by the lecturer using lecture slides. Tutorials were also run during some of the lecture hours in the control class where the instructor went through example questions with students in a step-by-step manner in the conventional teaching style.

A pre-test and post-test were administered in both classes before and after the optics module to assess students’ improvement in conceptual knowledge and its relationship with the exam score. More details are included in Sec. 2.3.

An in-class exam on Optics was conducted immediately after the Optics module ended. This exam involved 15 multiple choice questions and 2 open questions. Most of the questions in the exam were numerical, with some requiring short written explanations. Students also had access to previous years’ exams on which they could practice.

2.2

Participants

There were about 80 students in the flipped and control classes respectively. About 90% of the students in the two classes were Chinese by ethnicity. Due to absence of students, not all students in the two classes completed both the pre-test and post test. In this study, we only analyse statistically the results from students who completed both tests.

There were N_flpd = 40 and N_ctrl = 47 students who completed both the pre-test and post-test in the flipped and control class, respectively. The results from students who were late to the tests were considered invalid and were not included in this analysis. Out of the students who completed the pre-test and post-test, 39 and 46 students attended the exam at the end of the module in the flipped and control class, respectively. Chinese students made up 93% and 96% of these participants in the flipped and control class, respectively.

2.3

Pre-test and Post-test

Similar to the Force Concept Inventory (FCI)²² and Force and Motion Conceptual Evaluation (FMCE)²³ which are well known among the Physics Education Research (PER) community, an Optics equivalent of such evaluations named The Light and Optics Conceptual Evaluation (LOCE) was used as both the pre-test and the post-test for both classes. The LOCE was developed as part of the Active Learning in Optics and Photonics Program (ALOP) initiated by UNESCO in 2003. The LOCE was further refined over the years and the one used in this study was the 2007 version by D. Sokoloff.²⁴

The LOCE consists of 50 multiple choice questions and 1 diagram drawing question. All questions were concept-based and minimal calculations were needed. The pre-test and post-test were administered during the same lectures for the flipped and control class. The pre-test was administered in the first lecture and the post-test was conducted after the in-class exam. Students were NOT advised about the two tests, hence not all students participated in both tests.

2.4

Out of class activities: Interactive videos

Twenty-three interactive videos were developed for this study.²⁵ The videos took the form of a recorded Microsoft Powerpoint presentation converted into mp4 format, with inserted interactive questions using the H5P “interactive video” content type (https://h5p.org). The cover screen of one of the videos is shown in Fig. 1a as an example. The interactive questions and notes were denoted by purple or blue dots on the play bars. These questions popped up at a set time within a video and students were required to make an attempt before moving on; an example question is shown in Fig. 1b. Retries and answers were available after one attempt. These videos were embedded in the school’s learning management system (Blackboard Learn) for student access. Three types of interactive questions were used to give students variation in the questioning style, including multiple choice, single choice set and true/false question.

Figure 1.

(a) The cover screen of a pre-class video with interactive questions and notes embedded, as denoted by the purple and blue dots in the play bar. (b) A “single choice” interactive question that pops up in the video with the correct answer selected.

Each video included a standalone topic or sub-topic such that students could obtain a complete picture of the content delivered. Therefore, depending on the complexity of the topic content, the lengths of the different videos vary between 3 to 16 minutes.

Students in the flipped-class were asked to watch between 5-8 videos per week before attending the lectures while these videos were available to the control class as supplementary materials. Identical traditional lecture slides were also available to both classes as supplementary materials.

2.5

In-class activities

During the lecture hours of the flipped class, students participated in several interactive activities, including clicker questions, tutorial type long question and students discussions. These interactive activities were run based on the peer instruction framework proposed by Mazur.⁸

Clicker questions were administered using an in-house developed student response system called uReply. Realtime online polls were run for conceptual questions using uReply where students could vote for their answers using either their hand-held mobile devices or laptops. The responses of the whole class were then presented as bar graphs, so students could see their peers’ responses while remaining anonymous. Students were encouraged to discuss the questions with each other and convince each other of their own answers when strongly divided opinions were observed as a class. Students were often asked to explain their choice of answer to the class after the polls, a tablet connected to the projector was also provided for students to draw diagrams to explain their answers.

Tutorial type questions which required more calculations and a step-by-step solution were also posed to the students. Students were encouraged to work in groups and given slightly longer time to solve the problems and provide a numerical answer at the end. For this type of questions, the lecturer would ask different students to suggest a general direction in solving the problem, then go through the solution with the class. The tutorial type questions were provided in order to help students with the homework assignments and exams which consisted of similar type questions.

2.6

Student survey

A student survey was conducted at the conclusion of the optics module, after the post-test. The survey consisted of 24 Likert scale questions from 1 (strongly disagree) to 5 (strongly agree) and 5 open questions. These questions were designed to grasp an idea of students’ opinion on the flipped-classroom teaching style and the use of interactive videos instead of conventional educational videos.

The questions in the survey are provided at the end of this article in Appendix A. There were 81 valid surveys received. The surveys of the students who have shown not to have attended any flipped lectures despite being on the roll were discarded.

2.7

Result analysis

The scores of the pre-tests, post-tests and in-class exams were analysed using IBM SPSS Statistics 23. The comparison between the control and the flipped class was conducted using the independent sample t-test, whereas the performance of the same group of students in different tests was analysed using the paired sample t-test.

3.1

Tests and exam results

The statistics (mean with standard deviation in parentheses) of the pre-test, post-test, score difference, normalized gain and exam score for the flipped and control class is presented in Table 1.

Table 1.

Statistics of the flipped and control class.

All of these scores were comparable statistically between the two classes as analysed by Levene’s test. In this table, all scores are presented as percentages of the respective full score. The score difference is calculated as

The normalized gain measures the percentage increase of the students’ scores between the pre-test and post-test with reference to their possible improvement range, and is calculated as

The difference in scores of the pre-test (t(85) = 1.26, p = .212), post-test (t(85) = –1.94, p = .056) and exam (t(83) = 0.50, p = .622) between the two classes were statistically insignificant as calculated using the independent sample t-test, meaning that the two classes performed similarly in all tests and exam.

The score differences between the pre-test and post-test for the flipped class (t(39) = –9.04, p < .001) and the control class (t(46) = –4.25, p < .001) were both statistically significant, meaning that the students did improve in their conceptual understanding of optics after the course in both classes. This was of course an expected result since students should have gained new knowledge and learnt new concepts after taking 4 weeks of lectures.

The differences in the score difference (t(85) = –3.18, p = .002) and normalized gain (t(85) = –3.74, p < .001) between the two classes were statistically significant, meaning that the flipped class improved significantly more than the control class. This suggests that despite a significant improvement in both classes, flipped classroom teaching was more effective in improving students’ conceptual knowledge than traditional lecturing. This could be explained by both the active learning segment of the in-class activities in flipped classroom and the interactive questions in the pre-class videos. Active learning has been shown to lead to more comprehensive understanding^{26, 27} while interactive questions allowed students to pick up misconception and misunderstanding in a more memorable way while learning the material for the first time.^{28, 29} Moreover, the similar exam results suggest that the concept-focused lectures in flipped classroom did not negatively affect students’ exam performance; this was a concern of the course instructor and some students since the exam questions were typically more numerical than conceptual.

The correlation between the pre-test score and normalized gain, as well as that between the post-test and exam scores can provide insights into the two different teaching styles. The pre-test score of each student in the flipped class is plotted against their respective normalized gain in Fig. 2a as a scatter plot. As can be seen in this plot, the data points do not show clear correlation between the two variables since the pre-test scores scatter quite uniformly across the positive region of the normalized gain. Indeed, our statistical analysis showed a lack of correlation between them with r = 0.12, p = .465. Since p > 0.05, the correlation between the pre-test score and the normalized gain of the flipped class was statistically insignificant. A similar plot is shown in Fig. 2b for the control class. A strong negative correlation (r = –0.55, p < .001) was found and is indicated by the line of best fit in the plot.

Figure 2.

Scatter plots of the pre-test score against normalized gain of the (a) flipped class and (b) control class. The correlation between the pre-test score and normalized gain for the flipped class is statistically insignificant, whereas that of the control class shows a strong negative correlation (r = –0.55, p < .001). The grey dashed lines denote the point of zero normalized gain for the two plots. The red straight slope in (b) is the line of best fit for the data plotted, showing the correlation between the two variables visually.

The lack of correlation in the flipped classroom case indicated that the level of prior knowledge or understanding of students in Optics did not play a significant role in predicting the students’ improvement with the new teaching style. Moreover, and importantly, since almost all students achieved a positive normalized gain, it suggested that this teaching style allowed for non-discriminatory improvement for students of different initial academic levels, which could be explained by the active learning nature of flipped classroom. We note here that the negative normalized gain obtained by a few of the students could be due to random guessing of answers by the individual students.

On the other hand, the strong negative correlation for the control class suggested that the better the initial conceptual understanding the student had, the less he/she improved under traditional teaching. While this was not necessarily a bad trend, the peculiar observation of retrogression of numerous high achieving students (pretest score > 55%) in the control class was alarming, as shown by the data points below the zero normalized gain dashed line. It implied that students who had good initial understanding of the course material became confused and misunderstood some of the concepts during the traditionally taught lectures. The general negative trend might suggest the loss of motivation of students. For example and as a speculation, the pace of the traditional lectures is usually fixed, which could lead to boredom or frustration in students with greater and lower ability, respectively. This would in turn affect students’ attitude towards the subject and motivation to learn during the course.

Figure 3 shows the correlations between the post-test and exam scores of the two classes. By looking at the correlation between the post-test and exam scores, we can see how the students’ conceptual knowledge related to their numerical problem-solving skills, and whether their exam results were a good indicator for their conceptual understanding of the course materials. Figure 3a shows that there was a very strong positive correlation (r = 0.72, p < .001) between the post-test and the exam scores of the flipped class. On the other hand, the control class shows a moderate positive correlation between the two variables (r = 0.37, p = .013), as can be seen in the more scattered data points in Fig. 3b. This meant that the higher the post-test score, the higher the exam score of the students in both classes. This implied that the better the conceptual understanding a student had, the better he/she was in tackling exam type questions, and vice versa.

Figure 3.

Scatter plots of the post-test score against the exam of the (a) flipped class and (b) control class. A strong positive correlation between the post-test and the exam score for the flipped class is observed (r = 0.72, p < .001), while a moderate correlation is observed in the control class (r = 0.37, p = .013). The red straight slopes are the lines of best fit for the data plotted, showing the correlation between the two variables visually. The grey shaded areas show the spread of students’ post-test scores for students with an exam score over 70%.

We should note that this correlation was much stronger in the flipped class than in the control class, suggesting that the exam scores were a better indicator for students’ level of understanding in the underlying scientific principles in the flipped class than in the traditionally taught class. It is imperative that exam results are able to reflect the students’ understanding of the taught material since exam scores are the most important measure in the current education system. Students’ exam scores are used by lecturers to judge the effectiveness of teaching as well as to identify students who are struggling with the content of the course. Naturally, we would expect students with good fundamental understanding of the subject to perform well in exams. However, our results suggested that this was not necessarily the case. For example, the students who achieved an exam score of over 70% in the control class possessed a post-test score range from just under 50% to above 80% (grey shaded area in Fig. 3b). Whereas, the flipped class followed closely the expected positive trend with a smaller post-test score range of 70% – 90%, which was also on the high end of the score spectrum (grey shaded area in Fig. 3a). While the reason leading to this difference between the flipped and control class was unclear, these results suggested that students struggling with the basic concepts or understanding of the course material could not be identified easily with the exam results. Hence, flipped classroom style teaching allowed for the exam scores to be a better indicator of students’ actual understanding of the course content, leading to more timely support for the students requiring extra help.

3.2

Survey results

Out of the 24 Likert scale questions [from 1 (strongly disagree) to 5 (strongly agree)] in the student survey, most of the responses were quite neutral (2.7 ≤ M ≤ 3.3). There were five statements on which students had stronger opinions (M < 2.7 or M > 3.3) that were worth noting. These statements and their means in parentheses are listed below:

Statements 1 and 2 showed that students were happy with flipped classroom due to the flexibility it provided. For example, students were able to watch the pre-class videos at a time of the day that was most comfortable to and convenient for them, as well as pausing and re-watching the videos as needed. This flexibility allowed student to learn and take notes at their own pace at a time and location that they could focus best, hence providing a better learning environment as well as promoting self-regulated learning for all students.

Statements 3 - 5 suggested that the pre-class videos aid the students’ learning and understanding of the course material. More importantly, the students valued the interactive questions in the videos, which are not a typical inclusion in flipped classroom teaching. In addition to statement 5, a related question was posed in one of the open questions to understand students’ preference or attitude towards the interactive component of the pre-class lecture videos. Students were asked to select the resources they would like to have available to help them prepare for the in-class activities if the flipped classroom learning style were to be continued. The students could choose out of the options of (with multiple selections allowed):

The selections of the 80 students surveyed are summarized in Fig 4. Most students (70%) selected options including lecture videos (red, blue and cyan sectors), while 67% preferred lecture videos with interactive questions (red and blue sectors). Just over a quarter of the students preferred static resources ((c) and (d) in the list above) only (magenta sector). This observation was consistent with the results obtained from the Likert scale questions discussed above. The combination of these results from the survey suggested that a majority of the students not only preferred interactive videos as pre-class activities, they also recognized how the use of interactive videos had assisted their learning.

Figure 4.

A majority of students prefers the use of interactive videos as pre-class preparation materials. 21% (red) of students prefer lecture videos with interactive questions only. 46% (blue) prefer lecture videos with interactive questions in addition to static resources, such as traditional lecture slides. 3% prefers static resources with videos that contain no interactive components. 28% prefer static resources only (i.e. no videos at all).

In the open question section of the survey, mixed opinions were seen from the students’ comments. While most students provided constructive feedback specific to the way we ran flipped classroom, several general comments on the flipped classroom teaching style were also received. Some negative feedback showed that students disliked changes and speaking up in class, or were exam focused rather than learning focused, such as “I prefer the traditional way because I have been taught in this way for many years,” “[I’m] nervous, because [the] teacher may pick students to answer questions. I feel shy and ashamed if I [am] picked and [give] wrong answers,” “theory questions are not asked in exam…” and “[name of instructor] is not teaching.”

Positive feedback tended to talk about how flipped classroom had helped with their learning, such as “[flipped classroom] strengthen my self-learning ability,” “flipped classroom is a good experience and the interactive videos [are] very helpful.[We] can [replay] again and again since [it’s] hard to understand some [parts],” “[I] just hope that this kind of teaching can be continued,” and “I think it’s an unique and novel experience for me.”

The survey results show that continuous improvement on the flipped classroom teaching style will be needed. In particular, the negative feedback demonstrated quite noticeably the stereotypical Asian students’ passive learning mentality^12–14 which hinders their acceptance of flipped classroom. As a future task, the in-class activities will need to be carefully constructed to balance between the need for active learning and students’ reservation in activities requiring students to speak or hold discussion in class.