Categories
Delivery and assessment of the curriculum Digital technologies to enhance learning and teaching and assessment. Enabling student development and achievement Online learning Student engagement in learning Student experience Theories of learning, teaching and assessment

Learning and Satisfaction in Webinar, Online, and Face-to-Face Instruction: A Meta-Analysis

Blog Authors: Tracey Howe, Anthony Adams, Lisa Shields, Sarah Janette Robertson, Walter Patterson, John McVeigh, David Cullen, Kate Cotter, Joe Wilson.

Here’s what they did

Since the middle of the 1990s, there has been considerable increase in eLearning resources and educational technologies within higher education and professional training contexts. One such method being the Webinar with the advantages it confers in terms of flexibility. It negates the need for a ‘classroom space’ and means students can learn from their own homes, or from other geographically suitable, convenient places. The authors sought to investigate, using meta-analysis, the effectiveness of Webinars in the promotion of online learning as compared to traditional classroom based, ‘face to face’ teaching and with online asynchronous online learning systems. In addition they aimed to test Kirkpatrick’s four-level training evaluation model which assumes that a positive correlation exists between student satisfaction and learning. They wanted to ascertain the levels of satisfaction and learning with respect to Webinars, ‘face to face teaching’ and asynchronous online learning systems. The predictive validity of a positive association between satisfaction levels and learning had not thus far been established.

A meta-analysis of randomised controlled trials was performed for which a 2 step literature search was conducted based on pre-defined inclusion and exclusion criteria. An electronic search was undertaken of 4 databases, ERIC, PsycINFO, PubMed and Scopus using relevant keywords, followed by screening and removal of duplicates. 403 paper were excluded as they either reported qualitative research, were review papers or did not fully focus on the required topics under scrutiny. Both authors read the remaining 51 to ensure eligibility, and from these 2 papers were selected. The second step involved cross referencing using a backward and forward literature searching process for potentially eligible papers – a further 3 were selected, thus a total of 5 were papers were included in the meta-analysis. These were coded independently and in-duplicate using statistical testing to ensure interrater reliability. Various inferential statistical tests were conducted to complete the meta-analysis and address the research questions.

Here’s what they found

With regard to student learning, Webinars were found to be more effective in promoting knowledge than traditional, ‘face to face’ teaching and asynchronous online learning. However, the researchers point out that the difference between Webinars and the other groups were minimal and statistically insignificant, thus leading to the assumption that all 3 modalities tend to be equally effective for student learning.

In terms of student satisfaction, it seems that Webinars are inferior to ‘face to face’ teaching but produce higher satisfaction than asynchronous online instruction. However, again, differences were negligible in size; so it can be assumed that satisfaction is similar in all 3 modalities. Results of correlation analysis to determine the association between student satisfaction and student knowledge showed negative relationships between these 2 variables in all learning modalities. Therefore, the researchers found that Kirkpatrick’s predicted positive causal link between satisfaction and learning could not be confirmed.

The author/s concluded

The researchers concluded that the results of this meta-analysis provide insight and indications as to the practical application of e-leaning modalities in higher education and professorial learning contexts. All 3 modalities were roughly equal in terms of the outcomes of learning and satisfaction – therefore the use of one of them may well be justified without concern for major negative consequences. As traditional ‘face to face’ instruction seems to be slightly superior to online learning environments generally, if there is no need for flexibility (time or location), ‘face to face’ classroom education seems to be an appropriate learning environment for higher education and professional training contexts. But, if flexibility is required, Webinars can be used as an alternative with only slightly reduced student satisfaction. Asynchronous learning environments also offer a viable alternative, for example if students are in different time zones. ELearning modalities generally and Webinars in particular, are useful tools for extending the traditional leaning environment and creating a more flexible environment for both students and tutors.

Our Journal Club’s views

Who are the authors of the paper and where do they work? For the primary author, Christian Ebner, this was his first published article; perhaps as a result of PhD studies or an ‘early researcher’ article. Andreas Gegenfurtner has 39 publications, the focus of these being ‘knowledge transfer’ and ‘learning through technology’.

What do we know about the journal? This is a academic, peer reviewed journal, the 5th most cited publication and readily available via Open Access.

What about the methodology used? The title of this paper is self-explanatory, encouraging the reader to engage, and the abstract is concise but informative. The 2 stated research questions are clear and focused. The choice of meta-analysis, allowing the pooling of data from primary research studies is a suitable methodological choice, enabling the researchers to explore and investigate a wider data set. This methodological approach being at the top of ‘the food chain’ in terms of evidence hierarchy. PRISMA guidelines were acknowledged and followed in the execution of this systematic review of the literature. Identification of relevant inclusion and exclusion criteria was evident as was a comprehensive search of the literature, including retrospective searching. The authors ensured that all methodological detail was presented in order for their study to be reproduced, to obtain the same results. Throughout the paper, decision-making has been documented, making this cognitive process transparent to the reader. The inclusion of the PRISMA statement provided a concise but illuminative overview and the chosen coding scheme was also clearly outlined. Data extraction was fully aligned to research questions. Interrater reliability was employed in an attempt to reduce bias and minimise subjectivity, which revealed a high level of consistency between raters. Statistical calculations were made available, as were the Forest plots (allowing the results of the studies to be combined), showing results which were then faithfully presented in textual form. The researchers also acknowledged the limitations of their study and the project was funded by an Educational establishment.

Our conclusions are – that this evidence has a low risk of bias.

Implications for our practice

The trustworthy evidence from this paper could help with decision making about our educational programmes when planning for the Academic year of 2021-22. This could inform decisions about which level of students receive which modality of learning, ‘face to face’, Webinar or Asynchronous online systems. Having a relevant evidence base to inform College policy will be advantageous. The motivation of different levels of student could also be considered when reaching this decision.

This paper offers direction as to the use of synchronous Webinars, enabling immediate responses and spontaneity of feedback. These also confer benefits in terms of connectively and belonginess on the part of the student. However, there could be a potential ‘burden’ placed on lecturers when preparing and participating in these live events, and perhaps there needs to be guidelines provided as to how much time from the timetable in devoted to an online synchronous presence.

There needs to be recognition of the ‘digital divide’ on the part of students, in terms of potential limited internet connectively and not having a suitable environment from which to engage in online learning.

Next steps

To consider the other papers within this series – one Journal Club member to review, read and disseminate relevant information, perhaps via Webinar.

The paper, Olson, J. S., and McCracken, F. E. (2015). Is it worth the effort? The impact of incorporating synchronous lectures into an online course. Online Learn. J. 19, 73–84. doi: 10.24059/olj.v19i2.499 is recommended for Journal club members to read.

To make full use of the support and guidance to the Learning and Teaching Academy provided by the College.

When we return, we may well experience a very different College, and knowing that online learning ‘ works’, confers a greater flexibility when planning and executing teaching and learning.

View from

What do you think?

References

Ebner, C and Gegenfurtner, A. (2019) Learning and Satisfaction in Webinar, Online, and Face-to-Face Instruction: A Meta-Analysis. Front. Educ. 4:92. doi: 10.3389/feduc.2019.00092

Keywords: Adult learning, computer-mediated communication, distance education and telelearning, distributed learning environments, media in education.

Our Blog Posts are written by staff at City of Glasgow College to inform and inspire our practice. We meet together at the Journal Club to consider the latest evidence to provide insights on hot topics related to learning and teaching, quality assurance and subject needs. It forms part of our activity for General Teaching Council Scotland registration and Professional Standards for lecturers in Scotland’s Colleges demonstrating that we are a self-critical staff community.

css.php