Blog Authors: Tracey Howe, Lisa Shields, Sarah Janette Robertson, Walter Patterson, John McVeigh, David Cullen, Kate Cotter, Joe Wilson.
Here’s what they did
Moocs (Massive Open Online Courses) enable learning to take place anytime and anywhere and have created more accessible educational opportunities for the ‘masses’. There are however discrepancies between enrolment and completion rates in Moocs, suggesting that learning online presents its own challenges and that learners may require support to succeed. Prior studies suggest that learners find this form of learning challenging as they do not effectively use self-regulated learning (SRL) strategies. Research thus far demonstrates that the provision of SRL strategies is likely to result in greater online academic success. The authors of this systematic review considered the role of SRL in online academic success and the influence of human factors, (for example, motivation and experience) and aimed to investigate approaches to support SRL in online environments. Their goal being to use any insights gleaned to inform further research into the development of Moocs.
The authors reviewed empirical studies in Moocs and also included studies conducted in other online learning environments, using this as an ‘umbrella term’ in order to include all related learning taking place on the internet. They sought to investigate the current provision and type of SRL support provided in the online leaning environment. Additionally, they examined the impact of human factors as identified and addressed in the studies selected. Their primary research question being to determine the effectiveness of approaches used to support SRL strategies in online learning environments, and whether these approaches take account of the role of human factors.
The authors followed the 5-step methodology of Khan, Kunz, Kleijnen and Antes (2003) which includes framing the question for the review, identification and assessment of relevant studies, summarising the evidence and interpreting the findings. Keywords were decided upon and the literature search was conducted using the databases; Scopus, Web of Science, ERIC over the timeframe of the last 10 years. Searches via Google Scholar supplemented this as did manual searches of reference lists. A total of 398 articles were selected having browsed their titles and predefined inclusion criteria were applied to further screen these results. These were then fully examined separately and then discussed by 3 of the authors to determine the final selection of 35 studies to be included in the review.
Here’s what they found
The studies included in the review had been conducted over varying educational levels, and of the 35 reviewed, 23 focused on undergraduate level. These studies also covered a wide range of subjects, e.g. psychology chemistry, biology and medical sciences. Results were presented in 4 main sections, ‘Prompt’, ‘Feedback’, ‘Integrated support systems’ and ‘Other’ approaches; a further section examined the human factors investigated.
Prompt – Prompting appears to be an effective approach to support SRL strategies and academic success. However, the studies reviewed explored varying modes of prompting intervention and differences existed between the range of prompts, operationalisation and measurement.
Feedback – Only 2 studies considered feedback alone as an approach to support SRL activities, so the authors found it difficult to reach a conclusion as to its efficacy in this context. When feedback and prompting were combined, results were more promising, but the authors recommended that more studies should be undertaken to investigate the independent effect of feedback.
Integrated Support Systems – If students use the support systems made available to them, there are positive effects on their SRL strategies. However, these tools and support are only effective if utitlised, and used appropriately. The authors point out that human factors could however impact the uptake of these support systems, for instance if the student feels overwhelmed by the plethora of support offered to them.
Other – this section included the results relating to approaches that could not be categories in the aforementioned sections. These results were variable and not definitive in nature.
Human factors – 12 studies were examined, and the authors concluded that the effectiveness of an approach to support SRL and academic success is dependent on these human factors. Findings suggest that additional or differential support should be in place to assist learners with different levels of prior knowledge, cognitive and metacognitive ability. Approaches adapted to suit the different needs of learners will assist them to become better at regulating their own learning and thereby achieve greater academic success.
The author/s concluded
The authors concluded that it is important to encourage and assist learners to use the tools and strategies in place to support their SRL, as effective approaches cannot benefit the learner if they are not sufficiently and appropriately utitlised. Adaptive support should be provided to meet the learners’ diverse learning needs and the impact of human factors should be acknowledged when considering which approaches to use to support SRL strategies.
Our Journal Club’s views
Who are the authors of the paper and where do they work?
There are 5 named authors, (some of whom are PhD students), whose background is split between educational psychology, focusing on motivational aspects, and technical contributors, whose background is Web Information systems. This instils us with a sense of confidence with the authors, as the topic of the study is within their particular area of interest.
What do we know about the journal?
This is a peer-reviewed journal; external experts reviewing articles prior to being accepted for publication. It has a 1.354 Impact Factor, which has increased by approximately 7% from last year. It is an established journal but did close for 2 years over recent times.
What about the methodology used?
The title provides enough interest to invite one to read further and the abstract is clear and informative about the paper’s subject matter and what the researchers set out to do. There is not a great deal of information available about this aspect in relation to Moocs, and although it is recognised that there is a high drop-out rate, how to maintain a student’s presence in a Mooc is less well investigated. Within the abstract, there would normally be more detail about the authors’ results and methodology along with some conclusions; these are missing somewhat, bringing the abstract to an abrupt ending. Detail within the abstract is important as many people will decide whether to read further, and sometimes purchase the article, depending on the abstract’s content.
When explaining their methodological process, the authors acknowledge the use of the 5 Step (Khan, Kunz, Kleijnen and Antes, 2003) process, which was instigated by experienced reviewers. However, the PRISMA reporting guidance postdates this process and could have been used to inform this systematic review. The researchers’ objectives and methodology were made clear which gave us sufficient information to believe that the study could be replicated. When searching the literature they used Google Scholar as an adjunct which was useful, as this enabled them to access papers not yet included in the chosen databases; they also reported that they stopped searching Google Scholar after 200 papers were sourced, as no new data became available. They clearly stated the Inclusion Criteria used and the steps undertaken to conduct the review were fully explained and reported – thereby, their methodology was made evident.
Within the results section, the authors could have taken a more analytical stance and ‘stood back’ in order to appraise their chosen papers more objectively and succinctly. The approach taken simply amassed the information from the chosen papers and presented it back in a summarised form rather than it being an analytical discourse of the findings. It is only when the reader reaches the end of the different sections, that there is a structured and organised overview of the results. It therefore appears that when considering the multiple studies they had sourced, they merely described the results of each of them, rather than ‘pooling’ these individual results thereby pulling them together. In addition, quality assessment of each individual paper was not evident, which resulted in each paper’s findings being given equal weighting, possibly erroneously. Within the general discussion, an attempt is made to bring the results back to the research questions – thereby relating them successfully to one another. Some time is spent in making recommendations for ‘going forward’ and the authors demonstrate an awareness of the limitations of their study. The conclusion is abrupt and does not fully reflect the findings comprehensively. It is almost as if a conclusion had not actually been reached. This research was supported financially by one of the institutions of the authors – thus internally funded, which is not unusual.
Our conclusions are – that this evidence has a moderate risk of bias. There is no evidence of quality assessment of every paper included in the review and thus each paper being credited with equal weighting when drawing conclusions.
Implications for our practice
This paper has lead us to consider the necessity to use integrated support systems for our online teaching and learning environment, taking note of the individual learning needs of our various students – maturity, prior learning, existing knowledge and skills and experience, level of study and so forth. The introduction of the support intervention of ‘combined prompt and feedback’ seems to be a sound path to follow. However, we realise that the impact of the ‘human face’, discussion and debate should not be underestimated. The flexibility of online learning has been reinforced and this is something we should consider due to the changing nature of the demography of our student population.
Next steps
It would be useful to ascertain the existing support for SRL learning within the College, and using that as a baseline, make plans to build this in. The use of a ‘Tool kit’ might be an advantageous asset, but developing the right support for SRL is imperative. The categories and headings used in the paper might be something we could use to build and develop our ‘toolkit’ and the planned roadshows with curricula teams would be a medium through which to disseminate this information to teaching and support staff. At the moment, supporting students learning and participation in Moocs is not a priority but supporting them to become self-directed students within the online environment is something to focus on in the planning for the next academic year.
View from
What do you think?
References
Wong J., Baars, M., Davis, D., Van Der Zee, T., Houben, G. and Paas, F. (2019). Supporting Self-Regulated Learning in Online Learning Environments and MOOCs: A Systematic Review. International Journal of Human–Computer Interaction,35 (4-5), 356–73.
Khan, K. S., Kunz, R., Kleijnen, J., & Antes, G. (2003). Five steps to conducting a systematic review. Journal of the Royal Society of Medicine, 96(3), 118–121.
Keywords: learning, environments, review, moocs, online, self-regulated, supporting, systematic.
Our Blog Posts are written by staff at City of Glasgow College to inform and inspire our practice. We meet together at the Journal Club to consider the latest evidence to provide insights on hot topics related to learning and teaching, quality assurance and subject needs. It forms part of our activity for General Teaching Council Scotland registration and Professional Standards for lecturers in Scotland’s Colleges demonstrating that we are a self-critical staff community.
