Category: Systematic Review

Categories
Delivery and assessment of the curriculum Design, development and approval of programmes Enabling student development and achievement Political, social and economic drivers influencing educational policy and strategy. Quality assurance Student experience Systematic Review

Work-based learning in technical and vocational education

Photo by Science in HD on Unsplash

Blog Authors: Tony Adams, Fiona Balloch, David Cullen, Ian Hamilton, John McVeigh, Walter Patterson, Jan Robertson, Joe Wilson, Tracey Howe

City of Glasgow College provides over 2000 courses across a diverse range of technical, business, and professional curriculum areas. Our unique Industry Academy model channels our curriculum and staff expertise, along with external industry partner collaboration, to match the needs of students with the needs of employers. As a result, our students graduate with industry-relevant skills and highly valuable qualifications sought after by industry. We were therefore interested in a recently published systematic review of Application Of Work-Based Learning Model In Technical And Vocational Education (TVET).

Here’s what they did

The authors searched five databases: Scopus, PsycINFO, Springer, Google Scholar, and ScienceDirect using search terms “Work-Based Learning” “Conceptual Model of WBL” “WBL in TVET” and “Implementation of WBL in TVET”. They included and reviewed 16 research based articles published from 2000 to 2018.

Here’s what they found

  • The extent of the implementation of WBL in TVET in tertiary institutions including universities is low.
  • Emphasis was given to aspects such as Student Industrial Work Experience Scheme (SIWES) leaving other aspects unexploited. cooperative work experience, job shadowing, youth apprenticeship programme, internships etc.
  • Factors affecting the implementation of WBL in TVET included curriculum defects, poor policy framework, inadequate trained manpower to supervise the proper implementation and lack of WBL learning implementation framework in institution of learning.

The author/s concluded

WBL is beneficial to students and attainment of the goal of TVET, however, the several obstacles to the proper implementation of the WBL contradicts the effectiveness of the WBL in TVET ……… However, there is no high quality evidence with which to provide robust answers to questions about the effectiveness of WBL.

Our Journal Club’s views

Who are the authors of the paper and where do they work?The authors work at Department of Technical and Engineering Education, School of Education, Universiti Teknologi Malaysia, Johor Bahru and Department of Electrical and Electronic Technology Education, SOSE (Technical), Federal College of Education (Technical) Bichi, Kano, Nigeria.

What do we know about the journal? Education, Sustainability and Society (ESS) is a peer-reviewed, open access trans- and interdisciplinary e-journal. Volume 1 was published in 2018.

What about the methodology used? The authors state that this was a systematic review. However, there were no definitions of the population of interest other than ‘schools’ and ‘workplace’. They did not offer the definition of ‘work based learning’ that they used to include papers. Neither did they define any specific measurable outcomes they were interested in. It is unclear whether papers were independently reviewed and how disagreements were resolved.

The analysis section was weak giving no information on how themes were established or whether they were indeed defined a priori by the authors. No indication is given on whether quality appraisal took place. The findings are descriptive and lack detailed critique.

The discussion is a summary and is not reflective and does not draw upon the extensive wider literature. Instead it focuses on issues relating to the authors own contexts.

The authors could have benefiting from using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses

Our conclusions are – that this evidence has a high risk of bias.

Implications for our practice

This paper evoked much discussion around this topic. There is literature from North America and Europe and UNESC and the World Bank on this area, examples include [Joe to add].

In terms of our own practice we should be cognisant of National – and for World Skills – International occupational standards and we considered whether our staff are up to date and aware of the latest versions and integrate these into curriculum design, delivery and evaluation. This could be facilitated by the engagement of members from industry professional organisations at all phases in the curriculum cycle. Our Industry Academy model would provide an obvious platform for this.

We discussed the validation cycle for rapidly moving areas such as computing [other examples] which are currently 5 years. Should we be looking a faster cycles such as 18 months? This could be addressed within the refresh of our student experience strategy. Other organisations have considered graduate skills and given the dramatic change since the Covid-19 pandemic we also need to consider skills FOR and AT work.

Other areas discussed included the College’s role as a Civic Anchor and how we could benefit the wider community for example linking with Volunteer Scotland.

Next steps

  • Review the integration of new staff from industry into education
  • Review staff expertise and identify opportunities for updating skills
  • Redefine Industry Academies and their operationalisation
  • Deliver new opportunities for professional practice deriving from work placements –  building on staff internal capacities.
  • Anticipate demand for Active Blended Learning and Active  Distance Learning  programmes in preparation for and during work as part of an ongoing engagement with Industry and Commerce.
  • Finalise the ‘Appointment of visiting and honorary staff’ scheme
  • Engage externals in our governance process e.g. Faculty Boards
  • Engage externals in all aspects of the curriculum process
  • Address some of these issues within the refresh of our student experience strategy.

View from

What do you think?

References

Keywords: systematic review, technical, vocational, work-based, education, learning

Our Blog Posts are written by staff at City of Glasgow College to inform and inspire our practice. We meet together at the Journal Club to consider the latest evidence to provide insights on hot topics related to learning and teaching, quality assurance and subject needs. It forms part of our activity for General Teaching Council Scotland registration and Professional Standards for lecturers in Scotland’s Colleges demonstrating that we are a self-critical staff community.

Categories
Digital technologies to enhance learning and teaching and assessment. Enabling student development and achievement Online learning Student engagement in learning Student experience Systematic Review Technology and digital literacies

Supporting Self-Regulated Learning in Online Learning Environments and MOOCs: A Systematic Review

Blog Authors: Tracey Howe, Lisa Shields, Sarah Janette Robertson, Walter Patterson, John McVeigh, David Cullen, Kate Cotter, Joe Wilson.

Here’s what they did

Moocs (Massive Open Online Courses) enable learning to take place anytime and anywhere and have created more accessible educational opportunities for the ‘masses’. There are however discrepancies between enrolment and completion rates in Moocs, suggesting that learning online presents its own challenges and that learners may require support to succeed. Prior studies suggest that learners find this form of learning challenging as they do not effectively use self-regulated learning (SRL) strategies.  Research thus far demonstrates that the provision of SRL strategies is likely to result in greater online academic success. The authors of this systematic review considered the role of SRL in online academic success and the influence of human factors, (for example, motivation and experience) and aimed to investigate approaches to support SRL in online environments. Their goal being to use any insights gleaned to inform further research into the development of Moocs.

The authors reviewed empirical studies in Moocs and also included studies conducted in other online learning environments, using this as an ‘umbrella term’ in order to include all related learning taking place on the internet. They sought to investigate the current provision and type of SRL support provided in the online leaning environment.  Additionally, they examined the impact of human factors as identified and addressed in the studies selected. Their primary research question being to determine the effectiveness of approaches used to support SRL strategies in online learning environments, and whether these approaches take account of the role of human factors.

The authors followed the 5-step methodology of Khan, Kunz, Kleijnen and Antes (2003) which includes framing the question for the review, identification and assessment of relevant studies, summarising the evidence and interpreting the findings. Keywords were decided upon and the literature search was conducted using the databases; Scopus, Web of Science, ERIC over the timeframe of the last 10 years. Searches via Google Scholar supplemented this as did manual searches of reference lists.  A total of 398 articles were selected having browsed their titles and predefined inclusion criteria were applied to further screen these results. These were then fully examined separately and then discussed by 3 of the authors to determine the final selection of 35 studies to be included in the review.

Here’s what they found

The studies included in the review had been conducted over varying educational levels, and of the 35 reviewed, 23 focused on undergraduate level. These studies also covered a wide range of subjects, e.g. psychology chemistry, biology and medical sciences. Results were presented in 4 main sections, ‘Prompt’, ‘Feedback’, ‘Integrated support systems’ and ‘Other’ approaches; a further section examined the human factors investigated.

Prompt – Prompting appears to be an effective approach to support SRL strategies and academic success. However, the studies reviewed explored varying modes of prompting intervention and differences existed between the range of prompts, operationalisation and measurement.

Feedback – Only 2 studies considered feedback alone as an approach to support SRL activities, so the authors found it difficult to reach a conclusion as to its efficacy in this context. When feedback and prompting were combined, results were more promising, but the authors recommended that more studies should be undertaken to investigate the independent effect of feedback.

Integrated Support Systems – If students use the support systems made available to them, there are positive effects on their SRL strategies.  However, these tools and support are only effective if utitlised, and used appropriately.  The authors point out that human factors could however impact the uptake of these support systems, for instance if the student feels overwhelmed by the plethora of support offered to them.

Other – this section included the results relating to approaches that could not be categories in the aforementioned sections. These results were variable and not definitive in nature.

Human factors – 12 studies were examined, and the authors concluded that the effectiveness of an approach to support SRL and academic success is dependent on these human factors. Findings suggest that additional or differential support should be in place to assist learners with different levels of prior knowledge, cognitive and metacognitive ability. Approaches adapted to suit the different needs of learners will assist them to become better at regulating their own learning and thereby achieve greater academic success.

The author/s concluded

The authors concluded that it is important to encourage and assist learners to use the tools and strategies in place to support their SRL, as effective approaches cannot benefit the learner if they are not sufficiently and appropriately utitlised.  Adaptive support should be provided to meet the learners’ diverse learning needs and the impact of human factors should be acknowledged when considering which approaches to use to support SRL strategies.

Our Journal Club’s views

Who are the authors of the paper and where do they work?

There are 5 named authors, (some of whom are PhD students), whose background is split between educational psychology, focusing on motivational aspects, and technical contributors, whose background is Web Information systems. This instils us with a sense of confidence with the authors, as the topic of the study is within their particular area of interest.

What do we know about the journal?

This is a peer-reviewed journal; external experts reviewing articles prior to being accepted for publication. It has a 1.354 Impact Factor, which has increased by approximately 7% from last year. It is an established journal but did close for 2 years over recent times.

What about the methodology used?

The title provides enough interest to invite one to read further and the abstract is clear and informative about the paper’s subject matter and what the researchers set out to do. There is not a great deal of information available about this aspect in relation to Moocs, and although it is recognised that there is a high drop-out rate, how to maintain a student’s presence in a Mooc is less well investigated. Within the abstract, there would normally be more detail about the authors’ results and methodology along with some conclusions; these are missing somewhat, bringing the abstract to an abrupt ending. Detail within the abstract is important as many people will decide whether to read further, and sometimes purchase the article, depending on the abstract’s content.

When explaining their methodological process, the authors acknowledge the use of the 5 Step (Khan, Kunz, Kleijnen and Antes, 2003) process, which was instigated by experienced reviewers. However, the PRISMA reporting guidance postdates this process and could have been used to inform this systematic review. The researchers’ objectives and methodology were made clear which gave us sufficient information to believe that the study could be replicated. When searching the literature they used Google Scholar as an adjunct which was useful, as this enabled them to access papers not yet included in the chosen databases; they also reported that they stopped searching Google Scholar after 200 papers were sourced, as no new data became available.  They clearly stated the Inclusion Criteria used and the steps undertaken to conduct the review were fully explained and reported – thereby, their methodology was made evident. 

Within the results section, the authors could have taken a more analytical stance and ‘stood back’ in order to appraise their chosen papers more objectively and succinctly. The approach taken simply amassed the information from the chosen papers and presented it back in a summarised form rather than it being an analytical discourse of the findings. It is only when the reader reaches the end of the different sections, that there is a structured and organised overview of the results. It therefore appears that when considering the multiple studies they had sourced, they merely described the results of each of them, rather than ‘pooling’ these individual results thereby pulling them together. In addition, quality assessment of each individual paper was not evident, which resulted in each paper’s findings being given equal weighting, possibly erroneously. Within the general discussion, an attempt is made to bring the results back to the research questions – thereby relating them successfully to one another. Some time is spent in making recommendations for ‘going forward’ and the authors demonstrate an awareness of the limitations of their study. The conclusion is abrupt and does not fully reflect the findings comprehensively. It is almost as if a conclusion had not actually been reached. This research was supported financially by one of the institutions of the authors – thus internally funded, which is not unusual.

Our conclusions are – that this evidence has a moderate risk of bias. There is no evidence of quality assessment of every paper included in the review and thus each paper being credited with equal weighting when drawing conclusions.

Implications for our practice

This paper has lead us to consider the necessity to use integrated support systems for our online teaching and learning environment, taking note of the individual learning needs of our various students – maturity, prior learning, existing knowledge and skills and experience, level of study and so forth. The introduction of the support intervention of ‘combined prompt and feedback’ seems to be a sound path to follow. However, we realise that the impact of the ‘human face’, discussion and debate should not be underestimated. The flexibility of online learning has been reinforced and this is something we should consider due to the changing nature of the demography of our student population.

Next steps

It would be useful to ascertain the existing support for SRL learning within the College, and using that as a baseline, make plans to build this in. The use of a ‘Tool kit’ might be an advantageous asset, but developing the right support for SRL is imperative. The categories and headings used in the paper might be something we could use to build and develop our ‘toolkit’ and the planned roadshows with curricula teams would be a medium through which to disseminate this information to teaching and support staff. At the moment, supporting students learning and participation in Moocs is not a priority but supporting them to become self-directed students within the online environment is something to focus on in the planning for the next academic year.

View from

What do you think?

References

Wong J., Baars, M., Davis, D., Van Der Zee, T., Houben, G. and Paas, F. (2019). Supporting Self-Regulated Learning in Online Learning Environments and MOOCs: A Systematic Review. International Journal of Human–Computer Interaction,35 (4-5), 356–73.

Khan, K. S., Kunz, R., Kleijnen, J., & Antes, G. (2003). Five steps to conducting a systematic review. Journal of the Royal Society of Medicine, 96(3), 118–121.

Keywords: learning, environments, review, moocs, online, self-regulated, supporting, systematic.

Our Blog Posts are written by staff at City of Glasgow College to inform and inspire our practice. We meet together at the Journal Club to consider the latest evidence to provide insights on hot topics related to learning and teaching, quality assurance and subject needs. It forms part of our activity for General Teaching Council Scotland registration and Professional Standards for lecturers in Scotland’s Colleges demonstrating that we are a self-critical staff community.

Categories
Dementia Online learning Systematic Review Technology and digital literacies

E-learning as valuable caregivers’ support for people with dementia – A systematic review.

Blog Authors: Tracey Howe, John McVeigh, Lisa Shields, Walter Patterson, David Cullen, Sarah Jannette Robertson, Lynn Brown.

Here’s what they did

This is a systematic review of accessible peer reviewed papers retrieved from three reputable databases. The focus of the review was to study if eLearning could be an informal support tool for Informal Caregivers of people living with dementia. It aimed to identify both benefits and imitations of this tool.

Here’s what they found

• eLearning in its various forms helped Informal Caregivers feel more confident about dementia care.
eLearning:-
• enhanced their knowledge and skills
• relieved perceived stress
• enhanced feelings of empathy and understanding.

The author/s concluded

The use of eLearning as a support tool for Informal Caregivers may have some potential, but training in its use is required to enable Caregivers to fully utilise eLearning platforms.

Our Journal Club’s views

Who are the authors of the paper and where do they work?

All of the authors are reputable, (Associate Professors, Professor and a Department Head); between them they are responsible for 1500 publications and have been cited over 50,000 times. Our view is that this is a potentially trustworthy publication.

What do we know about the journal?

The journal of BMC Health Services Research has been in existence since 2001, it is in the top 25% of medical journals, has an impact factor of 1.932 and is Open Access. Our view is that this is a reputable journal.

What about the methodology used?

The authors performed a systematic literature review based on a focused aim which could perhaps have been articulated with more clarity. They chose relevant keywords to conduct their search but the concept of eLearning could have been widened with the use of alternative descriptive search terms. The search terms were therefore mostly relevant as were the inclusion and exclusion criteria.  The choice of only focusing on older adults with dementia was a little misleading as what was construed as an ‘older adult’ was not defined and dementia also affects younger adults. They also used the ‘backward search strategy. However, the results were presented in a rather descriptive, narrative manner rather than being analytical and there was no evidence that the quality of the literature sources used had been critically appraised. There was however recognition of results being consistent with other similar studies. The limitations of the paper include: only a small number of studies relating to the topic of enquiry; wider more encompassing search terms could have been used; only papers in English language were considered; there was a lack of detailed data from each of the studies reviewed; no defined outcome measures identified for the review.

Our conclusions are – that this study has a high risk of bias and even though a paper is published in a highly rated journal it may still contain some flaws.

Implications for our practice

We will consider the organisation and provision of distance, remote and eLearning from the College’s perspective.

Next steps

Review and reflect on the distance learning as provided by the College

Consider the scaffolding required to fully support online learning, both for students and staff.

Raise the profile and awareness of Dementia.

View from

What do you think?

References

Klimova, B., Valis, M., Kuca, K. et al. E-learning as valuable caregivers’ support for people with dementia – A systematic review. BMC Health Serv Res 19, 781 (2019). https://doi.org/10.1186/s12913-019-4641-9

Keywords: eLearning, Dementia, Caregivers, Support tool, benefits, limitations

Our Blog Posts are written by staff at City of Glasgow College to inform and inspire our practice. We meet together at the Journal Club to consider the latest evidence to provide insights on hot topics related to learning and teaching, quality assurance and subject needs. It forms part of our activity for General Teaching Council Scotland registration and Professional Standards for lecturers in Scotland’s Colleges demonstrating that we are a self-critical staff community.

css.php