performance management a systematic literature review

Open access
Published: 21 February 2020

Performance management methods and practices among nurses in primary health care settings: a systematic scoping review protocol

Cynthia Zandile Madlabana ORCID: orcid.org/0000-0003-0187-4263 1 ,
Tivani Phosa Mashamba-Thompson 2 &
Inge Petersen 1

Systematic Reviews volume 9 , Article number: 40 ( 2020 ) Cite this article

31k Accesses

12 Citations

6 Altmetric

Metrics details

Nurses make up the largest constituent of the health workforce. The success of health care interventions depends on nurses’ ability and willingness to provide quality health care services. A well-implemented performance management (PM) system can be a valuable asset in ensuring that nurses are motivated, promoted, trained and rewarded appropriately. Despite the significant benefits of effective PM such as improved motivation, job satisfaction and morale, PM systems are highly contested. Therefore, it is important to examine evidence on PM methods and practices in order to understand its consequences among nursing professionals in primary health care (PHC) settings.

The search strategy of this systematic scoping review will involve various electronic databases which include Academic Search Complete , PsycARTICLES . PsycINFO , Cumulative Index to Nursing and Applied Health Literature , Medline and Cochrane Library from the EbsocHost Database Platform. Electronic databases such as PubMed and Google Scholar, Union catalogue of theses and dissertations via SABINET online and WorldCat dissertations will be incorporated. A grey literature search will be conducted on websites such as the World Health Organization and government websites to find relevant policies and guidelines. The period for the search is from 1978 to 2018. This time period was chosen to coincide with the Declaration of Alma-Ata on PHC adopted in 1978. All references will be exported to Endnote library. Two independent reviewers will begin screening for eligible titles, abstracts and full articles. During title and abstract screening, duplicates will be removed. The Mixed Method Appraisal Tool will determine the quality of included studies. Thematic analysis will be used to analyse the included articles.

Evidence of preferences on PM methods and practices will generate insight on the use of PM systems in PHC and how this can be used for the purpose of improving nurses’ performance and in turn, the provision of quality health care. We hope to expose knowledge gaps and inform future research.

Peer Review reports

Growing health challenges have placed pressure on health management to monitor and evaluate human resource for health (HRH) in an effort to strengthen health systems response to evolving health challenges [ 1 ]. One such challenge is chronic conditions. Chronic conditions present the largest public health challenge of the twenty-first century [ 2 ]. It is projected that by 2020, heart disease, stroke, depression and cancer will be the greatest contributors of non-communicable diseases (NCDs), with mental disorders accounting for 60% of total mortality in the world. The number of people that require daily health care is rapidly growing, and it is projected that NCDs will continue to increase at a higher rate in lower-socio economic groups [ 2 ]. This has created a need for NCDs surveillance, prevention and control [ 3 ]. If not managed appropriately, chronic multiple NCDs will become the most expensive problem faced by health care systems globally [ 3 ]. This has resulted in the need for the re-organisation of health care systems to cater for chronic conditions, with people-centred care identified as the optimal approach to cater for multimorbid chronic conditions [ 4 ]. Noticeably, the ability of a country to strengthen its health system in order to meet its health goals depends largely on its human capital [ 5 ]. The six core components or ‘building blocks’ of the World Health Organization (WHO)’s analytical framework of health systems includes the health workforce as the people responsible for organising and delivering quality health services [ 6 , 7 ]. Quality health care refers to services that consistently deliver care that improves or maintains health, is valued and trusted by recipients and is responsive to changing population needs [ 5 , 6 ], with people-centred services identified as central to this endeavour globally given the changing disease profile towards chronic multimorbidity (see Table 1 for definition of quality care). In order to achieve the above, the health workforce must possess the knowledge, skills, motivation and preparedness to engage in actions with the primary intent to improve the provision of quality health services for people-centred services. Therefore, it is of vital importance that health workers are motivated and supported with the relevant capacities, thereby ensuring that they significantly contribute to attaining health objectives set nationally and globally [ 6 , 7 ]. One of the key human resource (HR) processes used to facilitate training and motivation of any workforce is a performance management (PM) system.

PM is described as a continuous process to identify, measure and improve the performance of individuals, teams and organisation, which involves aligning performance activities with the strategic goals of the organisation [ 7 ]. An important component of a PM system is performance appraisal (PA). PA refers to the formal process of assessing performance at work. PA is also sometimes referred to as performance review [ 9 ].

Accordingly, PA is a necessary component of PM systems. Some researchers argue that due to previous research not distinguishing between these two concepts, these terms are generally used interchangeably [ 9 , 10 ]. This study will do the same.

Accordingly, PM systems primarily serve three broad functions:

Strategically, PM systems aim to achieve the strategic objectives of the organisation, which is achieved by linking the organisation’s goals with individual performance goals [ 11 ].

Administratively, PM provides essential information to help managers take important decisions regarding salary increments, promotions and rewards [ 12 ].

The developmental function is facilitated through the provision of feedback on evaluated performance. Through the feedback mechanism, remedial action and steps to improve performance should be discussed. This presents an opportunity for managers to coach employees and aid improvement in performance on an ongoing basis [ 13 ].

In order to re-configure health care systems to support people-centred care for chronic multimorbid conditions, there is a need to initially identify methods and practices that promote effective PM that can be harnessed to this end [ 14 ]. Methods refer to standard processes and procedures used by a PM system (this is usually prescribed by policy). Practices refer to the formal and informal application or execution of ideas, beliefs and methods. Such re-configured systems require a focus on training, motivation and readiness of health professionals who are at the forefront of facilitating changes in health care best practices, such as nursing staff who constitute the largest sector of health workers across the globe [ 15 ] (see Table 1 for definitions of PM methods and practices). As an important managerial tool, PM systems are a critical tool for facilitating health system reforms as they determine if health workers are working diligently, trained appropriately and adequately rewarded for providing quality health care interventions in line with the health systems reforms [ 16 ].

Contribution to the field

Pm methods and practices.

PM systems are generally housed as part of role of human resource management (HRM), within the health care sector. The benefits of HRM practices to employee well-being and improved health outcomes have become a topical discussion among human resource practitioners and health care systems researchers around the world [ 17 ]. However, the impact of PM systems in health care settings has not received as much attention. While the nature of each health system and the use of HRM differ depending on national context, regardless of the context, it has become evident across national settings that HR is crucial in terms of its impact on patient outcomes and health care expenditures [ 18 ]. In order to determine how current health care delivery and reforms in health care systems may fully utilise HRM processes and systems such as a PM system to improve quality health care for people-centred care and promote better health outcomes [ 19 ], there is a need to initially examine evidence on PM methods and practices, as well as its consequences on the delivery of quality care among nurses in PHC settings.

PM opportunities and challenges

Some identified challenges include a world-wide shortage of nurses, health worker’s commitment and job satisfaction [ 8 , 13 , 14 , 15 , 16 , 17 , 18 , 19 ]. These factors have an impact on patient care and the provision of quality service delivery. Generally, there is a limited understanding of how a PM system impacts on managing health workers, more specifically nurses and how it may be used to improve care delivery and ultimately patient outcomes. Researchers opine that the purpose of PM systems is to monitor employees’ performance, motivate staff through providing opportunities for skills development and improving morale through rewarding and incentivising good performance. Their argument is that a PM system is one of the most important components of HRM. It provides justifications for decisions regarding recruitment and selection, training and development needs of existing employees and how to optimise the quality of work and efficiency within individual health care centres as well as the health system in general [ 18 ]. Accordingly, a poorly implemented PM system can be detrimental to staff morale, overall job satisfaction and result in high staff turnover rates [ 20 , 21 ]. The extent to which this has been investigated in health care settings is not clear. Some experts have varying opinions and approaches to PM systems that may add to HRM outcomes and quality of care [ 22 ]. Consequently, there is a need to review what is available on this topic for the purpose of creating a greater understanding of PM systems, as well as to identify knowledge gaps and providing recommendations on how future research may fill these gaps.

The aim of this scoping review was to systematically map the available evidence on the PM of nurses in PHC settings in order to enhance our understanding of the role of PM systems in improving ‘quality of care’, so as to understand how PM systems need to be strengthened for managing performance in PHC settings that encourages improving quality people-centred care and improve patients’ outcomes. This review offers a broad overview of managing the performance of nurses working at various PHC settings. In addition, it provides an analysis of international methods and practices used to manage nurses. From these methods, it is possible to identify best practices for suitable PM approaches.

The review of primary research has gained popularity, as evidence-based practices gain recognition as a benchmark for care and primary research sources continue to grow [ 23 ]. A scoping review is considered as a relatively new method for reviewing literature, with the first such framework published in 2005. This method of literature review is an advantage for synthesising research evidence and mapping existing literature in a given field in terms of its prevalence and key features. Hence, it is also referred to as a ‘mapping’ review [ 23 , 24 ].

Methodology

Systematic scoping review

We will conduct a systematic scoping review of grey and peer-reviewed literature on PM and its influence on quality of care among nurses in PHC settings. The review will be guided by the Arksey and O’Malley’s scoping review framework [ 25 ], which outlines the following steps:

Stage 1. Identifying the research question

Stage 2. Identifying relevant studies

Stage 3. Study selection

Stage 4. Charting the data

Stage 5. Collating, summarising and reporting the results

The recommendations of Levac et al. (2010) will be used to improve the transparency of each step pertaining to the conduct of the systematic scoping review [ 26 ].

Stage 1: Identifying the research question

The central research question of the study is as follows:

What is the existing evidence on the influence of PM methods and practices on quality of care among nurses in PHC?

The sub-research questions are as follows:

What are the common challenges and opportunities reported on various PM methods and practices?

What are the key gaps in literature on the contribution of effective PM on quality of care among nurses in PHC settings?

The study will use the broad population, concept and context (PCC) framework recommended by the Joanna Briggs Institute for Scoping Reviews [ 27 , 28 ]. The design of the search strategy will be underpinned by a key inclusion criteria (see Table 2 ).

The PCC framework to determine the research questions is illustrated in Fig. 1 .

PCC Framework

Stage 2: Identifying relevant studies

We will identify relevant studies by conducting a comprehensive search on the following electronic databases: Academic Search Complete , PsycARTICLES . PsycINFO , Cumulative Index to Nursing and Applied Health Literature ( CINAHL ), Medline , Cochrane Library and PubMed . Literature will include published peer-reviewed journal articles with primary studies which have a transparent empirical base utilising qualitative, quantitative and mixed method research design and grey literature addressing the research questions.

To achieve a comprehensive search, websites such as the WHO and governmental websites will be used to gather policies and guidelines on PM for the respective health care sectors. Databases such as Google Scholar , Union Catalogue of Theses and Dissertations (UCTD) via SABINET Online and WorldCat Dissertations and Theses via OCLO will also be used to find relevant literature.

A hand search through the main published texts used in PM systems and its outcomes will also be conducted. In addition, articles will be searched through the ‘cited by’ search as well as citations included in the reference lists of included articles. The search terms will include Nurse OR Nurse Practitioners OR Registered Nurse AND, Performance Management OR Performance Appraisal OR Performance Review OR Performance Management and Appraisal Systems AND Primary Health Care or Clinics. This search strategy was piloted to check the suitability of selected electronic databases and key words (see Table 2 ).

Stage 3: Study selection

Following the keyword search, relevant citations must be selected through title, abstract and full-text screening. The study selection process involves the elimination of studies that do not address the main research question. Developing an inclusion and exclusion criteria at the outset of the study ensures there are clear guidelines enforced, so each researcher is consistent in decision-making on the relevance for each citation [ 25 ]. An inclusion and exclusion criteria reduce the risk of bias in the review, thereby minimising the risk of error and promoting credibility of the findings.

In Table 3 , information is provided about the inclusion and exclusion criteria that will be adhered to.

An Endnote™ library will be created for the aforementioned review. The primary investigator (CZM) will conduct a comprehensive database search and screen titles from the previously mentioned databases with the assistance from a senior librarian at the University of KwaZulu-Natal (UKZN) library services to assist with the search. All references screened will be exported to the Endnote library; title and abstract screening will be conducted. Once the initial screening is completed, eligible references are kept, and duplicates will be removed. The full text of eligible abstracts will be retrieved. To optimise the full article search procedure, the reviewers will further consult with the librarian to assist with locating and retrieving articles that will be included in the full article screening. In cases where the reviewers are unable to retrieve the articles from the databases, a request will be lodged with the relevant authors. Two reviewers (CZM and TS) will discuss eligible and ineligible studies to identify if there are any discrepancies [ 27 ]. Should the reviewers be unable to resolve disagreements through discussion, a third reviewer will be consulted (TPM-T). The screening results will be reported accordingly using the PRISMA chart as depicted in Fig. 2 [ 30 , 31 ].

Example of PRISMA-ScR chart. Source: The PRISMA-ScR = preferred reporting item for systematic reviews and meta-analyses extension for scoping reviews [ 27 , 29 ]

Stage 4: Charting the data

The process of extracting data aims to generate a descriptive summary of the results that corresponds to the aim and research question of the scoping review at hand. A draft data charting table (see Table 4 ) has been developed to facilitate the collection and sorting of key pieces of information from articles that have made the selection [ 32 ]. A data charting form, highlighting the important aspects for the study will be developed and piloted. The variables and themes included will answer each of the research questions. One reviewer will be involved in data extraction (CZM). Once completed, this process will be verified by the two other reviewers (TS and TPM-T). The data charting form will be updated as and when required.

Stage 5: Collating, summarising and reporting the results

To provide a narrative account of the data extracted from the included studies, data will be analysed using content and thematic analysis. Content and thematic analysis is useful as it provides a descriptive presentation of data. Through the identification of common themes in the text, the researcher is able to analyse the data. The data will be extracted around the following themes: PM initiatives, managing performance of nurses in PHC settings and the use of PM to influence the improvement of the quality of health care.

Quality appraisal

The Mixed Method Appraisal Tool (MMAT) will be used to assess the quality of the studies [ 32 , 33 ]. Each section is divided by research design type. During the appraisal process, the following will be used:

Section 1 of the MMAT will be used to review the quality of a qualitative study;

Section 2 is for quantitative randomised controlled studies;

Section 3 will be used for non-randomised studies;

Section 4 is for descriptive studies;

Section 5 is for mixed-method research methodology studies.

[Note: for a mixed methods study, we will use section 1 for appraising the qualitative component, the appropriate section for the quantitative component (2 or 3 or 4) and section 5 for the mixed methods component]. This tool is valuable in examining the suitability of an objective of a study, its methodology, the appropriateness of the study design, the data collection, the study selection, the data analysis, the findings presentation as well as the discussion and conclusion. The results from the scrutiny of the above-mentioned aspects will determine the quality of the articles and if the studies will be included after the extraction of the data [ 33 ]. The quality of the articles will be graded per domain on a percentage basis. For qualitative (QUAL) and quantitative (QUAN) studies, the grading of each study will be based on the number of criteria met divided by 4, the score ranging from 25 (*only one criterion was met) to 100% (****all criteria were met). For mixed methods (MM) studies, the quality of the combination cannot exceed the quality of the weakest component. Therefore, the overall quality score is the lowest score of the study components. Thus, the score of 25% (*) is gained when QUAL = 1 or QUAN = 1 or MM = 0, 50% (**) when QUAL = 2 or QUAN = 2 or MM = 1, it is 75% when QUAL = 3 or QUAN = 3 or MM = 2 and it is 100% when QUAL = 4 or QUAN = 4 and MM = 3. For the purpose of this study, 25% is considered low quality, and above 80% is considered high [ 31 , 34 ]. Grey literature will be assessed using the Joanna Briggs Institution (JBI) Narrative, Opinion, Text Assessment and Review Instrument (NOTARI) systematic reviews. Using the JBI Reviewer’s Manual 2014, any issues relating to the including suitability of topic selection, critical appraisal, data extraction and synthesis will be addressed. Textual evidence requires three levels of credibility. Therefore, the reviewers are required to determine if, when comparing the conclusion with the argument, the conclusion represents evidence that is Unequivocal (U) (relates to evidence beyond reasonable doubt), Credible (C) and Unsupported (findings that are not supported by the data) [ 35 ].

PM systems are a significant element of HRM. The growing need for improved clinical outcomes and quality of care has highlighted the importance of standards of care and managing the performance of health workers. However, poor practices in the implementation of PM systems within the health sector have been shown to have a negative impact on employees’ perceptions of fairness and accountability, which in turn leads to high staff turnover and poor clinical outcomes. Literature on the PM of nurses in health care is abundant. With the shift towards PHC and its well-documented benefits, the reviewers will aim to map literature around the evidence, preferences and practices of the PM of nurses, in light of the need to ensure health workers are adequately trained and rewarded for meeting the needs of existing health care systems. Enhancing methods and practices of PM will help inform decisions on how the practice of people-centred care may be improved, by ensuring good performance is rewarded and health workers are equipped with tools that assist and facilitate effective chronic care practices in PHC settings.

The reviewers anticipate this scoping review finding will assist in mapping evidence of best practices and preferences on PM methods and practices. Likewise, the reviewers hope to expose knowledge gaps and limitations, as well as inform future research. Findings will be disseminated electronically, in print, through peer presentations and conferences on strengthening health systems, HRH or conference proceedings, symposia and other research contributions that examine investing in health care human capital.

Availability of data and materials

All data generated or analysed during this study will be included in the published systematic scoping review article and will also be made available upon request.

Abbreviations

Health care workers

Human resource

Human resources for health

Human resource management

Human resource practitioner

Mixed Method Appraisal Tool

Non-communicable disease

Performance appraisals

Population concept context

Primary health care
Performance management

Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews

Quality improvement

Quality of car

Thematic analysis

University of Kwa-Zulu Natal

World Health Organization

World Health Organisation. Global strategy on human resources for health: workforce 2030. Geneva: World Health Organisation; 2016. https://www.who.int/hrh/resources/pub_globstrathrh-2030/en/ . Accessed 25 Jan 2019.

Epping-Jordan JE, Bengoa R, Yach D. Chronic conditions-the new health challenge. SAMJ. 2008;93(8):585.

Google Scholar

Tinetti ME, Naik AD, Dodson JA. Moving from disease-centred to patient goals–directed care for patients with multiple chronic conditions: patient value-based care. JAMA Cardiology. 2016;1(1):9–10.

Article Google Scholar

Kuluski K, Peckham A, Williams AP, Upshur RE. What gets in the way of person-centred care for people with multimorbidity? Lessons from Ontario. Canada. Healthc Q. 2016;19(2):17–23.

Nxumalo N, Goudge J, Gilson L, Eyles J. Performance management in times of change: experiences of implementing a performance assessment system in a district in South Africa. Int J Equity Health. 2018;17(1):141.

Kruk ME, Pate M, Mullan Z. Introducing the Lancet Global Health Commission on high-quality health systems in the SDG era. The Lancet Global health. 2017;5(5):e480–1.

World Health Organisation. Monitoring the building blocks of health systems: a handbook of indicators and their measurement strategies. Geneva: World Health Organisation; 2010. https://www.who.int/healthinfo/systems/monitoring/en/ . Accessed 24 Jan 2019

World Health Organisation. Quality of care in the context of universal health coverage (UHC). Lilongwe. Malawi: World Health Organisation; 2017. https://www.who.int/maternal_child_adolescent/topics/quality-of-care/quality-of-care-brief-uhc.pdf?ua=1 . Accessed: 31 October 2019

DeNisi AS, Murphy KR. Performance appraisal and performance management: 100 years of progress? J Appl Psychol. 2017;102(3):421.

Aguinis H. Performance management. Boston, MA: Pearson; 2013.

Lutwama GW, Roos JH, Dolamo BL. Assessing the implementation of performance management of health care workers in Uganda. BMC Health Serv Res. 2013;13(1):355.

Lee TW, Steers RM. Facilitating effective performance appraisals: the role of employee commitment and organisational climate. In: Landy EJ, Zedeck S, Cleveland J, editors. Performance measurement and theory. New York, NY: Routledge; 2017. p. 75–93.

DeNisi AS. Managing performance to change behaviour. J Organ Behav Man. 2011;31(4):262–76.

Bartram T, Dowling PJ. An international perspective on human resource management and performance in the health care sector: toward a research agenda. Int J Hum Resour Man. 2013;24(16):3031–7.

Ball J, Day T, Murrells T, Dall’Ora C, Rafferty AM, Griffiths P, Maben J. Cross-sectional examination of the association between shift length and hospital nurses job satisfaction and nurse reported quality measures. BMC Nursing. 2017;16(1):26.

Platis C, Reklitis P, Zimeras S. Relation between job satisfaction and job performance in health care services. Procedia Soc Behav Sci. 2015;175:480–7.

Bitton A, Ratcliffe HL, Veillard JH, Kress DH, Barkley S, Kimball M, Secci F, Wong E, Basu L, Taylor C, Bayona J. Primary health care as a foundation for strengthening health systems in low-and middle-income countries. J Gen Intern Med. 2017;32(5):566–71.

Choudhary G, Puranik S. Study on employee performance appraisal in health care. Asian J Manag Sci. 2014;2(03 Special Issue):59-64.

O’Malley AS, Rich EC. Measuring comprehensiveness of primary care: challenges and opportunities. J Gen Intern Med. 2015;30(3):568–75.

Gedif G, Sisay Y, Alebel A, Belay YA. Level of job satisfaction and associated factors among health care professionals working at University of Gondar Referral Hospital, Northwest Ethiopia: a cross-sectional study. BMC Research Notes. 2018;11(1):824.

Skinner N, Van Dijk P, Stothard C, Fein EC. “It breaks your soul”: an in-depth exploration of workplace injustice in nursing. J Nurs Manag. 2018;26(2):200–8.

Scheffelaar A, Bos N, Hendriks M, van Dulmen S, Luijkx K. Determinants of the quality of care relationships in long-term care-a systematic review. BMC Health Serv Res. 2018 Dec;18(1):903.

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015;13(3):141–6.

Khalil H, Peters M, Godfrey CM, McInerney P, Soares CB, Parker D. An evidence-based approach to scoping reviews. Worldviews Evid Based Nurs. 2016;13(2):118–23.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5:69.

Hannes K, Macaitis K. A move to more systematic and transparent approaches in qualitative evidence synthesis: update on a review of published papers. Qual Res. 2012;12(4):402–42.

Ng L, Pitt V, Huckvale K, Clavisi O, Turner T, Gruen R, Elliott JH. Title and abstract screening and evaluation in systematic reviews (TASER): a pilot randomised controlled trial of title and abstract screening by medical students. Syst Rev. 2014;3(1):121.

Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo M, Dagenais P, Gagnon MP, Griffiths F, Nicolau B, O’Cathain A, Rousseau MC, Vedel I. The Mixed Methods Appraisal Tool (MMAT) version 2018. User guide. Montreal: McGill University, Department of Family Medicine; 2018. http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/fetch/127425851/MMAT_2018_criteria-manual_2018-04-04.pdf . Accessed 25 Jan 2019

Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLOS Medicine. 2009;6(7):e1000097.

Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, Moher D, Peters MD, Horsley T, Weeks L, Hempel S. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Armstrong R, Hall BJ, Doyle J, Waters E. Cochrane Update. ‘Scoping the scope’ of a cochrane review. J Public Health. 2011;33(1):147–50.

Pace R, Pluye P, Bartlett G, Macaulay AC, Salsberg J, Jagosh J, Seller R. Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review. Int J Nurs Stud. 2012;49(1):47–53.

Tricco AC, Lillie E, Zarin W, O’Brien K, Colquhoun H, Kastner M, Levac D, Ng C, Sharpe JP, Wilson K. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16(1):15.

The Joanna Briggs Institute. Joanna Briggs Institute Reviewers’ Manual: 2014 edition. The University of Adelaide. South Australia: The Joanna Briggs Institute; 2014.

Download references

Acknowledgements

The authors would like to thank the College of Health Sciences, Systematic Review Services and University of KwaZulu-Natal for financially supporting the development of this research study.

The College of Health Sciences, Systematic Review Services and University of KwaZulu-Natal funded this research study. They also provided the resources used in the development of this protocol.

Author information

Authors and affiliations.

School of Applied Human Sciences, Discipline of Psychology, University of KwaZulu-Natal, Durban, 4001, Republic of South Africa

Cynthia Zandile Madlabana & Inge Petersen

Discipline of Public Health, University of KwaZulu-Natal, Durban, 4001, Republic of South Africa

Tivani Phosa Mashamba-Thompson

You can also search for this author in PubMed Google Scholar

Contributions

CZM contributed by conceptualising the study and prepared the draft proposal under the guidance and supervision of IP and TPM-T. All three authors contributed to the development of the background and planned output of the research as well as the design of the study. TPM-T contributed to the development process. CZM prepared the manuscript, and IP and TPM-T reviewed it. All three authors contributed to the reviewed draft version of the manuscript and approved the final version.

Corresponding author

Correspondence to Cynthia Zandile Madlabana .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Madlabana, C.Z., Mashamba-Thompson, T.P. & Petersen, I. Performance management methods and practices among nurses in primary health care settings: a systematic scoping review protocol. Syst Rev 9 , 40 (2020). https://doi.org/10.1186/s13643-020-01294-w

Download citation

Received : 15 March 2018

Accepted : 10 February 2020

Published : 21 February 2020

DOI : https://doi.org/10.1186/s13643-020-01294-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Performance review
Quality of care
Human resources management

Systematic Reviews

ISSN: 2046-4053

Submission enquiries: Access here and click Contact Us
General enquiries: [email protected]

performance management a systematic literature review

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List

Performance management methods and practices among nurses in primary health care settings: a systematic scoping review protocol

Cynthia zandile madlabana.

1 School of Applied Human Sciences, Discipline of Psychology, University of KwaZulu-Natal, Durban, 4001 Republic of South Africa

Tivani Phosa Mashamba-Thompson

2 Discipline of Public Health, University of KwaZulu-Natal, Durban, 4001 Republic of South Africa

Inge Petersen

Associated data.

All data generated or analysed during this study will be included in the published systematic scoping review article and will also be made available upon request.

Growing health challenges have placed pressure on health management to monitor and evaluate human resource for health (HRH) in an effort to strengthen health systems response to evolving health challenges [ 1 ]. One such challenge is chronic conditions. Chronic conditions present the largest public health challenge of the twenty-first century [ 2 ]. It is projected that by 2020, heart disease, stroke, depression and cancer will be the greatest contributors of non-communicable diseases (NCDs), with mental disorders accounting for 60% of total mortality in the world. The number of people that require daily health care is rapidly growing, and it is projected that NCDs will continue to increase at a higher rate in lower-socio economic groups [ 2 ]. This has created a need for NCDs surveillance, prevention and control [ 3 ]. If not managed appropriately, chronic multiple NCDs will become the most expensive problem faced by health care systems globally [ 3 ]. This has resulted in the need for the re-organisation of health care systems to cater for chronic conditions, with people-centred care identified as the optimal approach to cater for multimorbid chronic conditions [ 4 ]. Noticeably, the ability of a country to strengthen its health system in order to meet its health goals depends largely on its human capital [ 5 ]. The six core components or ‘building blocks’ of the World Health Organization (WHO)’s analytical framework of health systems includes the health workforce as the people responsible for organising and delivering quality health services [ 6 , 7 ]. Quality health care refers to services that consistently deliver care that improves or maintains health, is valued and trusted by recipients and is responsive to changing population needs [ 5 , 6 ], with people-centred services identified as central to this endeavour globally given the changing disease profile towards chronic multimorbidity (see Table Table1 1 for definition of quality care). In order to achieve the above, the health workforce must possess the knowledge, skills, motivation and preparedness to engage in actions with the primary intent to improve the provision of quality health services for people-centred services. Therefore, it is of vital importance that health workers are motivated and supported with the relevant capacities, thereby ensuring that they significantly contribute to attaining health objectives set nationally and globally [ 6 , 7 ]. One of the key human resource (HR) processes used to facilitate training and motivation of any workforce is a performance management (PM) system.

Key concepts and definitions

Accordingly, PM systems primarily serve three broad functions:

i. Strategically, PM systems aim to achieve the strategic objectives of the organisation, which is achieved by linking the organisation’s goals with individual performance goals [ 11 ].
ii. Administratively, PM provides essential information to help managers take important decisions regarding salary increments, promotions and rewards [ 12 ].
iii. The developmental function is facilitated through the provision of feedback on evaluated performance. Through the feedback mechanism, remedial action and steps to improve performance should be discussed. This presents an opportunity for managers to coach employees and aid improvement in performance on an ongoing basis [ 13 ].

In order to re-configure health care systems to support people-centred care for chronic multimorbid conditions, there is a need to initially identify methods and practices that promote effective PM that can be harnessed to this end [ 14 ]. Methods refer to standard processes and procedures used by a PM system (this is usually prescribed by policy). Practices refer to the formal and informal application or execution of ideas, beliefs and methods. Such re-configured systems require a focus on training, motivation and readiness of health professionals who are at the forefront of facilitating changes in health care best practices, such as nursing staff who constitute the largest sector of health workers across the globe [ 15 ] (see Table Table1 1 for definitions of PM methods and practices). As an important managerial tool, PM systems are a critical tool for facilitating health system reforms as they determine if health workers are working diligently, trained appropriately and adequately rewarded for providing quality health care interventions in line with the health systems reforms [ 16 ].

Contribution to the field

Pm methods and practices.

PM opportunities and challenges

Some identified challenges include a world-wide shortage of nurses, health worker’s commitment and job satisfaction [ 8 , 13 – 19 ]. These factors have an impact on patient care and the provision of quality service delivery. Generally, there is a limited understanding of how a PM system impacts on managing health workers, more specifically nurses and how it may be used to improve care delivery and ultimately patient outcomes. Researchers opine that the purpose of PM systems is to monitor employees’ performance, motivate staff through providing opportunities for skills development and improving morale through rewarding and incentivising good performance. Their argument is that a PM system is one of the most important components of HRM. It provides justifications for decisions regarding recruitment and selection, training and development needs of existing employees and how to optimise the quality of work and efficiency within individual health care centres as well as the health system in general [ 18 ]. Accordingly, a poorly implemented PM system can be detrimental to staff morale, overall job satisfaction and result in high staff turnover rates [ 20 , 21 ]. The extent to which this has been investigated in health care settings is not clear. Some experts have varying opinions and approaches to PM systems that may add to HRM outcomes and quality of care [ 22 ]. Consequently, there is a need to review what is available on this topic for the purpose of creating a greater understanding of PM systems, as well as to identify knowledge gaps and providing recommendations on how future research may fill these gaps.

Methodology

Systematic scoping review.

Stage 1. Identifying the research question
Stage 2. Identifying relevant studies
Stage 3. Study selection
Stage 4. Charting the data
Stage 5. Collating, summarising and reporting the results

The recommendations of Levac et al. (2010) will be used to improve the transparency of each step pertaining to the conduct of the systematic scoping review [ 26 ].

Stage 1: Identifying the research question

The central research question of the study is as follows:

What is the existing evidence on the influence of PM methods and practices on quality of care among nurses in PHC?

The sub-research questions are as follows:

i. What are the common challenges and opportunities reported on various PM methods and practices?
ii. What are the key gaps in literature on the contribution of effective PM on quality of care among nurses in PHC settings?

Pilot database search results

The PCC framework to determine the research questions is illustrated in Fig. Fig.1 1 .

An external file that holds a picture, illustration, etc.
Object name is 13643_2020_1294_Fig1_HTML.jpg

PCC Framework

Stage 2: Identifying relevant studies

Stage 3: Study selection

In Table Table3, 3 , information is provided about the inclusion and exclusion criteria that will be adhered to.

Inclusion and exclusion criteria

An external file that holds a picture, illustration, etc.
Object name is 13643_2020_1294_Fig2_HTML.jpg

Example of PRISMA-ScR chart. Source: The PRISMA-ScR = preferred reporting item for systematic reviews and meta-analyses extension for scoping reviews [ 27 , 29 ]

Stage 4: Charting the data

The process of extracting data aims to generate a descriptive summary of the results that corresponds to the aim and research question of the scoping review at hand. A draft data charting table (see Table Table4) 4 ) has been developed to facilitate the collection and sorting of key pieces of information from articles that have made the selection [ 32 ]. A data charting form, highlighting the important aspects for the study will be developed and piloted. The variables and themes included will answer each of the research questions. One reviewer will be involved in data extraction (CZM). Once completed, this process will be verified by the two other reviewers (TS and TPM-T). The data charting form will be updated as and when required.

Data extraction/data charting tool

Stage 5: Collating, summarising and reporting the results

Quality appraisal

Section 1 of the MMAT will be used to review the quality of a qualitative study;
Section 2 is for quantitative randomised controlled studies;
Section 3 will be used for non-randomised studies;
Section 4 is for descriptive studies;
Section 5 is for mixed-method research methodology studies.

Acknowledgements

The authors would like to thank the College of Health Sciences, Systematic Review Services and University of KwaZulu-Natal for financially supporting the development of this research study.

Abbreviations

Authors’ contributions.

The College of Health Sciences, Systematic Review Services and University of KwaZulu-Natal funded this research study. They also provided the resources used in the development of this protocol.

Availability of data and materials

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

To read this content please select one of the options below:

Please note you do not have access to teaching notes, humanitarian supply chain performance management: a systematic literature review.

Supply Chain Management

ISSN : 1359-8546

Article publication date: 2 September 2014

This paper aims to identify the state of the art of performance measurement and management in humanitarian supply chains; to categorize performance measurement indicators in the five supply chain phases of Gunasekaran and Kobu (2007) and evaluate them based on the evaluation criteria of Caplice and Sheffi (1995); and to define gaps and challenges in this field and give insights for future research in this domain.

Design/methodology/approach

A literature review has been conducted using a structured method based on Denyer and Tranfield (2009) and Rousseau et al. (2008). The state of the art on humanitarian supply chain performance management with a focus on measurement frameworks and indicators and their applications in practice is classified in three categories. The first category is the definition and measurement of success in humanitarian supply chains. The second category is managing performance, which focuses on describing and analyzing the actual practice of managing performance. The third category shows the challenges in performance management that humanitarian supply chain actors deal with.

Findings reveal that performance measurement and management in humanitarian supply chains is still an open area of research, especially compared to the commercial supply chain sector. Furthermore, the research indicates that performance measurement and management in humanitarian supply chains has to be developed in support of the supply chain strategy. Based on the findings of the literature review on performance measurement and management in the commercial and humanitarian field, a first classification of 94 performance measurement indicators in humanitarian supply chains is presented. Furthermore, the paper shows key problems why performance measurement and management systems have not been widely developed and systematically implemented in humanitarian supply chains and are not part of the supply chain strategy. The authors propose performance measurement guidelines that include input and output criteria. They develop a research agenda that focuses on four research questions for designing, deploying and disseminating performance measurement and management in humanitarian supply chains.

Practical implications

The result helps the humanitarian supply chain community to conduct further research in this area and to develop performance measurement frameworks and indicators that suit humanitarian supply chains.

Originality/value

It is the first systematic approach to categorize research output regarding performance measurement and management in humanitarian supply chains. The paper shows the state of the art in performance measurement and management in humanitarian supply chains and develops a research agenda.

Performance measurement
Performance management
Humanitarian logistics
Humanitarian supply chains

Abidi, H. , de Leeuw, S. and Klumpp, M. (2014), "Humanitarian supply chain performance management: a systematic literature review", Supply Chain Management , Vol. 19 No. 5/6, pp. 592-608. https://doi.org/10.1108/SCM-09-2013-0349

Emerald Group Publishing Limited

We’re listening — tell us what you think, something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

Search Menu
Volume 8, Issue 2, 2024 (In Progress)
Volume 8, Issue 1, 2024
Volume 6, Issue Supplement_1, October 2022 (In Progress)
Advance Articles
Supplements
Collections
Current and future advances in practice series
BSR Registers Papers
General Instructions
Submission Site
Open Access
Self-Archiving Policy
Trainee Publishing Program
About Rheumatology Advances in Practice
About British Society for Rheumatology
Editorial Board
Advertising & Corporate Services
Journals Career Network
Journals on Oxford Academic
Books on Oxford Academic

Article Contents

Lay summary, introduction, physical activity and exercise, mental health, supplementary material, data availability, authors’ contributions, acknowledgements.

< Previous

Lifestyle interventions in the management of systemic sclerosis: a systematic review of the literature

Article contents
Figures & tables
Supplementary Data

Ioannis Parodis, Alexander Tsoi, Alvaro Gomez, Jun Weng Chow, Charlotte Girard-Guyonvarc’h, Tanja Stamm, Carina Boström, Lifestyle interventions in the management of systemic sclerosis: a systematic review of the literature, Rheumatology Advances in Practice , Volume 8, Issue 2, 2024, rkae037, https://doi.org/10.1093/rap/rkae037

Permissions Icon Permissions

We aimed to investigate the efficacy of lifestyle interventions for the management of SSc.

We searched the MEDLINE, Embase, Web of Science and CINAHL databases in June 2021. We included studies conducted on five or more patients with SSc published between 1 January 2000 and the search date evaluating lifestyle interventions, excluding systematic reviews without meta-analyses. Critical appraisal was conducted using critical appraisal tools from the Joanna Briggs Institute. Thirty-six studies were included for full-text evaluation.

A total of 17 studies evaluated the effect of physical exercise alone, whereas 14 studies evaluated educational interventions for mental health management, often with physical exercise as a central component. At an aggregated level, these studies support patient education and physical exercise for the improvement of physical function, in particular hand and mouth function. Studies on diet and nutrition were few ( n = 5) and pertained to gastrointestinal as well as anthropometric outcomes; these studies were insufficient to support any conclusions.

Physical exercise and patient education should be considered for improving physical function in patients with SSc. These interventions can be provided alongside pharmacotherapy, but there is no evidence supporting that they can be a substitute. Further research should aim at assessing the effects of reductions of harmful exposures, including tobacco smoking and alcohol, improving sleep and enhancing social relations, three hitherto underexplored facets of lifestyle in the context of SSc.

What does this mean for patients?

For individuals living with systemic sclerosis, a rare autoimmune disease affecting the skin and internal organs, managing symptoms and maintaining quality of life can be challenging. The present systematic review delves into lifestyle interventions, including exercise and dietary changes, aiming to improve patient outcomes. While exercise interventions showed promise in enhancing mobility and overall well-being, evidence regarding dietary modifications was limited. However, combining interventions targeting physical function with various aspects of self-management could further amplify their impact on quality of life. For patients, this research underscores the potential benefits of incorporating tailored lifestyle changes alongside drug treatments. It suggests that regular exercise could alleviate symptoms such as fatigue and pain, thereby improving daily functioning. Moreover, it hints at the importance of a holistic approach to self-care, with pharmacotherapy as just one, albeit irreplaceable, part of a whole. Ultimately, this highlights avenues for patients to actively participate in managing their condition, enhancing their overall well-being and quality of life. Further research is needed to explore the full scope of lifestyle interventions and their potential long-term benefits for patients with systemic sclerosis.

The efficacy of lifestyle interventions in systemic sclerosis is underexplored.

Patient education enhances outcomes and physical exercise improves physical function in systemic sclerosis.

Lifestyle interventions constitute a supplement, not a substitute, to pharmacotherapy.

SSc is a chronic connective tissue disease that primarily affects women, commonly in their fifth decade of life, and can manifest with limited cutaneous involvement, diffuse cutaneous involvement or with no cutaneous involvement [ 1 ]. The estimated prevalence ranges from <150–443 cases per million population, being higher in regions such as southern Europe, North America and Australia [ 1 ]. Areas commonly affected by skin fibrosis are the hands and face, which often results in impaired hand function [ 2 ] and microstomia [ 3 ]. Although advances in pharmacotherapy for rheumatic diseases have been achieved during the 21st century, the guidelines for non-pharmacological management in general, and lifestyle interventions in particular, are ill-defined.

Upon examination of the literature, the definition of a lifestyle intervention itself is not clearly characterized. The American College of Lifestyle Medicine (ACLM) defines lifestyle medicine as ‘a medical specialty that uses therapeutic lifestyle interventions as a primary modality’ and lists six fundamental domains as targets of lifestyle medicine: nutrition, exercise, stress, substance abuse, sleep and relationships [ 4 ]. Analogously, the British Society of Lifestyle Medicine specifies six ‘pillars of lifestyle medicine’: healthy eating, physical activity, mental well-being, minimizing harmful substances, sleep and healthy relationships [ 5 ]. As such, a lifestyle intervention can be any intervention that covers any or all six domains, i.e. physical activity and exercise, diet and nutrition, mental health, harmful exposures, sleep and social relations.

The importance of lifestyle in the management of rheumatic diseases is gaining recognition. The EULAR recently published recommendations regarding lifestyle behaviours and work participation aimed at preventing disease progression in patients with rheumatic and musculoskeletal diseases [ 6 ]. These 18 recommendations, accompanied by five overarching principles, were derived from systematic literature reviews geared toward six ‘lifestyle exposures’, i.e. exercise, diet, weight, alcohol, smoking and work participation [ 6 ]. Recently the EULAR also issued guidelines for the non-pharmacological management of SLE and SSc [ 7 ], following a thorough systematic literature review [ 8 ]. However, this review, while comprehensive, did not distinctly isolate lifestyle interventions from other approaches. Consequently, valuable insights into lifestyle interventions targeting modifiable health factors were obscured among the multitude of non-pharmacological management strategies examined. To bridge this gap in the literature, we herein conducted a systematic literature review to address the efficacy of lifestyle interventions in different aspects of the disease course in people living with SSc.

Inclusion and exclusion criteria

Inclusion criteria for studies included a date of publication between 1 January 2000 and the search date, having a cohort of patients with SSc (as defined by classification criteria and/or International Classification of Diseases codes) as a population under investigation and evaluation of a lifestyle intervention. Studies were excluded if they had fewer than five participants, if they were systematic reviews without a meta-analysis, had no data on a distinct SSc patient population, were duplicates, were written in a language other than English, Spanish, or Swedish or if they did not assess an intervention that comprised one or more of the following: physical activity and exercise, diet and nutrition, mental health, harmful exposures, sleep and social relations.

Search strategy

On 22 June 2021, the MEDLINE, Embase, Web of Science and CINAHL databases were searched for studies concerning non-pharmacological management for SSc. Two investigators (A.G. and J.C.) screened the 11 089 initial hits under supervision of one senior investigator (I.P.). Conflicts were solved upon discussion with two investigators (I.P. and C.B.). The search and study selection was documented according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses statement ( Fig. 1 ) [ 9 ].

Flowchart of study selection

Data extraction

Data extraction was conducted by one researcher (J.C.) under the supervision of one senior investigator (I.P.). Data extracted included the number of participants for each study, interventions or management strategies given to both experimental and control groups, the characteristics of the comparator group, outcomes and the efficacy of the intervention. These data are provided in Supplementary Table S1 , available at Rheumatology Advances in Practice online.

Categorization

After data extraction and risk of bias (RoB) assessment, the studies were grouped by the category of lifestyle intervention they assessed. Studies combining exercise protocols with other educational interventions were separated from studies evaluating only physical exercise protocols, which in turn constituted their own group. Studies within the two above categories specifically evaluating the hand and mouth were further subgrouped into their own categories.

Quality assessment and evidence grading

RoB assessment for all included articles was conducted by one researcher (A.T.) using the Joanna Briggs Institute critical appraisal (CA) tools (checklists) [ 10 ]. Since all articles were already included before quality assessment for this review, the alternatives for overall appraisal ‘include’, ‘exclude’ and ‘seek further info’ were modified to ‘robust’, ‘weak’ and ‘intermediate’, respectively. The appropriate checklist for each study was selected based on the study design. A study was deemed weak if there were six or more checklist items it did not clearly fulfil, intermediate if there were three to five checklist items it did not clearly fulfil or robust if it clearly fulfilled all checklist items but two or fewer. After CA, studies were graded by level of evidence (LoE) according to the Oxford Centre for Evidence-Based Medicine [ 11 ].

Study characteristics

Thirty-six studies were included. Of these, 17 evaluated physical activity and exercise alone rather than in combination with other interventions [ 12–28 ]. Fourteen studies evaluated the efficacy of mental health management [ 29–42 ] and five studies assessed diet and nutrition [ 43–47 ]. Fig. 1 presents a flowchart of the study selection. Studies and their characteristics, including the number of participants, interventions, characteristics of the comparator groups and outcomes are provided in Supplementary Table S1 , available at Rheumatology Advances in Practice online.

A randomized controlled trial (RCT) examining the effect of a tailored home-based exercise program (CA: intermediate; LoE: 2) found improvements in 6-min walking distance (6MWD) [ 48 ], the physical component score (PCS) of the 36-item Short Form Health Survey (SF-36) [ 49 ] and the HAQ Disability Index (HAQ-DI) [ 15 , 50 ]. Two RCTs evaluating the effect of exercise on microcirculation (CA: weak; LoE: 3) found no significant impact on cutaneous vascular conductance (CVC) after 12 weeks of high-intensity interval training alone [ 21 ]; however, this produced a significant effect when combined with endurance training [ 22 ]. An RCT evaluating tai chi (CA: weak; LoE: 3) found improvements in scores relating to balance (Berg Balance Scale [ 51 ]), sleep (Pittsburgh Sleep Quality Index [ 52 ]) and fatigue (Fatigue Severity Scale [ 53 ]), but not trunk lateral endurance (trunk lateral endurance test [ 23 , 54 ]. Observational studies found that lower quadriceps strength associated with worse HAQ-DI scores (CA: robust; LoE: 3) [ 19 ] and that exercise habits associated with improved scores on the HAQ-DI and the Patient-Reported Outcomes Measurement Information System [ 55 ] (PROMIS; CA: intermediate; LoE: 3) [ 20 ]. Aerobic exercise improved maximum oxygen consumption (VO 2 max) without exacerbation of skin induration, RP or digital ulcers at an 8-week follow-up (CA: intermediate; LoE: 3) [ 17 ].

An RCT by Rannou et al. [ 13 ] (CA: robust; LoE: 2) provided 4 weeks of personalized physical therapy to the experimental group and found an improvement in the HAQ-DI and hand function measured by the Cochin Hand Function Scale (CHFS) [ 56 ] after 4 weeks compared with patients receiving usual care. However, these improvements disappeared after 12 months. Stretching programs for hands improved scores in the Canadian Occupational Performance [ 57 ] after 3 months (CA: intermediate; LoE: 2) [ 12 ] but did not improve Hand Mobility in Scleroderma (HAMIS) test [ 2 ] scores at 9 or 18 weeks, regardless of adjunct treatment with paraffin baths (CA: intermediate; LoE: 2) [ 14 ]. Two RCTs evaluating functional impairment (CA: weak; LoE: 3) showed that app-delivered occupational therapy and stretching exercises administered through a telemedicine system were efficacious in improving hand function [ 25 ] as measured by a shortened version of the Disabilities of the Arm, Shoulder and Hand questionnaire (QuickDASH) [ 58 ] and HAMIS [ 18 ]. A controlled quasi-experimental study (CA: intermediate; LoE: 3) found daily stretching exercises improved the range of motion in each finger in patients with SSc 1 month after baseline, and this improvement was maintained or further increased after 1 year from baseline [ 16 ].

The RCT by Rannou et al. [ 13 ] (CA: robust; LoE: 2) showed that 1 month of personalized physical therapy produced a sustained improvement in oral aperture up to 1 year from baseline. Another RCT (CA: weak; LoE: 3) found 12 weeks of an orofacial exercise protocol improved scores in the Mouth Handicap in Systemic Sclerosis index [ 3 ] up to 20 weeks from baseline [ 24 ]. An uncontrolled quasi-experimental study (CA: robust; LoE: 4) found daily mouth stretching exercises improved oral aperture 18 weeks after baseline.

An RCT evaluating the efficacy of a self-management website (CA: weak; LoE: 3) found no differences compared with issuing an educational patient-focused book when assessing PROMIS (primary outcome) scores at 16 weeks [ 35 ]. A controlled quasi-experimental study evaluating 3 weeks of patient education through occupational therapy (CA: robust; LoE: 3) found improvements in the HAQ-DI up to 24 weeks after baseline [ 34 ].

An RCT evaluating an educational program for self-management (CA: intermediate; LoE: 2) noted improvements in hand-related measures such as the HAMIS, Duruoz Hand Index [ 56 ], HAQ-DI and handgrip strength after 8 weeks [ 32 ]. A controlled quasi-experimental study (CA: intermediate; LoE: 3) found an educational self-management program for hands reduced the pain experienced by patients assessed using a visual analogue scale as well as improved CHFS scores after 24 weeks [ 36 ]. The protocol in this study was based on an uncontrolled study published the year before (CA: robust; LoE: 4), which also found amelioration of pain experienced by patients as well as improvements in CHFS scores after 8 weeks [ 42 ].

An RCT evaluating the effect of patient education with emphasis on orofacial exercises (CA: intermediate; LoE: 2) found that face-to-face training increased oral aperture more than educational material alone at 12 months after baseline in per-protocol analysis [ 30 ]. Another RCT (CA: intermediate; LoE: 2) found an increase in oral aperture after 1 month of orofacial exercise, regardless of whether they received oral hygiene advice before or after [ 31 ]. Two RCTs by Yuen et al. [ 29 , 33 ] examined the effects of oral health interventions, including instruction on dental product use and orofacial exercises. One of these studies [ 33 ] found an increase in oral aperture at 3 months, but not 6 months after baseline (CA: intermediate; LoE: 2), noting low adherence to the exercise program in particular, while the other study (CA: weak; LoE: 3) assessed gingival health and found significant improvements in the Löe–Silness gingival index [ 59 ] in both groups at 6 months after baseline, but a larger improvement in the intervention group compared with controls. Yet another multifaceted oral hygiene intervention was evaluated in an uncontrolled study by Poole et al. [ 39 ] (CA: intermediate; LoE: 4) and incorporated instruction of hand exercises on top of dental hygiene and orofacial exercise instruction. After a 6-month intervention, this study noted improvements in the Patient Hygiene Performance Index (PHP) [ 60 ] after 12 months from baseline, but no improvements in upper extremity measures such as the Keitel Function Test [ 61 ] or oral aperture [ 39 ].

Diet and nutrition

Five selected studies examined the effect of diet and nutrition (CA: four intermediate, one weak). Two RCTs on this topic examining the effects of probiotics found no significant changes in the University of California, Los Angeles Scleroderma Clinical Trial Consortium Gastrointestinal Tract Instrument (GIT-score) [ 62 ] compared with placebo after 60 days (CA: intermediate; LoE: 2) [ 43 ] and 8 weeks (CA: weak; LoE: 3) [ 44 ], respectively. However, the former study found an improvement in the GIT-score reflux component after 120 days [ 43 ] and the in latter study a decrease in Th17 cells after 8 weeks compared with placebo [ 44 ]. Conversely, one uncontrolled quasi-experimental study (CA: intermediate; LoE: 4) found that the use of probiotics associated with a significant reduction in total GIT-score as well as the reflux and bloating/distention component scores after 2 months [ 45 ].

Two quasi-experimental studies evaluated nutritional therapy. One found that nutritional support had no significant improvement in weight, body mass index (BMI) [ 63 ], energy intake or SF-36, with follow-up time points up to 12 months (CA: intermediate; LoE: 4) [ 46 ]. The other study found improvement in the abridged Patient-Generated Subjective Global Assessment [ 64 ] and a reduction in the number of patients classified as sarcopenic by DXA after 18 months (CA: intermediate; LoE: 4) [ 47 ]. The study did not find significant changes in caloric intake or macronutrient distribution in the enrolled patients [ 47 ].

This systematic review of the literature assessed the current evidence for lifestyle interventions as viable management strategies for people living with SSc. The main categories of intervention were physical activity and exercise, mental health and diet and nutrition. Physical exercise in general improved functional impairment and aerobic capacity, while stretching exercises of the hands and mouth efficaciously ameliorated hand impairment and microstomia. Stretching exercises of the hands and mouth were in turn often central components of educational interventions, which in principle focused on different facets of self-management. Studies on diet and nutrition showed sparse efficacy of probiotics in alleviating gastrointestinal symptoms and limited use of nutritional therapy for improving body composition. Overall, there was no rigorous investigation as to how lifestyle affects global disease activity in SSc. Furthermore, none of the included studies aimed to replace pharmacotherapy with lifestyle interventions.

These findings are largely in line with the comprehensive body of evidence compiled in the EULAR recommendations for lifestyle behaviours and work participation [ 6 ], which conclude that physical exercise can be a safe and beneficial way to improve functional impairment. However, factors such as comorbidities and disease severity warrant caution when recommending physical exercise as a part of disease management, which, as always, should be tailored to the patient. Furthermore, the findings in this review also agree that the evidence for recommending specific diets for the management of rheumatic and musculoskeletal diseases is sparse [ 6 ]. Relating to the principal components of lifestyle medicine [ 4 , 5 ], there are gaps in knowledge regarding the effect of sleep, social relations and the use of harmful substances (such as nicotine, tobacco and alcohol) on SSc specifically. For these lifestyle domains, there exist EULAR recommendation sets [ 6 , 7 , 65 ] and other systematic reviews [ 8 , 66 ].

The categorization of interventions was not absolute, as many studies employed a combination of many different intervention categories. For example, studies on nutritional therapy [ 46 , 47 ] consisted of counselling and informative meetings, as in the studies on patient education. Similarly, educational interventions often had exercise programs as a central constituent [ 29–33 , 36 , 39 , 42 ]. The complexity of stratifying studies by category implies a tendency in current research toward examining multimodal approaches when evaluating lifestyle interventions. This may be based on mechanistic reasoning that certain lifestyle interventions should produce a larger effect size when made concurrently but complicates the assessment when trying to discern the efficacy of individual interventions in isolation.

A limitation that we encountered while compiling the evidence was the lack of a structured synthesis or meta-analysis. Moreover, the overall CA was derived from assessment by only one investigator, which potentially reduces the reliability of the RoB assessment. Despite this, there are strengths to this review in the form of a generous inclusion of studies spanning a period of >2 decades with varied study designs and a conservative approach in the CA of studies, treating unclearly fulfilled criteria as unfulfilled.

Considering that most of the studies included in this review were conducted in Europe (particularly Italy) and the USA, caution should be exercised when generalizing the findings to other regions, particularly those with different healthcare systems, demographics and environmental factors. While a focus on Western countries may provide valuable insights into lifestyle interventions in SSc, extrapolating these findings to populations worldwide should be approached with caution.

In conclusion, this systematic review found physical exercise and mental health management to be efficacious lifestyle interventions for improving functional impairment in patients with SSc, which we therefore advocate should be considered for patients suffering from hand or face involvement, reduced muscle function and reduced physical fitness. Importantly, it is worth mentioning that current evidence overall supports lifestyle interventions as a complement and not a substitute to pharmacotherapy. Future studies, preferably of RCT design, are needed for exploring other aspects of lifestyle interventions, namely concerning diet and nutrition, sleep, harmful exposures and social relations, and how these potentially impact the disease course and patient experience, particularly the degree of disease activity.

Supplementary material is available at Rheumatology Advances in Practice online.

The data underlying this article are available in the article and in its online supplementary material .

I.P. and C.B. were responsible for study conception and design, supervision of study selection and data extraction. I.P., C.G., T.S. and C.B. were responsible for the methodology. I.P., A.T., A.G. and J.C. were responsible for study selection and data extraction. A.T. was responsible for risk of bias assessment. I.P. and A.T. were responsible for drafting the manuscript. All authors reviewed and approved the final version of the manuscript and are responsible for its content.

I.P. has received grants from the Swedish Rheumatism Association (R-969696), King Gustaf V’s 80-year Foundation (FAI-2020–0741), Swedish Society of Medicine (SLS-974449), Nyckelfonden (OLL-974804), Professor Nanna Svartz Foundation (2021-00436), Ulla and Roland Gustafsson Foundation (2021-26), Region Stockholm (FoUI-955483) and Karolinska Institutet. C.B. has received grants from the Swedish Rheumatism Association and Norrbacka-Eugeniastiftelsen.

Disclosure statement : I.P. has received research funding and/or honoraria from Amgen, AstraZeneca, Aurinia, Bristol Myers Squibb, Eli Lilly, Gilead, GSK, Janssen, Novartis, Otsuka and Roche. The remaining authors have declared no conflicts of interest. The funders had no role in the design of the study; the collection, analyses or interpretation of data; writing of the manuscript or in the decision to publish the results.

The authors express gratitude to Emma-Lotta Säätelä, librarian at the KI Library, for her help with the search strategy.

Allanore Y , Simms R , Distler O et al. Systemic sclerosis . Nat Rev Dis Primers 2015 ; 1 : 15002 .

Google Scholar

Sandqvist G , Eklund M. Hand Mobility in Scleroderma (HAMIS) test: the reliability of a novel hand function test . Arthritis Care Res 2000 ; 13 : 369 – 74 .

Mouthon L , Rannou F , Bérezné A et al. Development and validation of a scale for mouth handicap in systemic sclerosis: the Mouth Handicap in Systemic Sclerosis scale . Ann Rheum Dis 2007 ; 66 : 1651 – 5 .

American College of Lifestyle Medicine . What is lifestyle medicine? 2022 . https://lifestylemedicine.org/ACLM/ACLM/About/What_is_Lifestyle_Medicine_/Lifestyle_Medicine.aspx?hkey=26f3eb6b-8294-4a63-83de-35d429c3bb88

British Society of Lifestyle Medicine . The pillars of lifestyle medicine. 2022 . https://bslm.org.uk/

Gwinnutt JM , Wieczorek M , Balanescu A et al. 2021 EULAR recommendations regarding lifestyle behaviours and work participation to prevent progression of rheumatic and musculoskeletal diseases . Ann Rheum Dis 2022 ; 82 : 48 – 56 .

Parodis I , Girard-Guyonvarc’h C , Arnaud L et al. EULAR recommendations for the non-pharmacological management of systemic lupus erythematosus and systemic sclerosis . Ann Rheum Dis 2023 ;doi:10.1136/ard-2023-224416.

Parodis I , Gomez A , Tsoi A et al. Systematic literature review informing the EULAR recommendations for the non-pharmacological management of systemic lupus erythematosus and systemic sclerosis . RMD Open 2023 ; 9 : e003297 .

Moher D , Liberati A , Tetzlaff J , Altman DG , PRISMA Group . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement . PLoS Med 2009 ; 6 : e1000097 .

Aromataris E , Munn Z , eds. JBI manual for evidence synthesis. 2020 . https://synthesismanual.jbi.global (1 April 2024, date last accessed).

Howick J , Chalmers I , Glasziou P et al. The Oxford levels of evidence 2. 2011 . https://www.cebm.ox.ac.uk/resources/levels-of-evidence/ocebm-levels-of-evidence (1 April 2024, date last accessed).

Stefanantoni K , Sciarra I , Iannace N et al. Occupational therapy integrated with a self-administered stretching program on systemic sclerosis patients with hand involvement . Clin Exp Rheumatol 2016 ; 34(Suppl 100) : 157 – 61 .

Rannou F , Boutron I , Mouthon L et al. Personalized physical therapy versus usual care for patients with systemic sclerosis: a randomized controlled trial . Arthritis Care Res 2017 ; 69 : 1050 – 9 .

Gregory WJ , Wilkinson J , Herrick AL. A randomised controlled trial of wax baths as an additive therapy to hand exercises in patients with systemic sclerosis . Physiotherapy 2019 ; 105 : 370 – 7 .

Filippetti M , Cazzoletti L , Zamboni F et al. Effect of a tailored home-based exercise program in patients with systemic sclerosis: a randomized controlled trial . Scand J Med Sci Sports 2020 ; 30 : 1675 – 84 .

Mugii N , Hasegawa M , Matsushita T et al. The efficacy of self-administered stretching for finger joint motion in Japanese patients with systemic sclerosis . J Rheumatol 2006 ; 33 : 1586 – 92 .

Oliveira NC , dos Santos Sabbag LM , de Sá Pinto AL , Borges CL , Lima FR. Aerobic exercise is safe and effective in systemic sclerosis . Int J Sports Med 2009 ; 30 : 728 – 32 .

Piga M , Tradori I , Pani D et al. Telemedicine applied to kinesiotherapy for hand dysfunction in patients with systemic sclerosis and rheumatoid arthritis: recovery of movement and telemonitoring technology . J Rheumatol 2014 ; 41 : 1324 – 33 .

Lima TR , Guimaraes FS , Carvalho MN et al. Lower limb muscle strength is associated with functional performance and quality of life in patients with systemic sclerosis . Braz J Phys Ther 2015 ; 19 : 129 – 36 .

Azar M , Rice DB , Kwakkenbos L et al. Exercise habits and factors associated with exercise in systemic sclerosis: a Scleroderma Patient-centered Intervention Network (SPIN) cohort study . Disabil Rehabil 2018 ; 40 : 1997 – 2003 .

Mitropoulos A , Gumber A , Crank H , Akil M , Klonizakis M. The effects of upper and lower limb exercise on the microvascular reactivity in limited cutaneous systemic sclerosis patients . Arthritis Res Ther 2018 ; 20 : 112 .

Mitropoulos A , Gumber A , Akil M , Klonizakis M. Exploring the microcirculatory effects of an exercise programme including aerobic and resistance training in people with limited cutaneous systemic sclerosis . Microvasc Res 2019 ; 125 : 103887 .

Cetin SY , Calik BB , Ayan A. Investigation of the effectiveness of Tai Chi exercise program in patients with scleroderma: a randomized controlled study . Complement Ther Clin Pract 2020 ; 40 : 101181 .

Maddali Bongi S , Passalacqua M , Landi G et al. Rehabilitation of the face and temporomandibular joint in systemic sclerosis . Ther Adv Musculoskelet Dis 2021 ; 13 : 1759720X211020171 .

Murphy SL , Barber M , Huang S et al. Intensive and app-delivered occupational therapy to improve upper extremity function in early diffuse cutaneous systemic sclerosis: a pilot two-arm trial . Rheumatology (Oxford) 2021 ; 60 : 5002 – 11 .

Pizzo G , Scardina GA , Messina P. Effects of a nonsurgical exercise program on the decreased mouth opening in patients with systemic scleroderma . Clin Oral Invest 2003 ; 7 : 175 – 8 .

Pinto AL , Oliveira NC , Gualano B et al. Efficacy and safety of concurrent training in systemic sclerosis . J Strength Cond Res 2011 ; 25 : 1423 – 8 .

Defi IR , Gultom C , Chorman MJ , Jennie J. High-intensity interval training can improve hand grip strength, inspiratory muscle, and quality of life in systemic sclerosis subjects . Reumatologia 2021 ; 59 : 98 – 103 .

Yuen HK , Marlow NM , Reed SG et al. Effect of orofacial exercises on oral aperture in adults with systemic sclerosis . Disabil Rehabil 2012 ; 34 : 84 – 9 .

Uras C , Mastroeni S , Tabolli S et al. A comparison between two educational methods in the rehabilitation of the microstomia in systemic sclerosis: a randomized controlled trial . Clin Rehabil 2019 ; 33 : 1747 – 56 .

Cüzdan N , Türk İ , Çiftçi V et al. The effect of a home-based orofacial exercise program on oral aperture of patients with systemic sclerosis: a single-blind prospective randomized controlled trial . Arch Rheumatol 2021 ; 36 : 176 – 84 .

Gokcen N , Badak SO , Sarpel T , Sertdemir Y , Erken E. The efficacy of a home-based, self-administered hand exercise program for patients with systemic sclerosis: a randomized controlled, evaluator-blind, clinical trial . J Clin Rheumatol 2022 ; 28 : e422–9 .

Yuen HK , Weng Y , Bandyopadhyay D et al. Effect of a multi-faceted intervention on gingival health among adults with systemic sclerosis . Clin Exp Rheumatol 2011 ; 29 (2 Suppl 65): S26 – 32 .

Zanatta E , Rodeghiero F , Pigatto E et al. Long-term improvement in activities of daily living in women with systemic sclerosis attending occupational therapy . Br J Occup Ther 2017 ; 80 : 417 – 22 .

Khanna D , Serrano J , Berrocal VJ et al. Randomized controlled trial to evaluate an internet-based self-management program in systemic sclerosis . Arthritis Care Res 2019 ; 71 : 435 – 47 .

Landim S , Bertolo M , Del Rio A et al. Sustained efficacy of a concise self-management programme for hands in systemic sclerosis: a longitudinal case-control observational study . Rheumatology (Oxford) 2020 ; 59 : 3330 – 9 .

Samuelson UK , Ahlmen EM. Development and evaluation of a patient education program for persons with systemic sclerosis (scleroderma) . Arthritis Care Res 2000 ; 13 : 141 – 8 .

Brown SJ , Somerset ME , McCabe CS , McHugh NJ. The impact of group education on participants’ management of their disease in lupus and scleroderma . Musculoskelet Care 2004 ; 2 : 207 – 17 .

Poole J , Conte C , Brewer C et al. Oral hygiene in scleroderma: the effectiveness of a multi-disciplinary intervention program . Disabil Rehabil 2010 ; 32 : 379 – 84 .

Poole JL , Skipper B , Mendelson C. Evaluation of a mail-delivered, print-format, self-management program for persons with systemic sclerosis . Clin Rheumatol 2013 ; 32 : 1393 – 8 .

Poole JL , Mendelson C , Skipper B , Khanna D. Taking charge of systemic sclerosis: a pilot study to assess the effectiveness of an internet self-management program . Arthritis Care Res 2014 ; 66 : 778 – 82 .

Landim SF , Bertolo MB , Marcatto de Abreu MF et al. The evaluation of a home-based program for hands in patients with systemic sclerosis . J Hand Ther 2019 ; 32 : 313 – 21 .

Low AHL , Teng GG , Pettersson S et al. A double-blind randomized placebo-controlled trial of probiotics in systemic sclerosis associated gastrointestinal disease . Semin Arthritis Rheum 2019 ; 49 : 411 – 9 .

Marighela TF , Arismendi MI , Marvulle V et al. Effect of probiotics on gastrointestinal symptoms and immune parameters in systemic sclerosis: a randomized placebo-controlled trial . Rheumatology (Oxford) 2019 ; 58 : 1985 – 90 .

Frech TM , Khanna D , Maranian P et al. Probiotics for the treatment of systemic sclerosis-associated gastrointestinal bloating/distention . Clin Exp Rheumatol 2011 ; 29 (2 Suppl 65): S22 – 5 .

Ortiz-Santamaria V , Puig C , Soldevillla C et al. Nutritional support in patients with systemic sclerosis . Reumatol Clin 2014 ; 10 : 283 – 7 .

Doerfler B , Allen TS , Southwood C et al. Medical Nutrition Therapy for Patients With Advanced Systemic Sclerosis (MNT PASS): a pilot intervention study . J Parenter Enteral Nutr 2017 ; 41 : 678 – 84 .

Guyatt GH , Sullivan MJ , Thompson PJ et al. The 6-minute walk: a new measure of exercise capacity in patients with chronic heart failure . Can Med Assoc J 1985 ; 132 : 919 – 23 .

Ware JE Jr , Sherbourne CD. The MOS 36-item Short-Form Health Survey (SF-36). I. Conceptual framework and item selection . Med Care 1992 ; 30 : 473 – 83 .

Fries JF , Spitz P , Kraines RG , Holman HR. Measurement of patient outcome in arthritis . Arthritis Rheum 1980 ; 23 : 137 – 45 .

Berg KO , Wood-Dauphinee SL , Williams JI , Maki B. Measuring balance in the elderly: validation of an instrument . Can J Public Health 1992 ; 83(Suppl 2) : S7 – 11 .

Buysse DJ , Reynolds CF 3rd , Monk TH , Berman SR , Kupfer DJ. The Pittsburgh Sleep Quality Index: a new instrument for psychiatric practice and research . Psychiatry Res 1989 ; 28 : 193 – 213 .

Krupp LB , LaRocca NG , Muir-Nash J , Steinberg AD. The Fatigue Severity Scale. Application to patients with multiple sclerosis and systemic lupus erythematosus . Arch Neurol 1989 ; 46 : 1121 – 3 .

McGill SM , Childs A , Liebenson C. Endurance times for low back stabilization exercises: clinical targets for testing and training from a normal database . Arch Phys Med Rehabil 1999 ; 80 : 941 – 4 .

Cella D , Yount S , Rothrock N et al. The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap Cooperative Group during its first two years . Med Care 2007 ; 45 (5 Suppl 1): S3 – 11 .

Duruöz MT , Poiraudeau S , Fermanian J et al. Development and validation of a rheumatoid hand functional disability scale that assesses functional handicap . J Rheumatol 1996 ; 23 : 1167 – 72 .

Law M , Baptiste S , McColl M et al. The Canadian occupational performance measure: an outcome measure for occupational therapy . Can J Occup Ther 1990 ; 57 : 82 – 7 .

Gummesson C , Ward MM , Atroshi I. The shortened disabilities of the arm, shoulder and hand questionnaire (QuickDASH): validity and reliability based on responses within the full-length DASH . BMC Musculoskelet Disord 2006 ; 7 : 44 .

Löe H , Silness J. Periodontal disease in pregnancy. I. Prevalence and severity . Acta Odontol Scand 1963 ; 21 : 533 – 51 .

Podshadley AG , Haley JV. A method for evaluating oral hygiene performance . Public Health Rep 1968 ; 83 : 259 – 64 .

Keitel W , Hoffmann H , Weber G , Krieger U. [Evaluation of the percentage of functional decrease of the joints using a motor function test in rheumatology] . Dtsch Gesundheitsw 1971 ; 26 : 1901 – 3 .

Khanna D , Hays RD , Maranian P et al. Reliability and validity of the University of California, Los Angeles Scleroderma Clinical Trial Consortium Gastrointestinal Tract Instrument . Arthritis Rheum 2009 ; 61 : 1257 – 63 .

Keys A , Fidanza F , Karvonen MJ , Kimura N , Taylor HL. Indices of relative weight and obesity . J Chronic Dis 1972 ; 25 : 329 – 43 .

Vigano AL , di Tomasso J , Kilgour RD et al. The abridged patient-generated subjective global assessment is a useful tool for early detection and characterization of cancer cachexia . J Acad Nutr Diet 2014 ; 114 : 1088 – 98 .

Rausch Osthoff AK , Niedermann K , Braun J et al. 2018 EULAR recommendations for physical activity in people with inflammatory arthritis and osteoarthritis . Ann Rheum Dis 2018 ; 77 : 1251 – 60 .

Wieczorek M , Gwinnutt JM , Ransay-Colle M et al. Smoking, alcohol consumption and disease-specific outcomes in rheumatic and musculoskeletal diseases (RMDs): systematic reviews informing the 2021 EULAR recommendations for lifestyle improvements in people with RMDs . RMD Open 2022 ; 8 : e002170 .

Supplementary data

Email alerts.

Rheumatology Twitter
BSR Twitter
BSR Facebook
Advertising and Corporate Services

Affiliations

Online ISSN 2514-1775
Copyright © 2024 British Society for Rheumatology
About Oxford Academic
Publish journals with us
University press partners
What we publish
New features
Open access
Institutional account management
Rights and permissions
Get help with access
Accessibility
Advertising
Media enquiries
Oxford University Press
Oxford Languages
University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

Copyright © 2024 Oxford University Press
Cookie settings
Cookie policy
Privacy policy
Legal notice

This Feature Is Available To Subscribers Only

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Influence of e-learning on the students’ of higher education in the digital era: A systematic literature review

Published: 16 April 2024

Cite this article

Rashmi Singh ORCID: orcid.org/0000-0001-9195-5301 1 ,
Shailendra Kumar Singh 1 &
Niraj Mishra 1

The integration of digital technologies into educational practices has reshaped traditional learning models, creating a dynamic and accessible global landscape for higher education. This paradigm shift transcends geographical boundaries, fostering a more interconnected and inclusive educational environment. This comprehensive literature analysis explores the impact of e-learning on higher education students in the digital era. A meticulous review of 53 studies, sourced from reputable databases including Web of Science, Taylor & Francis, Springer Link, ProQuest, Elsevier, and Scopus, was conducted. Employing the content analysis method, the selected studies spanning from November 2012 to April 2023 were systematically examined. Predominantly utilizing quantitative methods, the studies, largely originating from the United States, China, Malaysia, and India, focused on university students. Key variables such as student engagement, perception, and academic performance were consistently employed across diverse educational settings. The synthesis of findings revealed that e-learning technologies positively impacted academic achievement, student satisfaction, and collaborative efforts. Moreover, challenges associated with technology usage and internet access were identified, which impact e-learning implementation. The study proposes further investigation through a mixed-methods approach to explore students’ interactions with the educational environment while utilizing e-learning technology in institutions of higher education.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Data availability

Not applicable.

Abdullah, S. I. N. W., Arokiyasamy, K., Goh, S. L., Culas, A. J., & Manaf, N. M. A. (2022). University students’ satisfaction and future outlook towards forced remote learning during a global pandemic. Smart Learning Environments , 9 (1). https://doi.org/10.1186/s40561-022-00197-8 .

Abu Talib, M., Bettayeb, A. M., & Omer, R. I. (2021). Analytical study on the impact of technology in higher education during the age of COVID-19: Systematic literature review. Education and Information Technologies , 26 (6), 6719–6746. https://doi.org/10.1007/s10639-021-10507-1 .

Article Google Scholar

Abuhassna, H., & Alnawajha, S. (2023a). The Transactional Distance Theory and Distance Learning Contexts: Theory Integration, Research Gaps, and Future Agenda. In Education Sciences (Vol. 13, Issue 2). MDPI. https://doi.org/10.3390/educsci13020112 .

Abuhassna, H., & Alnawajha, S. (2023b). Instructional Design Made Easy! Instructional Design Models, Categories, Frameworks, Educational Context, and Recommendations for Future Work. In European Journal of Investigation in Health, Psychology and Education (Vol. 13, Issue 4, pp. 715–735). MDPI. https://doi.org/10.3390/ejihpe13040054 .

Abuhassna, H., Yahaya, N., Zakaria, M. A. Z. M., Zaid, N. M., Samah, N. A., Awae, F., Nee, C. K., & Alsharif, A. H. (2023). Trends on using the Technology Acceptance Model (TAM) for online learning: A bibliometric and content analysis. International Journal of Information and Education Technology , 13 (1), 131–142. https://doi.org/10.18178/ijiet.2023.13.1.1788 .

Adarkwah, M. A. (2020). I ’ m not against online teaching, but what about us ? ICT in Ghana post Covid-19. 2.

Adeshola, I., & Agoyi, M. (2022). Examining factors influencing e-learning engagement among university students during covid-19 pandemic: A mediating role of learning persistence. Interactive Learning Environments . https://doi.org/10.1080/10494820.2022.2029493 .

Aharony, N., & Gazit, T. (2020). Students’ information literacy self-efficacy: An exploratory study. Journal of Librarianship and Information Science , 52 (1), 224–236. https://doi.org/10.1177/0961000618790312 .

Al-Nimer, M., & Alsheikh, G. (2022). Unleashing the role of e-learning in student engagement practices and accounting professional competencies. Journal of Applied Research in Higher Education , 14 No (2), 829–851. https://doi.org/10.1108/JARHE-10-2020-0377 .

Al-Rahmi, W. M., Alias, N., Othman, M. S., Marin, V. I., & Tur, G. (2018). A model of factors affecting learning performance through the use of social media in Malaysian higher education. Computers and Education , 121 , 59–72. https://doi.org/10.1016/j.compedu.2018.02.010 .

Al-Teete, R., Hassan, I. I., Kadir, A., A., & AbuAlRub, R. (2023). Nursing lecturers’ perception toward E-learning approaches used in nursing colleges: Scoping review. In Journal of Professional Nursing (Vol. 46, pp. 102–110). W.B. Saunders. https://doi.org/10.1016/j.profnurs.2023.03.001 .

Alajmi, Q., Al-Sharafi, M. A., Alajmi, Q., & Abuali, A. (2020). Smart Learning Gateways for Omani HEIs Towards Educational Technology: Benefits, Challenges and solutions Cloud computing among HEIs in Oman View project 2nd International Conference on Emerging Technologies and Intelligent Systems (ICETIS) View project Smart Learning Gateways for Omani HEIs Towards Educational Technology: Benefits, Challenges and solutions. In International Journal of Information Technology and Language Studies (IJITLS) (Vol. 4, Issue 1). http://journals.sfu.ca/ijitls .

Alvi, I. (2021). College students’ reception of social networking tools for learning in India: An extended UTAUT model. Smart Learning Environments , 8 (1). https://doi.org/10.1186/s40561-021-00164-9 .

Alzahrani, N. M. (2020). Augmented reality: A systematic review of its benefits and challenges in e-learning contexts. In Applied Sciences (Switzerland) (Vol. 10, Issue 16). MDPI AG. https://doi.org/10.3390/app10165660 .

Ansari, J. A. N., & Khan, N. A. (2020). Exploring the role of social media in collaborative learning the new domain of learning. Smart Learning Environments , 7 (1). https://doi.org/10.1186/s40561-020-00118-7 .

Birla Institue of Technology, (Dept of Management).

Bizami, N. A., Tasir, Z., & Kew, S. N. (2022). Innovative pedagogical principles and technological tools capabilities for immersive blended learning: A systematic literature review. Education and Information Technologies . https://doi.org/10.1007/s10639-022-11243-w .

Bruguera, C., Guitert, M., & Romeu, T. (2022). Social media in the learning ecologies of communications students: Identifying profiles from students’ perspective. Education and Information Technologies , 27 (9), 13113–13129. https://doi.org/10.1007/s10639-022-11169-3 .

Cela, K. L., Sicilia, M., & Sánchez, S. (2015). Social Network Analysis in E-Learning Environments: A Preliminary Systematic Review. In Educational Psychology Review (Vol. 27, Issue 1, pp. 219–246). Springer New York LLC. https://doi.org/10.1007/s10648-014-9276-0 .

Chawla, D., & Joshi, H. (2012). Management education through e-learning in India: An empirical study. Campus-Wide Information Systems , 29 (5), 380–393. https://doi.org/10.1108/10650741211275134 .

Clark, M. (2017). Imposed-inquiry information-seeking self-efficacy and performance of College students: A review of the literature. Journal of Academic Librarianship , 43 (5), 417–422. https://doi.org/10.1016/j.acalib.2017.05.001 .

Cui, Y., Ma, Z., Wang, L., Yang, A., Liu, Q., Kong, S., & Wang, H. (2023). A survey on big data-enabled innovative online education systems during the COVID-19 pandemic. Journal of Innovation and Knowledge , 8 (1). https://doi.org/10.1016/j.jik.2022.100295 .

Damnjanovic, V., Jednak, S., & Mijatovic, I. (2015). Factors affecting the effectiveness and use of Moodle: Students’ perception. Interactive Learning Environments , 23 (4), 496–514. https://doi.org/10.1080/10494820.2013.789062 .

de Goulão, M. F., & Menedez, R. C. (2015). Learner autonomy and self-regulation in eLearning. Procedia - Social and Behavioral Sciences , 174 , 1900–1907. https://doi.org/10.1016/j.sbspro.2015.01.853 .

Eze, S. C., Chinedu-Eze, V. C., & Bello, A. O. (2018). The utilisation of e-learning facilities in the educational delivery system of Nigeria: A study of M-University. International Journal of Educational Technology in Higher Education , 15 (1). https://doi.org/10.1186/s41239-018-0116-z .

Fehrmann, P., Foster, M., Fowler, S. A., Glanville, J., & Young, S. (2021). PRISMA-S: An extension to the PRISMA Statement for reporting literature searches in systematic reviews. Systematic Reviews , 10 (1). https://doi.org/10.1186/s13643-020-01542-z .

Ferrer, J., Ringer, A., Saville, K., Parris, A., M., & Kashi, K. (2022). Students’ motivation and engagement in higher education: The importance of attitude to online learning. Higher Education , 83 (2), 317–338. https://doi.org/10.1007/s10734-020-00657-5 .

Gikas, J., & Grant, M. M. (2013). Mobile computing devices in higher education: Student perspectives on learning with cellphones, smartphones & social media. Internet and Higher Education , 19 , 18–26. https://doi.org/10.1016/j.iheduc.2013.06.002 .

Giray, G. (2021). An assessment of student satisfaction with e-learning: An empirical study with computer and software engineering undergraduate students in Turkey under pandemic conditions. Education and Information Technologies , 26 (6), 6651–6673. https://doi.org/10.1007/s10639-021-10454-x .

Han, H., Røkenes, F. M., & Krumsvik, R. J. (2023). Student teachers’ perceptions of flipped classroom in EFL teacher education. Education and Information Technologies . https://doi.org/10.1007/s10639-023-11839-w .

Hanson, J. (2016). Surveying the experiences and perceptions of undergraduate nursing students of a flipped classroom approach to increase understanding of drug science and its application to clinical practice. Nurse Education in Practice , 16 (1), 79–85. https://doi.org/10.1016/j.nepr.2015.09.001 .

Horng, J. S., Liu, C. H., Chou, S. F., Yu, T. Y., Fang, Y. P., & Huang, Y. C. (2022). Student’s perceptions of sharing platforms and digital learning for sustainable behaviour and value changes. Journal of Hospitality Leisure Sport and Tourism Education , 31 , https://doi.org/10.1016/j.jhlste.2022.100380 .

Hossain, S. F. A., Xi, Z., Nurunnabi, M., & Anwar, B. (2022). Sustainable academic performance in higher education: A mixed method approach. Interactive Learning Environments , 30 (4), 707–720. https://doi.org/10.1080/10494820.2019.1680392 .

Influence of e-learning on The students’ of higher education in the digital era: A systematic literature review.

Jarrah, H. Y. (2023). Global teaching style adaptations and curriculum adjustments for school students during COVID-19. Interactive Learning Environments , 1–13. https://doi.org/10.1080/10494820.2023.2186899 .

Kaliisa, R., Rienties, B., Mørch, A. I., & Kluge, A. (2022). Social learning analytics in computer-supported collaborative learning environments: A systematic review of empirical studies. Computers and Education Open , 3 , 100073. https://doi.org/10.1016/j.caeo.2022.100073 .

Kang, H. S., & Pak, Y. (2023). Student Engagement in Online Graduate Program in Education: A mixed-methods study. American Journal of Distance Education . https://doi.org/10.1080/08923647.2023.2175560 .

Kim, H. J., Hong, A. J., & Song, H. D. (2019). The roles of academic engagement and digital readiness in students’ achievements in university e-learning environments. International Journal of Educational Technology in Higher Education , 16 (1). https://doi.org/10.1186/s41239-019-0152-3 .

Koay, K. Y., & Poon, W. C. (2022). Understanding Students’ Cyberslacking Behaviour in e-Learning Environments: Is Student Engagement the Key? https://doi.org/10.1080/10447318.2022.2080154

Koh, J. H. L., & Kan, R. Y. P. (2021). Students’ use of learning management systems and desired e-learning experiences: Are they ready for next generation digital learning environments? Higher Education Research and Development , 40 (5), 995–1010. https://doi.org/10.1080/07294360.2020.1799949 .

Kowalczyk, N. & Truluck, C. (2013). Literature reviews and systematic reviews: What is the difference? Radiologic Technology, 85 , 219–222.

Lai, Y., Saab, N., & Admiraal, W. (2022). University students’ use of mobile technology in self-directed language learning: Using the integrative model of behavior prediction. Computers and Education , 179 , https://doi.org/10.1016/j.compedu.2021.104413 .

Lau, K. H., Lam, T., Kam, B. H., Nkhoma, M., Richardson, J., & Thomas, S. (2018). The role of textbook learning resources in e-learning: A taxonomic study. Computers and Education , 118 , 10–24. https://doi.org/10.1016/j.compedu.2017.11.005 .

Lee, J., Song, H. D., & Hong, A. J. (2019). Exploring factors, and indicators for measuring students’ sustainable engagement in e-learning. Sustainability (Switzerland) , 11 (4). https://doi.org/10.3390/su11040985 .

Lin, Y. N., Hsia, L. H., & Hwang, G. J. (2021). Promoting pre-class guidance and in-class reflection: A SQIRC-based mobile flipped learning approach to promoting students’ billiards skills, strategies, motivation and self-efficacy. Computers and Education , 160 , https://doi.org/10.1016/j.compedu.2020.104035 .

Mamedova, L., Rukovich, A., Likhouzova, T., & Vorona-Slivinskaya, L. (2023). Online education of engineering students: Educational platforms and their influence on the level of academic performance. Education and Information Technologies . https://doi.org/10.1007/s10639-023-11822-5 .

Mesra, Ranchi, (Jharkhand), India.

Müller, W., & Leyer, M. (2023). Understanding intention and use of digital elements in higher education teaching. Education and Information Technologies . https://doi.org/10.1007/s10639-023-11798-2 .

Neier, S., & Zayer, L. T. (2015). Students’ perceptions and experiences of Social Media in Higher Education. Journal of Marketing Education , 37 (3), 133–143. https://doi.org/10.1177/0273475315583748 .

Noorbhai, H., & Ojo, T. A. (2023). mHealth and e-Learning in health sciences curricula: A South African study of health sciences staff perspectives on utilisation, constraints and future possibilities. BMC Medical Education , 23 (1), 189. https://doi.org/10.1186/s12909-023-04132-4 .

Ouyang, F., Wu, M., Zhang, L., Xu, W., Zheng, L., & Cukurova, M. (2023). Making strides towards AI-supported regulation of learning in collaborative knowledge construction. Computers in Human Behavior , 142. https://doi.org/10.1016/j.chb.2023.107650 .

Palvia, S., Aeron, P., Gupta, P., Mahapatra, D., Parida, R., Rosner, R., & Sindhi, S. (2018). Online Education: Worldwide Status, challenges, trends, and implications. Journal of Global Information Technology Management (Vol , 21 (4), 233–241. https://doi.org/10.1080/1097198X.2018.1542262 . Taylor and Francis Inc.

Peng, Y., Wang, Y., & Hu, J. (2023). Examining ICT attitudes, use and support in blended learning settings for students’ reading performance: Approaches of artificial intelligence and multilevel model. Computers and Education , 203. https://doi.org/10.1016/j.compedu.2023.104846 .

Pospíšilová, L., & Rohlíková, L. (2023). Reforming higher education with ePortfolio implementation, enhanced by learning analytics. Computers in Human Behavior , 138 , https://doi.org/10.1016/j.chb.2022.107449 .

Rawashdeh, A. (2021). Advantages and Disadvantages of Using e-Learning in University Education: Analyzing Students’ Perspectives. 19(2), 107–117. www.ejel.org.

Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., Blunt, H., Brigham, T., Chang, S., Clark, J., Conway, A., Couban, R., de Kock, S., Farrah, K., Fehrmann, P., Foster, M., Fowler, S. A., Glanville, J., & Young, S. (2021). PRISMA-S: An extension to the PRISMA Statement for reporting literature searches in systematic reviews. Systematic Reviews , 10 (1). https://doi.org/10.1186/s13643-020-01542-z .

Robinson P, Daly J. (2014). Producing the evidence we need and validating the evidence we have. Journal of Applied Arts and Health, 5 (2), 245–53.

Samsul, S. A., Yahaya, N., & Abuhassna, H. (2023). Education big data and learning analytics: A bibliometric analysis. Humanities and Social Sciences Communications , 10 (1). https://doi.org/10.1057/s41599-023-02176-x .

Sarwar, B., Zulfiqar, S., Aziz, S., & Ejaz Chandia, K. (2019). Usage of Social Media Tools for collaborative learning: The Effect on Learning Success with the moderating role of Cyberbullying. Journal of Educational Computing Research , 57 (1), 246–279. https://doi.org/10.1177/0735633117748415 .

Scott, S. G., & Lane, V. R. (2000). A stakeholder approach to organizational identity. The Academy of Management Review, 25 (1), 43–62. https://doi.org/10.2307/259262

Sewandono, R. E., Thoyib, A., Hadiwidjojo, D., & Rofiq, A. (2022). Performance expectancy of E-learning on higher institutions of education under uncertain conditions: Indonesia context. Education and Information Technologies . https://doi.org/10.1007/s10639-022-11074-9 .

Shafiq, M., & Parveen, K. (2023). Social media usage: Analyzing its effect on academic performance and engagement of higher education students. International Journal of Educational Development , 98 . https://doi.org/10.1016/j.ijedudev.2023.102738 .

Sobaih, A. E. E., Hasanein, A., & Elshaer, I. A. (2022). Higher education in and after COVID-19: The impact of using Social Network Applications for E-Learning on students’ academic performance. Sustainability (Switzerland) , 14 (9). https://doi.org/10.3390/su14095195 .

Söderlund, A., Blazeviciene, A., Elvén, M., Vaskelyte, A., Strods, R., Blese, I., Paakkonen, H., Fernandes, A., Cardoso, D., Kav, S., Baskici, C., & Wiktsröm-Grotell, C. (2023). Exploring the activities and outcomes of digital teaching and learning of practical skills in higher education for the social and health care professions: A scoping review. Discover Education , 2 (1). https://doi.org/10.1007/s44217-022-00022-x .

Tandon, U., Mittal, A., Bhandari, H., & Bansal, K. (2022). E-learning adoption by undergraduate architecture students: Facilitators and inhibitors. Engineering Construction and Architectural Management , 29 (10), 4287–4312. https://doi.org/10.1108/ECAM-05-2021-0376 .

Taucean, I. M., & Tamasila, M. (2014). Research challenges for eLearning support in Engineering and Management Training. Procedia - Social and Behavioral Sciences , 124 , 210–218. https://doi.org/10.1016/j.sbspro.2014.02.479 .

Thathsarani, H., Ariyananda, D. K., Jayakody, C., Manoharan, K., Munasinghe, A. A. S. N., & Rathnayake, N. (2023). How successful the online assessment techniques in distance learning have been, in contributing to academic achievements of management undergraduates? Education and Information Technologies . https://doi.org/10.1007/s10639-023-11715-7 .

Turan, Z., & Karabey, S. C. (2023). The use of immersive technologies in distance education: A systematic review. Education and Information Technologies . https://doi.org/10.1007/s10639-023-11849-8 .

Turan, Z., Kucuk, S., & Cilligol Karabey, S. (2022). The university students’ self-regulated effort, flexibility and satisfaction in distance education. International Journal of Educational Technology in Higher Education , 19 (1). https://doi.org/10.1186/s41239-022-00342-w .

Wang, Q., Wen, Y., & Quek, C. L. (2022a). Engaging learners in synchronous online learning. Education and Information Technologies . https://doi.org/10.1007/s10639-022-11393-x .

Wang, Y. M., Wei, C. L., Lin, H. H., Wang, S. C., & Wang, Y. S. (2022b). What drives students’ AI learning behavior: A perspective of AI anxiety. Interactive Learning Environments . https://doi.org/10.1080/10494820.2022.2153147 .

Wang, S., Sun, Z., & Chen, Y. (2023). Effects of higher education institutes’ artificial intelligence capability on students’ self-efficacy, creativity and learning. Education and Information Technologies , 28 (5), 4919–4939. https://doi.org/10.1007/s10639-022-11338-4 .

Watt-Watson, J., McGillion, M., Lax, L., Oskarsson, J., Hunter, J., MacLennan, C., Knickle, K., & Victor, C., J (2019). Evaluating an innovative elearning pain education interprofessional resource: A pre-post study. Pain Medicine (United States) , 20 (1), 37–49. https://doi.org/10.1093/pm/pny105 .

Wei, H. C., & Chou, C. (2020). Online learning performance and satisfaction: Do perceptions and readiness matter? Distance Education , 41 (1), 48–69. https://doi.org/10.1080/01587919.2020.1724768 .

Xiao, M., Tian, Z., & Xu, W. (2023). Impact of teacher-student interaction on students’ classroom well-being under online education environment. Education and Information Technologies . https://doi.org/10.1007/s10639-023-11681-0 .

Yavuzalp, N., & Bahcivan, E. (2021). A structural equation modeling analysis of relationships among university students’ readiness for e-learning, self-regulation skills, satisfaction, and academic achievement. Research and Practice in Technology Enhanced Learning , 16 (1). https://doi.org/10.1186/s41039-021-00162-y .

Zaidi, S. F. H., Kulakli, A., Osmanaj, V., & Zaidi, S. A. H. (2023). Students’ perceived M-Learning quality: An evaluation and directions to improve the quality for H-Learning. Education Sciences , 13 (6), 578. https://doi.org/10.3390/educsci13060578 .

Download references

Author information

Authors and affiliations.

Department of Management, Birla Institue of Technology, Ranchi, Jharkhand, India

Rashmi Singh, Shailendra Kumar Singh & Niraj Mishra

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rashmi Singh .

Ethics declarations

Conflict of interest.

There is no conflict of interest. The contributors declare that they do not have any kind of financial interest in the subject content or information covered in this work.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Singh, R., Singh, S.K. & Mishra, N. Influence of e-learning on the students’ of higher education in the digital era: A systematic literature review. Educ Inf Technol (2024). https://doi.org/10.1007/s10639-024-12604-3

Download citation

Received : 10 August 2023

Accepted : 26 February 2024

Published : 16 April 2024

DOI : https://doi.org/10.1007/s10639-024-12604-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Student engagement
Higher education
Teaching and learning
Find a journal
Publish with us
Track your research

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Machine learning in internet financial risk management: A systematic literature review

Roles Conceptualization, Data curation, Methodology, Software, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliations Science and Technology Finance Key Laboratory of Hebei Province, Hebei Finance University, Baoding, Hebei, China, Faculty of Management, Universiti Teknologi Malaysia, Johor Baru, Malaysia, Faculty of Management, Hebei Finance University, Baoding, Hebei, China

Roles Project administration, Resources, Validation, Visualization, Writing – review & editing

Affiliation BoHai College, Hebei Agricultural University, Cangzhou, Hebei, China

Roles Methodology, Supervision, Writing – review & editing

Affiliation Faculty of Management, Universiti Teknologi Malaysia, Johor Baru, Malaysia

Roles Funding acquisition, Writing – review & editing

Affiliation Faculty of Management, Hebei Finance University, Baoding, Hebei, China

Xu Tian,
ZongYi Tian,
Saleh F. A. Khatib,

Published: April 16, 2024
https://doi.org/10.1371/journal.pone.0300195
Reader Comments

Internet finance has permeated into myriad households, bringing about lifestyle convenience alongside potential risks. Presently, internet finance enterprises are progressively adopting machine learning and other artificial intelligence methods for risk alertness. What is the current status of the application of various machine learning models and algorithms across different institutions? Is there an optimal machine learning algorithm suited for the majority of internet finance platforms and application scenarios? Scholars have embarked on a series of studies addressing these questions; however, the focus predominantly lies in comparing different algorithms within specific platforms and contexts, lacking a comprehensive discourse and summary on the utilization of machine learning in this domain. Thus, based on the data from Web of Science and Scopus databases, this paper conducts a systematic literature review on all aspects of machine learning in internet finance risk in recent years, based on publications trends, geographical distribution, literature focus, machine learning models and algorithms, and evaluations. The research reveals that machine learning, as a nascent technology, whether through basic algorithms or intricate algorithmic combinations, has made significant strides compared to traditional credit scoring methods in predicting accuracy, time efficiency, and robustness in internet finance risk management. Nonetheless, there exist noticeable disparities among different algorithms, and factors such as model structure, sample data, and parameter settings also influence prediction accuracy, although generally, updated algorithms tend to achieve higher accuracy. Consequently, there is no one-size-fits-all approach applicable to all platforms; each platform should enhance its machine learning models and algorithms based on its unique characteristics, data, and the development of AI technology, starting from key evaluation indicators to mitigate internet finance risks.

Citation: Tian X, Tian Z, Khatib SFA, Wang Y (2024) Machine learning in internet financial risk management: A systematic literature review. PLoS ONE 19(4): e0300195. https://doi.org/10.1371/journal.pone.0300195

Editor: Muhammad Usman Tariq, Abu Dhabi University, UNITED ARAB EMIRATES

Received: November 8, 2023; Accepted: February 22, 2024; Published: April 16, 2024

Copyright: © 2024 Tian et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the manuscript and its Supporting information files.

Funding: Hebei Social Science Fund (HB22YJ026); Open Fund Project of Science and Technology Finance Key Laboratory of Hebei Province (STFCIC202102;STFCIC202213); S&T Program of Hebei (22567630H); Baoding Science and Technology Bureau science and technology plan soft science project (2340ZZ013). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors declare no conflict of interest.

1. Introduction

With the rapid development of internet technology and the arrival of the intelligent era, traditional financial enterprises have gradually expanded their online business operations and are embracing the new format of internet financial services along with the internet financial platform companies that have emerged since 2012 [ 1 ]. Internet finance has rapidly developed due to its convenience, real-time nature, and no geographical limitations, resulting in the expansion of market size, number of participants, and services or products offered [ 2 ], However, it also faces significant risks, as evidenced by the large number of problems with P2P platforms in 2018. Compared with traditional financial services, internet finance has relatively low barriers to entry, smaller amounts, faster speeds, and more relaxed audits, which has led to higher requirements for credit risk control, fraud prediction, and other risk prevention measures in internet financial platforms [ 3 , 4 ]. Research related to risk identification, risk alert, and risk supervision based on big data [ 5 – 8 ], blockchain [ 9 , 10 ], artificial intelligence [ 4 , 11 , 12 ] and machine learning algorithms [ 1 , 2 ] is progressively unfolding.

The internet finance refers to a business model wherein traditional financial institutions or internet companies utilize internet technology to provide financial-related services such as financing, payment, investment, and information intermediation on the internet [ 13 ]. Over a span of two years starting from 2016, more than 200 internet finance companies in China alone faced instances of default, involving issues like borrower delinquency, platform fraud, and cyberattacks [ 2 ]. Only in 2018, the thriving P2P internet finance platforms in China plummeted from 6385 to 1595 by August, resulting in significant losses for investors [ 14 ]. Internet financial services offer rapid response times, thereby enhancing user satisfaction. Consequently, swift identification of potential risks is crucial [ 15 , 16 ]. Considering that the internet will remain a pivotal direction for the development of the financial industry for the foreseeable future, with an increasing number of services offered by major financial institutions, such as banks, being conducted through online channels, this paper focuses on the issue of financial risk prevention in the internet domain. Research on internet financial risk warning can effectively nip potential risks in the bud [ 17 ], as traditional credit scoring card models can no longer cater to the needs of business development and security balance [ 2 ]. The aim is to explore how different machine learning methods can better identify and mitigate internet finance risks, particularly when traditional credit rating methods are not well-suited for the rapid and efficient nature of the internet. This paper adopts a systematic literature review approach to examine the various machine learning models and algorithms utilized by different scholars in assessing internet finance risks. This comprehensive review aims to gain insights into the application of machine learning algorithms in this field and the outcomes across different contexts, thereby comparing the suitability of different algorithms in this domain.

The significance and main contributions of this paper are manifested in several aspects. Firstly, it innovatively employs a systematic literature review approach to delineate the landscape of machine learning models and algorithms in internet finance risk management. Through a systematic analysis of previous research achievements, this study comprehensively reviews and compares the approaches and outcomes of machine learning in internet financial risk warning and identification. Secondly, while traditional credit scoring methods and various machine learning algorithms are commonly used in risk management for internet finance platforms, previous literature has compared these methods in different contexts. This paper provides a clear and comprehensive classification and summary analysis of the application of these methods in internet finance platforms. Thirdly, building upon the existing landscape, we believe this paper provides a clear roadmap for future research on this topic, outlining research directions and themes to bridge knowledge gaps. Fourthly, from a practical standpoint, the various frameworks and methods for internet financial risk identification provided by this study can assist internet financial companies in identifying their weaknesses and enhancing risk prevention measures. This, in turn, can elevate their service quality, facilitating more widespread and stable financial services.

The subsequent structure of this study is outlined as follows. Section 2 presents the literature review; Section 3 introduces the methods and strategies of this paper; Section 4 shows the results; Section 5 discusses the findings; Section 6 presents the conclusions and the last section is the future research suggestions.

Due to scholars’ utilization of various data sets and scenarios in their research, coupled with the rapid development of machine learning model algorithms, including large models like Transformer, which currently lack research literature on internet finance risk, this paper cannot provide a unified conclusion. Instead, practitioners could select models and algorithms that best suit their own circumstances and data based on the evaluative findings presented in this paper.

2. Literature review

Scholars have proposed the utilization of machine learning techniques [ 14 , 18 , 19 ] to predict credit risks by collecting and mining internet data. This approach has yielded superior predictive outcomes compared to conventional methods. Even within the same data sources, machine learning models exhibit greater accuracy [ 2 , 8 ], stability [ 8 ], predictive precision [ 19 , 20 ], and efficiency [ 20 ] in contrast to traditional credit scoring models.

Mirza et al. [ 19 ] compared various methods such as Naïve Bayes, Random Forest, and DLNN, and computed the accuracy of different models, revealing an enhancement in the precision of internet finance credit detection and prediction. However, researchers have discovered variations in efficiency and outcomes among different machine learning models and algorithms. Thus, developing superior algorithms and more efficient, reliable machine learning models for internet financial risk prediction has become an urgent challenge to address.

The research on the topic of internet financial risk has a long history [ 21 ], encompassing both quantitative empirical analyses [ 22 ] and qualitative descriptions [ 1 ], as well as comprehensive review studies [ 13 ]. There are analyses employing quantitative platform data [ 2 , 14 ] and those conducted using textual data [ 23 , 24 ]. Studies have delved into various subtopics such as risk perception [ 22 ], risk identification [ 24 ], and risk regulation [ 12 ], rendering the research on internet financial risk quite extensive.

However, the exploration of internet financial risk from the perspective of machine learning models emerged relatively late. The application of this approach to internet financial risk warning and risk management research began as early as 2019 [ 15 ], gradually gaining momentum alongside technological development [ 11 , 19 , 25 ]. The primary focus of these studies lies in the selection of model methodologies [ 17 , 20 , 26 ] and the construction of risk systems [ 1 , 27 , 28 ]. However, to date, there has been no comprehensive review article or study systematically outlining the state of this emerging yet critical research field. This is precisely the contribution of the present study.

The primary object of this study is to elucidate the application and research status of various machine learning algorithms or models in identifying and warning about internet financial risks. Using a systematic literature review approach, a comprehensive analysis of relevant literature in this field is conducted. Currently, there are only a limited number of articles on this topic [ 11 , 25 , 27 ], and our study addresses the following three main questions through analysis, clarifying the current state of research advancement and literature gaps in this field, as well as the differences between various internet financial risk identification and warning methods.

Q1. What machine learning algorithms have been studied in the literature for internet financial risk identification and warning, and have these algorithms and models all shown improvement?
Q2. How is the application status of the aforementioned algorithms and models?
Q3. Is there a best-suited machine learning algorithm for most internet financial platforms?

In this study, a systematic literature review method is employed to investigate the above questions. This method is well-suited for concentrating on a specific topic, providing a panoramic view, offering a more comprehensive understanding of the chosen domain, and highlighting gaps and future research directions [ 29 – 31 ].

3. Methodology

Following the standardized Systematic Literature Review (SLR) [ 32 , 33 ], this study advanced its research. Initially, we opted for the Scopus and Web of Science (WOS) databases as sources, conducting searches for all publications related to "internet financial risk" across various years. Scopus, being the world’s largest abstract and citation database, provides an extensive repository of abstracts and citations. Web of Science, on the other hand, is a comprehensive, multidisciplinary, core journal citation indexing database. Both databases are globally authoritative and specialized platforms for data retrieval, offering advanced search functionalities. This facilitates our ability to obtain relevant search results quickly, efficiently, and comprehensively.

3.1 Sample identification

In this study, we employed a keyword-based literature retrieval strategy [ 29 , 34 ]. To gather all relevant literature and research, we formulated multiple search strings related to "internet financial risk". Considering the diverse expressions in English, where internet financial could also be referred to as "online finance," "network finance," or "Fintech," we compiled all potentially involved keywords listed in Table 1 and combined them through permutations using the Boolean operator "or". Furthermore, recognizing variations in the usage of terms like "finance" and "financial," we used the asterisk "*" to represent inconsistent parts, aiming to comprehensively cover the complete continuum of the phrase "internet financial risk". In the Scopus database, we employed the search method of "Title-Abstract-Keywords". In the Web of Science database, we used "Topic" as the search mode, and we narrowed down the search scope to three major citation databases: the Science Citation Index (SCI), Social Sciences Citation Index (SSCI), and Arts & Humanities Citation Index (A&HCI), to ensure the quality of the source journals. The final search strings are as presented in Table 1 . The search date for all the data is August 9, 2023, and all literature cited in this study is up to that date.

PPT PowerPoint slide
PNG larger image
TIFF original image

https://doi.org/10.1371/journal.pone.0300195.t001

3.2 Inclusion and exclusion criteria

Following the search using the aforementioned keyword strings, the initial results in the Scopus and Web of Science databases were 116 and 48 publications, respectively. After following the approach of Khatib et al. [ 31 ] and Khatib et al. [ 32 ], we refined the results by limiting the language to "English", reducing the counts to 113 and 48. Further refining to "journal articles" and resulted in 70 and 48 publications. Subsequently, in the Scopus database, we narrowed down the "Subject area" to categories including "Computer Science", "Economics, Econometrics and Finance", "Business, Management and Accounting", "Engineering", "Mathematics", "Social Sciences", "Decision Sciences" and "Multidisciplinary". In the WOS database, we limited the "research area" to "Business Economics", "Computer Science", "Mathematics", "Telecommunications", "Engineering", "Operations Research Management Science", "Environmental Sciences Ecology" and "Science Technology Other Topics", yielding 68 and 48 publications respectively.

Then we merged the above-mentioned literature while removing duplicates, resulting in 70 articles. Subsequently, we conducted title screening and excluded 7 articles. The remaining 63 publications were subjected to abstract reading and screening, yielding 47 relevant articles. Finally, we thoroughly read these remaining publications, retaining those that incorporated concepts related to machine learning and eliminating others unrelated to the subject. We have also excluded a paper that has been retracted. This led to the final selection of 17 literatures focusing on the application of machine learning for internet financial risk identification and warning.

Fig 1 illustrates the process conducted in this study, encompassing database searches, refinement, merging, deduplication, screening, and eligibility selection. Unlike existing articles that solely focus on "internet finance risk" [ 22 ], "financial technology" [ 35 ], or "credit risk" [ 13 ], this study employs a systematic review approach to concentrate on the application and exploration of various machine learning methodologies in the realm of "internet financial risk." Despite the limited number of publications, this review comprehensively assesses and evaluates literature in this field. It not only analyzes numerous models and algorithms applied in the domain of internet financial risk but also systematically examines aspects like annual publication trends, regional publication trends, relevant research methods, evaluation metrics, and more.

https://doi.org/10.1371/journal.pone.0300195.g001

For the aforementioned literature, this paper will focus on examining the machine learning models employed by scholars in the field of internet finance risk management, as well as how these models perform across different platforms and scenarios. Therefore, we will compare the applicability of different models and algorithms in this field based on the development history, application domains, and advantages of machine learning. The results will be presented in Section 4.5.

When evaluating and assessing model performance, the goal is to ensure that the model correctly classifies samples, meaning that the actual situation of the sample data matches the model’s predictions as closely as possible. Therefore, for binary classification problems, there are four different scenarios:

The model predicts positive, and the actual situation is also positive, indicating that the model prediction is true, known as the True Positive (TP) scenario.
The model predicts negative, but the actual situation is positive, indicating that the model prediction is false, known as the False Negatives (FN) scenario.
The model predicts positive, but the actual situation is negative, indicating that the model prediction is false, known as the False Positives (FP) scenario.
The model predicts negative, and the actual situation is also negative, indicating that the model prediction is true, known as the True Negatives (TN) scenario.

TP, FN, FP, and TN respectively represent the sample counts for the four scenarios described above. Therefore, machine learning model assessment is based on these four scenarios, and a series of metrics have been developed to judge the model’s performance. This paper will present the results and explanations based on the main metrics applied in the literature in the "Results" section.

Despite including data from WOS and Scopus, there is still a possibility of not encompassing all relevant literature. However, considering the authority of the literature research, this paper still relies on the two aforementioned databases, which are of higher quality and more authoritative in content.

4.1 Publication trends

The popularization of Internet financial services occurred around 2010, while research focusing on Internet financial risks began in 2012 [ 21 ]. Thanks to a plethora of algorithmic innovations in the field of computer algorithms, machine learning, deep learning, and other methods have gradually been applied to Internet financial risk analysis. This has led to a growing interest in the subject. In our sample literature, the earliest document on this topic dates back to 2019 which was conducted by Noor et al. [ 15 ], with only one publication. Subsequently, the number of publications started to increase gradually, reaching 6 by 2022. As of August 2023, there have been three more publications, indicating a relatively limited volume overall. This suggests that research on the application of these specific methods in this particular field is still relatively insufficient. The yearly publications volume shown in ( Fig 2 ).

https://doi.org/10.1371/journal.pone.0300195.g002

4.2 Geographical distribution

As shown in Table 2 , this section presents the annual regional distribution of all the references in this paper. It’s quite evident that out of the 17 documents, 11 of them are based on Chinese Internet financial data [ 14 , 27 ]. Chinese scholars or researchers using Chinese data for machine learning algorithms in Internet financial risk analysis stand out as the driving force behind research on this topic. The sources of all this data primarily fall into three categories: national-level data [ 26 ], data from Internet financial platforms or related enterprises [ 17 , 20 ], and individual lending data from platforms [ 2 , 14 ], all of which are also detailed in Table 2 . Regarding this subject, there are studies focused on Europe, the United States, and those utilizing global Internet financial platform data. Additionally, three articles do not precisely specify the regional focus of their research [ 7 , 25 , 36 ].

https://doi.org/10.1371/journal.pone.0300195.t002

This phenomenon might be attributed to the fact that in China, after a period of rapid and unchecked growth of Internet financial platforms [ 17 ], serious risk issues emerged, involving numerous defaults, platform escape with money, and other problems [ 23 ], the number of platform drop to 1/4 from the top year [ 14 ]. Although Internet financial is an emerging financial service model, it has not altered the fundamental nature of financial services. Risk prevention remains a crucial and central aspect [ 27 , 28 ]. Consequently, Chinese scholars and professionals in the financial industry have shown a great deal of concern about Internet financial risk. They aim to utilize various methods to mitigate these risks, promote the healthy development of the industry and Internet financial services, thus generating a heightened demand [ 8 ].

4.3 Literature focus

Upon reviewing all the literatures, it becomes evident that these documents broadly focus on two distinct core aspects. One category of literature primarily revolves around comparison. These papers compare the differences in final risk identification, risk prediction, and risk supervision using various algorithms or models [ 11 , 19 , 20 ]. The objective is to identify the most suitable approach for applying sample data, thereby better assisting platform companies or other entities in mitigating Internet financial risks. A total of 14 documents fall into this category. The other category of literature centers on designing or innovating Internet financial risk systems, applying relevant data to construct appropriate risk identification or risk prediction systems [ 27 , 28 , 36 ]. Although these two categories of literature emphasize slightly different core points, their ultimate goals are risk reduction and enhancing operational stability. Both categories utilize machine learning-related models or algorithms, leading to a convergence of approaches. This underscores the diverse perspectives and research angles in understanding the practical applications of computer technology in the realm of Internet financial risk. As shown in Table 3 .

https://doi.org/10.1371/journal.pone.0300195.t003

Currently, there are numerous sources of risk in internet finance, and the application of machine learning in internet finance risk management covers a wide range of areas and directions. From the literature reviewed, machine learning is primarily applied in the following five different types of risk management:

Internet financial platforms risk: This category focuses on analyzing and alerting various risks that may occur during the operation and management processes of internet finance platforms using different machine learning algorithms [ 7 , 23 , 28 ]. For instance, Feng and Qu [ 18 ] designed an RBF neural network model optimized by genetic algorithms and established an evaluation index system for internet finance risk. Han et al. [ 8 ] decomposed it into four major components: credit risk, liquidity risk, interest rate risk, and technology risk.
Credit risk assessment and early warning: This area primarily studies the early identification and prediction of borrower credit using various machine learning algorithms. It is believed that suitable machine learning algorithms can effectively promote the identification of credit risks in lending, leading to higher predictive accuracy [ 2 , 11 , 14 , 25 , 36 ].
Internet financial market risk: This category focuses on identifying and analyzing risks in the internet finance market to enhance the level of internet finance risk management [ 18 , 27 ].
Fraud Detection: This involves analyzing the efficacy of machine learning models in fraud detection, aiming to identify danger signals in economic datasets to detect future fraudulent activities [ 19 ].
Cyber threat: This area explores how machine learning models and algorithms can identify advanced network attack patterns and conduct automated network threat attribution analysis and prediction [ 15 ]. The distribution of different risk types in the literature is shown in Table 3 .

4.4 Fields of sciences

Fig 3 provides a detailed overview of the science subject areas in which the articles from the Scopus database are classified. According to the categorization method of the Scopus database, all the literature has been divided into a total of eight different subject areas. The highest number of papers falls under "Computer Science," followed by "Mathematics" and "Engineering," with no more than two papers in any other category. This indicates that although the theme of "Internet financial risk" leans more toward the field of economics and management, the literature predominantly focuses on the methodological aspects of risk identification and prediction. This aligns with the content discussed in the previous section.

https://doi.org/10.1371/journal.pone.0300195.g003

4.5 Machine learning methods

Based on the machine learning methods employed in the literature covered in this paper, they can be broadly categorized into five types: Traditional Machine Learning Algorithms, Deep Learning and Neural Networks, Optimization Algorithms, Data Preprocessing and Enhancement, and Other Methods. In the following sections, we will categorically discuss the methods utilized in the literature. Table 4 presents the annual distribution of all methods used in the sample literature. It should be noted that the same method can be classified into different categories based on various classification approaches. The above classification is solely aimed at facilitating the organization and expression of the literature content.

https://doi.org/10.1371/journal.pone.0300195.t004

4.5.1 Traditional machine learning algorithms.

Among all the literature, traditional machine learning methods are mentioned and utilized a total of 30 times, making it the most frequently used category among the four mentioned above. This suggests that methods introduced or adopted earlier have a higher frequency of use in the context of internet financial risk, implying their relatively mature applicability. Among these, the most commonly used method in the literature is the Logistic Model, appearing in 6 articles, followed by the Random Forest, Gaussian Naïve Bayes Model and Decision Tree methods are each used in 4 articles. It’s worth noting that among all the traditional machine learning methods, the more frequently used methods belong to the category of Classification Algorithms.

Due to its strong interpretability, the Logistic Model is the most frequently utilized model in credit scoring [ 2 , 36 ]. Since the Logistic Model’s predictions can output the probability of belonging to a certain category for a record [ 25 ], adopting methods like the logistic model reveals that machine learning models have an advantage in identifying key influencing factors affecting credit customer default performance. Bussmann et al. [ 11 ] and Wu et al. [ 1 ] have also compared the logistic model with other machine learning models. Scholars utilized data from European Credit Assessment Institutions (ECAIs) focusing on commercial loans for small and medium-sized enterprises (SMEs) obtained from P2P platforms to construct a logistic regression scoring model. This model incorporated financial data on assets and liabilities, as well as network centrality indicators derived from similarity networks, to estimate the default probability of each company. A comparison was made with the results of models employing the XGBoost tree algorithm, and it was found that for internet financial risk, newer deep learning methods generally exhibit higher predictive accuracy [ 11 ].

The Random Forest method is based on the decision tree approach, where each branch of the decision tree represents a potential decision, event, or response [ 15 ]. Decision trees can achieve very low bias, but they also exhibit strong instability and sensitivity [ 25 ]. Therefore, the Random Forest method employs random data sampling and replacement strategies to construct decision trees, mitigating the issue of inconsistent sample selection due to varying tree shapes [ 19 ]. When using the Random Forest method for risk stratification on internet financial platforms, its objective is to reduce variance. This method is applicable to machine learning tasks involving classification and regression, offering higher accuracy and robustness [ 19 ]. Compared to other machine learning methods, it yields more robust and accurate results [ 1 , 25 ]. For internet financial platforms, greater accuracy and robustness are crucial for identifying risks with greater precision and reliability. Therefore, the Random Forest method is widely applied across various platforms.

Due to the relatively simpler implementation of Naive Bayes models and their requirement of smaller training data, they are capable of handling both continuous and discrete data. Naive Bayes is a probabilistic classifier based on the principles of conditional probability in Bayes theorem [ 15 ]. They also offer rapid prediction capabilities, making them particularly suitable for real-time forecasting [ 15 ]. Furthermore, they can conduct sentiment analysis and scoring of online user information, effectively evaluating user eligibility [ 1 ]. Additionally, Bayes models exhibit a higher level of accuracy [ 19 , 25 ]. K-Nearest Neighbor (KNN) is a supervised machine learning algorithm that doesn’t require prior knowledge and can classify based on the majority vote of its neighbors. It’s particularly well-suited for large-scale financial service platforms [ 15 , 17 ]. The KNN method can be employed in conjunction with other techniques to obtain the fitness value. However, in the study by Mirza et al. [ 19 ], KNN was found to have the lowest accuracy among the five methods employed.

Of course, there are also other methods like traditional RBF-NN [ 18 , 27 ], and complementary-neural network (CMTNN) [ 27 ] applied in the existing literature as innovative models and approaches for internet financial risk prediction. Overall, these methods have the potential to enhance the accuracy and speed of traditional predictions. Consequently, models like the Logistic model, Bayes model, and Random Forest, have become more mature in the field of machine recognition [ 1 , 7 ], and have found wide application. However, from scholars’ perspectives, newer deep learning and reinforcement learning methods have shown superior performance on specific datasets compared to traditional machine learning algorithms [ 2 , 19 ]. These methods may find broader applications in the future in the field of internet financial risk identification and early warning.

4.5.2 Deep learning and neural networks.

Firstly, in terms of overall quantity, applications related to deep learning and neural network methods in the context of internet financial risk have appeared a total of 25 times, which is equal to the count of applications of traditional machine learning methods. Additionally, from a temporal perspective, deep learning and neural network models had only one literature in 2019, and the combined occurrences in 2020 and 2021 were merely 8. However, since 2022, the frequency has escalated to 17 occurrences, surpassing more than 2 times the occurrences in the preceding three years. Specifically, in 2022 alone, there were 8 occurrences, and by August 2023, there were already 9 instances, signifying a gradual and increasing integration and utilization of deep learning and neural network-related models in the domain of internet financial risk management. Turning to the specifics of method applications, the most utilized is the BP neural network, referenced and employed in a total of 7 literature sources. Following this, there are 4 instances mentioning the Deep Learning Neural Network (DLNN), and subsequently, for the XGBoost Model, Convolutional Neural Network (CNN) and Long- and Short-Term Memory (LSTM), each mentioned in 3, 2, and 2 literature sources, respectively.

In the analysis of internet financial risk, the Backpropagation (BP) neural network stands out as the most frequently applied method across all literature sources. Typically, a BP neural network comprises at least three layers: the input layer, hidden layer, and output layer [ 8 ]. This approach does not require a predefined mathematical expression between the input and output layers [ 25 ]. Its principle is rooted in the error backpropagation algorithm of a multi-layer feedback network, which involves adjusting thresholds and weights based on the error of results [ 20 , 26 ]. As a result, the structure of the BP neural network is simpler, while its predictive accuracy and nonlinear processing capabilities are stronger [ 18 , 20 ]. In the context of analyzing internet financial platform risk management issues, this approach has been widely adopted by scholars [ 1 , 18 , 27 ]. The study utilized data from 65 publicly listed Chinese companies to train optimized neural networks. Testing was conducted using big data from internet finance enterprises spanning from 2015 to 2018, with a comparison drawn against the actual development of the internet finance sector., it has been observed that compared to other models, although the BP neural network yields higher predictive accuracy, it requires the longest training time [ 18 ]. Therefore, as a foundational deep learning and neural network method, when combined with other algorithms in subsequent steps, it can produce improved outcomes [ 18 ].

In the literature on deep learning for internet financial risk, it is mentioned that the foundation of deep learning operates akin to the neural network systems in the human brain [ 15 ], capable of learning from unlabeled or unstructured data. It fundamentally follows a supervised learning approach, enabling a better understanding of the mapping relationship between x and y [ 26 , 37 ]. Thanks to significant advancements in algorithms and hardware, deep learning can leverage increased levels and neuron counts for modeling, thus making it feasible for application in internet financial risk management [ 1 ]. Mirza et al. [ 19 ] constructed a database spanning 10 years, comprising 95 companies, using KBW and Nasdaq Financial Technology Rankings, as well as the Nasdaq Insurance (IXIS) Index. The aforementioned data was then used to compare five algorithms, including Naive Bayes, KNN, Decision Tree, Random Forest, and DLNN and found that, in comparison to traditional machine learning methods, the accuracy of deep learning (DLNN) is the highest among all five methods. Scholars have been consistently combining foundational deep learning models with other algorithms in an attempt to explore more suitable deep learning algorithms.

The highly renowned XGBoost optimization model is also rooted in the decision tree algorithm, essentially utilizing the gradient boosting ensemble technique to combine multiple decision tree models [ 2 ]. Leveraging gradient descent methods to minimize errors [ 11 ], inappropriate trees are pruned, resulting in a high-accuracy gradient tree boosting model [ 19 ]. This uniqueness positions the XGBoost Model with a distinctive advantage in handling sparse data. Fan et al. [ 2 ] selected a P2P online lending platform in China as the research subject and utilized data from 30,225 short-term loans issued by the platform from August to December 2018. Logistic regression, GMDH, SVM, and XGBoost algorithms were compared for internet finance risk assessment. It was found that the XGBoost model achieved the highest overall accuracy, with a testing set accuracy of 90.1%. Similar conclusions were also drawn in the study by Bussmann et al. [ 11 ].

Convolutional Neural Networks (CNN), built upon the foundation of deep learning (DLNN), incorporate convolutional layers designed for data feature extraction [ 23 ]. These extracted features are then passed to different network nodes, allowing for layered representation and data learning, ensuring efficient learning processes. CNNs are characterized by sparse connections and weight sharing [ 23 ], and have been attempted for prediction tasks, demonstrating performance on par with human experts [ 19 ].

Scholars have employed Long Short-Term Memory (LSTM) for researching internet financial risk [ 19 ]. LSTM, a specialized Recurrent Neural Network, comprises three control units: input gate, output gate, and forget gate, enabling it to address the challenge of long sequence dependencies in neural networks [ 23 ]. Consequently, this enhances the predictive accuracy for high-risk groups in internet finance. Xia et al. [ 23 ] improved classification outcomes by incorporating an attention mechanism, and further elevated accuracy by introducing Bi-directional Long Short-Term Memory (BiLSTM) with reverse sequence information using 42,590 Q&A pairs text. This is because BiLSTM consists of both positive and negative LSTMs, enabling a thorough consideration of the contextual information’s influence on the current output. This facilitates the learning of more accurate semantic representations of text, leading to a more comprehensive understanding of its semantics [ 38 ]. The consideration of contextual information in the output led to even higher recognition accuracy.

Adaptive Boosting (AdaBoost) involves combining outputs from various methods to enhance classification performance, ensuring a reduction in overall classifier error after each iteration. As a result, this method has achieved high accuracy in internet financial risk models [ 19 ]. Methods such as Probabilistic-Neural Network (PNN) [ 27 ], general regression neural network (GR-NN) [ 27 ], and Restricted Boltzmann Machines (RBMs) can also accelerate the learning process, improving optimization efficiency [ 1 ]. By employing various algorithms based on deep learning and neural networks in internet financial risk management, scholars generally find that improvements in machine learning algorithms lead to enhanced accuracy, robustness on validation sets, and even reduced response times. Hence, it can be said that with the aid of more applicable machine learning algorithms, the capability of internet financial risk management is continuously improving, and this improvement process remains ongoing.

4.5.3 Optimization algorithms.

The literature also enumerates some optimization algorithms, with the most prominent being Genetic Algorithms(GA) and their enhanced variants based on genetic algorithms [ 20 ]. The Genetic Algorithm (GA) is a global optimization algorithm based on probabilistic optimization [ 20 ], known for its strong global search capabilities and wide adaptability [ 18 , 23 ]. The ACO-optimized RBF algorithm possesses high spatial mapping and generalization capabilities [ 18 ]. The GABP neural network adopts a distributed storage structure. It demonstrates fast iteration speed, accurate results, good redundancy, and robustness in financial risk identification [ 20 ].

Combining the aforementioned Genetic Algorithm and Simulated Annealing Algorithm, the GABP Algorithm Based on Simulated Annealing Optimization method was used, and it was found to have higher accuracy and predictive speed compared to BP neural networks and GABP networks. Guang et al. [ 20 ] selected 36 internet finance companies as samples and grouped them based on financial conditions for optimization using GA, GABP, and SA-GABP algorithms. They found that leveraging the global optimization capabilities of various genetic algorithms and applying the optimized networks to predict internet finance risks resulted in favorable prediction outcomes. This provides a scientific basis for credit decision-making and risk prevention in internet finance and banking.

4.5.4 Data preprocessing and enhancement.

This section primarily concerns data preprocessing and selection, encompassing three main methods: Synthetic Minority Over-sampling Technique Algorithm (SMOTE), Group Method of Data Handling (GMDH), and Weight of Evidence (WOE), with a total of only 5 applications. The SMOTE algorithm, based on synthetic sample synthesis, enhances data discriminability accuracy by generating new synthetic samples to form a new dataset [ 2 ]. The Group Method of Data Handling (GMDH) is a technique to extract significant information from vast and complex data, thereby improving analytical efficiency [ 14 ]. This is crucial for handling the substantial and intricate data inherent to open attributes in internet financial platforms. In the research by Fan et al. [ 2 ], GMDH achieved accuracy second only to XGBoost. Weight of Evidence (WOE) is primarily employed to assess the relationship between features and targets, examining default situations in internet financial platforms [ 7 , 36 ]. Through appropriate data preprocessing, feature selection, and effective algorithm integration, this serves as a pivotal step in ensuring accurate risk assessment for internet financial platforms.

4.5.5 Other methods.

Other methods mentioned in the literature include Named Entity Recognition (NER), which primarily involves text processing and entity identification. NER falls within the domain of text processing and natural language processing techniques and can identify names, specific locations, and other contextually significant content within text [ 23 ]. There is also the Fuzzy Analytic Hierarchy Process (FAHP), an analytical method used for multi-criteria decision-making problems [ 28 ], and methods related to big data and the Internet of Things (IoT) [ 27 ]. While not the main focus here, it’s evident that these methods, particularly NER in conjunction with emotional analysis, can effectively broaden the applicability of machine learning in internet financial risk identification.

4.6 Literature findings

Table 5 presents the titles and research findings of selected literature, aiming to comprehend the overall research conclusions, current status, and trends of this issue. This provides potential research directions for future studies. Based on the aforementioned analysis and the research findings listed in Table 5 . (1). It can be established that internet financial risk is a widely recognized and crucial latent issue. Machine learning, as a novel computational technology, whether through foundational algorithms or complex algorithm combinations, offers significant advancements in risk prevention compared to traditional credit scoring methods. (2). Different algorithms exhibit varying effectiveness in internet financial risk prediction. Overall, there is an improvement in prediction accuracy, time efficiency, and robustness with algorithm optimization. (3). Technological advancements also bring about technological risks [ 28 ], emphasizing the need for continuous improvement in risk anticipation and prevention.

https://doi.org/10.1371/journal.pone.0300195.t005

Therefore, future research should continue to explore and expand various machine learning algorithms, particularly the application of deep learning algorithms in the field of internet financial risk. A comprehensive and sustainable risk management strategy is imperative for internet financial platform companies, investors, borrowers, regulatory authorities, and even traditional institutions like banks engaged in internet financial operations.

4.7 Evaluation criteria

Table 6 lists all the evaluation metrics and the formula of the metrics used in the literature for assessing internet financial risks. These metrics are employed to gauge the strengths and weaknesses of various machine learning and other methods. TP represents the number of true positive predictions, FN represents the number of false negative predictions, FP represents the number of false positive predictions, and TN represents the number of true negative predictions. ROC is commonly used to evaluate the performance of binary classifiers, where the vertical axis represents the True Positive Rate (TPR) and the horizontal axis represents the False Positive Rate (FPR). The dashed line represents the baseline, indicating the lowest standard. ROC is used on this coordinate axis to measure the accuracy of the model. The closer the ROC curve is to the upper left corner, the higher the predictive accuracy of the model. Compared to other metrics, the ROC curve can more visually display the strengths and weaknesses of different models on a graph. The Area Under Curve (AUC) refers to the area enclosed by the Receiver Operating Characteristic (ROC) curve and the x-axis. Its maximum value is 1. A larger AUC indicates a higher efficiency of the model in identifying targets [ 2 ].

https://doi.org/10.1371/journal.pone.0300195.t006

From the perspective of the final evaluation metrics, undoubtedly, the most important evaluation metric is accuracy, which is mentioned in 16 articles. Accuracy refers to the proportion of correctly classified samples out of the total number of samples, i.e., the sum of the number of instances where the predicted value matches the actual value, divided by the total number of samples. This metric is the fundamental indicator for evaluating model performance. However, for imbalanced datasets, accuracy may not be reliable. Hence, although accuracy is widely used in literature, it is not considered the sole measure of performance.

Following that is true positive rate (Recall) which is used in 6 papers. Recall is the proportion of correctly classified positive samples out of the total number of true positive samples. Recall focuses on the statistical measure of some samples and emphasizes the correct identification of true positive samples. Then precision is used in 5 papers. Precision examines the probability of true positive samples among all predicted positive samples, indicating the confidence in correctly predicting positive samples. It measures the accuracy of positive predictions or the proportion of accurately identified positive samples. Recall focuses on how many positive instances were missed. The higher the recall, the stronger the model’s ability to distinguish positive samples. Precision, on the other hand, focuses on the proportion of predicted positives that are actually true positives. A higher precision indicates a stronger ability of the model to distinguish negative samples. Therefore, precision and recall have a trade-off relationship, each serving its purpose.

Then false positive rate (FPR) is used in 4 papers, which measures the percentage of all actual negative samples that were incorrectly classified as positive by the model. But a more comprehensive and objective evaluation metric and measurement method are the ROC curve and the AUC, which are the combined curves composed of true positive rate and false positive rate and the area under the curves, respectively. They are often used to assess the model overall, and these evaluation metrics can reduce interference from different test sets, providing a more objective measure of the model’s performance compared to individual metrics mentioned above. The ROC and AUC metrics were used in 4 and 3 articles, respectively.

F1 score is used in 3 articles. The F1 score integrates both precision and recall factors, achieving a balance between the two, ensuring both "precision" and "recall" are considered without bias. The F1 score is the harmonic mean of precision and recall, thus it simultaneously considers both the accuracy and recall of the model. However, because it is composed of the product of recall and precision, when the values of recall or precision are very small, the F1 score will also be very small. Regardless of how high one value is, if the other value is very small, the F1 score will be small as well. Therefore, it comprehensively reflects the effectiveness of the model. Using the F1 score as an evaluation metric can prevent the occurrence of extreme cases as mentioned above. Noor et al. [ 15 ] utilizes metrics such as Accuracy, Precision, Recall, F1-measure, False Positive Rate, etc., to assess and compare the effectiveness of Naïve Bayes, KNN, Decision Tree, Random Forest, and DLNN methods, thereby enabling a more comprehensive analysis and evaluation.

The results of these evaluation index indicate that higher accuracy or recall are the most intuitive indicator for assessing different methods, and it’s highly regarded by all researchers. This core metric is crucial in comparing various algorithms. The extensive and diverse set of metrics also provides us with analytical insights and frameworks for assessing the applicability of different methods in the future. Consequently, regardless of how far machine learning algorithms evolve in the future, these metrics and frameworks will continue to help us establish an effective judgment system.

5. Findings and discussion

The development of internet financial platforms has gone through initial rapid expansion followed by a period of gradual regulation, eventually transitioning into a stable operating phase guided by long-term goals. Given the rapid advancement of the internet and the significant role of finance in societal development, recognizing, anticipating, supervising, and managing internet financial risks have become critical topics. Utilizing techniques like machine learning to address the challenges of open internet environments and the abundance of data in financial risk prevention is both timely and necessary. In this study, we employed a systematic approach to review the internet financial risk research conducted using machine learning methods up to the present. This paper listed the machine learning models and algorithms currently used in internet finance risk management, addressing the first question posed. Future research can continue to explore areas such as research methods, data analysis, evaluation metrics, and research scope.

First and foremost, through our analysis, we have observed that whether it’s traditional machine learning algorithms, deep learning, neural networks, or other methods, all have the potential to improve prediction accuracy, surpassing traditional credit indicator calculation methods. This addresses the first question raised in this paper and also touches upon the effectiveness of machine learning methods applied to internet finance risk, addressing the second question. However, the accuracy of neural network models in predicting internet financial risks is contingent on factors such as model structure, sample data, and parameter settings [ 18 ]. There exist issues of data imbalance in the utilized datasets [ 23 ], and most algorithms exhibit certain biases in their final accuracy [ 27 ]. Hence, in the future, due to the specific requirements of the financial industry, ongoing optimization and improvement are necessary at both the algorithmic and data levels. This could involve the incorporation of new or updated algorithms more tailored to financial risks, especially algorithms suitable for extreme value research in risk identification. Although studies have developed models that are well-suited for handling fuzzy, heterogeneous, and incomplete data [ 17 ], currently, analysis of extreme cases is lacking, but financial risks or issues demand attention to extreme situations [ 23 ]. Simultaneously, in terms of data, the inherent nature of financial platforms makes obtaining timely, reliable, stable, and diverse data somewhat challenging. However, this aspect is crucial for enhancing the effectiveness of algorithms and models, given the limited quantity of research in this area at present.

Comparing different machine learning models and algorithms, the current state of affairs generally reflects that intelligent algorithms, represented by various deep learning algorithms, exhibit higher predictive accuracy compared to traditional machine learning models. They can address issues such as uncertainty, poor fault tolerance, and lack of self-learning capabilities in traditional warning models [ 8 ]. However, overall, scholars employ diverse platforms and datasets, and no study has comprehensively compared all mainstream machine learning models and algorithms. Consequently, there is no universally optimal model applicable to all platforms, addressing the third question posed in this paper.

Currently, there are multiple sources of risk in internet finance, including financial risk, legal risk, credit risk, market risk, and technological risk. Scholars primarily focus on credit risk [ 25 ] and technological risk [ 19 ]. Although some researchers have found that technological risk, ethical risk, and legal risk are the predominant factors affecting fintech risk [ 28 ], and even attempted to establish an internet finance risk control system based on deep learning algorithms [ 1 ], a considerable portion of literature still assesses machine learning algorithms from the perspective of credit risk. They evaluate whether single or multiple models can reduce expected losses [ 36 ], increase platform revenue [ 7 ], and obtain more reliable risk predictions [ 2 , 14 ].

Given the characteristics of the internet finance sector, which involve short timeframes and large quantities of data [ 2 ], it is inevitable to opt for artificial intelligence risk warning and management models based on machine learning algorithms. However, the mentioned literature predominantly focuses on data within internet financial platforms or companies, without considering the influence of the external environment and other external sources or third-party data [ 26 ], which limits the generalizability of prediction results. Few studies have concentrated on machine learning identification of textual data, even though in the operational process of internet financial platforms, effective communication among users can be enhanced. Developing more timely and effective sentiment analysis algorithms for textual data could improve risk identification strategies. Thus, from this perspective, the existing internet financial risk assessment metric system should be further refined. It should incorporate external environmental data, existing credit scoring factors, third-party data, and the evaluation metrics presented in this study [ 18 ]. Establishing a more comprehensive and rational internet financial risk assessment metric system can be a potential direction for future research.

Through the analysis of evaluation metrics used in all the literature reviewed, it is evident that most studies choose accuracy, recall, and precision as metrics for evaluation and comparison of results, while fewer studies apply metrics such as ROC, AUC, F-score, and even more comprehensive and complex indicators. None of the literature covered the use of newer models and algorithms like Transformer. These observations indicate that although machine learning has been extensively applied in many fields, research in the domain of internet finance risk management remains limited. Therefore, we outline potential research directions in the "Future Research" section.

Finally, it’s evident that the majority of current applications and research on machine learning in the field of internet financial risk are conducted by Chinese scholars, using Chinese data, and considering Chinese scenarios (11 articles). Therefore, the scope and focus of research are still quite limited. With the increasing adoption of financial technology, digital currencies, big data, the Internet of Things, artificial intelligence, cloud computing, and other technologies across various countries [ 28 ], a more extensive and diverse range of research scenarios and scopes should become a mainstream in future research. This would contribute to providing a safer internet financial environment for individuals, businesses, platforms, local governments, and regulatory authorities.

6. Conclusion

With the gradual penetration of internet financial services in society and the maturation of machine learning algorithms, this study systematically introduces the research of machine learning models and algorithms in the field of internet financial risk. The focus is on exploring various algorithms and their characteristics used in previous studies. While, in general, machine learning enhances the accuracy of internet financial risk identification, scholars’ conclusions vary due to different approaches, and research is overly concentrated in China. Using permutations and combinations of different expressions related to "internet," "finance," and "risk" as keywords, comprehensive searches were conducted in both the Scopus and Web of Science databases, yielding 116 and 48 articles respectively. After filtering by language, document type, topic, merging, deduplication, and focusing on reading and screening content related to "machine learning," the final sample was narrowed down to 17 articles. This paper provides a comprehensive analysis of the sample literature from aspects such as annual trends, regional distribution, literature focus, fields of sciences, used models and algorithms, research findings, and evaluation metrics. Subsequently, the findings of this paper are discussed. Ultimately, it identifies research gaps and proposes future research directions in this field.

The research findings of this paper reveal that although the overall quantity is limited, the research on this topic has tripled in the past three years, with two-thirds of the studies focusing on China. Looking at the machine learning algorithms employed by scholars, a range of traditional algorithms, deep learning algorithms, and novel algorithms like neural networks have been used. The research findings consistently show that compared to traditional credit evaluation methods, machine learning models and algorithms can significantly enhance the accuracy of internet financial risk identification. However, there are noticeable differences among different algorithms, and though conclusions differ with varying datasets, generally, more recent algorithms yield higher accuracy. Additionally, scholars evaluate the effectiveness of various algorithms from aspects such as learning efficiency, recall rate, true positive rate, and more. Our study provides a comprehensive review of the current state of research involving the application of machine learning to internet financial risk. We have identified certain limitations in existing literature, such as the restrictions in research methods, the limited application of various algorithms, incomplete data analysis, exclusion of external environmental data, optimization of evaluation metrics, and over-concentration on China.

7. Future research

The uniqueness of this study lies in its exploration of this emerging research field, offering a comprehensive review of the application of machine learning algorithms in internet financial risk management. Overall, research on machine learning in the field of internet finance risk management is not extensive, and the findings are inconsistent. Thus, it provides innovative analytical outcomes and future research suggestions for this area. Firstly, due to scholars using different platforms, data, models, and algorithms, there is no universally accepted best model. Hence, industry practitioners can categorize discussions on different machine learning algorithms in internet finance risk management based on our research, exploring the most suitable machine learning algorithms for their own specific scenarios. Secondly, a more detailed analysis of the application considerations of deep learning models and algorithms in internet finance risk management practice is needed, starting with data acquisition to improve model efficiency. Thirdly, as mentioned earlier, the literature used in this study comes from two databases, WOS and Scopus. Expanding the literature sources while ensuring quality could be beneficial. Fourthly, future research could gradually expand its scope by merging traditional statistical analysis with machine learning methods for studying internet financial risks. Lastly, some listed companies have claimed that models based on the Transformer architecture have been applied in vertical fields such as financial risk and public security, utilizing encoders and decoders for multi-step prediction. This is also an important research direction for future identification of internet finance risks. Additionally, attention could be directed towards the impact of emerging technologies or business models like digital currencies, metaverse, and blockchain on internet financial risks.

Supporting information

S1 file. prisma checklist..

https://doi.org/10.1371/journal.pone.0300195.s001

S2 File. Data search result of Scopus.

https://doi.org/10.1371/journal.pone.0300195.s002

S3 File. Data search result of WOS.

https://doi.org/10.1371/journal.pone.0300195.s003

Acknowledgments

The authors would like to thank Science and Technology Finance Key Laboratory of Hebei Province for their funding support.

View Article
Google Scholar
PubMed/NCBI
21. Li Q, Cai D, Wang H, editors. Study on network finance risk on the basis of logit model. Technology for Education and Learning; 2012. Berlin Heidelberg: Springer; 2012.

IMAGES

A Step by Step Guide for Conducting a Systematic Review
Systematic literature review phases.
The methodology of the systematic literature review. Four phases of the
Systematic reviews
How to Write A Systematic Literature Review?
Systematic Literature Review Methodology

VIDEO

Day 1 International Non Degree Training
FACULTY DEVELOPMENT PROGRAM ON SYSTEMATIC LITERATURE REVIEW
Speech Recording for Dietary Assessment A Systematic Literature Review
Online Extremism Detection A Systematic Literature Review With Emphasis on Datasets, Classification
FACULTY DEVELOPMENT PROGRAM ON SYSTEMATIC LITERATURE REVIEW
Performance Management

COMMENTS

Performance Management: A Systematic Review of the Literature and an
Performance Management (PM), in all its guises, is a core business practice across the majority of organizations; whether this occurs formally through an official organizational process or ...
Performance Management: A Scoping Review of the Literature and an
Performance management (PM), in all its guises, occurs across all organizations whether formally through an official organizational process or informally through daily dialogue. ... Human resource development in SMEs: A systematic review of the literature. International Journal of Management Reviews, 18, 85-107. Crossref. ISI. Google Scholar ...
Performance Management: A Systematic Review of the Literature and an
Performance Management (PM), in all its guises, is a core business practice across the majority of organizations; whether this occurs formally through an official organizational process or informally through daily dialogue. Given its inherent importance our aim was to understand how academics have explored PM and therefore we conducted a Systematic Literature Review (SLR) of the scholarly PM ...
Performance management: A scoping review of the literature and an
Performance management (PM), in all its guises, occurs across all organizations whether formally through an official organizational process or informally through daily dialogue. Given its inherent importance to the field of Human Resource Development (HRD), we conducted a scoping review of the PM literature over a period of more than 11 years, uncovering 230 articles from 41 different journals.
Performance Management: A Scoping Review of the Literature and an
Development (HRD). This link is further evidenced by Hamlin and Stewart' s (2011, p. p. 211) review of the literature in which they proposed four core purposes of HRD: . "improving individual ...
Performance management methods and practices among ...
Background Nurses make up the largest constituent of the health workforce. The success of health care interventions depends on nurses' ability and willingness to provide quality health care services. A well-implemented performance management (PM) system can be a valuable asset in ensuring that nurses are motivated, promoted, trained and rewarded appropriately. Despite the significant ...
Full article: Performance management systems: reviewing the rise of
The systematic literature search method makes use of an iterative procedure where relevant articles are searched, checked and reviewed for relevance until the whole literature review is complete (Tranfield, Denyer, & Smart, Citation 2003). Subsequent meta-analysis provides a quantitative method to increase the quality and comprehensiveness in ...
Performance Management System. A Literature Review
Following the systematic literature review approach (Huff 2009) and the model for systematic literature search (Duff 1996), a systematic literature review Footnote 2 of the studies on PMS design from 1980 to 2010 has been undertaken. The definition of the topic involved the identification of the following search terms: Performance management system
KEY PERFORMANCE INDICATORS: A SYSTEMATIC LITERATURE REVIEW
This study aims to identify key performance indicators (KPIs) and categorise them. based on performance measurement to improve a holistic performant management organisation. The Method is using ...
Guidance on Conducting a Systematic Literature Review
Literature reviews establish the foundation of academic inquires. However, in the planning field, we lack rigorous systematic reviews. In this article, through a systematic search on the methodology of literature review, we categorize a typology of literature reviews, discuss steps in conducting a systematic literature review, and provide suggestions on how to enhance rigor in literature ...
Factors affecting employee performance: a systematic literature review
The analysis can support the understanding of employee performance from a broader and more diverse view points; and help in providing insight into real-life opportunities, constraints and solutions in enhancing performance management.,This systematic literature review highlights important knowledge gaps which need to be explored especially in ...
Performance management of SMEs: a systematic literature review for
It categorizes the main antecedents for SMEs' performance, the dimensions of SMEs' performance and all variables used as moderators in the conceptual relationships.,This study undertakes a systematic literature review of 58 articles published in 33 peer-reviewed academic journals from 2000 to 2020 followed by a synthesis of individual ...
Performance management methods and practices among nurses in primary
Keywords: Performance management, Performance appraisals, Performance review, Nurses, Quality of care, Human resources management, Systematic scoping review ... This method of literature review is an advantage for synthesising research evidence and mapping existing literature in a given field in terms of its prevalence and key features ...
PDF Performance management in civil engineering: A systematic literature review
To gain a bibliometric perspective of performance management in civil engineering, several review methods can be used for analysing the existing literature and creating or extending the current body-of-knowledge, such as a critical review, literature review, meta-analysis, systematic search, and scientometric analysis [30-32].
Building a bridge between performance management, radical innovation
Some scholars even suggest that traditional PM may suppress radical innovation. Our study takes a different approach. It aims to bridge radical innovation and network performance management through a systematic review of the literature. Here, we adopt a hybrid approach that combines bibliometrics with content analysis.
Leadership styles and sustainable performance: A systematic literature
Abstract. The aim of this paper is to synthetize and critically analyze the linkage between leadership styles and sustainable performance (SP) through application of the rigor of systematic literature reviews. Bibliometric characterization of articles indexed in the Scopus database, network analysis and a manual in-depth review were carried out.
Performance management of SMEs: a systematic literature review for
DOI: 10.1108/ijppm-07-2020-0349 Corpus ID: 229428992; Performance management of SMEs: a systematic literature review for antecedents and moderators @article{Kafetzopoulos2020PerformanceMO, title={Performance management of SMEs: a systematic literature review for antecedents and moderators}, author={Dimitrios Kafetzopoulos}, journal={International Journal of Productivity and Performance ...
Drivers of Performance Information Use: Systematic Literature Review
Public Performance & Management Review Volume 38, 2015 - Issue 3. Submit an article Journal homepage. ... Drivers of Performance Information Use: Systematic Literature Review and Directions for Future Research. Alexander Kroll Florida International University Correspondence [email protected]. Pages 459-486 | Published online: 25 Mar 2015. Cite ...
PDF Sustainable performance assessment: A systematic literature review
The goal of this paper is two-fold. Firstly, a systematic literature review is conducted to highlight the importance and the need to assess the three pillars of sustainability and, finally, to encircle the sustainability concept by identifying the most used techniques and approaches in its evaluation.
Sample Selection in Systematic Literature Reviews of Management
The present methodological literature review (cf. Aguinis et al., 2020) addresses this void and aims to identify the dominant approaches to sample selection and provide insights into essential choices in this step of systematic reviews, with a particular focus on management research.To follow these objectives, I have critically reviewed systematic reviews published in the two most prominent ...
Humanitarian supply chain performance management: a systematic
A literature review has been conducted using a structured method based on Denyer and Tranfield (2009) and Rousseau et al. (2008). The state of the art on humanitarian supply chain performance management with a focus on measurement frameworks and indicators and their applications in practice is classified in three categories.
Employee engagement and performance: a systematic literature review
Bailey et al. [2017] conducted a systematic review, inv olving 214 studies that examined the meaning, antecedents, and consequences of enga gement (42 studies researched the performanc e outcomes ...
Review A systematic review of operations research and management
The systematic literature review presented in this paper highlights the existing research on the use of quantitative OR/MS modeling techniques in the context of higher education and highlights the potential of quantitative modeling techniques to support higher education policymakers and decision-makers in improving performance, enhancing ...
Back to the future revisited: A systematic literature review of
In order to assess the theoretical, empirical, and scientific reasoning for this interest we analyze the existing literature in order to identify the emerging discussions in this area and to provide a systematic review that can be used as guidance for future research.
(PDF) A Literature Review and Overview of Performance Management: A
Abstract. The underlining presuppo sition and the supposition of performance management as a st udy field have been controversial. or have a non -defined concept ever s ince the field was introd ...
Lifestyle interventions in the management of systemic sclerosis: a
This systematic review of the literature assessed the current evidence for lifestyle interventions as viable management strategies for people living with SSc. The main categories of intervention were physical activity and exercise, mental health and diet and nutrition.
Influence of e-learning on the students' of higher education in the
A systematic literature review is needed to comprehensively analyze and synthesize the existing research on how e-learning influences students in the digital age (Alajmi et al., 2020). Specifically, a systematic review would allow for an objective evaluation of the evidence regarding how e-learning affects factors like student performance ...
Machine learning in internet financial risk management: A systematic
Firstly, it innovatively employs a systematic literature review approach to delineate the landscape of machine learning models and algorithms in internet finance risk management. Through a systematic analysis of previous research achievements, this study comprehensively reviews and compares the approaches and outcomes of machine learning in ...
A systematic literature review on the economic impact of digitalization
Industry 4.0 and supply chain performance: A systematic literature review of the benefits, challenges, and critical success factors of 11 core technologies. Industrial Marketing Management ... A systematic literature review on knowledge management in SMEs: current trends and future directions. Manag Rev Q (2022), 10.1007/s11301-022-00299- ...
Sustainable Human Resource Management in the Hospital Sector: A Review
Additionally, she has a deep-rooted interest in research works related to service marketing, TQM and Industry 4.0. She has published and presented research papers in various national and international journals like Management and Labor Studies (Sage), International Journal of Public Sector Performance Management (Inderscience) to name a few.

Performance management methods and practices among nurses in primary health care settings: a systematic scoping review protocol

Contribution to the field

PM opportunities and challenges

Methodology

Stage 1: Identifying the research question

Stage 2: Identifying relevant studies

Stage 3: Study selection

Stage 4: Charting the data

Stage 5: Collating, summarising and reporting the results

Quality appraisal

Availability of data and materials

Abbreviations

Acknowledgements

Author information

Contributions

Corresponding author

Ethics declarations

Consent for publication

Additional information

Rights and permissions

About this article

Share this article

Systematic Reviews

Performance management methods and practices among nurses in primary health care settings: a systematic scoping review protocol

Tivani Phosa Mashamba-Thompson

Inge Petersen

Contribution to the field

PM opportunities and challenges

Methodology

Stage 1: Identifying the research question

Stage 2: Identifying relevant studies

Stage 3: Study selection

Stage 4: Charting the data

Stage 5: Collating, summarising and reporting the results

Quality appraisal

Acknowledgements

Abbreviations

Availability of data and materials

Consent for publication

To read this content please select one of the options below:

Design/methodology/approach

Practical implications

Originality/value

Related articles

All feedback is valuable

Join us on our journey

Questions & More Information

Article Contents

Lifestyle interventions in the management of systemic sclerosis: a systematic review of the literature

Inclusion and exclusion criteria

Search strategy

Data extraction

Categorization

Quality assessment and evidence grading

Study characteristics

Diet and nutrition

Supplementary data

Affiliations

This Feature Is Available To Subscribers Only

Influence of e-learning on the students’ of higher education in the digital era: A systematic literature review

Cite this article

Access this article

Data availability

Author information

Corresponding author

Ethics declarations

Additional information

Rights and permissions

About this article

Share this article

Machine learning in internet financial risk management: A systematic literature review

1. Introduction

2. Literature review

3. Methodology

3.1 Sample identification

3.2 Inclusion and exclusion criteria

4.1 Publication trends

4.2 Geographical distribution

4.3 Literature focus

4.4 Fields of sciences