phd thesis systematic review

• Open access
• Published: 12 December 2017

Acceptance of a systematic review as a thesis: survey of biomedical doctoral programs in Europe

• Livia Puljak   ORCID: orcid.org/0000-0002-8467-6061 1 , 2 , 3 &
• Damir Sapunar 3  

Systematic Reviews volume  6 , Article number:  253 ( 2017 ) Cite this article

15k Accesses

21 Citations

66 Altmetric

Metrics details

Systematic reviews (SRs) have been proposed as a type of research methodology that should be acceptable for a graduate research thesis. The aim of this study was to analyse whether PhD theses in European biomedical graduate programs can be partly or entirely based on SRs.

In 2016, we surveyed individuals in charge of European PhD programs from 105 institutions. The survey asked about acceptance of SRs as the partial or entire basis for a PhD thesis, their attitude towards such a model for PhD theses, and their knowledge about SR methodology.

We received responses from 86 individuals running PhD programs in 68 institutions (institutional response rate of 65%). In 47% of the programs, SRs were an acceptable study design for a PhD thesis. However, only 20% of participants expressed a personal opinion that SRs meet the criteria for a PhD thesis. The most common reasons for not accepting SRs as the basis for PhD theses were that SRs are ‘not a result of a PhD candidate’s independent work, but more of a team effort’ and that SRs ‘do not produce enough new knowledge for a dissertation’. The majority of participants were not familiar with basic concepts related to SRs; questions about meta-analyses and the type of plots frequently used in SRs were correctly answered by only one third of the participants.

Conclusions

Raising awareness about the importance of SRs and their methodology could contribute to higher acceptance of SRs as a type of research that forms the basis of a PhD thesis.

Peer Review reports

Systematic reviews (SRs) are a type of secondary research, which refers to the analysis of data that have already been collected through primary research [ 1 ]. Even though SRs are a secondary type of research, a SR needs to start with a clearly defined research question and must follow rigorous research methodology, including definition of the study design a priori, data collection, appraisal of study quality, numerical analyses in the form of meta-analyses and other analyses when relevant and formulation of results and conclusions. Aveyard and Sharp defined SRs as ‘original empirical research’ because they ‘review, evaluate and synthesise all the available primary data, which can be either quantitative or qualitative’ [ 2 ]. Therefore, a SR represents a new research contribution to society and is considered the highest level in the hierarchy of evidence in medicine [ 3 ].

SRs have been proposed as a type of research methodology that should be acceptable as the basis for a graduate research thesis [ 4 , 5 ]. To the best of our knowledge, there are no reports on the acceptance of SRs as the basis for PhD theses. A recent review addressed potential advantages and disadvantages of such a thesis type and presented opposing arguments about the issue [ 5 ]. However, there were no actual data that would indicate how prevalent one opinion is over another with regard to the acceptance of a SR as the primary research methodology for a PhD thesis. The aim of this cross-sectional study was to assess whether a PhD thesis in European biomedical graduate programs can be partly or entirely based on a SR, as well as to explore the attitudes and knowledge of individuals in charge of PhD programs with regard to a thesis of this type.

Participants

The Organization of PhD Education in Biomedicine and Health Sciences in the European System (ORPHEUS) includes 105 institutional members from 40 countries and six associate members from Canada, Georgia, Iran, Kyrgyzstan, Kazakhstan and the USA [ 6 ]. The ORPHEUS encompasses a network of higher education institutions committed to developing and disseminating best practice within PhD training programs in biomedicine, health sciences and public health. ORPHEUS approved the use of their mailing list for the purpose of this study. The mailing list had 1049 contacts. The study authors were not given the mailing list due to data protection and privacy. Instead, it was agreed that ORPHEUS officials would send the survey via email to the mailing list. The General Secretary of the ORPHEUS contacted individuals responsible for PhD programs (directors or deputy directors) among the institutional members, via e-mail, on 5th of July 2016. These individuals were sent an invitation to complete an online survey about SRs as the basis for PhD theses. We invited only individuals responsible for PhD programs (e.g., directors, deputy directors, head of graduate school, vice deans for graduate school or similar). We also asked them to communicate with other individuals in charge of their program to make sure that only one person per PhD program filled out the survey. If there were several PhD programs within one institution, we asked for participation of one senior person per program.

The survey was administered via Survey Monkey (Portland, OR, USA). The survey took 5–10 min to complete. One reminder was sent to the targeted participants 1 month after the first mail.

The ethics committee of the University of Split School of Medicine approved this study, which formed part of the Croatian Science Foundation grant no. IP-2014-09-7672 ‘Professionalism in Health Care’.

Questionnaire

The 20-item questionnaire, designed specifically for this study by both authors (LP and DS), was first tested for face validity and clarity among five individuals in charge of PhD programs. The questionnaire was then modified according to their feedback. The questionnaire included questions about their PhD program; whether PhD candidates are required to publish manuscript(s) before thesis defence; the minimum number of required manuscripts for defending a PhD thesis; the authorship requirements for a PhD candidate with regard to published manuscript(s); whether there is a requirement for a PhD candidate to publish manuscript(s) in journals indexed in certain databases or journals of certain quality, and how the quality is defined; the description about other requirements for defending a PhD thesis; whether a SR partly or fully meets requirements for approval of a PhD thesis in their graduate program; what are the rules related to the use of a SR as the basis for a PhD thesis; and the number of PhD theses based on SRs relative to other types of research methods.

Participants were also asked about their opinion with regard to the main reasons that SRs are not recognised in some institutions as the basis for a doctoral dissertation, and their opinion about literature reviews, using a four-item Likert scale, ranging from ‘agree’ to ‘disagree’, including an option for ‘don’t know’. In the last question, the participants’ knowledge about SR methodology was examined using nine statements; participants had to rate each statement as either ‘correct’, ‘incorrect’, ‘unsure’ or ‘I don’t know’. Finally, participants were invited to leave their email address if they wanted to receive survey results. The survey sent to the study participants can be found in an additional file (Additional file  1 ).

Data analysis

Survey responses were entered into a spreadsheet, checked by both authors and analysed using Microsoft Excel (Microsoft Inc., Redmond, WA, USA). Descriptive data are presented as frequencies and percentages. All raw data and analysed data sets used in the manuscript are available from authors on request. A point-biserial correlation (SPSS, IBM, Chicago, IL, USA) was used to measure the strength of the association between results on the knowledge test (continuous variable) and the attitude towards SRs as the basis for dissertations (dichotomous variable; we used the answer to the following question as this measure: ‘Do you agree that a systematic review, in whole or in part, meets the criteria for a publication on which a doctoral dissertation can be based?’).

Study participants

There are 105 institutions included in the ORPHEUS network. We received a response from 86 individuals representing 68 institutions from 37 countries (65% institutional response rate). There were more respondents than institutions because some institutions have several PhD programs and thus several program directors. Those responders were used as a unit of analysis in the analysis of attitudes and knowledge; institutions were the unit of analysis when analysing criteria for theses. Some of the questionnaires ( n  = 15) were only partly completed. In most cases, the missing data were related to knowledge about SR methodology.

Overview of requirements for a dissertation

Based on the information provided by the graduate program directors, in the majority of the included PhD programs, students were required to publish a research manuscript prepared within their PhD thesis prior to their thesis defence (83%; n  = 64). Among 13 programs (17%) that did not have this requirement, five respondents (38%) indicated that in their opinion their school’s rules related to a PhD thesis should be changed such as to specify that each thesis should be based on work that is already published in a journal.

The minimum number of published manuscripts necessary for the PhD thesis defence was prespecified in 94% ( n  = 60) of the programs that required publication of research manuscripts prior to the thesis defence. In most of the programs (37%; n  = 22), the number of required manuscripts was three or more. Two manuscripts were required in 30% ( n  = 18) and one was required in 33% ( n  = 20) of the programs. In four programs, there was no formal policy on this matter, but there was a strong expectation that the student will have contributed substantially to several manuscripts in peer-reviewed journals.

In most cases, the PhD candidates’ contribution to published manuscripts within the PhD thesis was determined through first authorship. A requirement that a PhD candidate should be the first author on a manuscript(s) that constitutes a PhD thesis was reported in 82% ( n  = 64) of the graduate programs.

In 60% ( n  = 52) of the graduate programs, the quality of the journals where a PhD candidate has to publish research manuscripts as a part of a PhD thesis was defined by the database in which these journals are indexed. The most commonly specified databases were Web of Science (41%; n  = 35) and MEDLINE/PubMed (13%; n  = 11), followed by Science Citation Index, Scopus, Current Contents, a combination of several databases or, in two cases, a combination of journals from a list defined by some governing body.

Systematic reviews as a PhD thesis

SRs, in whole or in part, met the criteria for acceptable research methodology for a PhD thesis in 47% ( n  = 40) of programs, whereas 53% ( n  = 46) of programs specifically stated that they did not accept SRs in this context (Fig.  1 a, b). Among the programs that accepted SRs, theses could be exclusively based on a SR in 42% ( n  = 17) of programs, while in the remaining programs, SRs were acceptable as one publication among others in a dissertation.

a European PhD programs that recognise a systematic review as a PhD thesis (green dot) and those that do not (red dot). Half red and half green dots indicate the five universities with institutions that have opposite rules regarding recognition of a systematic review as a PhD thesis. The pie chart presents b the percentage of the programs in which systematic reviews, in whole or in part, meet the criteria for a dissertation and c the opinion of participants about whether systematic reviews should form the basis of a publication within a PhD dissertation

The majority of participants (80%; n  = 69) indicated that SRs did not meet criteria for a publication on which a PhD dissertation should be based (Fig.  1 c). The main arguments for not recognising a SR as the basis for a PhD thesis are listed in Table  1 . The majority of respondents were neutral regarding the idea that scoping reviews or SRs should replace traditional narrative reviews preceding the results of clinical and basic studies in doctoral theses. Most of the respondents agreed that narrative or critical/discursive literature reviews preceding clinical studies planned as part of a dissertation should be replaced with systematic reviews (Table  2 ).

Most of the programs that accepted SRs as a research methodology acceptable for PhD theses had defined rules related to the use of an SR as part of a PhD thesis (Fig.  2 ). The most common rule was that a SR can be one publication among others within a PhD thesis. Some of the respondents indicated that empty (reviews that did not find a single study that should be included after literature search) or updated reviews could also be used for a PhD thesis (Fig.  2 ).

Frequency of different rules that define the use of systematic reviews as a part of a PhD thesis in European biomedical graduate programs

The results of the survey regarding knowledge about SR methodology indicated that the majority of respondents were not familiar with this methodology. Only three out of nine questions were correctly answered by more than 80% of the participants, and questions about meta-analyses and the type of plots frequently used in a SR were correctly answered by only one third of the participants (Table  3 ). The association between participants’ results on the knowledge test and attitudes towards SRs was tested using a point-biserial correlation; this revealed that lack of knowledge was not correlated with negative attitudes towards SRs ( r pb  = 0.011; P  = 0.94).

In this study conducted among individuals in charge of biomedical graduate programs in Europe, we found that 47% of programs accepted SRs as research methodology that can partly or fully fulfil the criteria for a PhD thesis. However, most of the participants had negative attitudes about such a model for a PhD thesis, and most had insufficient knowledge about the basic aspects of SR methodology. These negative attitudes and lack of knowledge likely contribute to low acceptance of SRs as an acceptable study design to include in a PhD thesis.

A limitation of this study was that we relied on participants’ responses and not on assessments of formal rules of PhD programs. Due to a lack of familiarity with SRs, it is possible that the respondents gave incorrect answers. We believe that this might be the case since we received answers from different programs in the same university, where one person claimed that SRs were accepted in their program, and the other person claimed that they were not accepted in the other program. We had five such cases, so it is possible that institutions within the same university have different rules related to accepted research methodology in graduate PhD programs. This study may not be generalisable to different PhD programs worldwide that were not surveyed. The study is also not generalisable to Europe, as there are no universal criteria or expectations for PhD theses in Europe. Even in the same country, there may be different models and expectations for a PhD in different higher education institutions.

A recent study indicated a number of opposing views and disadvantages related to SRs as research methodology for graduate theses, including lack of knowledge and understanding by potential supervisors, which may prevent them from being mentors and assisting students to complete such a study [ 5 ]. This same manuscript emphasised that there may be constraints if the study is conducted in a resource-limited environment without access to electronic databases, that there may be a very high or very low number of relevant studies that can impact the review process, that methods may not be well developed for certain types of research syntheses and that it may be difficult to publish SRs [ 5 ].

Some individuals believe that a SR is not original research. Indeed, it has been suggested that SRs as ‘secondary research’ are different than ‘primary or original research’, implying that they are inferior and lacking in novelty and methodological rigour as compared to studies that are considered primary research. In 1995, Feinstein suggested that such studies are ‘statistical alchemy for the 21st century’ and that a meta-analysis removes or destructs ‘scientific requirements that have been so carefully developed and established during the 19th and 20th centuries’ [ 7 ]. There is little research about this methodological issue. Meerpohl et al. surveyed journal editors and asked whether they consider SRs to be original studies. The majority of the editors indicated that they do think that SRs are original scientific contributions (71%) and almost all journals (93%) published SRs. That study also highlighted that the definition of original research may be a grey area [ 8 ]. They argued that, in an ideal situation, ‘the research community would accept systematic reviews as a research category of its own, which is defined by methodological criteria, as is the case for other types of research’ [ 8 ]. Biondi-Zoccai et al. pointed out that the main criteria to judge a SR should be its novelty and usefulness, and not whether it is original/primary or secondary research [ 9 ].

In our study, 80% of the participants reported negative attitudes, and more than half of the respondents agreed with a statement that SRs are ‘not a result of the candidate’s independent work since systematic reviews tend to be conducted by a team’. This opinion is surprising since other types of research are also conducted within a team, and single authorship is very rare in publications that are published within a PhD thesis. On the contrary, the mean number of authors of research manuscripts is continuously increasing [ 10 ]. At the very least, the authors of manuscripts within a PhD will include the PhD candidate and a mentor, which is a team in and of itself. Therefore, it is unclear why somebody would consider it a problem that a SR is conducted within a team.

The second most commonly chosen argument against such a thesis was that SRs ‘do not produce enough new knowledge for a dissertation’. The volume of a SR largely depends on the number of included studies and the available data for numerical analyses. Therefore, it is unfair to label a SR as a priori lacking in new knowledge. There are SRs with tens or hundreds of included studies, and some of them not only include meta-analyses, but also network meta-analyses, which are highly sophisticated statistical methods. However, limiting SRs within a thesis only to those with meta-analysis would be unfair because sometimes meta-analysis is not justified due to clinical or statistical heterogeneity [ 11 ] and the presence or absence of a meta-analysis is not an indicator of the quality of a SR. Instead, there are relevant checklists for appraising methodological and reporting quality of a SR [ 12 , 13 ].

The third most commonly chosen argument against SRs within PhD theses was ‘lack of adequate training of candidates in methodology of systematic reviews’. This could refer to either insufficient formal training or insufficient mentoring. The graduate program and the mentor need to ensure that a PhD candidate receives sufficient knowledge to complete the proposed thesis topic. Successful mentoring in academic medicine requires not only commitment and interpersonal skills from both the mentor and mentee, but also a facilitating institutional environment [ 14 ]. This finding could be a result of a lack of capacity and knowledge for conducting SRs in the particular institutions where the survey was conducted, and not general opinion related to learning a research method when conducting a PhD study. Formal training in skills related to SRs and research synthesis methods [ 15 , 16 ], as well as establishing research collaborations with researchers experienced in this methodology, could alleviate this concern.

One third of the participants indicated a ‘lack of appreciation of systematic review methodology among faculty members’ as a reason against such a thesis model. This argument, as well as the prevalent negative attitude towards SRs as PhD theses, perhaps can be traced to a lack of knowledge about SR methodology; however, although the level of knowledge was quite low in our study, there was no statistically significant correlation between knowledge and negative attitudes. Of the nine questions about SR research methodology, only three questions were correctly answered by more than half of the participants. This could be a cause for concern because it has been argued that any health research should begin with a SR of the literature [ 17 ]. It has also been argued that the absence of SRs in the context of research training might severely hamper research trainees and may negatively impact the research conducted [ 18 ]. Thus, it has been recommended that SRs should be included ‘whenever appropriate, as a mandatory part of any PhD program or candidature’ [ 18 ].

It has recently been suggested that the overwhelming majority of investment in research represents an ‘avoidable waste’ [ 19 ]. Research that is not necessary harms both the public and patients, because funds are not invested where they are really necessary, and necessary research may not be conducted [ 17 ]. This is valid not only for clinical trials, but also for other types of animal and human experiments [ 20 ]. SRs can help improve the design of new experiments by relying on current evidence in the field and by helping to clarify which questions still need to be addressed. SRs can be instrumental in improving methodological quality of new experiments, providing evidence-based recommendations for research models, reducing avoidable waste, and enabling evidence-based translational research [ 20 ].

Four respondents from three institutions indicated that empty SRs are accepted as a PhD thesis. While it makes sense to include such a SR as a part of the thesis to indicate lack of evidence in a certain field, it is highly unlikely that an entire thesis can be based on an empty SR, without a single included study.

There are many advantages of a SR as a graduate thesis [ 4 , 5 ], especially as a research methodology suitable for low-resource settings. A PhD candidate can prepare a Cochrane SR as a part of the PhD thesis, yielding a high-impact publication [ 4 ]. Non-Cochrane SRs can also be published in high-impact journals. A PhD candidate involved in producing a SR within a PhD thesis goes through the same research process as those conducting primary research, from setting up a hypothesis and a research question, to development of a protocol, data collection, data analysis and appraisal, and formulation of conclusions. Graduate programs can set limits, such as the prevention of empty reviews and the recognition of updated reviews as valid for a PhD thesis, and engage experienced researchers as advisors and within thesis evaluation committees, to ensure that a candidate will conduct a high-quality SR [ 4 ]. Conducting a SR should not be mandatory, but candidates and mentors willing to produce such research within a graduate program should be allowed to do so.

Further studies in this field could provide better insight into attitudes related to SRs as graduate theses and explore interventions that can be used to change negative attitudes and improve knowledge of SRs among decision-makers in graduate education.

Raising awareness about the importance of SRs in biomedicine, the basic aspects of SR methodology and the status of SRs as original secondary research could contribute to greater acceptance of SRs as potential PhD theses. Our results can be used to create strategies that will enhance acceptance of SRs among graduate education program directors.

Gopalakrishnan S, Ganeshkumar P. Systematic reviews and meta-analysis: understanding the best evidence in primary healthcare. J. Fam. Med Prim Care. 2013;2(1):9–14.

Article   CAS   Google Scholar  

Aveyard H, Sharp P. A beginner’s guide to evidence-based practice in health and social care. Glasgow: McGraw Open Press University; 2011.

Google Scholar  

Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126(5):376–80.

Article   CAS   PubMed   Google Scholar  

Puljak L, Sambunjak D. Cochrane systematic review as a PhD thesis: an alternative with numerous advantages. Biochemia Medica. 2010;20(3):319–2.

ten Ham-Baloyi W, Jordan P. Systematic review as a research method in post-graduate nursing education. Health SA Gesondheid. 2016;21:120–8.

Article   Google Scholar  

Organisation for PhD Education in Biomedicine and Health Sciences in the European System (ORPHEUS). Available at: http://www.orpheus-med.org/ .

Feinstein AR. Meta-analysis: statistical alchemy for the 21st century. J Clin Epidemiol. 1995;48(1):71–9.

Meerpohl JJ, Herrle F, Reinders S, Antes G, von Elm E. Scientific value of systematic reviews: survey of editors of core clinical journals. PLoS One. 2012;7(5):e35732.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Biondi-Zoccai G, Lotrionte M, Landoni G, Modena MG. The rough guide to systematic reviews and meta-analyses. HSR proc intensive care cardiovascular anesth. 2011;3(3):161–73.

CAS   Google Scholar  

Baethge C. Publish together or perish: the increasing number of authors per article in academic journals is the consequence of a changing scientific culture. Some researchers define authorship quite loosely. Dtsch. Arztebl. Int. 2008;105(20):380–3.

PubMed   PubMed Central   Google Scholar  

Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Statist Med. 2002;21:1539–58.

Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6(7):e1000100.

Article   PubMed   PubMed Central   Google Scholar  

Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10.

Sambunjak D, Straus SE, Marusic A. A systematic review of qualitative research on the meaning and characteristics of mentoring in academic medicine. J Gen Intern Med. 2010;25(1):72–8.

Article   PubMed   Google Scholar  

Balajic K, Barac-Latas V, Drenjancevic I, Ostojic M, Fabijanic D, Puljak L. Influence of a vertical subject on research in biomedicine and activities of the Cochrane collaboration branch on medical students’ knowledge and attitudes toward evidence-based medicine. Croat Med J. 2012;53(4):367–73.

Marusic A, Sambunjak D, Jeroncic A, Malicki M, Marusic M. No health research without education for research—experience from an integrated course in undergraduate medical curriculum. Med Teach. 2013;35(7):609.

Mahtani KR. All health researchers should begin their training by preparing at least one systematic review. J R Soc Med. 2016;109(7):264–8.

Olsson C, Ringner A, Borglin G. Including systematic reviews in PhD programmes and candidatures in nursing - ‘Hobson’s choice’? Nurse Educ Pract. 2014;14(2):102–5.

Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–9.

de Vries RB, Wever KE, Avey MT, Stephens ML, Sena ES, Leenaars M. The usefulness of systematic reviews of animal experiments for the design of preclinical and clinical studies. ILAR J. 2014;55(3):427–37.

Download references

Acknowledgements

The authors thank the ORPHEUS secretariat for administering the survey and the study participants for taking time to participate in the survey. We are grateful to Prof. Ana Marušić for the critical reading of the manuscript.

This research was funded by the Croatian Science Foundation, grant no. IP-2014-09-7672 ‘Professionalism in Health Care’. The funder had no role in the design of this study or its execution and data interpretation.

Availability of data and materials

The datasets used and/or analysed for the current study are available from the corresponding author on reasonable request.

Author information

Authors and affiliations.

Cochrane Croatia, University of Split School of Medicine, Šoltanska 2, 21000, Split, Croatia

Livia Puljak

Department for Development, Research and Health Technology Assessment, Agency for Quality and Accreditation in Health Care and Social Welfare, Planinska 13, 10000, Zagreb, Croatia

Laboratory for Pain Research, University of Split School of Medicine, Šoltanska 2, 21000, Split, Croatia

Livia Puljak & Damir Sapunar

You can also search for this author in PubMed   Google Scholar

Contributions

Both authors participated in the study design, data collection and analysis and writing of the manuscript, and both read and approved the final manuscript.

Corresponding author

Correspondence to Livia Puljak .

Ethics declarations

Ethics approval and consent to participate.

The Ethics Committee of the University of Split School of Medicine approved the study. All respondents consented to participate in the study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:.

Online survey used in the study. Full online survey that was sent to the study participants. (PDF 293 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Puljak, L., Sapunar, D. Acceptance of a systematic review as a thesis: survey of biomedical doctoral programs in Europe. Syst Rev 6 , 253 (2017). https://doi.org/10.1186/s13643-017-0653-x

Download citation

Received : 29 August 2017

Accepted : 30 November 2017

Published : 12 December 2017

DOI : https://doi.org/10.1186/s13643-017-0653-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Systematic review
• PhD program
• Biomedicine
• Study design

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Writing your thesis and conducting a literature review

• Writing your thesis

Your literature review

• Defining a research question
• Choosing where to search
• Search strings
• Limiters and filters
• Developing inclusion/exclusion criteria
• Managing your search results
• Screening, evaluating and recording
• Snowballing and grey literature
• Further information and resources

Most PhD and masters’ theses contain some form of literature review to provide the background for the research.  The literature review is an essential step in the research process. A successful literature review will offer a coherent presentation and analysis of the existing research in your field, demonstrating:

• Your understanding of the subject area
• Gaps in current knowledge (that may in turn influence the direction of your research) 
• Relevant methodologies

There are different approaches and methods to literature reviews, and you may have heard of terms like systematic, structured, scoping or meta-analysis. This is when the literature review becomes the research methodology in its own right, instead of forming part of the research process.

This table shows the differences between a traditional literature review and a structured or systematic literature review.

Structured vs traditional literature reviews

What is a traditional literature review?

A traditional literature review is a critical review of the literature on a particular topic. The aim of this type of literature review is to identify any background research on your topic and to evaluate the quality and relevance of the literature. You will use your literature review to understand what has already been researched, help develop your research questions and the methodology that you should follow to collect and to identify any areas that your research can explore. You want your research to be unique so you will use a literature review to prevent you duplicating any previous research but also identifying any errors or mistakes that you would want to avoid.

A literature review is aimed at Masters (MSc students) and research level.

What is a structured literature review?

A structured literature review involves bringing many research studies together to use them as the data to determine findings (known as secondary research). There is no other form of data collection involved such as creating your own surveys and questionnaires (primary research). This approach allows you to look beyond one dataset and synthesise the findings of many studies to answer your clearly formulated research question.

Sometimes a structured review can be described as being a systematic literature review. A structured review typically does not fulfil all of the criteria for a full systematic review but may take a similar approach by taking a systematic, step by step method to find literature. They tend to follow a set protocol for determining the research studies to be included and every stage is documented.

To help you prepare for your structured literature review please complete this interactive workbook.

For Logistics students only

To help you prepare for your systematic literature review please complete this interactive workbook.

What is a systematic literature review?

A systematic literature review is a specific research methodology to identify, select, evaluate, and synthesise relevant published and unpublished literature to answer a particular research question. The systematic literature review should be transparent and replicable, you should follow a predetermined set of inclusion and exclusion criteria to select studies and help minimise bias. A systematic literature review may be registered, so that others can discover and minimise duplication, and can take several years to complete.

The systematic literature review is aimed at research (PhD students) level.

Useful background reading

Cranfield Libraries have several books offering guidance on how to approach and conduct literature reviews, and structured or systematic literature reviews:

• Reading list for  literature review and study skills
• Reading list of items to support a structured or systematic literature review

Looking at previous structured and systematic literature reviews is an effective way to understand what is required and how they should be structured and written up. Structured literature reviews can be found in the Masters Thesis Archive (MTA) and systematic literature reviews can be found in the Cranfield University institutional Repository, CERES. Check out the Theses  link.

• << Previous: Writing your thesis
• Next: Defining a research question >>
• Last Updated: Apr 11, 2024 10:52 AM
• URL: https://library.cranfield.ac.uk/writing-your-thesis

• Reference Manager
• Simple TEXT file

People also looked at

Systematic review article, a systematic review of doctoral graduate attributes: domains and definitions.

• 1 Department of Psychology, University of the Western Cape, Bellville, South Africa
• 2 Research and Innovation, University of the Western Cape, Bellville, South Africa

Doctoral graduate attributes are the qualities, skills, and competencies that graduates possess, having completed their doctorate degree. Graduate attributes, in general, lack conceptual clarity, making the investigation into and quality assurance processes attached to doctoral outcomes challenging. As many graduate attributes are “unseen” or implicit, the full range of attributes that doctoral graduate actually possess needs to be synthesized, so that they may be recognized and utilized by educational stakeholders. The aim of this study was to establish and describe what attributes graduates from doctoral degrees possess. A systematic review of peer-reviewed, primary literature published between January 2016 and June 2021 was conducted, identifying 1668 articles. PRISMA reporting was followed, and after screening and full text critical appraisal, 35 articles remained for summation through thematic synthesis. The doctoral graduate attribute domains identified included knowledge, research skills, communication skills, organizational skills, interpersonal skills, reputation, scholarship, higher order thinking skills, personal resourcefulness, and active citizenship. Many of the domains were conceptualized as transferable or interdisciplinary, highlighting the relevance of the attributes doctoral graduates possess. The review findings align with existing frameworks yet extend those that tend to focus on generic “seen” attributes, and include a range of “unseen”, intrinsic qualities as outcomes of the doctoral degree. The review contributes to the conceptual development of doctoral graduate attributes, by synthesizing actual outcomes, as opposed to prospective attributes or attributes-in-process. Doctoral graduate attributes should be conceptualized to integrate both generic attributes alongside intrinsic qualities that are important for employability. Increased awareness as to the scope of doctoral graduate attributes among stakeholders, such as doctoral supervisors, students, graduates and employers, may facilitate improved educational outcomes and employability. Future research into the contextual relevance of the domains identified and how they are developed may be beneficial. Future research could involve the development of context-relevant scales to empirically measure doctoral graduate attributes among alumni populations, as a quality assurance outcome indicator. Such findings could inform program reform, improving the relevance of doctoral education and the employability of doctoral graduates.

Introduction

Doctoral graduate attributes are defined as the qualities or characteristics of a doctoral graduate, integrating skills, knowledge and competencies with doctoral identity ( Yazdani and Shokooh, 2018 ). Graduate attributes are of interest within the context of higher education quality assurance and the international focus on producing skilled and employable graduates ( Bitzer and Withering, 2020 ). Graduate attributes are typically defined institutionally and embedded in curriculum learning outcomes ( Bridgstock, 2009 ; Mashiyi, 2015 ). However, doctoral degrees often lack a standardized curriculum, with the primary focus being original research under supervision ( Elliot et al., 2016 ; Molla and Cuthbert, 2016 ), leaving no formal frame into which doctoral graduate attributes may be embedded. The transferability of graduate attributes is an important consideration for higher education institutions, so that the attributes instilled are relevant to multiple work contexts, enhancing graduate employability ( Kensington-Miller et al., 2018 ). This is particularly relevant in the context of doctoral education, with the shift away from the thesis as the primary outcome, and the increased demand for transferable skills to the increasingly competitive world of work ( Denicolo and Reeves, 2014 ). Doctoral graduate attributes, as outcomes of doctoral education, are important to consider for quality assurance, the employability of graduates, and the relevance of doctoral education.

Graduate attributes are generally critiqued as lacking conceptual clarity ( Mowbray and Halse, 2010 ; Bitzer and Withering, 2020 ). This conceptual ambiguity is reflected in the “range of adjectives such as “transferable”, “generic”, “soft”, “key”, “graduate” and “employability” [that] have been diversely paired with nouns such as “skills”, “attributes”, “outcomes” and “capabilities”” ( Kensington-Miller et al., 2018 , p. 1440). In short, the following differentiations between terms may be made: “competence” is a performance-based term, referring to successful or efficient performance whereas “competencies” refers to the knowledge and behaviors that, if utilized effectively, result in competent performance ( Potolea, 2013 ; Durette et al., 2016 ). “Skills” are typically learned abilities or expertise, but can be more broadly defined to include “the acquisition or development of specific capacities, abilities, aptitudes or competencies” ( Gilbert et al., 2004 , as cited by Mowbray and Halse, 2010 , p. 655). “Attributes” refers to the inherent characteristics or features of someone or something. By extension, doctoral graduate attributes would be the features or characteristics of doctoral graduates, and may thus be an umbrella term that integrates skills, knowledge and competencies, as expressed by Yazdani and Shokooh (2018) . This definition allows for the multidimensional and interrelated nature of the qualities and skills doctoral graduates possess ( Mowbray and Halse, 2010 ).

Measuring graduate attributes is complicated, partly due to the conceptual ambiguity noted. Graduate attributes are typically developed longitudinally, making them challenging to operationalize, and context dependent, limiting the generalizability of scales ( Hughes and Barrie, 2010 ; Cavanagh et al., 2015 ; Nell and Bosman, 2017 ). Graduate attributes often include a combination of skills, dispositions, values, and competencies that may be more abstract and difficult to quantify ( Hinchliffe and Jolly, 2011 ). Many graduate attributes are “unseen” or “invisible” as they may reflect the qualities of the person, and are not as explicit as those clearly embedded in the curriculum or formal degree processes ( Kensington-Miller et al., 2018 ). These “invisible” attributes are often implied, yet are important to consider for a graduates' overall profile, such as resilience ( Kensington-Miller et al., 2018 ). It is important to synthesize evidence on what attributes doctoral graduates actually do possess, in order to reconsider the scope of what is included in traditional lists of doctoral graduate attributes. In so doing, due consideration may be given to the “unseen” attributes that are outcomes of the doctoral degree. The notion of implicit attributes in parallel to those explicit to the “product” of the degree, aligns with the shift of focus in doctoral education from focusing exclusively on the production of the doctoral thesis, to include the holistic development and tacit learning involved in the doctoral journey ( Mowbray and Halse, 2010 ; Yazdani and Shokooh, 2018 ). Despite the challenges associated with conceptualizing graduate attributes, they remain pertinent to assess ( Bitzer and Withering, 2020 ).

There has been significant investment in improving postgraduate education, including the implementation of transferable skills development, particularly in the Global North ( Denicolo and Reeves, 2014 ). This has given rise to various models and frameworks related to researcher attributes and doctoral competencies. One of the most widely used frameworks, particularly in the United Kingdom, is the Researcher Development Framework ( Vitae, 2010 ). According to Reeves et al. (2012 , p. 4), “the purpose of the framework is to support the development of individual researchers while enhancing our capacity to build a workforce of world-class researchers within the UK higher education research base.” As such, it can be used to facilitate qualitative reflection on one's development, and encompasses a wider view of researcher career progression, beyond the doctorate degree. Similarly, the Researcher Skill Development Framework ( Willison and O'Regan, 2008/2015 ), developed in the Australian context, provides a developmental framework of research-related skills and agency, from first year undergraduate studies to established academics. The framework includes a qualitative matrix that may be well suited for developmental use in the context of doctoral supervision, or for curriculum development, rather than as an outcome indicator. The competence model for science, engineering and technology (SET) Ph.D. students and graduates ( Nikol and Lietzmann, 2019 ), in the broader European context, pertains to the doctorate, yet is focused on SET disciplines only and is curriculum focused, rather than outcomes focused. Notably, all of the above frameworks were developed in the Global North. In general, these models have been used with focus on the development of doctoral education and training, including formal curriculum and transferable skills development programs. They are well suited to use for personal, qualitative reflection on one's skill development. In the South African context, the Council for Higher Education ( CHE, 2018 ) has compiled the qualification standards for doctoral degrees, including a prospective list of graduate attributes, with five knowledge and four skill domains. However, these domains are not theoretically defined or clearly operationalized.

The Researcher Development Framework was validated prior to its launch in 2011 ( Reeves et al., 2012 ). Since then, there has been extensive ongoing work in the field of doctoral education and efforts to develop context and field specific models, such as CHE (2018) and Nikol and Lietzmann (2019) . Further, many institutional or governmental frameworks are aspirational ideals of the attributes institutions hope graduates will possess ( Kensington-Miller et al., 2018 ) that do not necessarily reflect the attributes that graduates actually do possess when they graduate. As such, there is a need to identify what recent evidence there is of the attributes doctoral graduates possess after completing their doctorate degrees. The focus of the models above aligns with the typical focus of doctoral education evaluation: the doctoral process and student experiences during the doctorate ( Luo et al., 2018 ). However, doctoral graduate attributes also need to be conceptualized as outcomes, rather than prospective qualities, in order to facilitate good, empirical quality assurance ( Harley, 2020 ). Therefore, the target population for evaluating graduate attributes as outcomes should be graduates who have completed their degree. As noted by Durette et al. (2016 , p. 1356):

Ph.D. students might not be the adequate population to study competencies developed during doctoral training since (1) they have not finished it entirely and (2) Ph.D. students might not be well aware of the competencies they have developed… in particular because they have not had the opportunity to exercise these competencies in other professional contexts.

There is a need to consolidate evidence of doctoral graduate attributes from the perspective of graduates only, excluding student populations. In so doing, a synthesis of the doctoral graduate attributes as outcomes, once the doctorate has been completed, may be possible. This may give preliminary indications of the extent to which developmental models used in curriculum and personal development, such as those identified above, have translated into real outcomes for doctoral graduates.

Limited empirical research exists attempting to synthesize graduate attributes ( Bridgstock, 2009 ). The Researcher Development Framework is an example of extensive work toward synthesizing doctoral graduate attributes ( Reeves et al., 2012 ). A recent example is the conceptual analysis of “doctorateness” by Yazdani and Shokooh (2018) that included literature published between 1995 and 2016. While the study appears to follow some review processes, it does not reflect the rigor required of a systematic review ( Page et al., 2021 ). The article provides a synthesis of “doctorateness” as a concept, however, the findings were limited to broad categories of attributes, without definitions or detail as to what these domains entail. Further, much literature has been published since 2016 that warrants consideration. Therefore, there is a gap in the consolidation of more recent literature that bears global evidence of the attributes that doctoral graduates possess.

The aim of this review was to establish and describe what attributes graduates from doctoral degrees possess, through a systematic review of recent, high-quality research literature. The objectives of the review were: (1) to identify doctoral graduate attribute domains and sub-domains, and (2) to clarify their theoretical and/or operational definitions.

Study design

A systematic review is a rigorous, systematic process used to filter and synthesize available evidence on a topic ( Laher and Hassem, 2020 ). There is a need to filter evidence to focus on a specific perspective, i.e., that of doctoral graduates, to the exclusion of doctoral students, in order to consolidate recent evidence of what attributes doctoral graduates actually possess, specifically focusing on the conceptualization of doctoral graduate attributes as outcomes of the doctoral degree. A systematic review is a suitable method to filter the available evidence and synthesize the various definitions and iterations of doctoral graduate attributes. Systematic reviews are recommended for use in scale development, as part of identifying and clarifying the scale construct ( Munnik and Smith, 2019 ), and thus are a suitable method for facilitating conceptual clarity of a constructs, such as graduate attributes.

Review question

The systematic review question was: what attributes do graduates from doctoral degrees possess? The formulation of PEO (population, exposure and outcome) was used ( Moola et al., 2020 ), with doctoral graduates as the population, the doctoral degree as the exposure, and doctoral graduate attributes being the outcome.

Eligibility

Inclusion and exclusion criteria were set a priori , to systematically determine which articles to include ( Gough et al., 2017 ). Articles had to meet the following criteria to be eligible for inclusion:

1. The participants were graduates from doctoral degree programs (any discipline, no geographic limitation, no specified timeframe since graduation);

2. A clear focus on the attributes of doctoral graduates must be present (e.g., skills, competencies, abilities);

3. Qualitative, quantitative and mixed-methods research were considered; and

4. Published between January 2016 and June 2021, to ensure that recent literature on the topic was included.

Articles were excluded if they were not published and peer reviewed. Gray literature, such as theses and conference proceedings, were excluded. Articles that were not accessible in full text and in English were excluded. The University of the Western Cape (UWC) library assisted in locating full text and English translations, where necessary.

Information sources

Databases were accessed through uKwazi, a composite search function available through the UWC library. Databases included: Academic Search Complete, Directory of Open Access Journals, EBSCOhost, Emerald Journals, JSTOR, Sabinet, SAGE, Springer, Taylor and Francis, and Wiley. The use of uKwazi allowed for a comprehensive, composite search of all databases simultaneously, using the same search terms and limiters, thus enhancing the systematic nature of the review. The use of this platform automatically excluded duplicate articles, streamlining the screening process.

Search strategy

The PEO model was used to develop search strings, as shown in Table 1 . The population of interest was doctoral graduates who completed their degree, thus excluding students, as the focus was specifically on attributes as outcomes. The term “graduate” was used to ensure specificity, as the inclusion of “student” would have resulted in too wide a search. The search terms related to exposure referred to the pursuit of the doctoral degree and was intentionally general to include various formats of doctoral degree study. The search terms included “doctorate OR doctoral OR Ph.D.”, which would include any combination of fields of study, for example, professional doctorate, Doctor of Education, or Doctor of Philosophy. This was intentionally kept broad, as many studies did not specify what kind of doctorate was reported on or included mixed groupings from various types of doctorates. The outcome of doctorate graduate attributes, for which the search string was kept general, to include various iterations of nouns used in relation to doctoral graduate attributes.

Table 1 . Search strategy.

The search strings were compiled into a Boolean phrase, as indicated in Table 1 that was entered into uKwazi for a composite search of all databases simultaneously. The first and third strings were searched in all fields, to include titles, abstract and subject. The second string was searched only in the subject field, to exclude irrelevant articles that were included in the search due to the authors' credentials (e.g., Ph.D.). The composite search was limited to include only articles, available in English, peer reviewed, with full text available online, and published between 2016 and 2021—in alignment with the inclusion and exclusion criteria. These limiters were applied on the search platform as part of the search process, prior to recording the search results.

A second search, using the single search term “doctorateness” (any field) was conducted through uKwazi. The same limiters were applied. The supplementary search was conducted to ensure that potentially relevant articles were not unintentionally excluded.

Study selection

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) was followed to ensure methodological rigor and transparency in reporting ( Page et al., 2021 ). The citation information of each article identified in the search was imported into Rayyan, an online systematic review platform ( Ouzzani et al., 2016 ). Rayyan was utilized to streamline the review process and enhance reporting, as it facilitated dual review and tracking of exclusion reasons. Two reviewers independently screened the titles and abstracts for relevance against the inclusion and exclusion criteria. Articles that did not meet all the inclusion criteria or met at least one of the exclusion criteria were excluded. The full texts of all remaining articles were retrieved and screened for eligibility, against the inclusion criteria. Reviewers independently screened each article, noting reasons for inclusion or exclusion. Disagreements were resolved through discussion.

Quality appraisal tool

The methodological quality of the remaining articles was assessed through the Smith, Franciscus and Swartbooi (SFS) full text critical appraisal tool, version E ( Smith et al., 2015 ). The tool includes three sections: (A) purpose, including problem statement, literature and theoretical framework (22 points); (B) methodological rigor, including design, sampling, data collection, data analysis, results, discussion and conclusion (52 points); and (C), general considerations of publication and peer review status (5 points). A minimum threshold score of 60% (strong to excellent quality) was set that must be met for inclusion in the review ( Smith et al., 2015 ), to ensure that only high quality research was included. The critical appraisals were independently conducted by two reviewers. Articles with scores that differed by five or more points were reviewed through discussion ( n = 4), until consensus was reached.

Data extraction and synthesis

A self-developed data extraction table was used to extract descriptive data (location, design, sample etc.) and analytic information relating to the doctoral graduate attribute domains and definitions. The review findings were analyzed using thematic synthesis ( Gough et al., 2017 ). Thematic synthesis is the equivalent of thematic analysis in primary research, and follows a similar process of coding and theme development ( Gough et al., 2017 ). The coding process employed was selective, focusing primarily on the findings reported ( Gough et al., 2017 ), for example, the attributes doctoral graduates indicated they possessed after having completed their studies. A hybrid inductive-deductive approach was used, with a primarily inductive approach was used for the initial coding, with codes emerging from the text ( Xu and Zammit, 2020 ). A deductive approach was utilized for subsequent readings of the articles, to identify potential codes that may have been overlooked in the initial coding, and for theme development to group the codes into subdomains and domains. The deductive approach drew on existing literature related to doctoral graduate attributes, such as Vitae (2010) , Yazdani and Shokooh (2018) , and Nikol and Lietzmann (2019) . In order to compare the findings of the review to existing models, selected frameworks were coded, using a deductive approach, to facilitate mapping against the domains and subdomains identified in the review. In some instances, the detail of descriptions differed, and so it was not always possible to map to the models exactly.

Ethics considerations

This review is part of a broader study, which obtained ethics clearance from the Humanities and Social Sciences Research Ethics Committee at UWC (HS21/7/19). Secondary data collection in the form of a systematic review does not require consent. However, consideration of the ethical practice and quality of each article under review, through critical appraisal, ensured the quality of the review findings. The authors of the original work were appropriately cited, so that there was no infringement on intellectual property or copyright. Only sources available in the public domain were utilized, and those that the researchers had legal access to through their institution. Furthermore, permission for the use of the SFS critical appraisal tool was obtained from the author.

The findings of the review include the outcome of the study selection process, the quality appraisal of articles, the descriptive characteristics of the studies in the review, and the doctoral graduate attributes domains, subdomains and descriptions that were identified.

Process results

A total of 1,701 articles were identified in the review ( Figure 1 ). Duplicates were automatically excluded in the comprehensive, integrated search through uKwazi. Duplicates between searches one and two were manually removed ( n = 33). Studies were manually excluded if they were secondary studies (e.g., reviews; secondary analysis), did not include reporting on primary data (e.g., letter to editor; theoretical papers), were outside of the publication range (first published prior to January 2016 or after June 2021); included the incorrect population (graduates from other degree levels; Ph.D. student populations without any Ph.D. graduates represented; or did not allow for differentiation between graduates and students); the incorrect exposure (e.g., did not explicitly relate to the Ph.D.); or the incorrect outcome (e.g., focus on doctoral experience or attributes needed for completion, without explicit mention of attributes possessed on completion). A total of 35 articles met all criteria and were included in the review.

Figure 1 . PRISMA flowchart of data collection and screening [Adapted from Page et al. (2021) ].

Quality appraisal results

All studies included in the review ( n = 35) exceeded the threshold score of 60% in the quality appraisal stage ( Table 2 ), and thus had strong to excellent methodological quality ( Smith et al., 2015 ). A common methodological gap identified in articles through the appraisal was not reporting on the analysis methods used ( n = 18). The focus of the review was on the domains relating to doctoral graduate attributes covered, so the actual results of the studies under review were not the primary focus. Further, the review was descriptive, thus the goal was not generalizability. There was sufficient evidence of methodological quality in the articles included for synthesis.

Table 2 . Quality appraisal scores of included articles.

Study characteristics

The studies included in the review ( n = 35) represented various universities internationally ( Table 3 ). Graduates from universities in the United States of America (USA) ( n = 9), Australia ( n = 8), and the United Kingdom (UK) ( n = 7) were most strongly represented. Graduates from universities across Europe (Belgium, France, Germany, Ireland, Poland, Portugal, Spain, Sweden, Switzerland, and the UK), North America (USA and Canada); Asia (Bhutan, China, Japan, and Malaysia), and Africa (South Africa) were represented. Most studies focused on a single country ( n = 28), if not a single university within that country. Cross-country comparisons were evident in seven studies. The studies reflected greater representation of graduates from institutions in the USA, Australia and the UK. This aligns with the USA and UK being among the top producers of doctoral graduates among Organization for Economic Co-operation and Development (OECD) countries ( OECD, 2019 ). There was low representation of graduates from universities in Asia, yet there has been noted growth in doctoral enrolments in China specifically since the early 2000s ( Luo et al., 2018 ). While there was lower representation of graduates from African institutions, it is unsurprising that the two studies included represented graduates from South African institutions, as there has been significant local investment in doctoral education in recent years, and South African higher education institutions attract doctoral students from various African countries ( Molla and Cuthbert, 2016 ). There was no representation of graduates from institutions in the Middle East or South America. Due to increased trends of mobility and internationalization in higher education ( OECD, 2019 ), it is likely that doctoral graduates from various nationalities were represented in the study, however, the nationalities of graduates were not consistently reported on.

Table 3 . Study characteristics.

The articles under review reported various methodologies, primarily interviews ( n = 13), surveys ( n = 11) or autoethnographies ( n = 7). Sample sizes ranged from 1 ( Boulos, 2016 ; Ai, 2017 ) to 2794 ( Durette et al., 2016 ). According to the inclusion criteria, all studies included doctoral graduates. In Table 3 , the graduate samples were classified as early career researchers (ECR) who graduated within the 5 years, mid-career researchers (MCR) who graduated between 6 and 15 years prior ( Nguyen et al., 2020 ), and those who graduated more than 15 years prior. Studies focused primarily on ECR and MCR. Samples of only doctoral graduates were most common ( n = 23). Mixed populations were included, where differentiation of the perspectives of graduates from other participants was possible. Some studies included faculty, supervisors, key persons in higher education, doctoral students and non-completers as participants. Most studies focused on a specific degree ( n = 26), with education-related fields having the highest frequency ( n = 10). The remaining studies ( n = 9) included cross-disciplinary samples of doctoral degrees in science, technology, engineering and mathematics (STEM) and humanities, arts, and social sciences (HASS). There was a range of disciplines and fields represented, with good representation of both HASS and STEM fields. In half of the studies ( n = 17), the doctoral degrees reported on were Doctor of Philosophy (Ph.D.). Professional doctorates were specified in a fifth of the articles ( n = 7), indicated by “Doctor of [field name]”, for example “Doctor of Education” or “Ed.D”. The remaining articles did not specify if it was a Ph.D. or professional doctorate ( n = 11). Given the greater prevalence of Ph.D. programs in comparison to professional doctorate programs, this may demonstrate good representation of both degree types. Neither type of doctorate nor field of study were gaps in the review. There were no notable trends in the type of doctorate, field of study, geographic location and emerging domains.

Scalar information on subscale(s) measuring dimensions related to doctoral graduate attributes were reported in six studies. Subscales addressing teaching ( Allgood et al., 2018 ; Shih et al., 2019 ) and research skills ( Luo et al., 2018 ; Shih et al., 2019 ) were used in two studies each. Subscales of general scientific competencies ( Grab-Kroll et al., 2019 ), and universal skills ( Luo et al., 2018 ) were identified. Two other studies ( Walker and Yoon, 2017 ; Bröchner and Sezer, 2020 ) provided nominal information on the scales utilized, with insufficient detail to identify what domains were covered. The scales were either adapted from other studies ( Allgood et al., 2018 ; Shih et al., 2019 ; Bröchner and Sezer, 2020 ), or self-developed for the study ( Walker and Yoon, 2017 ; Luo et al., 2018 ; Grab-Kroll et al., 2019 ). Shih et al. (2019) was the only study to report on psychometric properties. Where information was available, subscales had between 2 and 13 items per domain. Items were most often in Likert scale format ( Allgood et al., 2018 ; Luo et al., 2018 ; Grab-Kroll et al., 2019 ; Shih et al., 2019 ). Other formats used included continuous scales ( Bröchner and Sezer, 2020 ) and multiple-choice formats ( Walker and Yoon, 2017 ).

Doctoral graduate attributes

The studies under review rarely defined or mentioned doctoral graduate attributes explicitly, referring more often to the impact of the doctorate or the experiences of doctoral graduates. Related terms, such as competence, competency and competencies were noted. Greene et al. (2021) defined competence as “knowing negotiated within a single community of practice” (p. 95), suggesting that competence is context specific. Durette et al. (2016) defined competency by differentiating between the output, competent performance, and input—the “underlying attributes required for a person to achieve competent performance” ( Hoffmann, 1999 , as cited by Durette et al., 2016 , p. 1356). Durette et al. (2016) further defined competencies as “the resources available to doctorate holders to act competently” (p.1357). While these definitions differ, they all point to the definition of doctoral graduate attributes adopted for this study, which includes the qualities, skills, and competencies that doctoral graduates possess. Interestingly, of the frameworks and models identified previously, only two articles mentioned the Researcher Development Framework ( Bryan and Guccione, 2018 ; Creaton and Anderson, 2021 ), and this was in the introduction of the articles. These articles were two of the seven articles reflecting research conducted in the UK context, where the Researcher Development Framework was developed. As such, is appears that none of the articles under review explicitly drew on the conceptual and theoretical development that is evidenced in the existing models.

The studies addressed a wide range of doctoral graduate attributes identified as outcomes of the doctoral degrees mentioned in the studies. The domains include: knowledge, research, communication, organizational skills, interpersonal skills, scholarship, reputation, higher order thinking skills, personal resourcefulness, and active citizenship ( Table 4 ). Theoretical definitions of the actual domains identified in the studies were sparse, lacking theoretical or conceptual frameworks. These definitions were more general and process-oriented, not linked to a specific domain. Theoretical definitions for the identified domains and/or subdomains are provided, where available. Thereafter, descriptions of what these attributes entail are provided, i.e., how they may be operationalized.

Table 4 . Doctoral graduate attributes domains and subdomains.

The first domain of “knowledge” included codes that related to the knowledge doctoral graduates possessed ( n = 18). This excluded knowledge that explicitly related to “research”, as these were included under a subsequent domain. There was no explicit definition of “knowledge” in the articles under review. However, Adham et al. (2018) made reference to explicit and tacit knowledge, differentiating between knowledge that can be communicated or shared with others, and knowledge that is not as easily communicated. Knowledge development was defined as the processing of information within the individuals' “foundations, understanding, experience and beliefs” ( Adham et al., 2018 , p. 813). The various types of knowledge that doctoral graduates possessed were grouped into the following subdomains: discipline specific knowledge, interdisciplinary knowledge, and professional knowledge. In some instances, doctoral graduates were vaguely noted to possess “knowledge”, with no further explanation of what type of knowledge was indicated ( n = 2). In some studies, doctoral graduates were found to possess discipline specific knowledge, referring to the breadth and depth of their disciplinary knowledge base. Studies found that doctoral graduates' disciplinary expertise included the theoretical knowledge ( n = 14) and practical or technical skills of their discipline ( n = 3). Both their research and coursework were noted as sources of these forms of knowledge. In some instances, doctoral graduates had interdisciplinary knowledge from related or complementary disciplines ( n = 4), and had a unique and/or holistic perspective. Interdisciplinarity was defined as “integrating knowledge from two or more disciplines” ( Holley, 2018 , p. 107). Some studies found that doctoral graduates possessed professional knowledge relating to navigating the administrative and operational functioning of higher education institutions and/or work environment ( n = 5). The domain of knowledge, as a doctoral graduate attribute, thus includes subdomains of disciplinary and interdisciplinary knowledge, and professional knowledge.

Research skills

The domain of “research” included all codes that reflected skills utilized in the various stages of research, and included competencies related to research methods and processes, and attitudes related to research ( n = 21). This was labeled as the domain of research skills. There were no theoretical definitions related to research skills provided in the articles under review. In some instances, research skills were generally mentioned, without specific description ( n = 6). Studies identified that doctoral graduates were noted to possess skills related to the various stages of research: literature review ( n = 6), conceptualization ( n = 2), methods ( n = 4), and/or data collection and analysis ( n = 9). Literature skills reflected their ability to search, critically evaluate, synthesize, and write a literature review. Skills related to conceptualization included the ability to formulate research hypotheses, understand research ethics, and select suitable methods. Research methods included the knowledge of methods, and the ability to conduct quantitative and/or qualitative research. Data collection and data analysis skills included the context-relevant use of quantitative and/or qualitative data collection and data analysis methods. Consideration of discipline-specific methods skills ( n = 3), such as designing appropriate experimental controls, and the reflexive process of artistic research, and interdisciplinary research skills were noted ( n = 1). Doctoral graduates were noted to possess research expertise ( n = 1), and a research attitude ( n = 4) denoted by a respect for knowledge, a broadened outlook, research ownership and rigor. The domain of research skills that doctoral graduates possess included subdomains of range of methodological competencies, from conceptualization to data analysis, as well as research attitude and research expertise.

Communication skills

The domain of “communication” included codes that referenced various formats and forms of communication ( n = 16). There was no evidence of theoretical definitions of communication skills. In some studies, doctoral graduates were noted to possess language skills ( n = 3), and were articulate and confident in their communication skills ( n = 8). The written communication skills doctoral graduates possessed ( n = 8) included academic, scientific and technical writing skills, and being able to construct persuasive arguments. These writing skills ( n = 8) were utilized for various purposes and formats of written documents. Further, it was found that doctoral graduates possessed confidence in their written skills. Doctoral graduates' publication skills ( n = 3) were differentiated from their general writing skills, as this included knowledge of the journal landscape and publication process, and the skills to prepare an article, work with co-authors, negotiate and manage the publication process, and deal with rejection and reviewer feedback. Doctoral graduates possessed oral communication skills ( n = 4), including general presentation skills, and the dissemination of research findings through the presentation of scientific content. As with the domain of knowledge, some studies indicated that doctoral graduates possessed discipline specific communication skills ( n = 2), such as interviewing skills, and interdisciplinary communication skills ( n = 3), in their ability to communicate with non-academic audiences and produce non-academic outputs. The interdisciplinary nature of these communication skills is linked to the concept of research translation, that is defined as the “multidirectional nature of knowledge exchange between researchers and end-users” ( Merga and Mason, 2021 , p. 673). Communication skills as a domain thus included various modes and formats of communication, reflected in the subdomains.

Organizational skills

The domain of “organizational skills” reflects the skills that were learnt through managing the thesis project ( n = 8). In some studies, doctoral graduates possessed organizational skills, including project management and time management. Project management was defined as a transferable skill, that is developed “through a range of experiences… [as students] learned to determine priorities and achieve deadlines, became skillful in producing outcomes despite a limited budget, equipment failures or administrative impediment” ( Mowbray and Halse, 2010 , p. 661). Studies indicated that doctoral graduates were able to manage and run projects, and demonstrated coordinating skills, people skills, and goal-directed vision ( n = 7). Doctoral graduates possessed time management skills ( n = 3), being able to plan, work to deadlines and balance responsibilities. The organizational skills domain included subdomains of organizational and management skills at both a project and personal level.

Interpersonal skills

Interpersonal skills as a domain reflects a group of attributes that relate to interpersonal interaction in some form or another ( n = 14). Doctoral graduates possessed a range of interpersonal skills including collaboration and teamwork, networking, leadership, teaching and supervision. Collaboration was defined as “any type of joint effort of two or more people pursuing a common goal” ( Granata and Dochy, 2016 , p. 998). Collaboration and teamwork ( n = 5) were identified as being transferable skills. Doctoral graduates in the studies were able to demonstrate internal and external collaboration and teamwork, with clients, experts and industry. This involved the ability to work with people from different sectors and across research boundaries, including “working daily with close colleagues, data exchange with external partners and joint publications of findings with researchers in other faculties and universities” ( Granata and Dochy, 2016 , p. 998). Some doctoral graduates were able to network and connect with the scientific community ( n = 2), resulting in access to resources and information. Other studies highlighted that doctoral graduates demonstrated leadership capacity ( n = 2), which was cross-cutting of some other domains, including articulate communication skills (both written and verbal), the ability to work within structure, discipline of thought, investment in research, and university visibility through publication and collaboration. Doctoral graduates in some studies were noted to possess teaching skills ( n = 7), including being prepared to teach, the ability to deal with students, teaching at undergraduate level and facilitating groups effectively, and supervision skills ( n = 2) at under- and/or post-graduate level. The domain of interpersonal skills that doctoral graduates possessed, included subdomains that reflect various skills for collaborative engagement with others, for work, research and teaching.

Scholarship

The domain of “scholarship” included codes related to scholarly practice and identity that doctoral graduates possess ( n = 8). Studies mentioned scholarship in relation to doctoral graduates' practitioner scholarship and identity as scholar. Practitioner scholarship was defined as “professionals who bring theoretical, pedagogical, and research expertise to bear on identifying, framing and studying problems of practice and leading informed change in their [professional] contexts” ( Adams et al., 2014 , p.366, as cited by Kennedy et al., 2020 , p. 654). Practitioner scholarship was not exclusive to those who completed professional doctorates, but was also mentioned by a study focusing on an academic doctoral program ( Kowalczuk-Waledziak et al., 2017 ), and a study that did not specify the type of doctorate, but included a variety of fields ( Maxwell and Chophel, 2020 ). Theoretical definitions of processes related to scholarship and identity were noted. Professional identity development was defined as taking place through developmental networks. Similarly, socialization was defined as the process through which students “gain the knowledge, skills, and values necessary for successful entry into a professional career requiring an advanced level of specialized knowledge and skills” ( Weidman et al., 2001 , piii, as cited by Feldon et al., 2017 , p. 2574). Some studies identified that doctoral graduates demonstrated practitioner scholarship in their critical and reflexive approach to practice which is informed by theory and research, and using scholarship and research to respond to local needs ( n = 5). In some instances, doctoral graduates' scholarly identity included their independent academic identity as scholar, and, in some instances, balancing and navigating multiple roles as scholar and practitioner ( n = 3). The domain of scholarship thus speaks to the subdomains of the identity of doctoral graduates as scholars and professionals.

The domain of “reputation” included codes relating to the perceived shift of their reputation that doctoral graduates experienced, which may have shifted due to the title of “Doctor” ( n = 12). Conceptually, professional credibility was noted to have a positive impact on professional relationships with colleagues and clients. As a result, doctoral graduates were respected, with colleagues and students “looking up” to them. There was a level of status, prestige and respect associated with having achieved the doctoral degree. Doctoral graduates in some of the studies possessed academic, experiential and professional credibility, respect, and professional standing ( n = 8). This was linked to their experience of legitimacy, in terms of the recognition they received, and the internally perceived legitimization of graduates' role, work, and participation in the academic community ( n = 4). The domain of reputation relates to the impact of the doctorate on how graduates are received and/or perceived due to their doctoral title, reflected in the subdomains of credibility and legitimacy.

Higher order thinking skills

The domain of “higher order thinking skills” was defined by codes that reflected cognitive abilities and reasoning skills ( n = 14). Higher order thinking skills was an intrinsic doctoral graduate attribute that emerged in the review. While no theoretical definition for this domain was present in the review, articles in the review identified problem solving, cognitive abilities and innovation as transferable skills. In some instances, doctoral graduates possessed critical thinking skills ( n = 6), including critical reflection and analysis, questioning, justifying, and reflective and reflexive abilities. They possessed problem solving skills ( n = 6), being able to discover, analyze and solve problems, split problems into sub-problems, and having a problem-solving mindset. Doctoral graduates' cognitive abilities ( n = 4) included academic reasoning ability and the ability to construct an argument. Studies noted that doctoral graduates possessed the capacity to advance innovation, think outside the box and develop innovative research questions ( n = 3). Similarly, some doctoral graduates possessed creativity and curiosity ( n = 2), in their openness to new ideas, questioning stance and unique perspective. The domain of higher order thinking skills included various subdomains of cognitive skills, problem solving, critical thinking, innovation, creativity and curiosity.

Personal resourcefulness

The domain of “personal resourcefulness” collates various intrapersonal attributes and individual qualities that doctoral graduates possessed ( n = 22). Personal resourcefulness was defined by Mowbray and Halse (2010 , p. 657) as:

The acquisition of skills that enable students to become more assertive, confident, resilient, persistent and resolute in determining how to progress their Ph.D. while balancing their other commitments. Consequently, personal resourcefulness is the reflexive, perceptual, emotional and contextual capacity that students develop during their Ph.D. that they used to discern and guide their actions.

Personal resourcefulness, resilience and independence were defined as transferable skills. Doctoral graduates had self-efficacy ( n = 6), which is theoretically central to perseverance, because, according to Bandura (1989 , p. 1,176, as cited by Merga and Mason, 2021 , p. 681), “the acquisition of knowledge and competencies usually requires sustained effort in the face of difficulties and setbacks, it is resiliency of self-belief that counts.” Self-efficacy was demonstrated through persistence, self-discipline, self-organization, and dedication. Similarly, doctoral graduates in some studies had strategies for resilience, determination and tenancy that allowed them to persevere despite challenges ( n = 3). It was these personal qualities that facilitated the development of all other skills and knowledge.

Confidence was the subdomain with the second highest frequency count overall ( n = 12). Confidence was referred to in two ways: sources of confidence and the resultant confidence for action. Firstly, in some instances, doctoral graduates had confidence due to having experienced success in completing the doctorate, confidence in their research skills and knowledge, and confidence due to having successfully published. Secondly, and as a result of the first, doctoral graduates' achievements resulted in a sense of confidence in their professionalism, confidence to pursue innovative research, to defend their ideas, and confidence, in some instances, to bridge academic and professional worlds.

A group of intrapersonal qualities that are linked to doctoral graduates' organizational skills include agency, autonomy, adaptability and self-regulation. In some instances, doctoral graduates were noted to possess agency ( n = 3) in their decision-making capacity to manage priorities and work responsibilities, that may be reflected in their organizational skills. Some were autonomous ( n = 5), able to work independently as a scholar and researcher. Others were adaptable and flexible ( n = 3), with the versatility to manage and transition between multiple roles. Self-regulation was mentioned ( n = 3), and was theoretically defined based on the feedback loop model of self-regulated learning ( Devos et al., 2016 ), that included goal setting, goal operating and goal monitoring, thus referring to the ability to move toward a goal, adjusting behavior over time in order to achieve that goal. Doctoral graduates had the capacity to self-regulate, through goal setting, being able to see the big picture, and to manage stress. Their ability to self-regulate would thus be closely linked with the enactment of their organizational skills, such as project and time management. General life skills and study skills were nominally noted as attributes that some doctoral graduates possess ( n = 2). While not explicitly linked to self-regulation, these may be skills that facilitate or are used in self-regulation. The intrinsic qualities of doctoral graduates possess are thus closely linked to the skills they demonstrate. The domain of personal resourcefulness thus includes the subdomains of resilience, independence, agency, self-efficacy, confidence, self-regulation and general life and study skills.

Active citizenship

A cluster of attributes identified in the review were grouped under the domain “active citizenship,” although this term was not explicitly used in the articles ( n = 4). In other literature, active citizenship is defined as “knowing and practicing your rights as well as uplifting others in the realization and practice of theirs” ( Isaacs et al., 2016 , p. 103), and often refers to issues of transformation and empowerment ( Gal and Gan, 2020 ). Some doctoral graduates demonstrated active citizenship in their advocacy through strategic planning and policy development ( n = 2), and increased awareness of injustice and inequality ( n = 2). Some doctoral graduates were noted to have a social justice-oriented disposition ( n = 1), that is defined as a proactive stance, or a “disposition of action that [drives] change efforts result[ing] from graduates' increased awareness and knowledge… regarding educational inequity, marginalization and White privilege as motivating their persistent efforts to address problems of practice” ( Kennedy et al., 2020 , p. 658). There is preliminary evidence of social justice-related subdomains, indicating a broader domain of active citizenship. The domain of active citizenship, while underdeveloped, include potential subdomains of advocacy, awareness and a social-justice oriented dispositions.

Mapping of findings against existing models

The coverage of the review findings and its alignment to existing models was investigated by mapping existing models onto the review findings, as shown in Table 5 . The models were coded, using a deductive approach, to identify instances of the domains and subdomains of the review. In some instances, models had indications of broad categories that may imply inclusion of some of the subdomains in the review. The mapping of attributes illustrates that there is variation in the existing models, including the level of detail provided, likely indicative of the common issue of conceptual ambiguity around graduate attributes. There is evidence of each of the domains identified in the review, reflected in each of the models, showing good alignment. However, the present review highlights subdomains that were either not included in previous models, or not delineated in detail.

Table 5 . Alignment of findings to existing frameworks.

There were some domains identified in the selected frameworks that were not mentioned in the articles under review. However, these aspects are aligned to the domains and/or subdomains, as indicated in Table 6 . While these attributes may indicate potential “gaps” in the coverage of the present review, these attributes are easily integrated under the domains identified, and thus may rather provide additional detail as to the scope of the domains and subdomains identified.

Table 6 . Attributes not mentioned in the review.

The review synthesized high-quality literature on the attributes that doctoral graduates possess, identifying various doctoral graduate attribute domains, subdomains, and definitions. The findings of the review are discussed, in relation to the “seen” and “unseen” nature of attributes, conceptual development and evidence for the relevance of doctoral graduate attributes. Thereafter, the strengths, limitations, implication and recommendations are presented.

“Seen” and “unseen” doctoral graduate attributes

The review findings are well-aligned with other models of research-related and doctoral graduate attributes. In general, existing models had a stronger focus on the knowledge, research, communication, organizational skills, interpersonal skills, and higher order thinking domains. These attributes are more easily “seen” ( Kensington-Miller et al., 2018 ), as they are explicit competencies observable in doctoral graduates. Doctorate degrees involve an advanced level of original research, with a novel contribution to knowledge in their field ( Denicolo and Park, 2013 ; CHE, 2018 ). It follows that doctoral graduates would possess depth of knowledge and expertise in their field of study (discipline specific knowledge), and a range of research skills, that may facilitate and/or result from their doctoral research. Existing models, with the exception of the Researcher Development Framework ( Vitae, 2010 ) had a greater focus on “seen” as opposed to “unseen” attributes.

The present review includes the intrapersonal domains of personal resourcefulness, scholarship and reputation. The review extends previous models, with a more comprehensive view of the attributes that doctoral graduates possess. The “unseen” or “invisible” attributes of the doctorate are those which reflect the qualities of the person and are often implied in the educational process ( Kensington-Miller et al., 2018 ). Existing models generally focused more on the seen attributes, with less focus on the intrinsic qualities, such as personal resourcefulness. This was particularly evident in models that are more curriculum focused, such as Nikol and Lietzmann (2019) . The evidence of intrinsic qualities aligns with research on doctoral education that has highlighted the shift in viewing the doctorate as a product or commodity, to viewing it as a developmental process ( Mowbray and Halse, 2010 ; Durette et al., 2016 ; Creaton and Anderson, 2021 ). Individual identity development and personal qualities are considered equally as important as the research product ( Denicolo and Park, 2013 ; Ai, 2017 ). The importance of intrinsic development was reflected in the emergence of various intrapersonal domains and subdomains in the review. The qualities graduates possess were sometimes framed as being developed because of the challenges endured in the doctorate ( Devos et al., 2016 ; Lemon et al., 2020 ; Merga et al., 2020 ; Rabe et al., 2021 ). This aligns with the findings of Mowbray and Halse (2010) and their definition of personal resourcefulness as being developed through balancing competing responsibilities and challenges. The internal capacity of the student/graduate to endure the challenges of the doctorate is particularly pertinent in a context of high attrition and extended degree duration ( McKenna, 2017 ; Lemon et al., 2020 ). However, these intrinsic factors need to be supported externally, for example, through good quality supervision, and peer and institutional support ( Granata and Dochy, 2016 ; Lemon et al., 2020 ). The intrapersonal domains identified thus reflect the personal growth and qualities the individual develops and/or utilizes during the doctorate process. The domain of reputation speaks to the perceived credibility that graduates have as doctorate holders. There is preliminary evidence that the sense of credibility and legitimacy stems from graduates' perceived competence and status as doctorate holder. Graduates' perceived credibility then informs their interpersonal interactions, such as pursuing opportunities, networking and/or collaboration ( Bryan and Guccione, 2018 ). Reputation may be a mediating attribute that connects or informs the realization of other attributes. The domain of reputation reflects the confidence with which graduates implement their learnings, and by extension, the broader impact of the doctorate. This domain is somewhat different than the other domains, as it has both an externalized and internalized component. Reputation is typically considered to relate to how others view the individual, but the focus of this domain, as expressed in the articles under review, related more to how the individual perceived this as a result of having completed their doctorate degree, which resulted in an improved internal sense of credibility and legitimacy. The emerging domains identified reflect more of the “unseen” dimensions of doctoral education, and thus areas that students and supervisors need to be made aware of, in order to ensure their active investment in their educational process and identity development ( Kensington-Miller et al., 2018 ).

There was preliminary evidence of subdomains that fall under the domain of active citizenship, yet it did not emerge strongly in this review. There is some alignment with existing models, as the Researcher Development Framework ( Vitae, 2010 ) has subdomains of engagement and impact that align with the definition of active citizenship. In the doctoral context, active citizenship is aligned with the expectation or requirement that research impact should extend beyond academia, especially in applied fields ( Creaton and Anderson, 2021 ). There is evidence of active citizenship as a graduate attribute ( UWC, 2009 ), however, active citizenship as a doctoral graduate attribute requires further investigation and exploration.

Conceptual development of doctoral graduate attributes

Graduate attributes, in general, are noted to lack conceptual clarity ( Mowbray and Halse, 2010 ; Bitzer and Withering, 2020 ). In the review, this was evidenced by the general lack of theoretical definitions for the graduate attributes discussed and/or measured. While extensive work has gone into developing models and frameworks, the review highlights that published journal articles do not necessarily make reference to or explicitly draw on these models, when referencing the outcomes of doctoral degrees or defining the attributes of doctoral graduates. This may be because frameworks are more often utilized by researchers in their own or their students' development, or by policy makers, trainers and curriculum designers in the development of doctoral education training, as indicated, for example, by the target groups specified for the Researcher Development Framework ( Reeves et al., 2012 ). As such, there is a need to connect these developmental frameworks to outcomes-focused research, such as the studies included under review. This would provide theoretical and conceptual grounding to such research, and a good base on which graduate outcomes could be compared and/or measured. The nominal reporting of psychometric properties relating to scales used in the quantitative studies, gives further evidence of the lack of conceptual grounding of the studies under review. As scale development and validation requires conceptual clarity ( Munnik and Smith, 2019 ), this provides further indication of the noted conceptual ambiguity relating to the measurement of graduate attributes. The review contributed a synthesis of recent evidence of the doctoral graduate attributes and the domains in which they are operationalized. These domains were well-aligned to existing models, despite the articles not making explicit mention of these models. The review itself contributed to the conceptual development of what these attributes are, extending the common themes in existing models, to include additional “unseen” attributes alongside the “seen” or more traditionally conceptualized attributes. The review findings may provide tangential evidence of the developmental impact that existing models may have had on doctoral education, as many of the studies took place in the UK and Australia, where the Researcher Development Framework and the Research Skill Development Framework, respectively, are used. However, a further exploration of the use and impact of these frameworks is warranted.

Relevance of doctoral graduate attributes

The employability of graduates is a concern for higher education institutions, and thus the transferability of graduate attributes is an important consideration ( Kensington-Miller et al., 2018 ). This review highlighted a range of transferable and interdisciplinary attributes that doctoral graduates possess, in alignment with other models. Studies explicitly identified subdomains of communication, organizational skills, interpersonal skills, higher order thinking skills and personal resourcefulness as transferable skills. This review extends previous research that narrowly conceptualized the knowledge base of doctoral graduates as being discipline specific expertise, to include transferable interdisciplinary knowledge and professional knowledge. The inclusion of research translation and communication with various audiences aligns with the increased focus on interdisciplinarity in doctoral education for improving employability ( Holley, 2018 ). There is good evidence that doctoral graduates are “T-shaped” individuals, with depth of knowledge and skills in their discipline, and cross-cutting transferable skills ( Granata and Dochy, 2016 ). This finding reflects the shifts that took place in doctoral education to include the “development of broader workplace skills and experiences” ( Bryan and Guccione, 2018 , p. 1,125), and is in alignment with previous work, highlighting the importance of transferable skills for doctoral education and training ( Denicolo and Reeves, 2014 ). However, stakeholders may not yet be aware of the shift and the improved transferability of doctoral skills. Further, there are contextual differences, at a national and institutional level, which may account for gaps in training relevance and doctoral employability. The range of skills and qualities evidenced in the review provides evidence against the critiques of the doctorate being overly specialized and lacking relevance to the workplace ( Boulos, 2016 ; Maxwell and Chophel, 2020 ). This concern reflects a lack of awareness from employers and graduates themselves, as to the wide range of skills and qualities doctoral graduates possess ( Durette et al., 2016 ). It is important for graduates to first be aware of the range of skills they have developed, in order to market their skills, not just their specialized field of study ( Denicolo and Reeves, 2014 ; Holley, 2018 ).

Strengths and limitations

The focus of the review on a specific perspective of doctoral graduate attributes, i.e., the attributes that doctoral graduates actually possess, is a strength. The findings synthesize actual outcomes, as opposed to prospective attributes or attributes-in-process. Therefore, the findings may translate into quality assurance outcome indicators, for example, through the development of scales to measure doctoral graduate attributes. The review findings are well aligned with previous literature and models, and thus provides good evidence that the attributes identified in the review are common across contexts and disciplines. The wide range of literature sources accessed indicates good scope. All reviews are by nature limited to the search criteria used. In order to include only highest quality evidence, potentially relevant gray literature and non-peer reviewed literature was excluded, based on the search criteria and exclusion criteria. For example, much of the work around doctoral education and training, particularly utilizing models and frameworks for improving curriculum and skills development, may have taken place more informally or been published in technical reports, and thus would have been excluded from the present review.

Implications and recommendations

The synthesis of recent evidence of the domains, subdomains and definitions in the review provides preliminary evidence of the impact of the frameworks that are used to improve doctoral education and training, as there is good alignment between the review findings and some of these frameworks. The review therefore contributes to the conceptual development of doctoral graduate attributes. As such, the review findings may support efforts to measure and assess doctoral graduate attributes, for example, for quality assurance. Institutions can use these domains and definitions to develop context-relevant charters of doctoral graduate attributes and use these to guide curriculum development and support programs for doctoral students. These doctoral graduate attributes can be used at an institutional level as part of quality assurance and institutional marketing.

The lack of awareness among doctoral education stakeholders as to the wide range of doctoral graduate attributes possessed needs addressing. The “unseen” nature of many doctoral graduate attributes requires greater support to increase awareness in order to facilitate development ( Kensington-Miller et al., 2018 ). Increased awareness as to the scope of doctoral graduate attributes among stakeholders, such as doctoral supervisors, students, graduates and employers, may facilitate improved educational outcomes and employability ( Denicolo and Reeves, 2014 ). Supervisors should consider appropriate support for the multifaceted and holistic development of doctoral graduates in supervision. Students' awareness and active engagement in their attribute development may facilitate growth and their capacity to market themselves in their curriculum vitae and/or in job interviews. The various tools available related to the Researcher Development Framework ( Vitae, 2010 ; Denicolo and Reeves, 2014 ) and the Researcher Skills Development Framework ( Willison and O'Regan, 2008/2015 ) could be used to support such activities. Prospective doctoral students will be able to enter the doctoral program with more realistic expectations of what growth they can anticipate during the process and can use the attributes to benchmark their progress throughout the degree. Employers' awareness of the multifaceted attributes associated with completion of a doctorate may counteract the misconception that the doctorate has limited relevance or transferability beyond discipline-specific knowledge ( Boulos, 2016 ; Maxwell and Chophel, 2020 ). Doctoral graduates are key knowledge and innovation creators ( Molla and Cuthbert, 2016 ). Therefore, it is important to ensure that doctoral graduates are equipped and positioned to actively engage in good quality and contextually relevant research that benefits society at multiple levels. Increased awareness among stakeholders may improve the mobility of doctoral graduates across fields, disciplines and workplaces, thereby improving employability.

Future research around doctoral education should utilize existing models or frameworks as conceptual frameworks, adding much needed grounding and cohesion to the literature on doctoral graduate attributes. The review draws on many contexts yet is descriptive in nature and so the findings are not generalizable. Further research into doctoral graduate attributes is recommended for underrepresented contexts in the review, including Africa, Asia, South America, and the Middle East. Given the contextual differences in doctoral education and higher education systems, research in these varied contexts needs to be conducted and shared. The focus on doctoral education in the African context reflects the policy imperative to improve the capacity of the higher education sectors, the national systems of innovation, and facilitate engagement in the knowledge economy ( Molla and Cuthbert, 2016 ). In this context, there is an ongoing need to increase doctoral outputs. However, in the Global North, such as Australia, the academic job market is nearing saturation. Therefore, the questions around doctoral graduate production shift to employability, particularly beyond academia ( Guerin, 2020 ). In line with the recommendations of other authors ( Durette et al., 2016 ; Yazdani and Shokooh, 2018 ), it is recommended that context-relevant, conceptually sound scales be developed, to reliably and validly measure doctoral graduate attributes among alumni populations. Such scales could be used as quality assurance outcome indicators, or in graduate tracer study research ( Senekal and Munro, 2019 ), to explore issues of employment, employability and the relevance of the doctorate within specific contexts. Further contextual investigation into how the identified domains in this review are developed, including the supportive factors and potential barriers may further inform supervisory practice and institutional support provision. Research into the reflection of these doctoral graduate attributes in thesis examination processes may be useful, to develop more holistic examination guidelines.

Doctoral graduates (across disciplines and countries) possess a wide range of attributes, including knowledge, research, communication and organizational skills, and a variety of inter- and intrapersonal skills. Many of the attributes identified are transferable and inter- or transdisciplinary. Doctoral graduates—from a wide variety of disciplines and countries—possess a broad range of transferable knowledge and skills that align with those required for the workforce, in both academia and beyond. The review adds to the body of research by consolidating recent findings on the topic, including descriptions and definitions not just domains, and conceptualizing doctoral graduate attributes as outcomes—from the perspective of graduates. The domains identified represent the attributes that doctoral graduates actually do possess, as opposed to an aspirational list of what stakeholders hope students will develop. Doctoral graduate attributes include both the “seen” generic attributes that are more commonly identified in graduate attribute frameworks, such as disciplinary expertise, research skills and communication skills, together with “unseen” intrinsic qualities, such as personal resourcefulness. Doctoral graduate attributes should be conceptualized to integrate both the generic attributes, together with the intrinsic qualities that are invaluable both during the doctoral degree and beyond.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

Author contributions

JS contributed to the conceptualization of the review, fieldwork, data extraction, data analysis, draft write up, revisions, editing, writing and technical aspects of the article, and approved the submitted version. EM and JF contributed to the conceptualization of the review as well as the coordination of the review processes, provided leadership and input to the review team at each stage of the project and the conceptualization of the manuscript, contributed to the write up and technical aspects of the article, and approved the submitted version. All authors contributed to the article and approved the submitted version.

The financial assistance of the South African Department of Higher Education, Science and Technology (DHEST) via the University Capacity Development Grant, toward this research is hereby acknowledged. Opinions expressed, and conclusions arrived at, are those of the authors and are not necessarily to be attributed to DHEST.

Acknowledgments

The contributions of the following individuals is appreciated and acknowledged: Prof. Ruth Albertyn for her critical reading in reviewing the article for publication and Ms. Kendall Byne-Ross for her assistance in the screening and evaluation stages of the fieldwork for this research.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Adams, A., Bondy, E., Ross, D., Dana, N., and Kennedy-Lewis, B. L. (2014). Implementing an online professional practice doctoral program in a PhD environment: Managing the dilemmas. J. School Public Relat . 35, 363–382. doi: 10.3138/jspr.35.3.363

CrossRef Full Text | Google Scholar

Adham, K. A., Ha, H., Mohd Nor, S., and Yazid, Z. (2018). Learning to complete the Ph.D. thesis. Issues Educ. Res. 28, 811–829. Available online at: http://www.iier.org.au/iier28/adham.pdf

Google Scholar

Ai, B. (2017). Constructing an academic identity in Australia: an autoethnographic narrative. High. Educ. Res. Dev. 36, 1095–1107. doi: 10.1080/07294360.2017.1303459

Allgood, S., Hoyt, G., and McGoldrick, K. (2018). Teacher training for Ph.D. students and new faculty in economics. J. Econ. Educ. 49, 209–219. doi: 10.1080/00220485.2018.1438947

Bandura, A. (1989). Human agency in social cognitive theory. Am. Psychol . 44, 1175–1184. doi: 10.1037/0003-066X.44.9.1175

PubMed Abstract | CrossRef Full Text | Google Scholar

Bitzer, E., and Matimbo, F. (2017). Cultivating African academic capital—Intersectional narratives of an African graduate and his Ph.D. study supervisor. Innov. Educ. Teach. Int. 54, 539–549. doi: 10.1080/14703297.2017.1304825

Bitzer, E., and Withering, M. (2020). Graduate attributes: How some university students experience and learn them. S. Afr. J. High. Educ. 34, 13–31. doi: 10.20853/34-3-3504

Boud, D., Costley, C., Marshall, S., and Sutton, B. (2021). Impacts of a professional practice doctorate: a collaborative enquiry. High. Educ. Res. Dev. 40, 431–445. doi: 10.1080/07294360.2020.1765744

Boulos, A. (2016). The labour market relevance of Ph.D.s: An issue for academic research and policy-makers. Stud. High. Educ. 41, 901–913. doi: 10.1080/03075079.2016.1147719

Bridgstock, R. (2009). The graduate attributes we've overlooked: Enhancing graduate employability through career management skills. High. Educ. Res. Dev. 28, 31–44. doi: 10.1080/07294360802444347

Bröchner, J., and Sezer, A. A. (2020). Effects of construction industry support for Ph.D. projects: The case of a Swedish scheme. Indus. High. Educ. 34, 391–400. doi: 10.1177/0950422220904932

Bryan, B., and Guccione, K. (2018). Was it worth it? A qualitative exploration into graduate perceptions of doctoral value. High. Educ. Res. Dev. 37, 1124–1140. doi: 10.1080/07294360.2018.1479378

Caretta, M. A., Drozdzewski, D., Jokinen, J. C., and Falconer, E. (2018). “Who can play this game?” The lived experiences of doctoral candidates and early career women in the neoliberal university. J. Geogr. in High. Educ. 42, 261–275. doi: 10.1080/03098265.2018.1434762

Cavanagh, J., Burston, M., Southcombe, A., and Bartram, T. (2015). Contributing to a graduate-centred understanding of work readiness: An exploratory study of Australian undergraduate students' perceptions of their employability. Int. J. Manage. Educ. 13, 278–288. doi: 10.1016/j.ijme.2015.07.002

CHE (2018). Qualification Standard for Doctoral Degrees. Pretoria: Council on Higher Education.

Creaton, J., and Anderson, V. (2021). The impact of the professional doctorate on managers' professional practice. Int. J. Manage. Educ. 19, 100461. doi: 10.1016/j.ijme.2021.100461

Denicolo, P. M., and Park, C. (2013). “Doctorateness—an elusive concept?,” in Critical Issues in Higher Education , eds. M. Kompf and P.M. Denicolo. (Rotterdam: Sense Publishers).

Denicolo, P. M., and Reeves, J. (2014). Developing Transferable Skills: Enhancing Your Research Employment Potential . London, UK: SAGE.

Devos, C., Boudrenghien, G., Van der Linden, N., Azzi, A., Frenay, M., Galand, B., et al. (2016). Doctoral students' experiences leading to completion or attrition: a matter of sense, progress and distress. Eur. J. Psychol. Educ. 32, 61–77. doi: 10.1007/s10212-016-0290-0

Draper, P., and Harrison, S. (2018). Beyond a doctorate of musical arts: experiences of its impacts on professional life. Br. J. Music Educ. 35, 271–284. doi: 10.1017/S0265051718000128

Durette, B., Fournier, M., and Lafon, M. (2016). The core competencies of Ph.D.s. Stud. High. Educ. 41, 1355–1370. doi: 10.1080/03075079.2014.968540

Elliot, D. L., Baumfield, V., Reid, K., and Makara, K. A. (2016). Hidden treasure: successful international doctoral students who found and harnessed the hidden curriculum. Oxf. Rev. Educ. 42, 733–748. doi: 10.1080/03054985.2016.1229664

Feldon, D. F., Rates, C., and Sun, C. (2017). Doctoral conceptual thresholds in cellular and molecular biology. Int. J. Sci. Educ. 39, 2574–2593. doi: 10.1080/09500693.2017.1395493

Gal, A., and Gan, D. (2020). Transformative sustainability education in higher education: activating environmental understanding and active citizenship among professional studies learners. J. Transform. Educ. 18, 271–292. doi: 10.1177/1541344620932310

Gilbert, R., Balatti, J., Turner, P., and Whitehouse, H. (2004). The generic skills debate in research higher degrees. High. Educ. Res. Develop . 23, 375–388. doi: 10.1080/0729436042000235454

González-Ocampo, G., and Castelló, M. (2019). Supervisors were first students: Analysing supervisors' perceptions as doctoral students versus doctoral supervisors. Innov. Educ. Teach. Int. 56, 711–725. doi: 10.1080/14703297.2018.1531775

Goodall, H. J., Huggins, V. A., Webber, L. A., and Wickett, K. L. (2017). From student to graduate. Manage. Educ. 31, 180–186. doi: 10.1177/0892020617738178

Gough, D., Oliver, S., and Thomas, J. (2017). An Introduction to Systematic Reviews. London, UK: SAGE.

Grab-Kroll, C., Schneider, A., Keis, O., Mayer, B., Wirth, T., Barthd, T., et al. (2019). Welche Beitrage konnen strukturierte Promotionsprogramme zur Qualitatssicherung medizinischer Promotionen und wissenschaftlichen Karriereforderung/ Nachwuchsforderung leisten? Eine Evaluation am Beispiel der Programminitiative “Experimentelle Medizin” der Universitat Ulm. [What can structured doctoral programs contribute to ensure quality of medical dissertations and scientific careers/junior promotion? An evaluation using the “Experimental Medicine” program initiative of Ulm University as an example]. Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen 147-148, 110–119. doi: 10.1016/j.zefq.2019.10.001

Granata, S. N., and Dochy, F. (2016). Applied Ph.D. research in a work-based environment: an activity theory-based analysis. Stud. High. Educ. 41, 990–1007. doi: 10.1080/03075079.2014.966666

Greene, J. L. R., Cote, H. K., Koperniak, M., and Stanley, L. M. (2021). A Foot in Both Worlds: Navigating the Landscapes of P−12 Education Postdoctorate. J. Music Teach. Educ. 30, 93–107. doi: 10.1177/1057083720983944

Guerin, C. (2020). Stories of moving on HASS Ph.D. graduates' motivations and career trajectories inside and beyond academia. Arts Human. High. Educ. 19, 304–324. doi: 10.1177/1474022219834448

Hager, M. J., Turner, F., and Dellande, S. (2019). Academic and social integration: psychosocial support and the role of developmental networks in the DBA. Stud. Contin. Educ. 41, 241–258. doi: 10.1080/0158037X.2018.1551202

Harley, K. (2020). National Ph.D. Review—Matters of Quality and Accountability . University World News: Africa Edition. Available online at: https://www.universityworldnews.com/post.php?story=2020062909405560#:~:text=At%20the%20heart%20of%20the,integrity%20of%20the%20Qualification%20Standard (accessed December 3, 2021).

Hinchliffe, G. W., and Jolly, A. (2011). Graduate identity and employability. Br. Educ. Res. J. 37, 563–584. doi: 10.1080/01411926.2010.482200

Hoffmann, T. (1999). The meanings of competency. J. Eur. Ind. Train. 23, 275–86. doi: 10.1108/03090599910284650

Holley, K. A. (2018). The Longitudinal Career Experiences of Interdisciplinary Neuroscience Ph.D. Recipients. J. High. Educ. 89, 106–127. doi: 10.1080/00221546.2017.1341755

Hughes, C., and Barrie, S. C. (2010). Influences on the assessment of graduate attributes in higher education. Assess. Eval. High. Educ. 35, 325–334. doi: 10.1080/02602930903221485

Isaacs, S., Rose, J., and Davids, C. (2016). Transformative learning: postgraduate students' reflections on a community engagement program in South Africa. Soc. Behav. Pers. 44, 103–116. doi: 10.2224/sbp.2016.44.1.103

Kennedy, B. L., Bondy, E., Dana, N. F., Vescio, V., and Ma, V. W. (2020). The development and enactment of practitioner scholarship among graduates from one online Ed.D. programme. J. Further High. Educ. 44, 653–669. doi: 10.1080/0309877X.2019.1576858

Kensington-Miller, B., Knewstubb, B., Longley, A., and Gilbert, A. (2018). From invisible to SEEN: a conceptual framework for identifying, developing and evidencing unassessed graduate attributes. High. Educ. Res. Dev. 37, 1439–1453. doi: 10.1080/07294360.2018.1483903

Kilbourne, B. F., Mazerolle, S. M., and Bowman, T. G. (2018). Doctoral preparation influence on new faculty's perception of role transition. Athletic Train. Educ. J. 13, 340–347. doi: 10.4085/1304340

Kowalczuk-Waledziak, M., Lopes, A., Menezes, I., and Tormenta, N. (2017). Teachers pursuing a doctoral degree: motivations and perceived impact. Educ. Res. 59, 335–352. doi: 10.1080/00131881.2017.1345287

Laher, S., and Hassem, T. (2020). Doing systematic reviews in psychology. S. Afr. J. Psychol. 50, 450–468. doi: 10.1177/0081246320956417

Lemon, J., Pladsen, J., Tawill, S., Clayton-Wood, L., and Morgan-Sowada, H. (2020). The lived experience of marriage and family therapy doctoral graduates: a phenomenological study. Am. J. Family Ther. 48, 446–461. doi: 10.1080/01926187.2020.1741474

Lietzmann, A., and Nikol, P. (2019). European Core Curriculum in Transferable Skills for SET Disciplines. TU Berlin, WUT, NTUT and PoliMi.

Luo, Y., Liu, Z., Zhang, J., and Gu, J. (2018). A study on the quality of doctoral education based on the ipod framework: a case of agriculture-related disciplines. Chin. Educ. Soc. 51, 199–221. doi: 10.1080/10611932.2018.1454153

Mashiyi, F. N. (2015). Embedding graduate attributes into the foundation programme: Reflections on process and product. S. Afr. J. High. Educ. 29, 181–197. doi: 10.20853/29-1-456

Maxwell, T. W. (2019). Impact of education doctorates: the case of developing country, Bhutan. Stud. Contin. Educ. 41, 226–240. doi: 10.1080/0158037X.2018.1546688

Maxwell, T. W., and Chophel, D. (2020). The impact and outcomes of (non-education) doctorates: the case of an emerging Bhutan. High. Educ. 80, 1081–1102. doi: 10.1007/s10734-020-00531-4

McAlpine, L., Castello, M., and Pyhaltö, K. (2020). What influences Ph.D. graduate trajectories during the degree: a research-based policy agenda. High. Educ. 80, 1011–1043. doi: 10.1007/s10734-019-00448-7

McAlpine, L., Skakni, I., and Inouye, K. (2021). Ph.D. careers beyond the traditional: integrating individual and structural factors for a richer account. Eur. J. High. Educ. 11, 365–385. doi: 10.1080/21568235.2020.1870242

McKenna, S. (2017). Crossing conceptual thresholds in doctoral communities. Innov. Educ. Teach. Int. 54, 458–466. doi: 10.1080/14703297.2016.1155471

Merga, M. K., and Mason, S. (2021). Doctoral education and early career researcher preparedness for diverse research output production. J. Further High. Educ. 45, 672–687. doi: 10.1080/0309877X.2020.1807477

Merga, M. K., Mason, S., and Morris, J. E. (2020). ‘What do I even call this?' Challenges and possibilities of undertaking a thesis by publication. J. Further High. Educ. 44, 1245–1261. doi: 10.1080/0309877X.2019.1671964

Molla, T., and Cuthbert, D. (2016). In pursuit of the African Ph.D.: A critical survey of emergent policy issues in select sub-Saharan African nations, Ethiopia, Ghana and South Africa. Policy Futures Educ. 14, 635–654. doi: 10.1177/1478210316641567

Moola, S., Munn, Z., Tufanaru, C., Aromataris, E., Sears, K., Sfetcu, R., et al. (2020). “Chapter 7: Systematic reviews of etiology and risk,” in JBI Manual for Evidence Synthesis , eds. E. Aromataris and Z. Munn. Adelaide, Australia: The Joanna Briggs Institute.

Mowbray, S., and Halse, C. (2010). The purpose of the Ph.D.: theorising the skills acquired by students. High. Educ. Res. Dev. 29, 653–664. doi: 10.1080/07294360.2010.487199

Munnik, E., and Smith, M. R. (2019). Methodological rigour and coherence in the construction of instruments: The emotional social screening tool for school readiness. Afr. J. Psychol. Assess. 1, 2a. doi: 10.4102/ajopa.v1i0.2

Nell, I. A., and Bosman, J. P. (2017). Integrating graduate attributes into a Master of Divinity programme at a South African university. S. Afr. J. High. Educ. 31:175–90. doi: 10.20853/31-1-868

Nguyen, H. V., Phan, T. T. H., Nguyen, H., Nguyen, N., and Nguyen, M. H. (2020). What is a Good Journal? Perceptions of Vietnamese Early-Career and Mid-Career Researchers. Publish. Res. Q. 36, 296–303. doi: 10.1007/s12109-020-09718-0

Nikol, P., and Lietzmann, A. (2019). mindSET European Transferable Skills Training Demands Survey—Analysis Report . Washington, D.C: European Commission.

OECD (2019). Education at a Glance 2019: OECD Indicators . Paris, France: OECD Publishing.

Ouzzani, M., Hammady, H., Fedorowicz, Z., and Elmagarmid, A. (2016). Rayyan — a web and mobile app for systematic reviews. Syst. Rev. 5, 210. doi: 10.1186/s13643-016-0384-4

PubMed Abstract | CrossRef Full Text

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., et al. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372, n71. doi: 10.1136/bmj.n71

Potolea, D. (2013). Doctoral Studies and Research Competences. Procedia - Social and Behavioral Sciences 76, 935–946. doi: 10.1016/j.sbspro.2013.04.238

Rabe, M., Agboola, C., Kumswa, S., Linonge-Fontebo, H., and Mathe, L. (2021). Like a bridge over troubled landscapes: African pathways to doctorateness. Teach. High. Educ. 26, 306–320. doi: 10.1080/13562517.2021.1896490

Reeves, J., Denicolo, P. M., Metcalfe, J., and Roberts, J. (2012). The Vitae Researcher Development Framework and Researcher Development Statement: Methodology and Validation Report . UK: The Careers Research and Advisory Centre Limited.

Senekal, J., and Munro, N. (2019). Lessons learnt from two decades of graduate tracer research: Recommendations for the South African context. S. Afr. J. High. Educ. 33, 230–248. doi: 10.20853/33-2-2628

Shih, J. C., Reys, R. E., Reys, B. J., and Engledowl, C. (2019). A profile of mathematics education doctoral graduates' background and preparation in the United States. Invest. Math. Learn. 11, 16–28. doi: 10.1080/19477503.2017.1375357

Smith, M. R., Franciscus, G., Swartbooi, C., Munnik, E., and Jacobs, W. (2015). “The SFS scoring system,” in Symposium on Methodological Rigour and Coherence: Deconstructing the Quality Appraisal Tool in Systematic Review Methodology. 21st National Conference of the Psychological Association of South Africa . South Africa: PsySSA.

UWC (2009). Charter of Graduate Attributes. Cape Town, South Africa: University of the Western Cape.

Vitae (2010). Researcher Development Framework. UK: The Careers Research and Advisory Centre Limited.

Walker, J., and Yoon, E. (2017). Becoming an academic: the role of doctoral capital in the field of education. High. Educ. Res. Dev. 36, 401–415. doi: 10.1080/07294360.2016.1207616

Weidman, J. C., Twale, D. J., Stein, E. L., and Leahy, E. (2001). Socialization of Graduate and Professional Students in Higher Education: A Perilous Passage?. Washington, DC: Office of Educational Research and Improvement. p. 1–139.

Willison, J., and O'Regan, K. (2008/2015). Research Skill Development Framework . Adelaide University. Available online at: https://www.adelaide.edu.au/melt/ua/media/51/rsd-framework.pdf

Xu, W., and Zammit, K. (2020). Applying thematic analysis to education: a hybrid approach to interpreting data in practitioner research. Int. J. Qual. Methods 19, 160940692091881. doi: 10.1177/1609406920918810

CrossRef Full Text

Yazdani, S., and Shokooh, F. (2018). Defining Doctorateness: A Concept Analysis. Int. J. Doctoral Stud. 13, 031–148. doi: 10.28945/3939

Keywords: doctoral education, doctoral graduate attributes, interdisciplinarity, systematic review, transferable skills, educational outcomes, postgraduate education and training, research skills

Citation: Senekal JS, Munnik E and Frantz JM (2022) A systematic review of doctoral graduate attributes: Domains and definitions. Front. Educ. 7:1009106. doi: 10.3389/feduc.2022.1009106

Received: 01 August 2022; Accepted: 05 October 2022; Published: 25 October 2022.

Reviewed by:

Copyright © 2022 Senekal, Munnik and Frantz. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Janine S. Senekal, jssenekal@uwc.ac.za

This article is part of the Research Topic

Education and Innovative Perspectives in Higher Education

Acceptance of a systematic review as a thesis: survey of biomedical doctoral programs in Europe

Affiliations.

• 1 Cochrane Croatia, University of Split School of Medicine, Šoltanska 2, 21000, Split, Croatia. [email protected].
• 2 Department for Development, Research and Health Technology Assessment, Agency for Quality and Accreditation in Health Care and Social Welfare, Planinska 13, 10000, Zagreb, Croatia. [email protected].
• 3 Laboratory for Pain Research, University of Split School of Medicine, Šoltanska 2, 21000, Split, Croatia. [email protected].
• 4 Laboratory for Pain Research, University of Split School of Medicine, Šoltanska 2, 21000, Split, Croatia.
• PMID: 29233170
• PMCID: PMC5727923
• DOI: 10.1186/s13643-017-0653-x

Background: Systematic reviews (SRs) have been proposed as a type of research methodology that should be acceptable for a graduate research thesis. The aim of this study was to analyse whether PhD theses in European biomedical graduate programs can be partly or entirely based on SRs.

Methods: In 2016, we surveyed individuals in charge of European PhD programs from 105 institutions. The survey asked about acceptance of SRs as the partial or entire basis for a PhD thesis, their attitude towards such a model for PhD theses, and their knowledge about SR methodology.

Results: We received responses from 86 individuals running PhD programs in 68 institutions (institutional response rate of 65%). In 47% of the programs, SRs were an acceptable study design for a PhD thesis. However, only 20% of participants expressed a personal opinion that SRs meet the criteria for a PhD thesis. The most common reasons for not accepting SRs as the basis for PhD theses were that SRs are 'not a result of a PhD candidate's independent work, but more of a team effort' and that SRs 'do not produce enough new knowledge for a dissertation'. The majority of participants were not familiar with basic concepts related to SRs; questions about meta-analyses and the type of plots frequently used in SRs were correctly answered by only one third of the participants.

Conclusions: Raising awareness about the importance of SRs and their methodology could contribute to higher acceptance of SRs as a type of research that forms the basis of a PhD thesis.

Keywords: Biomedicine; PhD program; PhD thesis; Study design; Systematic review.

Publication types

• Systematic Review
• Academic Dissertations as Topic / standards*
• Education, Graduate / methods*
• Research Design*
• Review Literature as Topic*
• Surveys and Questionnaires*

Grants and funding

• IP-2014-09-7672/Hrvatska Zaklada za Znanost

How to Write a Systematic Review Dissertation: With Examples

Writing a systematic review dissertation isn’t easy because you must follow a thorough and accurate scientific process. You must be an expert in research methodology to synthesise studies. In this article, I will provide a step-by-step approach to writing a top-notch systematic review dissertation.

Table of Contents

However, for students who may find this process challenging and seek professional assistance, I recommend exploring SystematicReviewPro —a reliable systematic review writing service. By signing up and placing a free inquiry and engaging with the admin team at any time, students can avail themselves of an exclusive offer of up to 50% off on their systematic review order. Additionally, there is already a 30% discount running on the website, making it an excellent opportunity to ease your dissertation journey.

As an Undergraduate or Master’s student, you’re are allowed to pick a systematic review for your dissertation. As a PhD student, you can use a systematic review methodology in the second chapter (literature review) of your dissertation. A systematic review is considered the highest level of empirical evidence, especially in clinical sciences like nursing and medicine. When developing new practice guidelines, new services, or new products, systematic reviews are searched and synthesised first on that topic or idea.

Factors to Consider When Writing a Systematic Review Dissertation

The nature of your research topic or research question.

Some research topics or questions strictly conform to qualitative or quantitative methods. For example, if you’re exploring the lived experiences, attitudes, perceptions, and meaning-making in a given population, you’ll need qualitative methods. However, you will require quantitative methods if looking into quantifiable variables like happiness, depression, academic performance, sleep, etc. That said, the nature of your research question should guide you. If your topic is qualitative, you’ll need qualitative studies only. If your topic is quantitative, you’ll need quantitative studies only. Systematic reviews of qualitative studies are less intricate than of quantitative studies. Still, they require a thoughtful approach in synthesizing findings from various qualitative studies.

If you choose to review quantitative studies, you might need to conduct a meta-analysis in your systematic review. A meta-analysis refers to statistical techniques used in pooling findings from various independent studies to compute a summary statistic. For example, in your dissertation, you may aim to investigate the effect of a student well-being programme embedded in university classes on the happiness of university students. Various studies that have investigated the same or a related intervention and quantitively measured happiness among university students must be synthesised together using a statistical technique. The ultimate outcome of that meta-analysis is to provide an overview of the overall trend of the effect of the intervention on university student’s happiness. For more information about how to formulate a research question for a systematic review with a meta-analysis, visit this link.

An example meta-analysis showing the statistical combination of findings from various studies to indicate the overall effect of a psychological intervention on the psychological well-being of university students.

Availability of primary studies

Finding primary studies for your systematic review is the hardest thing you can encounter with this approach. You can choose your topic and plan your journey so well. Upon reaching the point you need primary studies to answer your research question, you get stuck. Retrieving primary studies is challenging because it requires advanced search strategies on various online databases. Doing an advanced search strategy can be an uphill task for someone who has never done a systematic review. This is because, more often than not, depending on the topic, primary studies are not readily available on the Internet. Remember, secondary studies, like systematic reviews and literature reviews, are not eligible for systematic reviews.

Supervisor’s recommendation

Always confirm with your supervisor if you can do a systematic review dissertation. Some supervisors may feel it better for you to do a primary study. So, always confirm with your supervisor before doing much.

Your confidence

Always ensure you’re confident that you can do a systematic review on your own. Writing a systematic review isn’t easy. You need to be aware that doing a systematic review may even be harder than doing interviews or surveys in primary research. Why? A systematic review involves combining many primary studies together in a scientific manner. That means you must have expertise in various research methodologies to know the best way to integrate or synthesise the various studies.

Availability of time and resources

The main advantage of doing a systematic review dissertation is that it saves a lot of time. Conducting interviews or surveys can be time- and resource-consuming. However, with a systematic review, you do everything from your desk. It will save you a lot of time and resources. If you find that you meet many of the requirements of successfully conducting a systematic review, the next step is to engage in the actual process. The step-by-step approach used in writing systematic reviews is outlined below.

Step-by-Step Process in Writing a Systematic Review Dissertation

The following steps are iterative, meaning you can start over again and again until you meet your research objectives. The step-by-step guide on how to write a systematic review dissertation is summarized in the infographic shown below.

Step-by-step guide on how to write a systematic review dissertation

Step 1: Formulate the systematic review research question

The starting point of a systematic review is to formulate a research question. As stated above, the nature of your research question will help you make key decisions. For example, you will be able to know which design (quantitative versus qualitative) to consider in your inclusion and exclusion criteria.

Step 2: Do a preliminary search

The next step is to perform a preliminary search on the Internet to determine if another systematic review has been published. It is not acceptable to repeat what has already been done. Your research should be novel and contribute to a knowledge gap. However, if you find that another systematic review has already been published on your topic. You should consider the publication date.

In most cases, systematic reviews on given topics are outdated. They have not used recent studies published on that topic, thus missing important updates. That can be a good reason you’re conducting your study. Suppose there’s an updated systematic review on your topic. In that case, you should consider reformulating your research question to address a specific knowledge gap.

Step 3: Develop your systematic review inclusion and exclusion criteria

One unique thing about systematic reviews is that they must be based on a very specific population, intervention/exposure, and assess a specific outcome. Let’s say, for example, you write on Intervention A’s effectiveness in reducing depression symptoms in older frail people. In that case, you must retrieve studies that strictly assess the effectiveness of Intervention A, the outcome being depression symptoms and the population being older frail people.

Therefore, it will be against the principles of a systematic review to focus on Intervention B (different intervention/exposure) on anxiety (different outcomes) in younger people (different populations). Also, depending on your research question, you will need to determine the research design (qualitative versus quantitative) of the studies you will review. Other criteria to consider are the country of publication, the publication date, language, etc.

Step 4: Develop your systematic review search strategy

As said, the main challenge in writing a systematic review is to identify papers. Your literature search should be thorough so that you don’t leave out some relevant studies. Developing a literature search strategy isn’t easy because you must start identifying relevant keywords and search terms for your topic. You must start by knowing common terminologies used in your subject of interest.

Afterward, combine the keywords using Boolean connectors like “AND” & “OR.” For example, suppose my topic is the effectiveness of cognitive behavioural therapy in treating anxiety in adolescents. In that regard, I can combine my keywords as follows: (Cognitive behavioural therapy OR CBT) AND (anxiety) AND (adolescents OR youth). If you use terminologies unknown in your discipline, you will likely not find relevant studies for review.

Step 5: Plan and perform systematic review database selection

At this stage, you identify the databases you’ll use to execute your search strategy. When writing a systematic review dissertation, you also need to report the databases that you searched. Commonly searched ones in the field of social and health sciences include PubMed, Google Scholar, Cochrane, PsycInfo, and many others. You need to know how each database works. Also, apart from Google Scholar and PubMed, most of these databases require paid or institutional access. Liaise with your supervisor or librarian to help in identifying good databases for subject and discipline.

Step 6: Perform systematic review screening using titles and abstracts

When you execute your search strategy on each database, results or search hits will be displayed. This is also another difficult step because of tedious work involved. You start by screening the titles. Then, eliminate results that contain irrelevant titles. You need to be careful at this point because sometimes people eliminate even relevant studies. The title doesn’t need to contain exactly your keywords. Some titles appear totally irrelevant but they actually contain useful data inside.

After screening titles, the next step is to screen abstracts. You may be surprised at this point that the titles you thought were irrelevant actually contain relevant information. For instance, some studies may indicate in the title that their study focused on depression as an outcome when you’re interested in anxiety. However, reading the abstract may surprise you that depression was only a primary outcome. The authors also measured secondary outcomes, among them anxiety. In such an article, you can decide to focus on anxiety results only because they are relevant to your study.

Step 7: Do a manual search to supplement database search

After screening articles identified using various databases, the next step is to augment the search strategy with a manual search. This will ensure you don’t miss relevant studies in your systematic review dissertation. The manual search involves identifying more studies in the bibliographies of the identified articles using a database search. It is also about contacting the authors and experts sourced from the found articles to give access to more articles that may not be found online. Finally, you can also identify key journals from the articles and perform a hand search. For example, suppose I identify the Journal of Cognitive Psychology. In that case, I will visit that journal’s website and perform a manual search there. A properly done manual search can help you identify more articles that you couldn’t have identified using databases only.

Step 8: Perform systematic review screening using the full-body texts

After having all your articles intact, the next step is to screen for full-text bodies. In most cases, the titles and abstracts may not contain enough information for screening purposes. You must read the full texts of the articles to determine their full eligibility. At this point, you screen articles identified through database search and manual search altogether. For example, sometimes you may be interested in healthy adolescents. In the abstract, the author of the articles may only report adolescents without providing any specifics about them. Upon reading the full text, you may discover that the authors included adolescents with mental issues that are not within your study’s scope. Therefore, always do a full-text screening before you move to the next step.

Step 9: Perform systematic review quality assessment using PRISMA, etc

Systematic review dissertations can be used to inform the formulation of practice guidelines and even inform policies. You must strive to review only studies with rigorous methodological quality. The quality assessment tool will depend on your study’s design. The commonly used ones for student dissertations include CASP Checklists and Joanna Briggs Institute (JBI) Checklists. You can consult with your supervisor before arriving at the final decision. Transparently report your quality assessment findings. For example, indicate the score of each study under each item of each tool and calculate the overall score in the form of a percentage. Also, always have a cut-off of 65%, and studies whose methodological rigour is below the cut-off are excluded.

Step 10: Perform systematic review data extraction

The next step is to extract relevant data from your studies. Your data extraction approach depends on the research design of the studies you used. If you use qualitative studies, your data extraction can focus on individual studies’ findings, particularly themes. You can also extract data that can aid in-depth analysis, such as country of study, population characteristics, etc. Using quantitative studies, you can collect quantitative data that will aid your analysis, such as means and standard deviations and other crucial information relevant to your analysis technique. Always chart your data in a tabular format to facilitate easy management and handling.

Step 11: Carry on with systematic review data analysis

The data analysis approach used in your systematic review dissertation will depend on the research design. Using qualitative studies, you will rely on qualitative approaches to analyse your data. For example, you can do a thematic analysis or a narrative synthesis. If you used quantitative studies, you might need to perform a meta-analysis or narrative synthesis. A meta-analysis is done when you have homogenous studies (such as population, outcome variables, measurement tools, etc.) that are experimental in nature. Particularly, meta-analysis is performed when reviewing controlled randomized trials or other interventional studies. In other words, meta-analysis is appropriately used when reviewing the effectiveness of interventions. However, if your quantitative studies are heterogenous, such as using different research designs, you must perform a narrative synthesis.

Step 12: Prepare the written report

The final step is to produce a written report of your systematic review dissertation. One of the ethical concerns in systematic reviews is transparency. You can improve the transparency of your reporting by using an established protocol like PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses).

Approximate price: $ 22

Calculate the price of your order

• Free title page and bibliography
• Unlimited revisions
• Plagiarism-free guarantee

Money-back guarantee

• 24/7 support
• Systematic Review Service
• Meta Analysis Services
• Literature Search Service
• Literature Review Assistance
• Scientific Article Writing Service
• Manuscript Publication Assistance
• 275 words per page
• 12 pt Arial/Times New Roman
• Double line spacing
• Any citation style (APA, MLA, Chicago/Turabian, Harvard, etc)

Our guarantees

Delivering a high-quality product at a reasonable price is not enough anymore. That’s why we have developed 5 beneficial guarantees that will make your experience with our service enjoyable, easy, and safe.

You have to be 100% sure of the quality of your product to give a money-back guarantee. This describes us perfectly. Make sure that this guarantee is totally transparent.

Zero-plagiarism guarantee

Each paper is composed from scratch, according to your instructions. It is then checked by our plagiarism-detection software. There is no gap where plagiarism could squeeze in.

Free-revision policy

Thanks to our free revisions, there is no way for you to be unsatisfied. We will work on your paper until you are completely happy with the result.

Privacy policy

Your email is safe, as we store it according to international data protection rules. Your bank details are secure, as we use only reliable payment systems.

Fair-cooperation guarantee

By sending us your money, you buy the service we provide. Check out our terms and conditions if you prefer business talks to be laid out in official language.

• Department of Psychology
• Nur Islamiah defends h...

Nur Islamiah defends her PhD thesis

‘Parental Roles on Children’s Emotion Regulation’.

Time and place

2 May 2024 at 9:30 am (CEST).

The defence will take place online on Zoom. Click here to participate . Passcode: 638765. 

Assessment committee

• Professor Stig Poulsen, Department of Psychology, University of Copenhagen, Denmark (chair)
• Associate Professor Patrick Bender, Department of Communication and Psychology, Aalborg University, Denmark

Supervisors

• Professor Ingo Zettler, Department of Psychology, University of Copenhagen (principal supervisor)
• Associate Professor Sonja Breinholst, Department of Psychology, University of Copenhagen (co-supervisor)
• Assistant Professor Monika Walczak, Department of Psychology, University of Copenhagen (co-supervisor)

The purpose of this thesis is to advance understanding of the parental role in children’s emotion regulation (ER). The thesis consists of three academic papers: two original studies and a systematic review. In the first study, mothers’ characteristics (i.e., psychopathology and metacognitions) and parenting practices were examined as predictors of children’s nine specific cognitive ER strategies in a community sample. The second study, a systematic review, was conducted to explore the potential role of fathers in the development of ER in typically developing children. The review included 43 studies that investigated a range of paternal factors influencing ER in their children. Finally, in the last study, fathers’ and mothers’ cognitions of child anxiety were examined as a potential predictor of emotion dysregulation in clinically anxious children. In sum, the findings of the thesis suggest that parents’ psychopathology and negative cognitions may be an important risk factor for lower ER or maladaptive cognitive ER strategies in their children. This thesis discusses the findings and implications of the three studies and suggests directions for future research.

Time: 2 May 2024, 9:30-12:30

Place: Online on Zoom

Organizer: The Department of Psychology