public speaking skills research

• Even more »

Account Options

• Try the new Google Books
• Advanced Book Search

Get this book in print

• Taylor & Francis
• Barnes&Noble.com
• Books-A-Million
• All sellers  »

Selected pages

Other editions - View all

Common terms and phrases, about the author  (2024).

Stevie M. Munz is Associate Professor of Communication at Utah Valley University, USA.

Tim McKenna-Buchanan is Associate Professor of Communication Studies at Manchester University, USA.

Anna M. Wright is Instructional Assistant Professor at University of Illinois Urbana-Champaign, USA.

Bibliographic information

SYSTEMATIC REVIEW article

Assessing speaking proficiency: a narrative review of speaking assessment research within the argument-based validation framework.

• 1 Language Testing Research Centre, The University of Melbourne, Melbourne, VIC, Australia
• 2 Department of Linguistics, University of Illinois at Urbana-Champaign, Champaign, IL, United States

This paper provides a narrative review of empirical research on the assessment of speaking proficiency published in selected journals in the field of language assessment. A total of 104 published articles on speaking assessment were collected and systematically analyzed within an argument-based validation framework ( Chapelle et al., 2008 ). We examined how the published research is represented in the six inferences of this framework, the topics that were covered by each article, and the research methods that were employed in collecting the backings to support the assumptions underlying each inference. Our analysis results revealed that: (a) most of the collected articles could be categorized into the three inferences of evaluation, generalization , and explanation ; (b) the topics most frequently explored by speaking assessment researchers included the constructs of speaking ability, rater effects, and factors that affect spoken performance, among others; (c) quantitative methods were more frequently employed to interrogate the inferences of evaluation and generalization whereas qualitative methods were more frequently utilized to investigate the explanation inference. The paper concludes with a discussion of the implications of this study in relation to gaining a more nuanced understanding of task- or domain-specific speaking abilities, understanding speaking assessment in classroom contexts, and strengthening the interfaces between speaking assessment, and teaching and learning practices.

Introduction

Speaking is a crucial language skill which we use every day to communicate with others, to express our views, and to project our identity. In today's globalized world, speaking skills are recognized as essential for international mobility, entrance to higher education, and employment ( Fulcher, 2015a ; Isaacs, 2016 ), and are now a major component in most international and local language examinations, due at least in part to the rise of the communicative movement in language teaching and assessment ( Fulcher, 2000 ). However, despite its primacy in language pedagogy and assessment, speaking has been considered as an intangible construct which is challenging to conceptualize and assess in a reliable and valid manner. This could be attributable to the dynamic and context-embedded nature of speaking, but may be also due to the various forms that it can assume (e.g., monolog, paired conversation, group discussion) and the different conditions under which speaking happens (e.g., planned or spontaneous) (e.g., Luoma, 2004 ; Carter and McCarthy, 2017 ). When assessing speaking proficiency, multiple factors come into play which potentially affect test takers' performance and subsequently their test scores, including task features, interlocutor characteristics, rater effects, and rating scale, among others ( McNamara, 1996 ; Fulcher, 2015a ). In the field of language assessment, considerable research attention and efforts have been dedicated to researching speaking assessment. This is evidenced by the increasing number of research papers with a focus on speaking assessment that have been published in the leading journals in the field.

This prolonged growth in speaking assessment research warrants a systematic review of major findings that can help subsequent researchers and practitioners to navigate the plethora of published research, or provide them with sound recommendations for future explorations in the speaking assessment domain. Several review or position papers are currently available on speaking assessment, either reviewing the developments in speaking assessment more broadly (e.g., Ginther, 2013 ; O'Sullivan, 2014 ; Isaacs, 2016 ) or examining a specific topic in speaking assessment, such as pronunciation ( Isaacs, 2014 ), rating spoken performance ( Winke, 2012 ) and interactional competence ( Galaczi and Taylor, 2018 ). Needless to say, these papers are valuable in surveying related developments in speaking proficiency assessment and sketching a broad picture of speaking assessment for researchers and practitioners in the field. Nonetheless, they typically adopt the traditional literature review approach, as opposed to the narrative review approach that was employed in this study. According to Norris and Ortega (2006 , p. 5, cited in Ellis, 2015 , p. 285), a narrative review aims to “scope out and tell a story about the empirical territory.” Compared with traditional literature review which tends to rely on a reviewer's subjective evaluation of the important or critical aspects of the existing knowledge on a topic, a narrative review is more objective and systematic in the sense the results are usually based on the coding analysis of the studies that are collected through applying some pre-specified criteria. Situated within the argument-based validation framework ( Chapelle et al., 2008 ), this study is aimed at presenting a narrative review of empirical research on speaking assessment published in two leading journals in the field of language assessment, namely, Language Testing (LT) and Language Assessment Quarterly (LAQ). Through following the systematic research procedures of narrative review (e.g., Cooper et al., 2019 ), we survey the topics of speaking assessment that have been explored by researchers as well as the research methods that have been utilized with a view to providing recommendations for future speaking assessment research and practice.

Theoretical Framework

Emerging from the validation of the revised Test of English as a Foreign Language (TOEFL), the argument-based validation framework adopted in this study represents an expansion of Kane's (2006) argument-based validation model, which posits that a network of inferences needs to be verified to support test score interpretation and use. A graphic display of this framework is presented in Figure 1 . As shown in this figure, the plausibility of six inferences need to be verified to build a validity argument for a language test, including: domain definition, evaluation, generalization, explanation, extrapolation , and utilization . Also included in the framework are the key warrants that license each inference and its underlying assumptions. This framework was adopted as the guiding theoretical framework of this review study in the sense that each article collected for this study was classified into one or several of these six inferences in the framework. As such, it is necessary to briefly explain these inferences in Figure 1 in the context of speaking assessment. The explanation of the inferences, together with their warrants and assumptions, is largely based on Chapelle et al. (2008) and Knoch and Chapelle (2018) . To facilitate readers' understanding of these inferences, we use the TOEFL speaking test as an example to provide an illustration of the warrants, key assumptions, and backings for each inference.

Figure 1 . The argument-based validation framework (adapted from Chapelle et al., 2008 , p. 18).

The first inference, domain definition , links the target language use (TLU) domain to test takers' observed performance on a speaking test. The warrant supporting this inference is that observation of test takers' performance on a speaking test reveals the speaking abilities and skills required in the TLU domain. In the case of the TOEFL speaking test, the TLU domain is the English-medium institutions of higher education. Therefore, the plausibility of this inference hinges on whether observation of test takers' performance on the speaking tasks reveals essential academic speaking abilities and skills in English-medium universities. An important assumption underlying this inference is that speaking tasks that are representative of language use in English-medium universities can be identified and simulated. Backings in support of this assumption can be collected through interviews with academic English experts to investigate speaking abilities and skills that are required in English-medium universities.

The warrant for the next inference, evaluation , is that test takers' performance on the speaking tasks is evaluated to provide observed scores which are indicative of their academic speaking abilities. The first key assumption underlying this warrant is that the rating scales for the TOEFL speaking test function as intended by the test provider. Backings for this assumption may include: a) using statistical analyses (e.g., many-facets Rasch measurement, or MFRM) to investigate the functioning of the rating scales for the speaking test; and b) using qualitative methods (e.g., raters' verbal protocols) to explore raters' use of the rating scales for the speaking test. Another assumption for this warrant is that raters provide consistent ratings on each task of the speaking test. Backing for this assumption typically entails the use of statistical analyses to examine rater reliability on each task of the speaking test. The third assumption is that detectable rater characteristics do not introduce systematic construct-irrelevant variance into their ratings of test takers' performance. Bias analyses are usually implemented to explore whether certain rater characteristics (e.g., experience, L1 background) interact with test taker characteristics (e.g., L1 background) in significant ways.

The third inference is generalization . The warrant that licenses this inference is that test takers' observed scores reflect their expected scores over multiple parallel versions of the speaking test and across different raters. A few key assumptions that underlie this inference include: a) a sufficient number of tasks are included in the TOEFL speaking test to provide stable estimates of test takers' speaking ability; b) multiple parallel versions of the speaking test feature similar levels of difficulty and tap into similar academic English speaking constructs; and c) raters rate test takers' performance consistently at the test level. To support the first assumption, generalizability theory (i.e., G-theory) analyses can be implemented to explore the number of tasks that is required to achieve the desired level of reliability. For the second assumption, backings can be collected through: (a) statistical analyses to ascertain whether multiple parallel versions of the speaking test have comparable difficulty levels; and (b) qualitative methods such as expert review to explore whether the parallel versions of the speaking test tap into similar academic English speaking constructs. Backing of the third assumption typically entails statistical analyses of the scores that raters have awarded to test takers to examine their reliability at the test level.

The fourth inference is explanation . The warrant of this inference is that test takers' expected scores can be used to explain the academic English speaking constructs that the test purports to assess. The key assumptions for this inference include: (a) features of the spoken discourse produced by test takers on the TOEFL speaking test can effectively distinguish L2 speakers at different proficiency levels; (b) the rating scales are developed based on academic English speaking constructs that are clearly defined; and (c) raters' cognitive processes when rating test takers' spoken performance are aligned with relevant theoretical models of L2 speaking. Backings of these three assumptions can be collected through: (a) discourse analysis studies aiming to explore the linguistic features of spoken discourse that test takers produce on the speaking tasks; (b) expert review of the rating scales to ascertain whether they reflect relevant theoretical models of L2 speaking proficiency; and (c) rater verbal protocol studies to examine raters' cognitive processes when rating performance on the speaking test.

The fifth inference in the framework is extrapolation . The warrant that supports this inference is that the speaking constructs that are assessed in the speaking test account for test takers' spoken performance in English-medium universities. The first key assumption underlying this warrant is that test takers' performance on the TOEFL speaking test is related to their ability to use language in English-medium universities. Backing for this assumption is typically collected through correlation studies, that is, correlating test takers' performance on the speaking test with an external criterion representing their ability to use language in the TLU domains (e.g., teachers' evaluation of students' speaking proficiency of academic English). The second key assumption for extrapolation is that raters' use of the rating scales reflects how spoken performance is evaluated in English-medium universities. For this assumption, qualitative studies can be undertaken to compare raters' cognitive processes with those of linguistic laypersons in English-medium universities such as subject teachers.

The last inference is utilization . The warrant supporting this inference is that the speaking test scores are communicated in appropriate ways and are useful for making decisions. The assumptions that underlie the warrant include: (a) the meaning of the TOEFL speaking test scores is clearly interpreted by relevant stakeholders, such as admissions officers, test takers, and teachers; (b) cut scores are appropriate for making relevant decisions about students; and (c) the TOEFL speaking test has a positive influence on English teaching and learning. To collect the backings for the first assumption, qualitative studies (e.g., interviews, focus groups) can be conducted to explore stakeholders' perceptions of how the speaking test scores are communicated. For the second assumption, standard setting studies are often implemented to interrogate the appropriateness of cut scores. The last assumption is usually investigated through test washback studies, exploring how the speaking test influences English teaching and learning practices.

The framework was used in the validation of the revised TOEFL, as reported in Chapelle et al. (2008) , as well as in low-stakes classroom-based assessment contexts (e.g., Chapelle et al., 2015 ). According to Chapelle et al. (2010) , this framework features several salient advantages over other alternatives. First, given the dynamic and context-mediated nature of language ability, it is extremely challenging to use the definition of a language construct as the basis for building the validity argument. Instead of relying on an explicit definition of the construct, the argument-based approach advocates the specification of a network of inferences, together with their supporting warrants and underlying assumptions that link test takers' observed performances to score interpretation and use. This framework also makes it easier to formulate validation research plans. Since every assumption is associated with a specific inference, research questions targeting each assumption are developed ‘in a more principled way as a piece of an interpretative argument' ( Chapelle et al., 2010 , p. 8). As such, the relationship between validity argument and validation research becomes more apparent. Another advantage of this approach to test validation it that it enables the structuring and synthesis of research results into a logical and coherent validity argument, not merely an amalgamation of research evidence. By so doing, it depicts the logical progression of how the conclusion from one inference becomes the starting point of the next one, and how each inference is supported by research. Finally, by constructing a validity argument, this approach allows for a critical evaluation of the logical development of the validity argument as well as the research that supports each inference. In addition to the advantages mentioned above for test validation research, this framework is also very comprehensive, making it particularly suitable for this review study.

By incorporating this argument-based validation framework in a narrative review of the published research on speaking assessment, this study aims to address the following research questions:

RQ1. How does the published research on speaking assessment represent the six inferences in the argument-based validation framework?

RQ2. What are the speaking assessment topics that constituted the focus of the published research?

RQ3. What methods did researchers adopt to collect backings for the assumptions involved in each inference?

This study followed the research synthesis steps recommended by Cooper et al. (2019) , including: (1) problem formation; (2) literature search; (3) data evaluation; (4) data analysis; (5) interpretation of results; and (6) public presentation. This section includes details regarding article search and selection, and methods for synthesizing our collected studies.

Article Search and Selection

We collected the articles on speaking assessment that were published in LT from 1984 1 to 2018 and LAQ from 2004 to 2018. These two journals were targeted because: (a) both are recognized as leading high-impact journals in the field of language assessment; (b) both have an explicit focus on assessment of language abilities and skills. We understand that numerous other journals in the field of applied linguistics or educational evaluation also publish research on speaking and its assessment. Admittedly, if the scope of our review extends to include more journals, the findings might be different; however, given the high impact of these two journals in the field, a review of their published research on speaking assessment in the past three decades or so should provide sufficient indication of the directions in assessing speaking proficiency. This limitation is discussed at the end of this paper.

The PRISMA flowchart in Figure 2 illustrates the process of article search and selection in this study. A total of 120 articles were initially retrieved through manually surveying each issue in the electronic archives of the two journals, containing all articles published in LT from 1984 to 2018 and LAQ from 2004 to 2018. Two inclusion criteria were applied: (a) the article had a clear focus on speaking assessment. Articles that targeted the whole language test involving multiple skills were not included; (b) the article reported an empirical study in the sense that it investigated one or more aspects of speaking assessment through the analysis of data from either speaking assessments or designed experimental studies.

Figure 2 . PRISMA flowchart of article search and collection.

Through reading the abstracts carefully, 13 articles were excluded from our analysis, with two special issue editorials and 11 review or position papers. A further examination of the remaining 107 articles revealed that three of them involved multiple language skills, suggesting a lack of primary focus on speaking assessment. These three articles were therefore excluded from our analysis, yielding 104 studies in our collection. Of the 104 articles, 73 (70.19%) were published in LT and 31 (29.81%) were published in LAQ . All these articles were downloaded in PDF format and imported into NVivo 12 ( QSR, 2018 ) for analysis.

Data Analysis

To respond to RQ1, we coded the collected articles into the six inferences in the argument-based validation framework based on the focus of investigation for each article, which was determined by a close examination of the abstract and research questions. If the primary focus did not emerge clearly in this process, we read the full text. As the coding progressed, we noticed that some articles had more than one focus, and therefore should be coded into multiple inferences. For instance, Sawaki (2007) interrogated several aspects of an L2 speaking test that were considered as essential to its construct validity, including the interrelationships between the different dimensions of spoken performance and the reliability of test scores. The former was considered as pertinent to the explanation inference, as it explores the speaking constructs through the analysis of test scores; the latter, however, was deemed more relevant to the generalization inference, as it concerns the consistency of test scores at the whole test level ( Knoch and Chapelle, 2018 ). Therefore, this article was coded into both explanation and generalization inference.

To answer RQ2, the open coding method ( Richards, 1999 ) was employed to explore the speaking assessment topics that constituted the focus of each article in our collection. This means that a coding scheme was not specified a prior ; rather, it was generated through examining the abstracts or full texts to determine the topics and subtopics. RQ3 was investigated through coding the research methods that were employed by speaking assessment researchers. A broad coding scheme consisting of three categories was employed to code the research methods: (a) quantitatively oriented; (b) qualitatively oriented; and (c) mixed methods with both quantitative and qualitative orientations. Next, the open coding method was adopted to code the specific methods that were utilized under each broad category. Matrix coding analysis ( Miles et al., 2014 ) was subsequently implemented in NVivo to explore the relationships between the speaking assessment topics, research methods and the six inferences in the argument-based validation framework. This would enable us to sketch the broad patterns of: (a) which topics on speaking assessment tended be investigated under each of the six inferences; (b) which research methods were frequently employed to collect the backings for the assumptions that underlie each inference.

The coding process underwent three iterative stages to ensure the reliability of the coding results. First, both authors coded 10 articles selected randomly from the dataset independently and then compared their coding results. Differences in coding results were resolved through discussion. Next, the first author coded the rest of the articles in NVivo, using the coding scheme that was generated during the first stage while adding new categories as they emerged from the coding process. Finally, the second author coded 20 articles (19.23%) which were randomly selected from the dataset, using the coding scheme that was determined during the second stage. Intercoder agreement was verified through calculating Cohen's kappa statistic in NVivo ( k = 0.93), which suggested satisfactory coding reliability.

Results and Discussion

Overall, our coding results indicate that a wide range of research was conducted of speaking assessment to interrogate the six inferences in the argument-based validation framework. These studies cover a variety of research topics, employing quantitative, qualitative, and mixed research methods. In this section, we describe and discuss the analysis results through showcasing the broad patterns that emerged from our coding process. Illustrative studies are used as appropriate to exemplify the research that was undertaken in assessing speaking proficiency.

Representation of the Published Research in the Six Inferences

Table 1 presents the representation of the published research in the six inferences. As indicated in this table, most of our collected articles were categorized into the three inferences of evaluation ( n = 42, 40.38%), generalization ( n = 42, 40.38%), and explanation ( n = 50, 48.08%); in contrast, a much smaller number of studies targeted the other three inferences of domain description ( n = 4, 3.85%), extrapolation ( n = 7, 6.73%), and utilization ( n = 5, 4.81%). Despite the highly skewed representation of the published research in the six inferences, the findings were not entirely surprising. According to the argument-based validation framework ( Chapelle et al., 2008 ), backings in support of the assumptions that underlie the three inferences of evaluation, generalization , and explanation relate to almost all key components in the assessment of speaking proficiency, including rater effects, rating scale, task features or administration conditions, interlocutor effects in speaking tasks such as paired oral, interview or group discussion, and features of produced spoken discourse. These components essentially represent the concerns surrounding the development, administration, and validation of speaking assessment (e.g., McNamara, 1996 ; Fulcher, 2015a ). Take the inference of evaluation as an example. In the argument-based validation framework, this inference pertains to the link from the observation of test takers' performance on a speaking test to their observed scores. As mentioned previously (see section Theoretical Framework), backings in support of the key assumptions underlying this inference include an evaluation of rating scales as well as rater effects at the task level. Given the pivotal role that raters and rating scales play in speaking assessment (e.g., Eckes, 2011 ), it is not surprising to observe a reasonably high proportion of studies exploring the plausibility of this inference. Almost half of our collected articles ( n = 50, 48.08%) interrogated the explanation inference. This finding can be interpreted in relation to the centrality of understanding the construct in language test development and validation (e.g., Alderson et al., 1995 ; Bachman and Palmer, 1996 ), which lies at the core of the explanation inference.

Table 1 . Representation of the published research in the six inferences ( n = 104).

One possible explanation for the limited research on domain description is related to the journals that formed the basis for this review study. Both LT and LAQ have an explicit focus on language assessment, whereas in many cases, exploration of language use in TLU domains, which is the focus of domain description , might be reported as needs assessment studies in test development reports, which were beyond the purview of this study. Another plausible explanation, as pointed out by one of the reviewers, might lie in the lack of theoretical sophistication regarding this inference. The reason why few studies targeted the extrapolation inference might be attributable to the challenges in pinpointing the external criterion measure, or in collecting valid data to represent test takers' ability to use language in TLU domains. These challenges could be exacerbated in the case of speaking ability due to its intangible nature, the various forms that it may assume in practice, and the different conditions under which it happens. Similarly, very few studies focused on the utilization inference which concerns the communication and use of test scores. This could relate to the fact that test washback or impact studies have to date rarely focused exclusively on speaking assessment ( Yu et al., 2017 ). Speaking assessment researchers should consider exploring this avenue of research in future studies, particularly against the backdrop of the increasingly extensive application of technology in speaking assessment ( Chapelle, 2008 ).

Speaking Assessment Topics

Table 2 presents the matrix coding results of speaking assessment topics and the six inferences in the argument-based validation framework. It should be noted that some of the frequency statistics in this table are over-estimated because, as mentioned previously, some articles were coded into multiple inferences; however, this should not affect the general patterns that emerged from the results in a significant way. The topics that emerged from our coding process are largely consistent with the themes that Fulcher (2015a) identified in his review of speaking assessment research. One noteworthy difference is many-facets Rasch measurement (MFRM), a topic in Fulcher (2015a) but was coded as a research method in our study (see section Research Methods). In what follows, we will focus on the three topics which were most frequently investigated by speaking assessment researchers, namely, speaking constructs, rater effects, and factors that affect speaking performance, as examples to illustrate the research that was undertaken of speaking assessment.

Table 2 . Matrix coding results of inferences and speaking assessment topics ( n = 104).

Speaking Constructs

Table 2 shows that “speaking constructs” ( n = 47) is the topic that was investigated most frequently in our collected studies. Matrix coding results indicate that this topic area appears most frequently under the inference of explanation ( n = 39, 37.50%). The importance of a clear understanding of the construct cannot be overemphasized in language test development and validation (e.g., Alderson et al., 1995 ; Bachman and Palmer, 1996 ). Indeed, construct definition forms the foundation of several highly influential test validation frameworks in the field (e.g., Messick, 1989 ; Weir, 2005 ). Our analysis indicates that considerable research has been dedicated to disentangling various speaking constructs. Two topics that feature prominently in this topic area are the analysis of spoken discourse and interactional competence.

A common approach to investigate the speaking constructs is through the analysis of produced spoken discourse ( Carter and McCarthy, 2017 ), usually focusing on linguistic features that can distinguish test takers at different proficiency levels such as complexity, accuracy, and fluency (e.g., Iwashita, 2006 ; Gan, 2012 ; Bosker et al., 2013 ). Research in this area can provide substantial evidence concerning speaking proficiency. Iwashita (2006) , for instance, examined the syntactic complexity of the spoken performance of L2 Japanese learners. Results reveal that learner' oral proficiency could be predicted significantly by several complexity indicators, including T-unit length, the number of clauses per T-unit, and the number of independent clauses per T-unit. In another discourse analysis study, Gan (2012) probed the syntactic complexity of test takers' spoken discourse and examined the relationship between syntactic complexity and task type in L2 speaking assessment. Gan's results show that, compared with the group interaction task, test takers' discourses on the individual presentation task featured longer T-units and utterances as well as significantly greater number of T-units, clauses, verb phrases and words. These discourse analysis studies have implications for understanding speaking proficiency as well as its development and maturity among L2 learners.

International competence (IC) is yet another topic which features prominently in this topic area. Despite the recognized need of including IC in speaking assessment (e.g., Kramsch, 1986 , McNamara, 1997 ), how it should be conceptualized remains a contentious issue. Research has shown that this construct consists of multiple dimensions which is susceptible to the influence of a range of personal cognitive and contextual factors ( Galaczi and Taylor, 2018 ). Our review suggests that IC was approached through analyzing test takers' spoken discourse as well as exploring raters' perspectives. Galaczi (2008) , for instance, performed elaborate analyses of test takers' spoken discourse on the paired speaking task in the First Certificate in English (FCE) speaking test. The results led the researcher to conclude that test takers' interactions primarily featured three patterns on paired oral assessment tasks: collaborative, parallel and blended interaction (i.e., a mixture of collaborative/parallel or collaborative/asymmetric features). In a more recent study, Lam (2018) analyzed test takers' spoken discourse on a school-based group oral speaking assessment for the Hong Kong Diploma of Secondary Education (HKDSE) English Language Examination. Instead of exploring IC more broadly, as in Galaczi (2008) , this study targeted a particular feature of IC, namely, producing responses contingent on previous speakers' contributions. The analyses pointed to three kinds of conversational actions that underpinned a response contingent on previous speaker's contributions: formulating previous speakers' contributions, accounting for (dis)agreement with previous speakers' ideas and extending previous speakers' ideas.

Some other studies explored the construct of IC from raters' perspectives. A typical study was reported by May (2011) who explored the features that were salient to raters on a paired speaking test. The study identified a repertoire of features which were salient to raters, and hence were potentially integral to the IC construct. Such features include, for example, the ability to manage a conversation, ask for opinion or clarification, challenge or disagree with an interactional partner, and demonstrate effective body language, and interactive listening. While suggesting that IC is a highly complex and slippery construct, these studies have significant implications for clarifying the IC construct and promoting its valid operationalization in speaking assessment. The findings are particularly meaningful in the context where interactive tasks are increasingly used in speaking assessment.

Rater Effects

Raters play a significant role in speaking assessment; their performance is affected by a host of non-linguistic factors, which are often irrelevant to the speaking constructs of interest, hence causing construct-irrelevant variance ( Messick, 1989 ) or contamination ( AERA et al., 2014 ). Not surprisingly, the next topic area that was most frequently explored by speaking assessment researchers is rater effects ( n = 39). The studies that focused on this topic were mostly classified into the two inferences of evaluation ( n = 27, 25.96%) and generalization (n =23, 22.12%). Knoch and Chapelle (2018) applied the argument-based validation framework to the analysis of rater effects and rating processes in language assessment research. They observed that several important aspects of rater effects could be mapped onto evaluation and generalization inferences. The key assumptions of the evaluation inference relate to the raters' consistency at the task level, the bias that raters display against task types or other aspects of the assessment situation, and the impact of raters' characteristics on the ratings that they assign. When it comes to the generalization inference, the key assumptions largely concern raters' consistency at the whole test level and the number of raters that is required to achieve the desired level of consistency. Research on rater effects has significant implications for enhancing both the validity and fairness of speaking assessment (e.g., McNamara et al., 2019 ).

Two topics that feature prominently in the study of rater effects are the impact of raters' characteristics on their rating behaviors and rater cognition, that is, the cognitive processes that raters engage when assigning scores to a spoken performance. Raters' characteristics such as language background, experience and qualifications may have appreciable impact on their ratings. This topic has attracted considerable research attention as it has implications for test fairness and rater training programs. One such study was reported by Kim (2009) who examined and compared the rating behaviors of native and non-native English teachers when assessing students' spoken performance. The results indicate that native-speaker (NS) and non-native-speaker (NNS) teachers on the whole exhibited similar severity levels and internal consistency; however, in comparison with NNS teachers, NS teachers provided more detailed and elaborate comments on students' performance. The findings generally concur with Zhang and Elder (2011) who compared the rating behaviors of NS and NNS teachers in the context of the College English Test - Spoken English Test (CET-SET), a large-scale high-stakes speaking test in China. Instead of focusing on raters' L1 background, Winke et al. (2013) examined whether raters' accent familiarity, defined as their L2 learning experience, constituted a potential source of bias when they rated test takers' spoken performance. In other words, if a rater studies Chinese as his or her L2, is he or she biased toward test takers who have Chinese as their L1? Their findings indicate that the raters with Spanish or Chinese as their L2 were significantly more lenient toward L1 Spanish and Chinese test takers than they were toward those from other L1 backgrounds. However, in both cases, the effect sizes were small, suggesting that such effect had minimal impact in practice. The results are largely consistent with some other studies in our collection (e.g., Yan, 2014 ; Wei and Llosa, 2015 ), which explored a similar topic.

Rater cognition or rating processes constitute yet another important topic under the topic area of “rater effects”. Studies along this line are typically implemented through analyzing raters' verbal protocols to explore their cognitive processes when applying the rating criteria or assigning scores to a spoken performance. Research into raters' cognitive processes can generate valuable insights into the validity of the rating scales as well as the speaking constructs that are being assessed in a speaking test. Findings from these studies have important implications for the revision of rating scales, improving rater training programs, and enhancing the validity and usefulness of the speaking test in focus. In a qualitative study, Kim (2015) explored the rating behaviors of three groups of raters with different levels of experience on an L2 speaking test by analyzing their verbal reports of rating processes. The study revealed that the three groups of raters exhibited varying uses of the analytic rating scales, hence suggesting that experience was an important variable affecting their rating behaviors. Furthermore, an analysis of their performance over time revealed that the three groups of raters demonstrated different degrees of improvement in their rating performance. It should be noted that several studies in our collection examined raters' rating processes with a view to either complementing or accounting for the quantitative analyses of speaking test scores. For instance, both Kim (2009) and Zhang and Elder (2011) , two studies which were reviewed previously, investigated raters' rating processes, and the findings significantly enriched our understanding of the rating behaviors of raters from different backgrounds.

Factors That Affect Spoken Performance

The third topic area that emerged from our coding process is “factors that affect spoken performance” ( n = 30). As shown in Table 3 , most of the studies in this topic area were classified into the inference of generalization ( n = 19, 18.27%). This is understandable as factors such as task features, administration conditions, and planning time might affect the generalizability of speaking test scores. Indeed, understanding factors that affect test performance has long since been one of the central concerns for language assessment research as a whole (e.g., Bachman, 1990 ; Bachman et al., 1995 ). Research along this line has implications for speaking test development and implementation, and for test score interpretation and use. Our coding analyses indicate that a range of factors have been explored by speaking assessment researchers, of which ‘interlocutor effects' features most prominently. This could be related to the increasingly widespread use of interviews, paired oral or group discussion tasks to assess speaking ability in applied linguistics and language pedagogy. A notable advantage with these assessment formats lies in the unscripted and dynamic nature of the interactions involved, which is key to increasing the authenticity of speaking assessments. Nonetheless, interlocutor characteristics, such as gender, proficiency levels, personality, and styles of interaction might have considerable impact on test takers' spoken performance, thus impinging on the validity, fairness and overall usefulness of these tasks.

Table 3 . Matrix coding results of research methods and inferences ( n = 104).

An earlier study on interlocutor effects was reported by McNamara and Lumley (1997) who examined the potential impact of interlocutor characteristics on test scores in the context of the Occupational English Test (OET), a high-stakes speaking test for health professionals in Australia. Their study indicated that interlocutor characteristics had some influence on the ratings that test takers received. For example, they found that raters tended to compensate for interlocutors' incompetence in conducting the speaking test; in other words, if an interlocutor was perceived as less competent, test takers tended to receive higher ratings than expected. In addition, they also observe that an interlocutor's ability to build rapport with test takers had a positive effect on the ratings that test takers received. In another study, Brown (2003) probed the effects of interlocutor characteristics on test takers' performance in the context of a conversational interview. She performed elaborate analyses of the interactions between the interviewers (i.e., interlocutors) and test takers, revealing that the interlocutors differed quite significantly in terms of: (a) how they structured topical sequences; (b) their questioning technique; and (c) how they provided feedback and built rapport with test takers. Further analyses uncovered that interviewer styles had quite significant impact on the ratings that test takers received. Resonating with McNamara and Lumley (1997) , the findings of this study again call for the reconceptualization of speaking proficiency.

Several other studies focused on the effects of interaction partners in paired or group oral tasks on spoken performance. ( Ockey, 2009 ), for instance, investigated the potential effects of group member's assertiveness levels on spoken performance on a group discussion task. Results confirmed that test takers' assertiveness levels had an impact on the scores that they received. Specifically, assertive test takers were awarded higher scores than expected when grouped with non-assertive test takers; this trend, however, was reversed when they were grouped with test takers with similar assertiveness levels. A plausible explanation could be that raters viewed assertive test takers more positively when other members in the groups were non-assertive, whereas more negatively when other group members, who were also assertive, competed to be the leaders in the interactions. This study reiterates the co-constructed nature of speaking proficiency. Despite the research that has been undertaken of interlocutor effects, controversy remains as to whether this variation is part of the speaking construct and therefore should be incorporated in the design of a speaking test or it should be controlled to such an extent that it poses minimal threat to the reliability and fairness of speaking test scores ( Fulcher, 2015a ).

In addition to the three topics above, researchers also explored speaking test design ( n = 14) in terms of the task features (e.g., Wigglesworth and Elder, 2010 ; Ahmadi and Sadeghi, 2016 ) and the use of technology in speaking test delivery (e.g., Nakatsuhara et al., 2017 ; Ockey et al., 2017 ). The next topic is test score generalizability ( n = 7), typically investigated through G-theory analysis (e.g., Lee, 2006 ; Sawaki, 2007 ; Xi, 2007 ). Furthermore, six studies in our collection evaluated the rating scales for speaking assessments, including comparing the effectiveness of different types of rating scales (e.g., Hirai and Koizumi, 2013 ), and examining whether a rating scale functioned as intended by the test developer (e.g., Isaacs and Thomson, 2013 ). Finally, five studies focused on the use of speaking assessments, mainly relating to test takers' perceptions of speaking assessments (e.g., Scott, 1986 ; Qian, 2009 ) and standard setting studies to determine the cut scores for certain purposes (e.g., Pill and McNamara, 2016 ).

Research Methods

Table 3 presents the matrix coding results of research methods and inferences. As indicated in this table, quantitative research methods were more frequently employed by speaking assessment researchers ( n = 50), in comparison with qualitative methods ( n = 23). It is worth noting that a number of studies ( n = 31) utilized mixed methods design, which features a combination of both quantitative and qualitative orientations.

Table 3 indicates that quantitative methods were most frequently used to collect backings in support of the evaluation ( n = 21, 20.19%) and generalization inferences ( n = 27, 25.96%). This finding can be interpreted in relation to the key assumptions that underlie these two inferences (see section Theoretical Framework). According to the argument-based validation framework, the assumptions of these two inferences largely concern rater consistency at task and whole-test level, the functioning of the rating scales, as well as the generalizability of speaking test scores across tasks and raters. Understandably, quantitative methods are widely used to collect the backings to test these assumptions. In addition to the overall representation of quantitative methods in speaking assessment research, we also went a step further to examine the use of specific quantitative methods. As shown in Table 3 , while traditional data analysis methods such as ANOVA or regression ( n = 34) continued to be utilized, mainly in the interrogation of the inferences of evaluation ( n = 13, 12.50%), generalization ( n = 14, 13.46%), and explanation ( n = 15, 14.42%), Rasch analysis methods were also embraced by speaking assessment researchers ( n = 28). Note that Rasch analysis is an overarching term which encompasses a family of related models, among which the many-facets Rasch model (MFRM) is frequently used in speaking assessment (e.g., McNamara and Knoch, 2012 ). As an extension of the basic Rasch model, the MFRM allows for the inclusion of multiple aspects or facets in a speaking context (e.g., rater severity, task difficulty, difficulty of rating scales). Furthermore, compared with traditional data analysis methods such as correlation and ANOVA which can only provide results at the group level, the MFRM can provide both group- and individual-level statistics ( Eckes, 2011 ). This finding concurs with Fulcher (2015a) who identified the MFRM as an important theme in speaking assessment. It also resonates with the observation of Fan and Knoch (2019 , p. 136) who commented that Rasch analysis has indeed become “one of the default methods or analysis techniques to examine the technical quality of performance assessments.” The power of Rasch analysis in speaking assessment research is best illustrated by studies such as Bonk and Ockey (2003) , Eckes (2005) , and Winke et al. (2013) , among others, all of which examined rater effects on speaking assessments in different contexts. Finally, G-theory ( n = 7) and structural equation modeling ( n = 5), two complicated quantitative methods, were also utilized by speaking assessment researchers.

In terms of qualitative research methods, discourse analysis is the one which was most frequently employed by speaking assessment researchers ( n = 25). Matrix coding results indicate that this method features most prominently under the inference of explanation ( n = 20, 19.23%). This finding is aligned with the key assumptions that underlie the explanation inference, namely, (a) features of the spoken discourse produced by test takers can effectively distinguish L2 speakers at different proficiency levels, and (b) raters' cognitive processes are consistent with the theoretical models of L2 speaking, both entailing the use of discourse analysis method to explore test takers' spoken responses and raters' rating processes. Importantly, our analysis results indicate that conversation analysis (CA) was the method that appeared frequently under the category of “discourse analysis.” This is best represented by studies such as Galaczi (2008) , Lam (2018) , and Roever and Kasper (2018) , all endeavoring to elucidate the construct of interactional competence. As a data analysis method, CA provides speaking researchers with a principled and intricate approach to analyze the interactions between test takers and examiners in interview, paired oral, or group discussion tasks. Table 3 shows that some other qualitative methods were also quite frequently used by speaking researchers, including interview/focus groups ( n = 11), written comments ( n = 11), and verbal protocol reports ( n = 10). These research methods were typically adopted following the quantitative analyses of test takers' scores, which explains the increasingly widespread use of mixed methods in speaking assessment research ( n = 31). The finding could find resonance in the observation that mixed method research has been gaining momentum in language assessment research more broadly (e.g., Turner, 2013 ; Jang et al., 2014 ; Moeller et al., 2016 ). As shown in Table 3 , mixed-methods design is most frequently employed to collect backings in support of the inferences of evaluation ( n = 17, 16.35%) and explanation ( n = 16, 15.38%). For the evaluation inference, mixed method design was often utilized to research rater effects where quantitative and qualitative analyses were used sequentially to examine rating results and processes. When it comes to the explanation inference, researchers tended to use a combination of quantitative and qualitative analyses to explore the differences in test takers' speaking scores as well as the spoken discourse that they produced.

Conclusions and Implications

In this study, we conducted a narrative review of published empirical research on assessing speaking proficiency within the argument-based validation framework ( Chapelle et al., 2008 ). A total of 104 articles on speaking assessment were collected from LT (1984–2018) and LAQ (2004–2018), two highly influential journals in the field of language assessment. Following the coding of the collected articles, matrix coding analyses were utilized to explore the relationships between the speaking assessment topics, research methods, and the six inferences in the argument-based validation framework.

The analysis results indicate that speaking assessment was investigated from various perspectives, primarily focusing on seven broad topic areas, namely, the constructs of speaking ability, rater effects, factors that affect spoken performance, speaking test design, test score generalizability, rating scale evaluation, and test use. The findings of these studies have significantly enriched our understanding of speaking proficiency and how assessment practice can be made more reliable and valid. In terms of research methods, it was revealed that quantitative research methods were most frequently utilized by speaking assessment researchers, a trend which was particularly pronounced in the inferences of evaluation and generalization . Though traditional quantitative methods such as ANOVA, regression, and correlation continued to be employed, Rasch analysis played a potent role in researching speaking assessment. In comparison, qualitative methods were least frequently used, mainly for the interrogation of the explanation inference. Mixed-methods design, recognized as “an alternative paradigm” ( Jang et al., 2014 , p. 123), ranked in the middle in terms of frequency, suggesting its increasingly widespread use in speaking assessment research. This is noteworthy when it comes to the evaluation and explanation inference.

Despite the abundance of research on speaking assessment and the variety of research topics and methods that emerged from our coding process, we feel that there are several areas which have not been explored extensively by language assessment researchers, and therefore warrant more future research endeavors. First, more studies should be conducted to interrogate the three inferences of domain description, extrapolation , and utilization in the argument-based validation framework. As indicated in our study, only a small fraction of studies have been dedicated to examining these three inferences in comparison with evaluation, generalization , and explanation (see Table 2 ). Regarding domain description , we feel that more research could be undertaken to understand task- and domain-specific speaking abilities and communicative skills. This would have significant implications for enhancing the authenticity of speaking assessment design, and for constructing valid rating scales for evaluating test takers' spoken performance. The thick description approach advocated by Fulcher et al. (2011) could be attempted to portray a nuanced picture of speaking ability in the TLU domains, especially in the case of Language for Specific Purposes (LSP) speaking assessment. When it comes to the extrapolation inference, though practical difficulties in collecting speaking performance data in the TLU domains are significant indeed, new research methods and perspectives, as exemplified by the corpus-based register analysis approach taken by LaFlair and Staples (2017) , could be attempted in the future to enable meaningful comparisons between spoken performance on the test and speaking ability in TLU domains. In addition, the judgments of linguistic layperson may also be employed as a viable external criterion (e.g., Sato and McNamara, 2018 ). The utilization inference is yet another area that language assessment researchers might consider exploring in the future. Commenting on the rise of computer-assisted language assessment, Chapelle (2008 , p. 127) argued that “test takers have needed to reorient their test preparation practices to help them prepare for new test items.” As such, it is meaningful for language assessment researchers to explore the impact of computer-mediated speaking assessments and automated scoring systems on teaching and learning practices.

Next, though the topic of speaking constructs has attracted considerable research attention from the field, as evidenced by the analysis results of this study, it seems that we are still far from achieving a comprehensive and fine-grained understanding of speaking proficiency. The results of this study suggest that speaking assessment researchers tended to adopt a psycholinguistic approach, aiming to analyze the linguistic features of produced spoken discourse that distinguish test takers at different proficiency levels. However, given the dynamic and context-embedded nature of speaking, there is a pressing need for a sociocultural perspective to better disentangle the speaking constructs. Using pronunciation as an example, Fulcher (2015b) argued convincingly the inadequacy of a psycholinguistic approach in pronunciation assessment research; rather, a sociocultural approach, which aims to demystify rationales, linguistic or cultural, that underlie (dys)fluency, could significantly enrich our understanding of the construct. Such an approach should be attempted more productively in future studies. In addition, as the application of technology is becoming prevalent in speaking assessment practices ( Chapelle, 2008 ), it is essential to explore whether and to what extent technology mediation has altered the speaking constructs and the implications for score interpretation and use.

We also found that several topics were under-represented in the studies that we collected. Important areas that received relatively limited coverage in our dataset include: (a) classroom-based or learning-oriented speaking assessment; (b) diagnostic speaking assessment; and (c) speaking assessment for young language learners (YLLs). The bulk of the research in our collection targeted large-scale high-stakes speaking assessments. This is understandable, perhaps, because results on these assessments are often used to make important decisions which have significant ramifications for stakeholders. In comparison, scanty research attention has been dedicated to speaking assessments in classroom contexts. A recent study reported by May et al. (2018) aimed to develop a learning-oriented assessment tool for interactional competence, so that detailed feedback could be provided about learners' interactional skills in support of their learning. More research of such a nature is needed in the future to reinforce the interfaces between speaking assessment with teaching and learning practices. In the domain of L2 writing research, it has been shown that simply using analytic rating scales does not mean that useful diagnostic feedback can be provided to learners ( Knoch, 2009 ). Arguably, this also holds true for speaking assessment. In view of the value of diagnostic assessment ( Lee, 2015 ) and the call for more integration of learning and assessment (e.g., Alderson, 2005 ; Turner and Purpura, 2015 ), more research could be conducted to develop diagnostic speaking assessments so that effective feedback can be provided to promote L2 learners' speaking development. Finally, young language learners (YLLs) have specific needs and characteristics which have implications for how they should be assessed (e.g., McKay, 2006 ). This is particularly challenging with speaking assessment in terms of task design, implementation and score reporting. This topic, however, has rarely been explored by speaking assessment researchers and therefore warrants more future research.

In terms of research methods, we feel that speaking assessment researchers should consider exploring more the potentials of qualitative methods which are well-suited to investigating an array of research questions related to speaking assessment. Our analysis results indicate that despite the quite frequent use of traditional qualitative methods such as interviews and focus groups, new qualitative methods that are supported by technology (e.g., eye-tracking) have only recently been utilized by speaking assessment researchers. For example, a recent study by Lee and Winke (2018) demonstrated the use of eye-tracking in speaking assessment through examining test-takers' cognitive processes when responding to computer-based speaking assessment tasks. Eye-tracking is advantageous in the sense that as opposed to traditional qualitative methods such as introspective think-aloud protocols, it causes minimal interference of the test taking process. Our final comment concerns the use of mixed-methods design in speaking assessment research. Despite it being applied quite frequently in researching speaking assessment, it appears that only the sequential explanatory design (i.e., the use of qualitative research to explain quantitative findings) was usually employed. Speaking assessment researchers may consider other mixed methods design options (e.g., convergent parallel design or embedded mixed methods design, see Moeller et al., 2016 ) to investigate more complex research questions in speaking assessment.

We acknowledge a few limitations with this study. As mentioned previously, we targeted only two highly influential journals in the field of language assessment, namely, LT and LAQ while aware that numerous other journals in applied linguistics or educational evaluation also publish research on speaking and its assessment. As such, caution needs to be exercised when interpreting the relevant research findings that emerged from this study. Future studies could be undertaken to include more journals and other publication types (e.g., research reports, PhD dissertations) to depict a more representative picture of speaking assessment research. In addition, given the sheer volume of published research on speaking assessment available, our research findings can only be presented as indications of possible trends of the wider publishing context, as reflected in the specific articles we explored. Arguably, the findings might be more revealing if we zoomed in on a few key topics in speaking assessment (e.g., rater effects, speaking constructs), analyzed specific studies on these topics in detail, and compared their findings. Finally, it would be worthwhile to explore how the research on some key topics in speaking assessment has been evolving over time. Such analysis could have provided a valuable reference point to speaking assessment researchers and practitioners. Such a developmental trend perspective, however, was not incorporated in our analysis and could be attempted in future research.

Data Availability Statement

The datasets generated for this study are available on request to the corresponding author.

Author Contributions

JF designed the study, collected and coded the data, and drafted the article. XY collected and coded the data, and drafted this article together with JF.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor declared a past collaboration with one of the authors JF.

Acknowledgments

The preparation of this manuscript was supported by the National Planning Office for Philosophy and Social Sciences (NPOPSS) of the People's Republic of China under the project title Reform of English speaking assessment and its impact on the teaching of English speaking (19BYY234). We would like to thank Angela McKenna and three reviewers for their insightful and perspicacious comments on the previous draft of this article.

1. ^ LT and LAQ made their debut in 1984 and 2004, respectively.

AERA, APA, and NCME (2014). Standards for Educational and Psychological Testing . Washtington, DC: AERA.

Ahmadi, A., and Sadeghi, E. (2016). Assessing English language learners' oral performance: a comparison of monologue, interview, and group oral test. Lang. Assess. Q. 13, 341–358. doi: 10.1080/15434303.2016.1236797

CrossRef Full Text | Google Scholar

Alderson, J. C. (2005). Diagnosing Foreign Language Proficiency: The interface between Learning and Assessment . London: Bloomsbury.

Google Scholar

Alderson, J. C., Clapham, C., and Wall, D. (1995). Language Test Construction and Evaluation . Cambridge: Cambridge University Press.

Bachman, L. F. (1990). Fundamental Considerations in Language Testing . Oxford England; New York, NY: Oxford University Press.

Bachman, L. F., Davidson, F., Ryan, K., and Choi, I.-C. (1995). An Investigation into the Comparability of Two Tests of English as a Foreign Language , Vol. 1 (Cambridge: Cambridge University Press).

Bachman, L. F., and Palmer, A. S. (1996). Language Assessment in Practice: Designing and Developing Useful Language Tests . Oxford: Oxford University Press.

Bonk, W. J., and Ockey, G. J. (2003). A many-facet rasch analysis of the second language group oral discussion task. Lang. Test. 20, 89–110. doi: 10.1191/0265532203lt245oa

Bosker, H. R., Pinget, A.-F., Quené, H., Sanders, T., and De Jong, N. H. (2013). What makes speech sound fluent? The contributions of pauses, speed and repairs. Lang. Test. 30, 159–175. doi: 10.1177/0265532212455394

Brown, A. (2003). Interviewer variation and the co-construction of speaking proficiency. Lang. Test. 20, 1–25. doi: 10.1191/0265532203lt242oa

Carter, R., and McCarthy, M. (2017). Spoken grammar: where are we and where are we going? Appl. Linguistics 38, 1–20. doi: 10.1093/applin/amu080

Chapelle, C. A. (2008). Utilizing Technology in Language Assessment Encyclopedia of Language and Education, Vol. 7 (New York, NY: Springer), 123–134.

Chapelle, C. A., Cotos, E., and Lee, J. (2015). Validity arguments for diagnostic assessment using automated writing evaluation. Lang. Test. 32, 385–405. doi: 10.1177/0265532214565386

Chapelle, C. A., Enright, M. K., and Jamieson, J. (2010). Does an argument-based approach to validity make a difference? Educ. Meas. Iss. Pract. 29, 3–13. doi: 10.1111/j.1745-3992.2009.00165.x

Chapelle, C. A., Enright, M. K., and Jamieson, J. M. (2008). Building a Validity Argument for the Test of English as a Foreign Language . New York, NY; London: Routledge; Taylor & Francis Group.

Cooper, H., Hedges, L. V., and Valentine, J. C. (2019). The Handbook of Research Synthesis and Meta-Analysis. New York, NY:Russell Sage Foundation.

Eckes, T. (2005). Examining rater effects in TestDaF writing and speaking performance assessments: a many-facet Rasch analysis. Lang. Assess. Q: Int. J. 2, 197–221. doi: 10.1207/s15434311laq0203_2

Eckes, T. (2011). Introduction to Many-Facet Rasch Measurement . Frankfurt: Peter Lang.

Ellis, R. (2015). Introduction: complementarity in research syntheses. Appl. Linguistics 36, 285–289. doi: 10.1093/applin/amv015

Fan, J., and Knoch, U. (2019). Fairness in language assessment: what can the Rasch model offer? Pap. Lang. Test. Assess. 8, 117–142. Available online at: http://www.altaanz.org/uploads/5/9/0/8/5908292/8_2_s5_fan_and_knoch.pdf

Fulcher, G. (2000). The ‘communicative' legacy in language testing. System 28, 483–497. doi: 10.1016/S0346-251X(00)00033-6

Fulcher, G. (2015a). Assessing second language speaking. Lang. teaching 48, 198–216. doi: 10.1017/S0261444814000391

Fulcher, G. (2015b). Re-Examining Language Testing: a Philosophical and Social Inquiry . New York, NY: Routledge.

Fulcher, G., Davidson, F., and Kemp, J. (2011). Effective rating scale development for speaking tests: performance decision trees. Lang. Test. 28, 5–29. doi: 10.1177/0265532209359514

Galaczi, E., and Taylor, L. (2018). Interactional competence: conceptualisations, operationalisations, and outstanding questions. Lang. Assess. Q. 15, 219–236. doi: 10.1080/15434303.2018.1453816

Galaczi, E. D. (2008). Peer-peer interaction in a speaking test: the case of the First Certificate in English examination . Lang. Assess. Q. 5, 89–119. doi: 10.1080/15434300801934702

Gan, Z. (2012). Complexity measures, task type, and analytic evaluations of speaking proficiency in a school-based assessment context. Lang. Assess. Q. 9, 133–151. doi: 10.1080/15434303.2010.516041

Ginther, A. (2013). “Assessment of speaking,” in The Encyclopedia of Applied Linguistics , ed C. A. Chapelle (New York, NY: Blackwell Publishing Ltd.), 1–7.

Hirai, A., and Koizumi, R. (2013). Validation of empirically derived rating scales for a story retelling speaking test. Lang. Assess. Q. 10, 398–422. doi: 10.1080/15434303.2013.824973

Isaacs, T. (2014). “Assessing pronunciation,” in The Companion to Language Assessment , Vol. 1, ed A. J. Kunnan (New York, NY: John Wiley & Sons), 140–155.

Isaacs, T. (2016). “Assessing speaking,” in Handbook of Second Language Assessment, Vol. 12 , eds D. Tsagari and J. Banerjee (Boston, MA; Berlin, Germany: De Gruyter), 131–146.

Isaacs, T., and Thomson, R. I. (2013). Rater experience, rating scale length, and judgments of L2 pronunciation: revisiting research conventions. Lang. Assess. Q. 10, 135–159. doi: 10.1080/15434303.2013.769545

Iwashita, N. (2006). Syntactic complexity measures and their relation to oral proficiency in Japanese as a foreign language. Lang. Assess. Q. Int. J. 3, 151–169. doi: 10.1207/s15434311laq0302_4

Jang, E. E., Wagner, M., and Park, G. (2014). Mixed methods research in language testing and assessment. Annu. Rev. Appl. Linguistics 34, 123–153. doi: 10.1017/S0267190514000063

Kane, M. T. (2006). “Validation,” in Educational Measurement , ed R. L. Brennan (Westport, CT: American Council on Education), 17–64.

Kim, H. J. (2015). A qualitative analysis of rater behavior on an L2 speaking assessment. Lang. Assess. Q. 12, 239–261. doi: 10.1080/15434303.2015.1049353

Kim, Y.-H. (2009). An investigation into native and non-native teachers' judgments of oral english performance: a mixed methods approach. Lang. Test. 26, 187–217. doi: 10.1177/0265532208101010

Knoch, U. (2009). Diagnostic assessment of writing: a comparison of two rating scales. Lang. Test. 26, 275–304. doi: 10.1177/0265532208101008

Knoch, U., and Chapelle, C. A. (2018). Validation of rating processes within an argument-based framework. Lang. Test. 35, 477–499. doi: 10.1177/0265532217710049

Kramsch, C. (1986). From language proficiency to interactional competence. Mod. Lang. J. 70, 366–372. doi: 10.1111/j.1540-4781.1986.tb05291.x

LaFlair, G. T., and Staples, S. (2017). Using corpus linguistics to examine the extrapolation inference in the validity argument for a high-stakes speaking assessment. Lang. Test. 34, 451–475. doi: 10.1177/0265532217713951

Lam, D. M. (2018). What counts as “responding”? Contingency on previous speaker contribution as a feature of interactional competence. Lang. Test. 35, 377–401. doi: 10.1177/0265532218758126

Lee, S., and Winke, P. (2018). Young learners' response processes when taking computerized tasks for speaking assessment. Lang. Test. 35, 239–269. doi: 10.1177/0265532217704009

Lee, Y.-W. (2006). Dependability of scores for a new ESL speaking assessment consisting of integrated and independent tasks. Lang. Test. 23, 131–166. doi: 10.1191/0265532206lt325oa

Lee, Y.-W. (2015). Diagnosing diagnostic language assessment. Lang. Test. 32, 299–316. doi: 10.1177/0265532214565387

Luoma, S. (2004). Assessing Speaking . Cambridge: Cambridge University Press.

May, L. (2011). Interactional competence in a paired speaking test: features salient to raters. Lang. Assess. Q. 8, 127–145. doi: 10.1080/15434303.2011.565845

May, L., Nakatsuhara, F., Lam, D. M., and Galaczi, E. (2018). “Learning-oriented assessment feedback for interactional competence: developing a checklist to support teachers and learners,” Paper presented at the Language Testing Research Colloquium (Auckland).

McKay, P. (2006). Assessing Young Language Learners . Cambridge: Cambridge University Press.

McNamara, T. (1996). Measuring Second Language Proficiency . London: Longman.

McNamara, T., and Knoch, U. (2012). The Rasch wars: the emergence of Rasch measurement in language testing. Lang. Test. 29, 553–574. doi: 10.1177/0265532211430367

McNamara, T., Knoch, U., and Fan, J. (2019). Fairness, Justice and Langauge Assessment . Oxford: Oxford University Press.

McNamara, T. F. (1997). ‘Interaction' in second language performance assessment: whose performance? App. Linguistics 18, 446–466. doi: 10.1093/applin/18.4.446

McNamara, T. F., and Lumley, T. (1997). The effect of interlocutor and assessment mode variables in overseas assessments of speaking skills in occupational I settings. Lang. Test. 14, 140–156. doi: 10.1177/026553229701400202

Messick, S. (1989). “Validity,” in Educational Measurement , 3rd Edn, ed R. L. Linn (New York, NY: McMillan; American Council on Education), 13–103.

Miles, M. B., Huberman, A. M., and Saldaña, J. (2014). Qualitative Data Analysis: A Methods Sourcebook, 3rd Edn. Thousand Oaks, CA: Sage.

Moeller, A. K., Creswell, J. W., and Saville, N. (2016). Second Language Assessment and Mixed Methods Research . Cambridge: Cambridge University Press.

Nakatsuhara, F., Inoue, C., Berry, V., and Galaczi, E. (2017). Exploring the use of video-conferencing technology in the assessment of spoken language: a mixed-methods study. Lang. Assess. Q. 14, 1–18. doi: 10.1080/15434303.2016.1263637

Norris, J. M., and Ortega, L. (2006). Synthesizing Research on Language Learning and Teaching , Vol. 13, Amsterdam; Philadelphia, PA: John Benjamins Publishing.

Ockey, G. J. (2009). The effects of group members' personalities on a test taker's L2 group oral discussion test scores. Lang. Test. 26, 161–186. doi: 10.1177/0265532208101005

Ockey, G. J., Gu, L., and Keehner, M. (2017). Web-based virtual environments for facilitating assessment of L2 oral communication ability. Lang. Assess. Q. 14, 346–359. doi: 10.1080/15434303.2017.1400036

O'Sullivan, B. (2014). “Assessing speaking,” in The Companion to Language Assessment , Vol. 1, ed A. J. Kunnan (Chichester: Wiley and Sons Inc), 156–171.

Pill, J., and McNamara, T. (2016). How much is enough? Involving occupational experts in setting standards on a specific-purpose language test for health professionals. Lang. Test. 33, 217–234. doi: 10.1177/0265532215607402

Qian, D. D. (2009). Comparing direct and semi-direct modes for speaking assessment: affective effects on test takers. Lang. Assess. Q. 6, 113–125. doi: 10.1080/15434300902800059

QSR (2018). NVivo Qualitative Data Analysis Software . Melbourne, VIC: QSR International Pty Ltd.

Richards, L. (1999). Using NVivo in Qualitative Research . Thousand Oaks, CA: Sage.

Roever, C., and Kasper, G. (2018). Speaking in turns and sequences: interactional competence as a target construct in testing speaking. Lang. Test. 35, 331–355. doi: 10.1177/0265532218758128

Sato, T., and McNamara, T. (2018). What counts in second language oral communication ability? The perspective of linguistic laypersons. Appl. Linguist. 40, 894–916. doi: 10.1093/applin/amy032

Sawaki, Y. (2007). Construct validation of analytic rating scale in speaking assessment: reporting a score profile and a composite. Lang. Test. 24, 355–390. doi: 10.1177/0265532207077205

Scott, M. L. (1986). Student affective reactions to oral language tests. Lang. Test. 3, 99–118. doi: 10.1177/026553228600300105

Turner, C., and Purpura, J. (2015). “Learning-oriented assessment in the classroom,” in Handbook of Second Language Assessment , eds D. Tsagari and J. Banerjee (Berlin; Boston, MA: DeGruyter Mouton), 255–273.

Turner, C. E. (2013). “Classroom assessment,” in The Routledge Handbook of Language Testing , eds G. Fulcher and F. Davidson (London; New York, NY: Routledge), 79–92.

Wei, J., and Llosa, L. (2015). Investigating differences between American and Indian raters in assessing TOEFL iBT speaking tasks. Lang. Assess. Q. 12, 283–304. doi: 10.1080/15434303.2015.1037446

Weir, C. J. (2005). Language Testing and Validation: An Evidence-Based Approach . Basingstoke: Palgrave Macmillan.

Wigglesworth, G., and Elder, C. (2010). An investigation of the effectiveness and validity of planning time in speaking test tasks. Lang. Assess. Q. 7, 1–24. doi: 10.1080/15434300903031779

Winke, P. (2012). “Rating oral language,” in The Encyclopedia of Applied Linguistics , ed C. A. Chapelle (New York, NY: Blackwell Publishing Ltd.).

Winke, P., Gass, S., and Myford, C. (2013). Raters' L2 background as a potential source of bias in rating oral performance. Lang. Test. 30, 231–252. doi: 10.1177/0265532212456968

Xi, X. (2007). Evaluating analytic scoring for the TOEFL ® Academic Speaking Test (TAST) for operational use. Lang. Test. 24, 251–286. doi: 10.1177/0265532207076365

Yan, X. (2014). An examination of rater performance on a local oral Englissh proficiency test: a mixed-methods approach. Lang. Test. 31, 501–527. doi: 10.1177/0265532214536171

Yu, G., He, L., Rea-Dickins, P., Kiely, R., Lu, Y., Zhang, J., et al. (2017). Preparing for the speaking tasks of the TOEFL iBT ® test: an investigation of the journeys of Chinese test takers. ETS Res. Rep. Ser. 2017, 1–59. doi: 10.1002/ets2.12145

Zhang, Y., and Elder, C. (2011). Judgments of oral proficiency by non-native and native English speaking teacher raters: competing or complementary constructs? Lang. Test. 28, 31–50. doi: 10.1177/0265532209360671

Keywords: speaking assessment, speaking proficiency, argument-based validation framework, research methods, narrative review

Citation: Fan J and Yan X (2020) Assessing Speaking Proficiency: A Narrative Review of Speaking Assessment Research Within the Argument-Based Validation Framework. Front. Psychol. 11:330. doi: 10.3389/fpsyg.2020.00330

Received: 20 November 2019; Accepted: 11 February 2020; Published: 27 February 2020.

Reviewed by:

Copyright © 2020 Fan and Yan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Jason Fan, jinsong.fan@unimelb.edu.au ; Xun Yan, xunyan@illinois.edu

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Adv Pract Oncol
• v.9(5); Jul-Aug 2018

Presenting With Confidence

Wendy h. vogel.

1 Wellmont Cancer Institute, Kingsport, Tennessee;

Pamela Hallquist Viale

2 University of California, San Francisco, San Francisco, California

Often, advanced practitioners must give clinical presentations. Public speaking, which is a major fear for most individuals, is a developed skill. Giving an oral presentation is a good way to demonstrate work, knowledge base, and expertise. Giving an effective presentation can help obtain recognition of skills and proficiency as an advanced practitioner or expert in the field. This paper will highlight skills and techniques that can help to improve presentation style and the ability to connect with an audience.

As an advanced practitioner, it is likely that you will be asked to deliver a lecture at some point in your career. Medical presentations can range from casual in-services to professional lectures given to audiences of thousands. Since public speaking is listed as one of the top fears of individuals living in the United States, it pays to develop skills as a speaker or presenter.

Giving an oral presentation is essential to demonstrating your work, knowledge base, and expertise. Giving an effective presentation can help you obtain recognition and acknowledgement of your skills and proficiency as an advanced practitioner or expert in the field. However, many presenters lack the skills to deliver a dynamic and persuasive lecture. Inadequate speaking skills can be detrimental to your ability to deliver an important message, or worse yet, bore your audience. This article will highlight skills and techniques that can help to improve your presentation style and ability to connect with your audience.

FEAR OF PUBLIC SPEAKING

If you are afraid of public speaking, you are not alone. Marinho, de Medeiros, Gama, and Teixeira ( 2016 ) studied college students to determine the prevalence of fear of public speaking. In a group of 1,135 undergraduate students (aged 17–58), over half of those surveyed (n = 63.9%) reported a fear of public speaking. Almost the entire group surveyed (89.3%) wanted classes to improve public speaking. Specific traits associated with a fear of speaking were reported as female gender, infrequent experience, and perception of poor voice quality.

Giving a bad presentation can alienate your audience from your lecture and the message you are trying to deliver. Table 1 lists ways to give a bad presentation. But, let us assume you do not want to give a bad presentation at all. In fact, you have an important message to share with your audience and you have been invited to give an hour-long lecture on the subject. How can you deliver that message in an effective and engaging manner?

Tips for Giving a Bad Presentation

HIGH-LEVEL TIPS

The first tip is to know your subject and know it well. In fact, should your audio-visual equipment malfunction (and if you speak often enough, this is likely to happen), you should have your presentation memorized. However, it is a good idea to make a hard copy of your slides and use them in case of equipment failure. Your audience might not be able to see a graph in detail, but you’ll be able to speak to a study and deliver the results without panicking about your lost slide deck or incompatible presentation equipment.

The second tip is to know your audience. If you are speaking to a group of nurses on a unit, your speaking style and delivery message will be more casual than when you speak to a room of 500 people. Nonetheless, you need to know who you are talking to and what they expect from your lecture. Table 2 lists some information you will want to know about your audience. Researching and knowing your audience will make your message more pertinent and personal.

What to Know About Your Audience

Understanding who your audience is will enable you to engage your audience. Look excited and enthusiastic. If you are motivated about your topic, then they will be too. Show your interest in your subject and your excitement about sharing the data with your audience.

Another tip is to develop your stage presence. Actors rehearse their roles until they can do it in their sleep, creating their best and most polished dramatic performances. You aren’t in a Broadway musical, but you need to have a stage presence. Recording your lecture and then examining ways to improve your delivery is a great way to develop your speaking skills. Utilize who you are and capitalize on that. Practice in front of a friend or mentor for feedback on your delivery

Your audience will develop an impression of you within the first 15 seconds. Develop an impactful opening to start off right. Table 3 gives some examples of impactful openings. For example, if you wanted to demonstrate the effect that tanning booths have had on the incidence of melanoma in young women, you could open with a photo of a tanning booth, followed by the daunting statistics in melanoma and an example of a case of melanoma. This slide becomes the "hook" that captures your audience’s interest.

Examples of Impactful Openings

When giving a medical presentation, advanced practitioners have a wonderful chance to share a patient story or vignette that will demonstrate the medical problem and its impact on practice ( Moffett, Berezowski, Spencer, & Lanning, 2014 ). You can do this easily by showing a patient radiological study or lab values, or a picture of a particularly challenging side effect. The net result is that your audience will be intrigued and relate to your story, especially if they take care of that patient population. Tell the story of the patient and describe the significance of the side effect or disease state. Clinical presentations often benefit from case studies that your audience may recognize from their own practices. Some of the most successful presentations use case studies followed by examples of right or wrong approaches to a patient problem, asking the audience to decide best practice and thereby engaging the audience fully. Tell your audience why this topic is important and why they need to know about it ( Moffett et al., 2014 ). Then, share the data supporting the importance of your story and how your audience can use the information to affect or change practice. You want to capture the attention of your audience at the very beginning of your presentation and then hold it. Humor may also be used for openings, but care must be taken with this and should be directed at yourself and not anyone else. Keep the attention of the audience by developing your delivery skills. Lastly, and perhaps the most important advice, is to "practice, practice, practice."

DEVELOPING SLIDES

Most medical speakers use PowerPoint to illustrate their talk and data. Using your slides effectively can make an important difference in your presentation and how your audience will respond. Develop your presentation and topic first, then create your slides. The 5/5/5 rule calls for no more than five words per line of text, five lines of text per slide, or five text or data-heavy slides in a row ( LearnFree.org, 2017 ). See Table 4 for tips for using PowerPoint.

PowerPoint Tips

Adding images to your slides can create visual interest. Pictures of patients with side effects or complications can immediately show the audience what you are trying to communicate. As with data slides, appropriate referencing of images must be added to each of your slides. If you are using clip art to add interest or humor to your presentation, be mindful of possible distractions to your main message. Use these kinds of imagery sparingly.

Using slides during your presentation can enhance the message you are giving, but it is vital that you use the slide and not let it use you. Know your slides well enough that you do not have to read them. The title of the slide should give the key message of that slide. You do not have to tell your audience everything on the slide; instead, give them an overview of what they are looking at. Never read a slide to an audience. Do not present to the slide; present to your audience.

KEEPING THE AUDIENCE’S INTEREST

If your presentation is longer than 20 minutes, you may have a "mid-talk slump." This is a great time to check in with your audience: Do they understand your message thus far? Pause for a moment and engage your audience with a question or anecdote, or perhaps a patient story. Ask your audience if they have something to share regarding the topic. Change the pace and change the inflection of your voice.

Taking questions from your audience can be daunting. Table 5 gives some tips on how to answer questions. Determining when to take questions will depend upon your audience size and makeup, and the setting of your presentation. The most important tip is to listen carefully to the question and be honest if you do not know the answer.

Handling Questions From Your Audience

Your delivery skills can determine how the audience perceives you and your message. Eye contact, voice, pace, inflection, gestures, and posture are all important aspects of your delivery. Eye contact establishes rapport and a feeling of being genuine. Although you shouldn’t stare someone down, making eye contact while making a statement, then moving to your next audience member and giving another statement fosters engagement. Scanning, which is running your eyes over the audience and not focusing on any one person, should be avoided.

Your voice should be loud and animated. Generally, however loud you think you should be, be louder. Convey your enthusiasm, and vary your pace and inflection.

Gestures can enhance or take away from your talk. Be natural with an open-body approach. Keep your hands at your sides if you’re not using them. Avoid pointing; instead, use open-handed gestures. Your posture should be good, with your shoulders back and weight equally balanced on both feet. When you move, move with purpose; do not sway, rock, or pace ( Butterfield, 2015 ).

It is very normal to feel anxious or nervous. But let that feeling work for you, not against you. When you are faced with a challenging situation, cortisol and adrenaline are released, causing dry mouth, difficulty getting words out, shallow breaths, tremors, sweating, and nervous behaviors like laughter or fidgeting. To combat this, take some deep breaths, which reduces adrenaline output. Slow down and look around. Take a moment, take a sip of water, and smile. Look confident even if you do not feel it. Utilize every resource you can find to further your skills (see Table 6 for further reading).

Resources for Presenters

Advanced practitioners have many opportunities to give medical presentations, both as part of their job and as a way to advance in their professional practice. The tools provided in this article can help you develop a presentation that will be meaningful and impactful to your audience. It is a great feeling when audience members come to you after your presentation to share with you how much they enjoyed and learned from your talk. With practice, your presentations can make a difference. And remember—your audience wants you to succeed.

The authors have no conflicts of interest to disclose.

10 Tips for Improving Your Public Speaking Skills

Few are immune to the fear of public speaking. Marjorie North offers 10 tips for speakers to calm the nerves and deliverable memorable orations.

Marjorie North

Snakes? Fine. Flying? No problem. Public speaking? Yikes! Just thinking about public speaking — routinely described as one of the greatest (and most common) fears — can make your palms sweat. But there are many ways to tackle this anxiety and learn to deliver a memorable speech.

In part one of this series,  Mastering the Basics of Communication , I shared strategies to improve how you communicate. In part two, How to Communicate More Effectively in the Workplace , I examined how to apply these techniques as you interact with colleagues and supervisors in the workplace. For the third and final part of this series, I’m providing you with public speaking tips that will help reduce your anxiety, dispel myths, and improve your performance.

Here Are My 10 Tips for Public Speaking:

1. nervousness is normal. practice and prepare.

All people feel some physiological reactions like pounding hearts and trembling hands. Do not associate these feelings with the sense that you will perform poorly or make a fool of yourself. Some nerves are good. The adrenaline rush that makes you sweat also makes you more alert and ready to give your best performance.

The best way to overcome anxiety is to prepare, prepare, and prepare some more. Take the time to go over your notes several times. Once you have become comfortable with the material, practice — a lot. Videotape yourself, or get a friend to critique your performance.

Communication Strategies: Presenting with Impact

Search all Communication programs.

2. Know Your Audience. Your Speech Is About Them, Not You.

Before you begin to craft your message, consider who the message is intended for. Learn as much about your listeners as you can. This will help you determine your choice of words, level of information, organization pattern, and motivational statement.

3. Organize Your Material in the Most Effective Manner to Attain Your Purpose.

Create the framework for your speech. Write down the topic, general purpose, specific purpose, central idea, and main points. Make sure to grab the audience’s attention in the first 30 seconds.

4. Watch for Feedback and Adapt to It.

Keep the focus on the audience. Gauge their reactions, adjust your message, and stay flexible. Delivering a canned speech will guarantee that you lose the attention of or confuse even the most devoted listeners.

5. Let Your Personality Come Through.

Be yourself, don’t become a talking head — in any type of communication. You will establish better credibility if your personality shines through, and your audience will trust what you have to say if they can see you as a real person.

6. Use Humor, Tell Stories, and Use Effective Language.

Inject a funny anecdote in your presentation, and you will certainly grab your audience’s attention. Audiences generally like a personal touch in a speech. A story can provide that.

7. Don’t Read Unless You Have to. Work from an Outline.

Reading from a script or slide fractures the interpersonal connection. By maintaining eye contact with the audience, you keep the focus on yourself and your message. A brief outline can serve to jog your memory and keep you on task.

8. Use Your Voice and Hands Effectively. Omit Nervous Gestures.

Nonverbal communication carries most of the message. Good delivery does not call attention to itself, but instead conveys the speaker’s ideas clearly and without distraction.

9. Grab Attention at the Beginning, and Close with a Dynamic End.

Do you enjoy hearing a speech start with “Today I’m going to talk to you about X”? Most people don’t. Instead, use a startling statistic, an interesting anecdote, or concise quotation. Conclude your speech with a summary and a strong statement that your audience is sure to remember.

10. Use Audiovisual Aids Wisely.

Too many can break the direct connection to the audience, so use them sparingly. They should enhance or clarify your content, or capture and maintain your audience’s attention.

Practice Does Not Make Perfect

Good communication is never perfect, and nobody expects you to be perfect. However, putting in the requisite time to prepare will help you deliver a better speech. You may not be able to shake your nerves entirely, but you can learn to minimize them.

Find related Communication programs.

Browse all Professional & Executive Development programs.

About the Author

North is a consultant for political candidates, physicians, and lawyers, and runs a private practice specializing in public speaking, and executive communication skills. Previously, she was the clinical director in the department of speech and language pathology and audiology at Northeastern University.

Why Gender Equity in the Workplace is Good for Business

Research indicates a correlation between gender equity and organizational success, yet it also points to obstacles for women in leadership.

Harvard Division of Continuing Education

The Division of Continuing Education (DCE) at Harvard University is dedicated to bringing rigorous academics and innovative teaching capabilities to those seeking to improve their lives through education. We make Harvard education accessible to lifelong learners from high school to retirement.

About Stanford GSB

• The Leadership
• Dean’s Updates
• School News & History
• Commencement
• Business, Government & Society
• Centers & Institutes
• Center for Entrepreneurial Studies
• Center for Social Innovation
• Stanford Seed

About the Experience

• Learning at Stanford GSB
• Experiential Learning
• Guest Speakers
• Entrepreneurship
• Social Innovation
• Communication
• Life at Stanford GSB
• Collaborative Environment
• Activities & Organizations
• Student Services
• Housing Options
• International Students

Full-Time Degree Programs

• Why Stanford MBA
• Academic Experience
• Financial Aid
• Why Stanford MSx
• Research Fellows Program
• See All Programs

Non-Degree & Certificate Programs

• Executive Education
• Stanford Executive Program
• Programs for Organizations
• The Difference
• Online Programs
• Stanford LEAD
• Seed Transformation Program
• Aspire Program
• Seed Spark Program
• Faculty Profiles
• Academic Areas
• Awards & Honors
• Conferences

Faculty Research

• Publications
• Working Papers
• Case Studies

Research Hub

• Research Labs & Initiatives
• Business Library
• Data, Analytics & Research Computing
• Behavioral Lab

Research Labs

• Cities, Housing & Society Lab
• Golub Capital Social Impact Lab

Research Initiatives

• Corporate Governance Research Initiative
• Corporations and Society Initiative
• Policy and Innovation Initiative
• Rapid Decarbonization Initiative
• Stanford Latino Entrepreneurship Initiative
• Value Chain Innovation Initiative
• Venture Capital Initiative
• Career & Success
• Climate & Sustainability
• Corporate Governance
• Culture & Society
• Finance & Investing
• Government & Politics
• Leadership & Management
• Markets & Trade
• Operations & Logistics
• Opportunity & Access
• Organizational Behavior
• Political Economy
• Social Impact
• Technology & AI
• Opinion & Analysis
• Email Newsletter

Welcome, Alumni

• Communities
• Digital Communities & Tools
• Regional Chapters
• Women’s Programs
• Identity Chapters
• Find Your Reunion
• Career Resources
• Job Search Resources
• Career & Life Transitions
• Programs & Services
• Career Video Library
• Alumni Education
• Research Resources
• Volunteering
• Alumni News
• Class Notes
• Alumni Voices
• Contact Alumni Relations
• Upcoming Events

Admission Events & Information Sessions

• MBA Program
• MSx Program
• PhD Program
• Alumni Events
• All Other Events

A Big Data Approach to Public Speaking

Key takeaways from analyzing 100,000 presentations.

April 04, 2016

How do you better resonate with your audience? A Stanford GSB lecturer uses big data to explain what works. | Reuters/Jonathan Ernst

Students in my strategic communication class often ask how they can become more engaging, competent communicators. This question is in no way new — rhetoricians dating back to the ancient Greeks have explored this issue. However, unlike Cicero and Aristotle, we now have big data tools and machine-learning techniques to examine the core characteristics of effective communicators.

One person leveraging this technology (and one of my most popular guest lecturers) is Noah Zandan , founder and CEO of Quantified Communications , which offers one of the first analytics platforms to measure, evaluate, and improve corporate executives’ communication skills.

Zandan’s team of data scientists analyzed more than 100,000 presentations from corporate executives, politicians, and keynote speakers. They examined behaviors ranging from word choices and vocal cues to facial expressions and gesture frequency. They then used this data to rate and rank important communication variables such as persuasiveness, confidence, warmth, and clarity.

Zandan grounds his team’s work in a communication scheme created by psychologist Albert Mehrabian. They expand upon Mehrabian’s original “Three V’s” — the verbal, vocal, and visual choices that a communicator makes — by adding a fourth V: the vital elements of communication.

Here’s what his team has learned through studying the best communicators, combined with concepts I cover in class:

The actual words you use, whether spoken or written, matter. Zandan and his team found that the language used in corporate earnings calls affects up to 2.5% of stock price movement. Based on data from the most successful communicators, here are three things to keep in mind.

First, word choice should be appropriate for your audience and conform to the context (e.g., formality). Relying on jargon is likely to confuse your audience. The best approach is always to take the time to define terms and technologies that some in your audience might not know. You would also be well-served to have someone review your content specifically to confirm that your word choices are appropriate.

Second, avoid hedging language. Qualifying phrases such as “kind of” and hesitant language like “I think” can be beneficial in interpersonal communication, where they invite contribution and adjust your status relative to the person with whom you are conversing. But in contexts like presenting in public, they can reduce your credibility. You will sound more confident when you remove qualifiers and say “I feel” or “I believe.” The best way to make yourself aware of how often you use hedging language is to have a trusted colleague alert you while giving a practice presentation. Once you’re aware, you will be better able to proactively eliminate this type of language.

Finally, speak clearly and concisely. Research suggests that succinct messages are more memorable. In fact, Zandan and his team found that effective communicators’ messages tend to be more concise than those from speakers who were rated as average or below average. Many presenters speak the way they write — that is, they use complex sentences with nested clauses and phrases. This works well in writing, but when you’re presenting, it’s hard for you to speak and challenging for your audience to follow. In writing, we don’t have to worry about pauses for breath. Nor do we need to worry about the audience understanding what we have written, as a reader can always reread a confusing passage. To be more concise, start by stripping away excess wording that might sound good when read silently but that adds limited value when spoken aloud. When you’re practicing, ask others to paraphrase your points to see if their wording can help you be more succinct.

Vocal elements include volume, rate, and cadence. The keys to vocal elements are variation and fluency. Think of your voice like a wind instrument. You can make it louder, softer, faster, or slower. We are wired to pay attention to these kinds of vocal change, which is why it is so hard to listen to a monotonous speaker. In fact, even just a 10% increase in vocal variety can have a highly significant impact on your audience’s attention to and retention of your message.

Less expressive speakers should vary their volume and rate by infusing their presentations with emotive words like “excited,” “valuable,” and “challenging,” and using variations in their voice to match the meaning of these words. If you’re speaking about a big opportunity, then say “big” in a big way. With practice, you will feel more comfortable with this type of vocal variety.

Disfluencies — all those “ums” and “uhs” — might be the most difficult vocal element to address. Not all disfluencies are distracting. “Ums” and “uhs” within sentences are not perceived as frequently, nor are they as bothersome, as those that occur between thoughts and phrases. Your audience often skips over midsentence disfluencies because they are more focused on your content than your verbal delivery. But as you move from one point to another, disfluencies stand out because your audience is no longer focused on what you are saying. In essence, you are violating your audience’s expectation of a silent pause by filling it.

To address these between-thought disfluencies, be sure to end your sentences, and especially your major points, on an exhalation. By ending your phrases on a complete exhalation, you necessarily start your next thought with an inhalation. It is nearly impossible to say “um” (or anything, for that matter) while inhaling. A useful way to practice this is to read out loud and notice your breathing patterns. In addition to eliminating between-thought disfluencies, your inhalation brings a pause with it. This unfilled pause has the added benefit of varying your rate.

Visual elements refer to what you do with your body. Zandan cites studies by educational researchers that suggest approximately 83% of human learning occurs visually. Your nonverbal behaviors such as stance, gestures, and eye contact are critical not only for conveying and reinforcing your messages, but they serve as the foundation of your audience’s assessments of your confidence. This is important because your audience equates your competence with their perceptions of your confidence.

Your stance is all about being big and balanced. Stand or sit so that your hips and shoulders are square (i.e., not leaning to one side) and keep your head straight, not tilted. Presenting from a balanced position not only helps you appear more confident, but it actually helps you feel more confident, too. When you make yourself big and balanced, you release neurochemicals that blunt anxiety-producing hormones.

Quote Even just a 10% increase in vocal variety can have a highly significant impact on your audience’s attention to and retention of your message Attribution Matt Abrahams

Gestures need to be broad and extended. When you’re gesturing, go beyond your shoulders rather than in front of your chest, which makes you look small and defensive. When you’re not gesturing, place your arms loosely at your sides or clasp your hands loosely right at your belly button level. Finally, remove any distracting items that you might futz or fiddle with, like jewelry, pens, and slide advancers.

Eye contact is all about connecting to your audience. In North American culture, audiences expect eye contact, and quickly feel ostracized when you fail to look out at them. While you need to spread your eye contact around so that you connect with your entire audience, you need not look at each member individually, especially if you are in front of a large crowd. A good strategy is to create quadrants and look in those various directions. Also, try to avoid repetitive patterns when you scan the room. Finally, as Zandan rightly advises his clients, if you are presenting remotely via video camera, imagine you’re speaking directly to people and look into the camera, not at your monitor or keyboard.

Vital elements capture a speaker’s true nature — it is what some refer to as authenticity. For authenticity, Zandan’s team has found that the top 10% of authentic speakers were considered to be 1.3 times more trustworthy and 1.3 times more persuasive than the average communicator. Authenticity is made up of the passion and warmth that people have when presenting.

Passion comes from exuding energy and enthusiasm. When you’re preparing and practicing your talk, be sure to reflect back on what excites you about your topic and how your audience will benefit. Reminding yourself of your motivation can help energize you (or reenergize you if it’s a presentation you give over and over again). Additionally, thinking about how you are helping your audience learn, grow, and achieve should ignite your spirits. This energy will manifest itself in how you relay your information. This doesn’t mean you have to be a cheerleader; you need to find a method for relaying your message that is authentic and meaningful for you.

Warmth can be thought of as operationalized empathy. It is a combination of understanding your audience’s needs and displaying that understanding through your words and actions. To be seen as warm, you should acknowledge your audience’s needs by verbally echoing them (e.g., “Like you, I once…”) and by telling stories that convey your understanding of their needs, such as the CEO who tells a story of the most difficult tech support call she had to deal with as she addresses her client services team. Further, maintain an engaged posture by leaning forward and moving toward people who ask questions.

Before your next speech, try out the Four V’s and the specific suggestions derived from big data and machine learning to see if they fit your needs. Only through reflection, practice, and openness to trying new things can you become an engaging, competent communicator.

Matt Abrahams is a Stanford GSB organizational behavior lecturer, author, and communications coach.

For media inquiries, visit the Newsroom .

Explore More

How to chat with bots: the secrets to getting the information you need from ai, you’re in charge: how to use ai as a powerful decision-making tool, is your business ready to jump into a.i. read this first., editor’s picks.

January 29, 2016 How to Manage Your Anxiety When Presenting Do you get nervous speaking in public? Learn how to mitigate your fear.

March 02, 2015 Matt Abrahams: Tips and Techniques for More Confident and Compelling Presentations A Stanford lecturer explains key ways you can better plan, practice, and present your next talk.

• Priorities for the GSB's Future
• See the Current DEI Report
• Supporting Data
• Research & Insights
• Share Your Thoughts
• Search Fund Primer
• Teaching & Curriculum
• Affiliated Faculty
• Faculty Advisors
• Louis W. Foster Resource Center
• Defining Social Innovation
• Impact Compass
• Global Health Innovation Insights
• Faculty Affiliates
• Student Awards & Certificates
• Changemakers
• Dean Jonathan Levin
• Dean Garth Saloner
• Dean Robert Joss
• Dean Michael Spence
• Dean Robert Jaedicke
• Dean Rene McPherson
• Dean Arjay Miller
• Dean Ernest Arbuckle
• Dean Jacob Hugh Jackson
• Dean Willard Hotchkiss
• Faculty in Memoriam
• Stanford GSB Firsts
• Certificate & Award Recipients
• Teaching Approach
• Analysis and Measurement of Impact
• The Corporate Entrepreneur: Startup in a Grown-Up Enterprise
• Data-Driven Impact
• Designing Experiments for Impact
• Digital Business Transformation
• The Founder’s Right Hand
• Marketing for Measurable Change
• Product Management
• Public Policy Lab: Financial Challenges Facing US Cities
• Public Policy Lab: Homelessness in California
• Lab Features
• Curricular Integration
• View From The Top
• Formation of New Ventures
• Managing Growing Enterprises
• Startup Garage
• Explore Beyond the Classroom
• Stanford Venture Studio
• Summer Program
• Workshops & Events
• The Five Lenses of Entrepreneurship
• Leadership Labs
• Executive Challenge
• Arbuckle Leadership Fellows Program
• Selection Process
• Training Schedule
• Time Commitment
• Learning Expectations
• Post-Training Opportunities
• Who Should Apply
• Introductory T-Groups
• Leadership for Society Program
• Certificate
• 2023 Awardees
• 2022 Awardees
• 2021 Awardees
• 2020 Awardees
• 2019 Awardees
• 2018 Awardees
• Social Management Immersion Fund
• Stanford Impact Founder Fellowships and Prizes
• Stanford Impact Leader Prizes
• Social Entrepreneurship
• Stanford GSB Impact Fund
• Economic Development
• Energy & Environment
• Stanford GSB Residences
• Environmental Leadership
• Stanford GSB Artwork
• A Closer Look
• California & the Bay Area
• Voices of Stanford GSB
• Business & Beneficial Technology
• Business & Sustainability
• Business & Free Markets
• Business, Government, and Society Forum
• Get Involved
• Second Year
• Global Experiences
• JD/MBA Joint Degree
• MA Education/MBA Joint Degree
• MD/MBA Dual Degree
• MPP/MBA Joint Degree
• MS Computer Science/MBA Joint Degree
• MS Electrical Engineering/MBA Joint Degree
• MS Environment and Resources (E-IPER)/MBA Joint Degree
• Academic Calendar
• Clubs & Activities
• LGBTQ+ Students
• Military Veterans
• Minorities & People of Color
• Partners & Families
• Students with Disabilities
• Student Support
• Residential Life
• Student Voices
• MBA Alumni Voices
• A Week in the Life
• Career Support
• Employment Outcomes
• Cost of Attendance
• Knight-Hennessy Scholars Program
• Yellow Ribbon Program
• BOLD Fellows Fund
• Application Process
• Loan Forgiveness
• Contact the Financial Aid Office
• Evaluation Criteria
• GMAT & GRE
• English Language Proficiency
• Personal Information, Activities & Awards
• Professional Experience
• Letters of Recommendation
• Optional Short Answer Questions
• Application Fee
• Reapplication
• Deferred Enrollment
• Joint & Dual Degrees
• Entering Class Profile
• Event Schedule
• Ambassadors
• New & Noteworthy
• Ask a Question
• See Why Stanford MSx
• Is MSx Right for You?
• MSx Stories
• Leadership Development
• Career Advancement
• Career Change
• How You Will Learn
• Admission Events
• Personal Information
• Information for Recommenders
• GMAT, GRE & EA
• English Proficiency Tests
• After You’re Admitted
• Daycare, Schools & Camps
• U.S. Citizens and Permanent Residents
• Requirements
• Requirements: Behavioral
• Requirements: Quantitative
• Requirements: Macro
• Requirements: Micro
• Annual Evaluations
• Field Examination
• Research Activities
• Research Papers
• Dissertation
• Oral Examination
• Current Students
• Education & CV
• International Applicants
• Statement of Purpose
• Reapplicants
• Application Fee Waiver
• Deadline & Decisions
• Job Market Candidates
• Academic Placements
• Stay in Touch
• Faculty Mentors
• Current Fellows
• Standard Track
• Fellowship & Benefits
• Group Enrollment
• Program Formats
• Developing a Program
• Diversity & Inclusion
• Strategic Transformation
• Program Experience
• Contact Client Services
• Campus Experience
• Live Online Experience
• Silicon Valley & Bay Area
• Digital Credentials
• Faculty Spotlights
• Participant Spotlights
• Eligibility
• International Participants
• Stanford Ignite
• Frequently Asked Questions
• Operations, Information & Technology
• Classical Liberalism
• The Eddie Lunch
• Accounting Summer Camp
• Videos, Code & Data
• California Econometrics Conference
• California Quantitative Marketing PhD Conference
• California School Conference
• China India Insights Conference
• Homo economicus, Evolving
• Political Economics (2023–24)
• Scaling Geologic Storage of CO2 (2023–24)
• A Resilient Pacific: Building Connections, Envisioning Solutions
• Adaptation and Innovation
• Changing Climate
• Civil Society
• Climate Impact Summit
• Climate Science
• Corporate Carbon Disclosures
• Earth’s Seafloor
• Environmental Justice
• Operations and Information Technology
• Organizations
• Sustainability Reporting and Control
• Taking the Pulse of the Planet
• Urban Infrastructure
• Watershed Restoration
• Junior Faculty Workshop on Financial Regulation and Banking
• Ken Singleton Celebration
• Marketing Camp
• Quantitative Marketing PhD Alumni Conference
• Presentations
• Theory and Inference in Accounting Research
• Stanford Closer Look Series
• Quick Guides
• Core Concepts
• Journal Articles
• Glossary of Terms
• Faculty & Staff
• Researchers & Students
• Research Approach
• Charitable Giving
• Financial Health
• Government Services
• Workers & Careers
• Short Course
• Adaptive & Iterative Experimentation
• Incentive Design
• Social Sciences & Behavioral Nudges
• Bandit Experiment Application
• Conferences & Events
• Reading Materials
• Energy Entrepreneurship
• Faculty & Affiliates
• SOLE Report
• Responsible Supply Chains
• Current Study Usage
• Pre-Registration Information
• Participate in a Study
• Founding Donors
• Location Information
• Participant Profile
• Network Membership
• Program Impact
• Collaborators
• Entrepreneur Profiles
• Company Spotlights
• Seed Transformation Network
• Responsibilities
• Current Coaches
• How to Apply
• Meet the Consultants
• Meet the Interns
• Intern Profiles
• Collaborate
• Research Library
• News & Insights
• Program Contacts
• Databases & Datasets
• Research Guides
• Consultations
• Research Workshops
• Career Research
• Research Data Services
• Course Reserves
• Course Research Guides
• Material Loan Periods
• Fines & Other Charges
• Document Delivery
• Interlibrary Loan
• Equipment Checkout
• Print & Scan
• MBA & MSx Students
• PhD Students
• Other Stanford Students
• Faculty Assistants
• Research Assistants
• Stanford GSB Alumni
• Telling Our Story
• Staff Directory
• Site Registration
• Alumni Directory
• Alumni Email
• Privacy Settings & My Profile
• Success Stories
• The Story of Circles
• Support Women’s Circles
• Stanford Women on Boards Initiative
• Alumnae Spotlights
• Insights & Research
• Industry & Professional
• Entrepreneurial Commitment Group
• Recent Alumni
• Half-Century Club
• Fall Reunions
• Spring Reunions
• MBA 25th Reunion
• Half-Century Club Reunion
• Faculty Lectures
• Ernest C. Arbuckle Award
• Alison Elliott Exceptional Achievement Award
• ENCORE Award
• Excellence in Leadership Award
• John W. Gardner Volunteer Leadership Award
• Robert K. Jaedicke Faculty Award
• Jack McDonald Military Service Appreciation Award
• Jerry I. Porras Latino Leadership Award
• Tapestry Award
• Student & Alumni Events
• Executive Recruiters
• Interviewing
• Land the Perfect Job with LinkedIn
• Negotiating
• Elevator Pitch
• Email Best Practices
• Resumes & Cover Letters
• Self-Assessment
• Whitney Birdwell Ball
• Margaret Brooks
• Bryn Panee Burkhart
• Margaret Chan
• Ricki Frankel
• Peter Gandolfo
• Cindy W. Greig
• Natalie Guillen
• Carly Janson
• Sloan Klein
• Sherri Appel Lassila
• Stuart Meyer
• Tanisha Parrish
• Virginia Roberson
• Philippe Taieb
• Michael Takagawa
• Terra Winston
• Johanna Wise
• Debbie Wolter
• Rebecca Zucker
• Complimentary Coaching
• Changing Careers
• Work-Life Integration
• Career Breaks
• Flexible Work
• Encore Careers
• Join a Board
• D&B Hoovers
• Data Axle (ReferenceUSA)
• EBSCO Business Source
• Global Newsstream
• Market Share Reporter
• ProQuest One Business
• Student Clubs
• Entrepreneurial Students
• Stanford GSB Trust
• Alumni Community
• How to Volunteer
• Springboard Sessions
• Consulting Projects
• 2020 – 2029
• 2010 – 2019
• 2000 – 2009
• 1990 – 1999
• 1980 – 1989
• 1970 – 1979
• 1960 – 1969
• 1950 – 1959
• 1940 – 1949
• Service Areas
• ACT History
• ACT Awards Celebration
• ACT Governance Structure
• Building Leadership for ACT
• Individual Leadership Positions
• Leadership Role Overview
• Purpose of the ACT Management Board
• Contact ACT
• Business & Nonprofit Communities
• Reunion Volunteers
• Ways to Give
• Fiscal Year Report
• Business School Fund Leadership Council
• Planned Giving Options
• Planned Giving Benefits
• Planned Gifts and Reunions
• Legacy Partners
• Giving News & Stories
• Giving Deadlines
• Development Staff
• Submit Class Notes
• Class Secretaries
• Board of Directors
• Health Care
• Sustainability
• Class Takeaways
• All Else Equal: Making Better Decisions
• If/Then: Business, Leadership, Society
• Grit & Growth
• Think Fast, Talk Smart
• Spring 2022
• Spring 2021
• Autumn 2020
• Summer 2020
• Winter 2020
• In the Media
• For Journalists
• DCI Fellows
• Other Auditors
• Academic Calendar & Deadlines
• Course Materials
• Entrepreneurial Resources
• Campus Drive Grove
• Campus Drive Lawn
• CEMEX Auditorium
• King Community Court
• Seawell Family Boardroom
• Stanford GSB Bowl
• Stanford Investors Common
• Town Square
• Vidalakis Courtyard
• Vidalakis Dining Hall
• Catering Services
• Policies & Guidelines
• Reservations
• Contact Faculty Recruiting
• Lecturer Positions
• Postdoctoral Positions
• Accommodations
• CMC-Managed Interviews
• Recruiter-Managed Interviews
• Virtual Interviews
• Campus & Virtual
• Search for Candidates
• Think Globally
• Recruiting Calendar
• Recruiting Policies
• Full-Time Employment
• Summer Employment
• Entrepreneurial Summer Program
• Global Management Immersion Experience
• Social-Purpose Summer Internships
• Process Overview
• Project Types
• Client Eligibility Criteria
• Client Screening
• ACT Leadership
• Social Innovation & Nonprofit Management Resources
• Develop Your Organization’s Talent
• Centers & Initiatives
• Student Fellowships

Enhancing Public Speaking Skills - An Evaluation of the Presentation Trainer in the Wild

• Conference paper
• Open Access
• First Online: 07 September 2016
• Cite this conference paper

You have full access to this open access conference paper

• Jan Schneider 16 ,
• Dirk Börner 16 ,
• Peter van Rosmalen 16 &
• Marcus Specht 16  

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9891))

Included in the following conference series:

• European Conference on Technology Enhanced Learning

15k Accesses

6 Citations

The increasing accessibility of sensors allows the study and development of multimodal learning tools that create opportunities for learners to practice while receiving feedback. One of the potential learning scenarios addressed by these learning applications is the development of public speaking skills. First applications and studies showed promising empirical results in laboratory conditions. In this article we present a study where we explored the use of a multimodal learning application called the Presentation Trainer, supporting learners with a real public speaking task in the classroom. The results of this study help to understand the challenges and implications of testing such a system in a real-world learning setting, and show the actual impact compared to the use in laboratory conditions.

You have full access to this open access chapter,  Download conference paper PDF

Similar content being viewed by others

Presentation Trainer: A Toolkit for Learning Non-verbal Public Speaking Skills

Stand Tall and Raise Your Voice! A Study on the Presentation Trainer

Presentation trainer: polishing your communication skills.

• Evaluation in the wild
• Sensor-based learning support
• Public speaking
• Multimodal learning application

1 Introduction

Experiencing a great presenter delivering a novel idea is an inspiring event. Therefore, at least for the last 2500 years humans have been studying the art of the oratory [ 1 ]. Currently the ability to present effectively is considered to be a core competence for educated professionals [ 2 – 5 ]. This relevance in learning how to communicate effectively is reinforced by the thought that ideas are the currency of the twenty first century [ 6 ]. Research on how to develop public speaking skills is a topic that has already been extensively studied. One of the conclusions to be drawn out of these studies is that practice and feedback are key aspects for the development of these skills [ 7 ]. Whereas it is possible to attend different courses and seminars on public speaking, opportunities to practice and receive feedback from tutors or peers under realistic conditions are limited.

Sensors have lately become increasingly popular [ 8 ], showing to be a technology with great potential to enhance learning, by providing users with feedback in scenarios where human feedback is not available or to give access to data sources to enhance learning [ 9 ]. This has led to the development and research of new sensory technologies designed to support users with the development of their public skills [ 10 – 13 ]. These technologies have not been widespread yet, and so far their impact has not been tested outside from controlled laboratory conditions. One of these technologies is the Presentation Trainer (PT), a multimodal tool designed to support the development of basic public speaking skills, by creating opportunities for learners to practice their presentations while receiving feedback [ 13 ]. This paper describes a field study where we took the PT outside of the laboratory and tested it in a classroom. The paper discusses the implications of using such a system in the wild, and identifies which of the findings in a lab setting [ 13 ] also hold in the real world.

2 Background Work

Educational interventions such as feedback are needed to develop public speaking skills [ 14 ]. Having a human tutor available to give feedback on these skills is neither always feasible nor affordable. Therefore, technological interventions designed to provide this feedback are desirable. Public speaking skills require from presenters a coherent use of their verbal and nonverbal channels. Timely measurement of these multimodal performances with an acceptable accuracy is challenging. However, in recent years driven by the rising availability of sensors, research on multimodal learning applications designed to support the development of public speaking skills has been undertaken.

During a presentation, the presenters communicate their messages using their voice together with their full body language, e.g., body posture, use of stage, eye contact, facial expressions, hand gestures, etc. Multimodal learning applications supporting the development of public speaking skills [ 10 – 16 ] generally use a depth sensor such as the Microsoft Kinect Footnote 1 in order to capture the body language of the user, and microphone devices to capture the user’s voice.

Studies on applications designed to support public speaking skills have been exploring effective strategies to provide feedback to users. In [ 11 ] feedback indicating whether the energy, body posture and speech rate is correct or not, is displayed on a Google Glass Footnote 2 . Another feedback strategy employed in [ 10 , 15 ] is the use of a virtual audience. Members of the virtual audience change postures and behaviors depending on the nonverbal communication of the user. Besides the display of the virtual audience the prototype in [ 10 ] also provides the user with direct visual indications regarding her own body posture. The applications in [ 12 , 16 ] provide the user with a dashboard interface that displays a mirrored image of the user together with modules indicating the use of nonverbal communication aspects such as use of gestures, voice, etc. In line with that, the feedback interface of the PT shows a mirror image of the user and displays at maximum one instruction to the user regarding her nonverbal communication at a given time (see Fig.  1 ). This instruction is communicated to the user through a visual and a haptic channel [ 13 ].

PT telling the user to correct the posture.

The impact of this type of applications on learners has also been studied, showing positive results in laboratory conditions. In the study of [ 10 ] the feedback of the system, regarding the closeness or openness of the learner’s body posture, helped learners to become more aware of their body posture. The impact of the PT’s feedback on learners has also been studied in controlled setups. The study in [ 13 ] showed, through objective measures made by the system, that after five practice sessions receiving feedback from the PT learners on average reduced 75 % of their nonverbal mistakes.

In this study we tested the PT in a classroom setting following an exploratory research approach [ 17 ], focusing on three main objectives:

Objective 1 : The first objective of this study is to explore the implications of investigating the use of a tool such as the PT in a regular learning scenario outside of a laboratory setup.

Objective 2: Studies on multimodal learning applications for public speaking have shown promising results in laboratory conditions according to quantified and timely machine measurements [ 10 , 13 ]. However, the purpose of a presentation is to transmit the desired message and provide the desired impact to a human audience, in contrast of improving a machine-based score. Studies showing evidence that an improved performance according to machine measurements is reflected in a better presentation according to a human audience are still missing. Therefore, the second objective of this study is to gain insights on how the improvements obtained by a learner using the PT to practice for a presentation relate to the impact that this trained presentation has on the audience. In other words, to what extent does an audience agree with the PT that a presentation improved.

Objective 3: A core competence for current professionals is having good public speaking skills [ 2 – 5 ]; therefore teaching these skills has become a common target for different courses. Feedback is a key aspect for learning and developing public speaking skills [ 7 ], therefore current courses in public speaking include well-established feedback practices to help learners with the development of these skills. The effectiveness of this feedback depends on various variables. One of these variables concerns the source where the feedback comes from. Feedback provided by a tutor in combination with feedback provided by peer students has proven to be more effective than feedback provided only by a human tutor [ 18 ]. The third objective of this study, researches the introduction of the PT to the already established practices for teaching public speaking skills, exploring whether its use and feedback contribute to the creation of more comprehensive learning scenarios for students.

4 Methodology

4.1 study context.

We conducted this study in the setting of a course in entrepreneurship for master students in a university. In this course students were divided in two teams, where each team is represented as an entrepreneurial business. During the course the teams have to develop and present their project. Thus, the students of the course receive some presentation training guidance. The teams have to give a presentation about their projects twice, at the middle and at the end of the course. The middle term presentations are recorded and in following sessions these recordings are used to give feedback to the students regarding their presentation skills, both by tutors and peers.

4.2 Study Procedure

This study was conducted some sessions after the students have already presented their project and received feedback. Nine participants, seven males and two females between the age of 24 and 28 years old took part in the study. A sketch of the study is shown in Fig.  2 . To prepare for the study, students got the homework to individually prepare a 60–120 s long pitch regarding their project. One week later the study was conducted during a two-hour session slot.

Study procedure

The study started with students individually presenting their pitch in front of their peers and course teachers. The objective of this first pitch was to obtain a baseline of the students’ performance. Peers evaluated the pitch by filling in a presentation assessment questionnaire.

After presenting the pitch each student moved to another room for the practice sessions. Before the practice sessions, students received a small briefing regarding the PT’s feedback. The purpose of this small briefing was to reduce the exploration time needed to understand the feedback given by the PT. After this short briefing time, participants were supposed to know how to correctly react to the feedback given by the PT. The practice sessions consisted delivering the pitch two consecutive times while receiving feedback from the PT. During the practice session students stood between 1.5 and 3 m in front of the Microsoft Kinect sensor and a 13-inches display laptop running the PT.

For the next phase of the study, the student returned to the classroom and presented the pitch once more to their peers. The objective of this second pitch was to explore the effects of the practice sessions. To observe these effects, peers evaluated this final presentation once more by filling in the presentation assessment questionnaire. The PT was also used to assess these pitches. However, due to a technical failure only the pitches given by the last three participants were assessed by the PT. After delivering this final pitch, students were asked to fill in a questionnaire regarding the experience of using the PT to practice.

4.3 Apparatus and Material

To evaluate the pitches done by the students, peers filled in a presentation assessment questionnaire. The questionnaire consists of eleven Likert-scaled items. The first seven items refer to a general assessment of the presentation including: the overall quality of the presentation, delivery of the presentation, speaker knowledge about the topic, confidence of the speaker, enthusiasm of the speaker, understandability of the pitch, and fun factor of the pitch. The last four items consisted of some of the specific nonverbal behaviors that can be trained using the PT: posture, use of gestures, voice quality, and use of pauses.

To practice for the second presentation of the pitch students used the current version of the PT. This version of the PT uses the immediate feedback mechanism described in [ 13 ], providing users with the maximum of one corrective feedback at the time regarding their body posture, use of gestures, voice volume, phonetic pauses or filler sounds, use of pauses, and facial expressions (45 s without smiling). The PT logs all the recognizable behaviors (mistakes and good practices) as events. It displays these events at the end of each practice the session a timeline (see Fig.  3 ) allowing learners to get an overall picture of their performance. These logs are stored into files that can later be used for data analysis.

Timeline displaying all tracked events, showed to the user after the presentation.

A user experience questionnaire was used to capture the impressions of the students regarding the use of the PT. This questionnaire consists of seven items in total, five Likert-scale items and two open questions. The purpose of this questionnaire was to inquire the learning perception, usefulness of the system, and comparison between human assessment and system assessment.

The peer evaluation of the first pitches is shown in Fig.  4 . Regarding the general aspects of the pitch, the item with the best score was the knowledge about the topic displayed by the presenter with an average score of 3.76 and the item with the lowest score was the entertaining factor of the pitch with an average score of 3.1. The nonverbal communication behavior with the highest score was the voice quality of the presenter with an average score of 3.73 and the behavior with the lowest score was the proper use of pauses during the pitch with an average score of 3.21.

Evaluation scores of the first pitches.

After giving the first pitch, students practiced it two times using the PT. We analyzed these practice sessions using the logged files created by the PT. To evaluate the impact of each of the identified behaviors captured by the PT, we used the percentage of time that this behavior was displayed during the training session (pTM). The pTM value for each behavior has a range from 0 to 1, where 0 indicates that the behavior was not displayed at all and 1 indicates that the behavior was identified throughout the whole presentation. The average pTM values for all the tracked behaviors are displayed in Table  1 . Results indicate that participants on average during the second practice session show an improvement in all trained aspects. The behavior that on average received the worst assessment for the first practice session was the use of gestures, followed by the voice volume and then posture. The pTM value for the other tracked behaviors was very low. In the second practice session voice volume received the worst assessment, followed by gestures and then posture. The area showing the biggest improvement was the use of gestures.

The peer evaluation of the pitches presented after the practice sessions is shown in Fig.  5 . Regarding the general assessment of the pitches the item with the highest score was the knowledge about the topic displayed by the speaker with an average score of 3.96. The item with the lowest score having an average of 3.55 was the entertaining factor of the pitch. Regarding the nonverbal communication aspects, the one with the highest score was the voice quality of the presenter with and average of 4.14 and the correct use of pauses was the lowest with and average of 3.71.

Evaluation scores of the second pitches.

To explore the relevance of having a tool designed to practice specifically the delivery of the pitch, we used Pearson’s r to measure the correlation between the scores of the overall quality of the pitch (content + delivery) and the scores of its delivery. These measurements show a correlation of [r = 0.94, n = 18, p < 0.01]. We also used Pearson’s r on the scores of the pitches to measure the correlation between the behaviors that can be trained using the PT and the overall quality of the presentations (see Table  3 ). This with the objective to explore the relevance of training these behaviors. The behavior displaying the strongest correlation was the use of pauses, followed by posture, voice quality and use of gestures.

Figure  6 shows the comparison in the evaluations between the first and second pitches. These comparisons show and improvement in all evaluated items. The general quality of the pitches increased on a 21.94 %. We calculated the significance of this difference using a t-test. The result of this t-test was t(14) = 3.6, p < .01. This indicates that the improvement observed is statistically significant. Regarding the general aspects of a presentation the delivery of the pitch was the item displaying the biggest improvement showing an increment of 24.27 %. The item showing the lowest improvement was the knowledge about the topic displayed by the presenter. This item had an improvement of only 14.37 %.

Comparison between first and second pitch

By examining the improvements on the nonverbal communication behaviors, the area that displayed the biggest improvements was the use of gestures with an increment of 27.89 %.

The PT’s assessment the second pitch for the last three speakers is shown in Table  2 Footnote 3 . Results from these tracked performances show that all of them had a total pTM value lower than 1.

Results from the user experience questionnaire are listed in Table  4 . These scores show that students would likely use the PT to prepare for future presentations. Results show that students perceived an increment of their nonverbal communication awareness. Students felt that the feedback of the PT is more useful as an addition rather than as a reinforcement of the feedback that peers and tutors can provide.

When asking students about the similarities between the PT’s and the feedback received in previous sessions by tutors and peers all students mentioned the correct use of pauses while presenting. Two of them also mentioned the use of gestures. Four students mentioned that, previously, they received the feedback of not given enough eye contact to the audience by their tutors and peers and that this aspect is missing in the PT’s feedback. Three students commented that receiving immediate feedback by the system makes it much more easy to identify and correct their behavior. One student mentioned that the PT gave feedback regarding the phonetic pauses while peers and tutors did not. One student mentioned a contradiction between the feedbacks regarding the use of voice. Peers and tutors in a previous presentation told the participant to speak louder, and during the training sessions the PT told the participant to speak softer.

6 Discussion

Studying the use of the PT outside of the laboratory in a real life formal learning scenario has several implications. In studies conducted in the lab, the setup of the experiment is carefully designed, allowing experimenters to have full control of variables such as time of each experimental session, location and instruments. This control allows the acquisition of reliable and replicable results. For this study we had to adapt our setup according to the restrictions of the ongoing course followed by the students. We encountered two main challenges while designing and conducting our study: time and location.

Regarding time, in previous laboratory studies participants had individual timeslots of sixty minutes, where they received all the briefing necessary and had five practice sessions with the PT. Moreover, experimenters had the chance to conduct their study with a large enough control and a treatment group, allowing them to assess significant results [ 13 ]. For this study we had two hours to conduct the whole experiment without knowing beforehand the amount of students that would show up that day for the course. Therefore, we reduced the training sessions from five to three and adapted to only two training sessions during the flow of the experiment. The act of training with the PT is individual and designed to be performed in a quiet room where the learner can focus on the task. That forced us to use a separate room where one student could do the practice session while the others waited in the lecture room. The room used for the practice sessions was not designed for the setup of the PT. The location of the power plugs, lighting conditions, place to position the Kinect and laptop screen running the PT were far from ideal. This problem of not having the ideal practice setup partially explains the difference between the average pTM values obtained in this study and the ones obtained in laboratory conditions [ 13 ]. In lab conditions the average values from the first and second training sessions were 0.51 and 0.32 respectively, while in this study they were 0.69 and 0.41. Nevertheless, despite the differences the values did show a similar trend displaying similar improvements in a less than ideal setting.

Previous studies showed that using the PT to practice for presentations improves the performance of the learner according to the measurements tracked by the PT [ 13 ]. The second objective of this study was to investigate whether using the PT to practice a presentation has also an influence in the way that the audiences perceives it. Results from this study showed that according to a human audience, all participants performed better in all aspects after having two practice sessions with the PT. The restricted time slot and restricted number of participants, did not allowed us to make use of a controlled and a treatment group. Therefore it is not possible to directly determine whether the improvements perceived by the audience are the results of practicing with the PT or just practicing. The results, however, revealed three key aspects suggesting the influence of the PT on this perceived improvement. The first key aspect is revealed by the assessed improvements regarding the general aspects of a presentation. The item showing the least improvement between the first and the second pitch is the knowledge that the presenter displayed regarding the topic. While on the other hand the item showing the biggest improvement was the delivery of the pitch. This aligns with the fact that the focus of the practice sessions using the PT was purely on the delivery of the pitch.

The second key aspect pointing out the influence of the PT has to do with the use of gestures. Use of gestures exhibited the biggest improvement from the first human assessed pitch to the second. This aligns with the computer assessment from the two practice sessions, where the aspect exhibiting the biggest improvements was also the use of gestures.

The third key aspect suggesting the influence of the PT is the PT’s assessment of the three of the nine final pitches. In previous studies the average total pTM for presentations of people who did not practice with the PT was close to 1.0, in contrast with the results shown in this study where all the three measured final pitches had total pTM below 0.67. Unfortunately, as mentioned before, due to technical and logistical difficulties we were not able to assess all pitches using the PT.

For the third objective of this study we investigated whether the introduction of a tool such as the PT can contribute to the creation of more comprehensive learning scenarios for the acquisition of public speaking skills. Results from our study support this. As seen in the evaluations of the first pitch, the highest evaluated aspect was the knowledge of the topic displayed by the presenter. This gives us a hint that when preparing for a presentation or a pitch, a common practice is to focus efforts on preparing only its content. This practice does not seem optimal according to the strong correlation measured in this study between the overall quality of a pitch and the quality of its delivery. The results illustrate how by practicing the pitch two times using the PT, students significantly improved the overall quality of it. The students also reported benefits regarding their experience of using the PT to practice. They affirmed that the practice sessions helped them to learn something about public speaking and increase their nonverbal communication awareness. It is interesting to note that according to the students the feedback of the PT complements the feedback received by tutors and peers. Three students stated that the immediate feedback received by the PT helped them to exactly identify and correct their behavior. One more important aspect to note is that students expressed the intention to use the PT in the future.

This study showed some benefits of using of a tool such as the PT to support common practices for learning public speaking skills. However, the introduction of such a tool is still a challenge. The Microsoft Kinect is not a product owned by many students, and it is not feasible to provide each student with a Kinect in order to train some minutes for their presentations. However, Intel is already working in the miniaturization of depth cameras that can be integrated to laptop computers Footnote 4 . Therefore, in a medium term it will become more feasible for students to have access to tools such as the PT and use them for home practice. In the meantime the introduction for dedicated places to practice the delivery of presentations would be needed in order to introduce the support of these types of tools to the current practices for teaching and learning public speaking skills.

7 Conclusion and Future Work

The creation of multimodal learning technologies to support the development of public speaking skills has been driven in recent years by the advances and availability in sensor technologies. In laboratory settings some of these technologies have already started to show promising results. In this study we took one of these technologies, the Presentation Trainer , outside of the lab and conducted some tests with students following an entrepreneurship course as part of the course agenda. The main purpose of this study was to start the exploration of the support that these technologies can bring to a formal learning scenario.

Studying the use of the PT for a real classroom task revealed that location and time constrains interfere with the straightforward conduction of research. Due to location constrains it was not possible to set up the PT in ideal conditions for its use. Due to time constrains it was not possible to have the students follow all the expected training sessions, and we were not able to use the PT to measure all the first and second pitches presented to the audience. These constrains do not allow us to determine the causes for some of the obtained results in this study. However, results from this study align to a large extend with results obtained in the lab [ 13 ].

Regarding the support that the use of a tool such as the PT can bring to the established practices of teaching and learning public speaking skills, results from this study show the following:

Students see themselves willingly using a tool such as the PT to practice for future presentations.

Students find the feedback of the PT to be a good complement to the feedback that peers and tutors can give.

Practicing with the PT leads to significant improvements in the overall quality of a presentation according to a human audience.

For future work we plan to show the results obtained in this study indicating the advantages of using the PT to coordinators of public speaking courses. This comes with a plan to deal with environmental constraints impeding the setup of PT and, hence, its use in the wild. Furthermore we plan to continue improving the PT. The purpose of the PT is to help humans give better presentation to humans. Hence, we plan to explore the relationship between human-based and machine-based assessment, and study how this information can later be used to provide learners with better feedback.

To conclude, there is still a lot of room for improvement for multimodal learning applications designed to support the development of public speaking skills. Introducing them to formal and non-formal educational scenarios still has some practical challenges. Though the application of the PT in a practical setting may not require equally strict conditions as in our research. In any case, studying the use of the PT in the wild has shown promising results regarding the support that such tools can bring to current practices for learning public speaking skills, indicating how courses on developing public speaking skills can be enhanced in the future.

https://dev.windows.com/en-us/kinect/hardware .

https://www.google.com/glass/start/ .

‘A technical failure prevented data capture of the first six participants’ (See Sect.  4.2 ).

http://www.intel.com/content/www/us/en/architecture-and-technology/realsense-overview.html .

DeCaro, P.A.: Origins of public speaking. In: The public speaking project (2011). http://www.publicspeakingproject.org/psvirtualtext.html . Chapter 2

Parvis, L.F. The Importance of Communication and public-speaking skills. J. Environ. Health 35–44 (2001)

Google Scholar  

Campbell, K.S., Mothersbaugh, D.L., Brammer, C., Taylor, T.: Peer versus self-assessment of oral business presentation performance. Bus. Commun. Q. 64 (3), 23–42 (2001)

Article   Google Scholar  

Hinton, J.S., Kramer, M.W.: The impact of self-directed videotape feedback on students’ self- reported levels of communication competence and apprehension. Commun. Edu. 47 (2), 151–161 (1998)

Smith, C.M., Sodano, T.M.: Integrating lecture capture as a teaching strategy to improve student presentation skills through self-assessment. Act. Learn. High Educ. 12 (3), 151–162 (2011)

Gallo, C.: Talk Like TED: The 9 Public Speaking Secrets of the World’s Top Minds. Pan Macmillan (2014)

Van Ginkel, S., Gulikers, J., Biemans, H., Mulder, M.: Towards a set of design principles for developing oral presentation competence: A synthesis of research in higher education. Educ. Res. Rev. 14 , 62–80 (2015)

Swan, M.: Sensor mania! the internet of things, wearable computing, objective metrics, and the quantified self 2.0. Journal of Sensor and Actuator. Networks 1 (3), 217–253 (2012)

MathSciNet   Google Scholar  

Schneider, J., Börner, D., Van Rosmalen, P., Specht, M.: Augmenting the Senses: a re-view on sensor-based learning support. Sensors 15 (2), 4097–4133 (2015)

Barmaki, R., Hughes, C.E.: Providing real-time feedback for student teachers in a virtual rehearsal environment. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 531–537 (2015)

Damian I., Tan C.S.S., Baur T., Schöning J., Luyten K., André E.: Augmenting social interactions: realtime behavioural feedback using social signal processing techniques. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI 2015, pp. 565–574 (2015)

Dermody, F., Sutherland, A.: A multimodal system for public speaking with real time feedback. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 369–370 (2015)

Schneider, J., Börner, D., Van Rosmalen, P., Specht, M.: Presentation trainer, your public speaking multimodal coach. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 539–546 (2015)

Kerby, D. Romine, J.: Develop oral presentation skills through accounting curriculum design and course- embedded assessment. J. Educ. Bus. 85 (3). (2009)

Wörtwein, T., Chollet, M., Schauerte, B., Morency, L. P., Stiefelhagen, R., Scherer, S.: Multimodal public speaking performance assessment. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 43–50 (2015)

Schneider, J., Börner, D., Van Rosmalen, P., Specht, M.: Stand tall and raise your voice! a study on the presentation trainer. In: Proceedings of the tenth European Conference on Technology enhanced learning, EC-TEL2015, pp. 311–324 (2015)

Shields, P.M., Rangarajan, N.: A playbook for research methods: Integrating conceptual frameworks and project management. New Forums Press, Stillwater (2013)

Mitchell, V.W., Bakewell, C.: Learning without doing enhancing oral presentation skills through peer review. Manage. Learn. 26 (3), 353–366 (1995)

Download references

Acknowledgment

The underlying research project is partly funded by the METALOGUE project. METALOGUE is a Seventh Framework Programme collaborative project funded by the European Commission, grant agreement number: 611073 ( http://www.metalogue.eu ).

Author information

Authors and affiliations.

Welten Institute, Open University of the Netherlands, Heerlen, The Netherlands

Jan Schneider, Dirk Börner, Peter van Rosmalen & Marcus Specht

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Jan Schneider .

Editor information

Editors and affiliations.

KU Leuven , Leuven, Belgium

Katrien Verbert

The Open University , Milton Keynes, United Kingdom

Mike Sharples

Jožef Stefan Institute , Ljubljana, Slovenia

Tomaž Klobučar

Rights and permissions

Open Access. This chapter is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, a link is provided to the Creative Commons license and any changes made are indicated.

The images or other third party material in this chapter are included in the work's Creative Commons license, unless indicated otherwise in the credit line; if such material is not included in the work's Creative Commons license and the respective action is not permitted by statutory regulation, users will need to obtain permission from the license holder to duplicate, adapt or reproduce the material.

Reprints and permissions

Copyright information

© 2016 The Author(s)

About this paper

Cite this paper.

Schneider, J., Börner, D., van Rosmalen, P., Specht, M. (2016). Enhancing Public Speaking Skills - An Evaluation of the Presentation Trainer in the Wild. In: Verbert, K., Sharples, M., Klobučar, T. (eds) Adaptive and Adaptable Learning. EC-TEL 2016. Lecture Notes in Computer Science(), vol 9891. Springer, Cham. https://doi.org/10.1007/978-3-319-45153-4_20

Download citation

DOI : https://doi.org/10.1007/978-3-319-45153-4_20

Published : 07 September 2016

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-45152-7

Online ISBN : 978-3-319-45153-4

eBook Packages : Computer Science Computer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Our Mission

4 Strategies for Developing Confident Student Speakers

Consistent support and low-stakes opportunities to practice go a long way to help students overcome the challenges of speaking in front of a group.

I belted out the opening line to Marc Antony’s funeral speech from The Tragedy of Julius Caesar , leaping up the stairs to the stage in my school’s auditorium. Thirteen ninth-grade students dressed in togas sat in the front rows, mouths agape, as they witnessed their goofy teacher unexpectedly launch into the 35-line monologue they had all just beautifully performed. 

Confession: I was nervous walking onto that stage—I sometimes get uncomfortable in the (literal) spotlight. To normalize feelings of anxiety during a performance, I revealed those emotions to my class afterward. 

Public speaking is hard, and it can be extremely daunting for our most introverted students ; however, with modeling and practice, I believe that teachers can cultivate confident speakers.

Strategy 1: Provide Direct Instruction

A poster inspired by Erik Palmer’s work on public speaking ( PVLEGS : Poise, Voice, Life, Eye Contact, Gestures, Speed) hangs in the back of my classroom. Well before we launch into a study of the funeral orations from Caesar , I explicitly teach those skills. I demonstrate for students appropriate eye contact—the speaker locks eyes briefly with individual audience members and scans the room as she speaks, establishing a connection with her audience. The speaker might also speed up his voice for a certain effect or raise his arms to command the attention of his audience.

Strategy 2: Provide Public Speaking Models

After we spend a class closely reading and analyzing the two funeral orations by Marc Antony and Brutus, I play clips of the speeches from the two film adaptations. We watch the performances and think about PVLEGS: Which speaking moves does each actor utilize in his performance? Which actor delivers the stronger performance and why? After we watch and students share their observations with partners, we discuss and debate the merits of each performance. 

When my classes work on memorizing and performing the monologues, I ask them to study the models and even borrow some of the actors’ techniques. They consider: Do I want to take an angrier approach to the Antony speech like Marlon Brando? Should I adopt a regretful tone like Jason Robards’ Brutus? This type of close viewing could be applied to any speaking performance. On a related note, I also hope that I serve as a public speaking model for my students as I stand in front of my English classes daily.

Strategy 3: Acknowledge and Coach through Anxiety

I remember stumbling through my senior research presentation in high school, well before the advent of interactive whiteboards and Google Slides. I’m vulnerable with my students about my former public speaking struggles. I want them to know that they’re not alone, and through practice and repetition, their self-doubt and fear can transform from an eardrum-rupturing siren into a quiet background hum.

I give a few bits of advice to nervous speakers as they prepare for any public speaking activity. First, I urge them to “practice, practice, practice!” If they know their stuff, they’ll be much more confident on game day. I also find that most students who report feeling very nervous while speaking don’t always appear nervous to others. Sharing this anecdotal evidence with them helps students tune out their inner critics and feel more calm. I also find lots of opportunities to confer with reluctant speakers and give them plenty of encouragement. This fosters stronger connections with my students and boosts their confidence. 

Strategy 4: Provide Lots of Low-Stakes Speaking Opportunities 

My Caesar unit usually takes place in the second semester, when students have had plenty of low-stakes speaking opportunities. In almost every class, I ask students to turn and talk to their partners to share a sentence from a quickwrite or to check in on their current drafts. Earlier in the year, I coach them on how to effectively talk to their partners, and the process quickly becomes routine. An effective technique to get all students speaking, even if it’s a quick response, is the whip-around. Teacher Marcus Luther asks a question with a short, one-word answer. He then goes around the room and has each student answer aloud, followed by a debrief with partners or groups. 

My favorite low-stakes speaking activity is the Pop-Up Debate , which teacher Dave Stuart Jr. invented and has been writing about for years. In this activity, the class is presented with a question, and they spend 10 minutes writing an answer to it. After that, the debate begins, and students simply “pop-up” at their desks to enter into the conversation. In my experience, this activity can transform a usually quiet class into an engaged one. Secondary students love to argue, even about literature! 

Flexibility, Support, and Incentives Develop Strong Speakers 

In many ways, the memorized funeral oration is the most high-stakes speaking task my ninth-grade honors students participate in throughout the year. I give a grade for this assessment, but to take the focus off of grades and to place that focus on improvement instead, I provide students with a few crutches to lean on. 

First, if students forget a line, a friend acts as a prompter in the pit with the text ready. Students can also revise the performance if they choose to. I tell students that they can redo the performance if it doesn’t go as planned. At the end of the class, if time allows, a handful of students always choose to give it another shot. I find that this also takes the pressure off. I think of it like a writing assignment; students can always revise a paper to improve it. Finally, I give them extra credit for wearing togas, which adds to the fun of the special day.

In a recent article in The Atlantic , “ End the Phone-Based Childhood Now ,” author Jonathan Haidt reports that Gen Z students are more anxious, shy, and risk-averse than the children of the past, which he attributes to social media and the introduction of the smartphone. Risk-taking, Haidt argues, “promote[s] competence, maturity, and mental health.” Public speaking, a type of risk-taking, can be very frightening for some of our students, but if teachers provide them with speaking opportunities each day, month, and school year, perhaps we can help mold them into confident, healthy young adults who venture into the world as strong public speakers. 

Thank you to my former department chair, Janet Matthews, for the Julius Caesar performance activity.