research question in journal article

Advertisement

• Previous Article
• Next Article

1. INTRODUCTION

2. background, 5. discussion, 6. conclusions, author contributions, competing interests, funding information, data availability, how common are explicit research questions in journal articles.

Handling Editor: Ludo Waltman

• Cite Icon Cite
• Open the PDF for in another window
• Permissions
• Article contents
• Figures & tables
• Supplementary Data
• Peer Review
• Search Site

Mike Thelwall , Amalia Mas-Bleda; How common are explicit research questions in journal articles?. Quantitative Science Studies 2020; 1 (2): 730–748. doi: https://doi.org/10.1162/qss_a_00041

Download citation file:

• Ris (Zotero)
• Reference Manager

Although explicitly labeled research questions seem to be central to some fields, others do not need them. This may confuse authors, editors, readers, and reviewers of multidisciplinary research. This article assesses the extent to which research questions are explicitly mentioned in 17 out of 22 areas of scholarship from 2000 to 2018 by searching over a million full-text open access journal articles. Research questions were almost never explicitly mentioned (under 2%) by articles in engineering and physical, life, and medical sciences, and were the exception (always under 20%) for the broad fields in which they were least rare: computing, philosophy, theology, and social sciences. Nevertheless, research questions were increasingly mentioned explicitly in all fields investigated, despite a rate of 1.8% overall (1.1% after correcting for irrelevant matches). Other terminology for an article’s purpose may be more widely used instead, including aims, objectives, goals, hypotheses, and purposes, although no terminology occurs in a majority of articles in any broad field tested. Authors, editors, readers, and reviewers should therefore be aware that the use of explicitly labeled research questions or other explicit research purpose terminology is nonstandard in most or all broad fields, although it is becoming less rare.

Academic research is increasingly multidisciplinary, partly due to team research addressing practical problems. There are also now large multidisciplinary journals, such as PLOS ONE and Nature Scientific Reports , with editorial teams that manage papers written by people from diverse disciplinary backgrounds. There is therefore an increasing need for researchers to understand disciplinary norms in writing styles and paradigms. The authors of a research paper need to know how to frame its central contribution so that it is understood by multidisciplinary audiences. One strategy for this is to base an article around a set of explicitly named research questions that address gaps in prior research. Employing the standard phrase “research question” gives an unambiguous signpost for the purpose of an article and may therefore aid clarity. Other strategies include stating hypotheses, goals, or aims, or describing an objective without calling it an objective (e.g., “this paper investigates X”). Similarly, structured abstracts are believed to help readers understand a paper ( Hartley, 2004 ), perhaps partly by having an explicit aim, objective, or goal section. A paper that does not recognize or value the way in which the central contribution is conveyed may be rejected by a reviewer or editor if they are unfamiliar with the norms of the submitting field. It would therefore be helpful for authors, reviewers, and editors to know which research fields employ explicitly labeled research questions or alternative standard terminology.

Purpose statements and research questions or hypotheses are interrelated elements of the research process. Research questions are interrogative statements that reflect the problem to be addressed, usually shaped by the goal or objectives of the study ( Onwuegbuzie & Leech, 2006 ). For example, a healthcare article argued that “a good research paper addresses a specific research question. The research question—or study objective or main research hypothesis—is the central organizing principle of the paper” and “the key attributes are: (i) specificity; (ii) originality or novelty; and (iii) general relevance to a broad scientific community” ( Perneger & Hudelson, 2004 ).

The choice of terminology to describe an article’s purpose seems to be conceptually arbitrary, with the final decision based on community norms, journal guidelines, and author style. For example, a research paper investigating issue X could phrase its purpose in the following ways: “research question 1: is X true?,” “this paper aims to investigate X,” “the aim/objective/purpose/goal is to investigate X,” or “X?” (as in the current paper). Implicit purpose statements might include “this paper investigates X” or just “X,” where the context makes clear that this is the purpose. Alternatively, the reader might deduce the purpose of a paper after reading it, with all these options achieving the same result with different linguistic strategies. Some research purposes might not be easily expressible as a research question, however. For example, a humanities paper might primarily discuss an issue (e.g., “Aspects of the monastery and monastic life in Adomnán’s Life of Columba ”) but even these could perhaps be expressed as research questions, if necessary (e.g., “Which are the most noteworthy aspects of the monastery and monastic life in Adomnán’s Life of Columba ?”).

In which fields are explicitly named research questions commonly used?

Has the use of explicitly named research questions increased over time?

Are research purposes addressed using alternative language in different fields?

Do large journals guide authors to use explicitly named research questions or other terminology for purpose statements in different fields?

2.1. Advice for Authors

There are some influential guidelines for reporting academic research. In the social sciences, Swales’ (1990 , 2004) Create A Research Space (CARS) model structures research article introductions in three moves (establishing a territory, establishing a niche, and occupying a niche), which are subdivided into steps. Within the 1990 model, move 3 includes the steps “outlining purposes” and “announcing present research,” but research questions are not explicitly included, being similar the “question raising” step in move 2. In the updated 2004 model, move 3 includes an obligatory step named “announcing present research descriptively and/or purposively” (that joins the steps “outlining purposes” and “announcing present research” from the 1990 model), whereas “listing research questions or hypotheses” is a new optional step.

In medicine, the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) initiative is a checklist of items that should be included to improve reporting quality. One of these is a statement of objectives that “may be formulated as specific hypotheses or as questions that the study was designed to address” or may be less precise in early studies ( Vandenbroucke, von Elm, et al., 2014 ). This description therefore includes stating research questions as one of a range of ways of specifying objectives. An informal advice article in medicine instead starts by arguing that the paper’s aim should be clearly defined ( McIntyrei, Nisbet, et al., 2007 ).

Researchers may also be guided about the language to use in papers by any ethical or other procedures that they need to follow before conducting their work. For example, clinical trials often need to be registered and declared in a standard format, which may include explicit descriptions of objectives (e.g., see “E.2.1: Main objective of the trial” at: https://www.clinicaltrialsregister.eu/ctr-search/trial/2015-002555-10/GB ).

2.2. Empirical Evidence

Journal article research questions and other purpose statements, such as aims, objectives, goals, and hypotheses ( Shehzad, 2011 ), are usually included within Introduction sections or introductory phases, sometimes appearing as separate sections ( Kwan, 2017 ; Yang & Allison, 2004 ). Some studies have analyzed research article introductions in different disciplines and languages based on the Swales’ (1990 , 2004) CARS model. Although these studies analyze small sets of articles, they seem to agree that the research article introduction structure varies across disciplines (e.g., Joseph, Lim & Nor, 2014 ) and subdisciplines within a discipline, including for engineering ( Kanoksilapatham, 2012 ; Maswana, Kanamaru, & Tajino, 2015 ), applied linguistics ( Jalilifar, 2010 ; Ozturk, 2007 ) and environmental sciences ( Samraj, 2002 ). Introductions in English seem to follow this pattern more closely than introductions in other languages ( Ahamad & Yusof, 2012 ; Hirano, 2009 ; Loi & Evans, 2010 ; Rahimi & Farnia, 2017 ; Sheldon, 2011 ), reflecting cultural differences. Research questions and other purpose terminology, such as aims, objectives, goals, or hypotheses, might also reappear within the Results or Discussion sections ( Amunai & Wannaruk, 2013 ; Brett, 1994 ; Hopkins & Dudley-Evans, 1988 ; Kanoksilapatham, 2005 ).

Previous research has shown that research questions and hypotheses are more common among English-language papers than non-English papers ( Loi & Evans, 2010 ; Mur Dueñas, 2010 ; Omidi & Farnia, 2016 ; Rahimi & Farnia, 2017 ; Sheldon, 2011 ), especially those written by English native speakers ( Sheldon, 2011 ). However, a study analyzing 119 English research article introductions from Iranian and international journals in three subdisciplines within applied linguistics found that “announcing present research” was more used in international journals whereas research questions were proclaimed explicitly more often in local journals ( Jalilifar, 2010 ).

In some fields the verbs examine , determine , evaluate , assess , and investigate are associated with the research purpose ( Cortés, 2013 ; Jalali & Moini, 2014 ; Kanoksilapatham, 2005 ) and the verbs expect , anticipate , and estimate are associated with hypotheses ( Williams, 1999 ). Some computer scientists seem to prefer to write the details of the method(s) used rather than stating the purpose or describing the nature of their research and use assumptions or research questions rather than hypotheses ( Shehzad, 2011 ). Moreover, scholars might state the hypotheses in other ways, such as “it was hypothesized that” ( Jalali & Moini, 2014 ).

A study analyzing lexical bundles (usually phrases) in medical research article introductions showed that the most frequent four-word phrases are related to the research objective, such as “the aim of the,” “aim of the present,” and “study was to evaluate” ( Jalali & Moini, 2014 ). Another study examined lexical bundles in a million-word corpus of research article introductions from several disciplines, showing that the main bundle used to announce the research descriptively and/or purposefully included the terms aim , objective , and purpose (e.g., “the aim of this paper,” “the objective of this study,” “the purpose of this paper”), but no bundles related to research questions or hypotheses were identified ( Cortés, 2013 ).

These findings are in line with other previous studies investigating the structure of research articles, especially the introduction section, which report a much higher percentage of journal papers specifying the research purpose than the research questions or hypotheses across disciplines, regardless of the language in which they are published, with the exception of law articles (see Table 1 ). These studies also show that research questions and hypotheses are much more frequent among social sciences articles (see Table 1 ), which has also been found in other genres, such as PhD theses and Master’s theses (see Table 2 ).

Reference to a wide research purposes, without specifying if they are objectives or RQs/hypotheses.

Restating RQs in the result section.

Note: Studies that have based their analysis on the Swales’s (1990) CARS model ( Anthony, 1999 ; Posteguillo, 1999 ; Mahzari & Maftoon, 2007 ) report the percentage related to “outlining purposes” and “announcing present research.” For these studies, the column “Present the research purpose” reports the higher value. Moreover, for these studies, the value reported in the RQs/hypotheses column refers to the “Question raising” information.

A few studies have focused exclusively on research purposes, research questions, and hypotheses. Some have discussed the development of research questions in qualitative ( Agee, 2009 ) or mixed method ( Onwuegbuzie & Leech, 2006 ) studies, whereas others have examined the ways of constructing research questions or hypotheses within some fields, such as organization studies ( Sandberg & Alvesson, 2011 ) or applied linguistics doctoral dissertations ( Lim, 2014 ; Lim, Loi, & Hashim, 2014 ). Shehzad (2011) examined the strategies and styles employed by computer scientists outlining purposes and listing research questions. She found an increase in the use of research nature or purpose statements and suggested that the “listing research questions or hypotheses” step of Swales’s model was obligatory in computing. No study seems to have examined how often journal guidelines give authors explicit advice about research questions or other purpose statements, however.

The PMC (Pub Med Central) Open Access subset ( www.ncbi.nlm.nih.gov/pmc/tools/openftlist/ ) was downloaded in XML format in November 2018. This is a collection of documents from open access journals or open access articles within hybrid journals. The collection has a biomedical focus, but includes at least a few articles from all broad disciplinary areas. Although a biased subset is not ideal, this is apparently the largest open access collection. Only documents declared in their XML to be of type “research article” were retained for analysis. This excludes many short contributions, such as editorials, that would not need research goals.

The XML of the body section of each article was searched for the test strings “research question,” “RESEARCH QUESTION,” “Research Question,” or “Research question,” recording whether each article contained at least one. This would miss papers exclusively using abbreviations, such as RQ1.

Full body text searches are problematic because terms could be mentioned in other contexts, depending on the part of an article. For example, the phrase “research question” in a literature review section may refer to an article reviewed. For a science-wide analysis it is not possible to be prescriptive about the sections in which a term must occur, however, because there is little uniformity in section names or orders ( Thelwall, 2019 ). Making simplifying assumptions about the position in a text in which a term should appear, such as that a research question should be stated in the first part of an article, would also not be defensible. This is because the structure of articles varies widely between journals and fields. For example, methods can appear at the end rather than the middle, and some papers start with results, with little introduction. There are also international cultural differences in the order in which sections are presented in some fields ( Teufel, 1999 ). The current paper therefore uses full-text searches without any heuristics to restrict the results for transparency and to give an almost certain upper bound to the prevalence of terms, given the lack of a high-quality alternative.

Articles were separated into broad fields using the Science-Metrics public journal classification scheme ( Archambault, Beauchesne, & Caruso, 2011 ), which allocates each journal into exactly one category. This seems to be more precise than the Scopus or Web of Science schemes ( Klavans & Boyack, 2017 ). The Science-Metrics classification was extended by adding the largest 100 journals in the PMC collection that had not been included in the original Science-Metrics classification scheme. These were classified into a Science-Metrics category by first author based on their similarity to other journals in the Science-Metrics scheme.

Five of the broad fields had too little data to be useful (Economics & Business; Visual & Performing Arts; Communication & Text Studies; General Arts, Humanities & Social Sciences; Built Environment & Design) and were removed. Years before 2000 were not included because of their age and small amount of data. Individual field/year combinations were also removed when there were fewer than 30 articles, since they might give a misleading percentage. Each of the 17 remaining categories contained at least 630 articles ( Table 3 ), with exact numbers for each field and year available in the online supplementary material (columns AE to AW: https://doi.org/10.6084/m9.figshare.10274012 ). For all broad fields, most articles have been published in the last 5 years (2014–2018), with the exception of Historical Studies, Chemistry, and Enabling & Strategic Technology.

For the third research question, alternative terms for research goals were searched for in the full text of articles. These terms might all be used in different contexts, so a match is not necessarily related to the main goal of the paper (e.g., the term “question” could be part of a discussion of a questionnaire), but the rank order between disciplines may be informative and the results serve as an upper bound for valid uses. The terms searched for were “research questions,” “questions,” “hypotheses,” “aims,” “objectives,” “goals,” and “purposes” in both singular and plural forms. These have been identified above as performing similar functions in research. For this exploration, the term “question” is used in addition to “research question” to capture more general uses.

Any of the queried terms could be included in an article out of context. For example, “research question” could be mentioned in a literature review rather than to describe the purpose of the new article. To check the context in which each term was used, a random sample of 100 articles (using a random number generator) matching each term (200 for each concept, counting both singular and plural, totaling 1,400 checks) was manually examined to ascertain whether any use of the term in the article stated the purpose of the paper directly (e.g., “Our research questions were…”) or indirectly (e.g., “This answered our research questions”), unless mentioned peripherally as information to others (e.g., “The study research questions were explained to interviewees”). There did not seem to be stock phrases that could be used to eliminate a substantial proportion of the irrelevant matches (e.g., “objective function” or “microscope objective”). There also was not a set of standard phrases that collectively could unambiguously identify the vast majority of research questions (e.g., “Our research questions were” or “This article’s research question is”).

Journal guidelines given to authors were manually analyzed to check whether they give advice about research questions and other purpose statements. Three journals with the most articles in each of the 17 academic fields were selected for this (see online supplement doi.org/10.6084/m9.figshare.10274012 ). This information is useful background context to help interpret the results.

4.1. RQ1 and RQ2: Articles Mentioning Research Questions

Altogether, 23,282 out of 1,314,412 articles explicitly mentioned the phrases “research question” or “research questions” (1.8%), although no field included them in more than a fifth of articles in recent years and there are substantial differences between broad fields ( Figure 1 ). When the terms are used in an article they usually (63%, from the 1,400 manual checks) refer to the article’s main research question(s). Other uses of these terms include referring to questions raised by the findings, and a discussion of other articles’ research questions in literature review sections or as part of the selection criteria of meta-analyses. Thus, overall, only 1.1% of PMC full-text research articles mention their research questions explicitly using the singular or plural form. There has been a general trend for the increasing use of these terms, however ( Figure 2 ).

The percentage of full-text research articles containing the phrases “research question” or “research questions” in the body of the text, 2014–2018, for articles in the PMC Open Access collection from 17 out of 22 Science-Metrics broad fields; 63% of occurrences of these terms described the hosting article’s research question(s) ( n = 801,895 research articles).

As for Figure 1 but covering 2000–2018 ( n = 1,314,412 research articles). (All fields can be identified in the Excel versions of the graph within the online supplement 10.6084/m9.figshare.10274012).

If the terms “question” or “questions” are searched for instead, there are many more matches, although for a minority of articles in most fields ( Figures 3 and 4 ). When these terms are mentioned, they rarely (17%) refer to the hosting article’s research questions (excluding matches with the exact phrases “research question” or “research questions” to avoid overlaps with the previous figure). Common other contexts for these terms include questions in questionnaires and questions raised by the findings. Sometimes the term “question” occurred within an idiomatic phrase or issue rather than a query (e.g., “considerable temperature gradients occur within the materials in question” and “these effects may vary for different medications. Future studies are needed to address this important question”). In Philosophy & Theology, the matches could be for discussions of various questions within an article, rather than a research question that is an article’s focus. Similarly for Social Sciences and Public Health & Health Services, the question mentioned might be in questionnaires rather than being a research question. After correcting for the global irrelevant matches, which is a rough approximation, in all broad fields fewer than 14% of research articles use these terms to refer to research questions. Nevertheless, this implies that the terms “question” or “questions” are used much more often than the phrases “research question” or “research questions” (1.8%) to refer to an article’s research purposes.

The percentage of full-text research articles containing the terms “question” or “questions” in the body of the text, 2014–2018, for articles in the PMC Open Access collection from 17 out of 22 Science-Metrics broad fields; 17% of occurrences of these terms described the hosting article’s main research question(s) without using the exact phrases “research question” or “research questions,” not overlapping with Figure 1(a) ( n = 801,895 research articles).

As for Figure 3 , but covering 2000–2018 ( n = 1,314,412 research articles).

4.2. RQ3: Other Article Purpose Terms

The terms “hypothesis” and “hypotheses” are common in Psychology and Cognitive Science as well as in Biology ( Figure 5 ). They are used in a minority of articles in all other fields, but, by 2018 were used in at least 15% of all (or 4% after correcting for irrelevant matches). The terms can be used to discuss statistical results from other papers and in philosophy and mathematics they can be used to frame arguments, so not all matches relate to an article’s main purpose, and only 28% of the random sample checked used the terms to refer to the articles’ main hypothesis or hypotheses.

The percentage of full-text research articles containing the terms “hypothesis” or “hypotheses” in the body of the text, 2014–2018, for articles in the PMC Open Access collection from 17 out of 22 Science-Metrics broad fields; 28% of occurrences of these terms described the hosting article’s main hypothesis or hypotheses. A corresponding time series graph showing little change is in the online supplement ( n = 801,895 research articles).

The use of the terms “aim” and “aims” is increasing overall, possibly in all academic fields ( Figures 6 and 7 ). Fields frequently using the term include Philosophy & Theology, Information & Communication Technologies (ICTs) and Public Health & Health Services, whereas it is used in only about 20% of Chemistry and Biomedical Research papers. Articles using the terms mostly use them (especially the singular “aim”) to describe their main aim (70%), so these are the terms most commonly used to describe the purpose of a PMC full-text article. The terms are also sometimes used to refer to wider project aims or relevant aims outside of the project (e.g., “The EU’s biodiversity protection strategy aims to preserve…”).

The percentage of full-text research articles containing the terms “aim” or “aims” in the body of the text, 2014–2018, for articles in the PMC Open Access collection from 17 out of 22 Science-Metrics broad fields; 70% of occurrences of these terms described the hosting article’s main aim(s) ( n = 801,895 research articles).

As for Figure 6 , but covering 2000–2018 ( n = 1,314,412 research articles).

The terms “objective” and “objectives” are reasonably common in most academic fields ( Figure 8 ) and are used half of the time (52%) for the hosting article’s objectives. Other common uses include lenses and as an antonym of subjective (e.g., “high-frequency ultrasound allows an objective assessment…”). It is again popular within ICTs, Philosophy & Theology, and Public Health & Health Services, whereas it is used in only about 12% of Physics & Astronomy articles.

The percentage of full-text research articles containing the terms “objective” or “objectives” in the body of the text, 2014–2018, for articles in the PMC Open Access collection from 17 out of 22 Science-Metrics broad fields; 52% of occurrences of these terms described the hosting article’s objective(s). A corresponding time series graph showing little change is in the online supplement ( n = 801,895 research articles).

The terms “goal” and “goals” follow a similar pattern to “aim” and “objective” ( Figure 9 ), but refer to the hosting paper’s goals in only 28% of cases. Common other uses include methods goals (“the overall goal of this protocol is…”) and field-wide goals (e.g., “over the last decades, attempts to integrate ecological and evolutionary dynamics have been the goal of many studies”).

The percentage of full-text research articles containing the terms “goal” or “goals” in the body of the text, 2014–2018, for articles in the PMC Open Access collection from 17 out of 22 Science-Metrics broad fields; 28% of occurrences of these terms described the hosting article’s research question(s). A corresponding time series graph showing little change is in the online supplement ( n = 801,895 research articles).

Some articles may also use the terms “purpose” or “purposes” rather than the arguably more specific terms investigated above, and there are disciplinary differences in the extent to which they are used ( Figure 10 ). These terms may also be employed to explain or justify aspects of an article’s methods. When used, they referred to main purposes in fewer than a third of articles (29%), and were often instead used to discuss methods details (e.g., “it was decided a priori that physical examination measures would not be collected for the purpose of this audit”), background information (e.g., “species are harvested through fishing or hunting, mainly for alimentary purposes”) or ethics (e.g., “Animal care was carried out in compliance with Korean regulations regarding the protection of animals used for experimental and other scientific purposes.”).

The percentage of full-text research articles containing the terms “purpose” or “purposes” in the body of the text, 2014–2018, for articles in the PMC Open Access collection from 17 out of 22 Science-Metrics broad fields; 29% of occurrences of these terms described the hosting article’s purpose(s). A corresponding time series graph showing little change is in the online supplement ( n = 801,895 research articles).

4.3. RQ4: Journal Guidelines

“The motivation or purpose of your research should appear in the Introduction, where you state the questions you sought to answer” ( zookeys.pensoft.net/about )

“Define the purpose of the work and its significance, including specific hypotheses being tested” ( www.mdpi.com/journal/nutrients/instructions )

“The introduction briefly justifies the research and specifies the hypotheses to be tested” ( www.ajas.info/authors/authors.php )

“A brief outline of the question the study attempts to address” ( onlinelibrary.wiley.com/page/journal/20457758/homepage/registeredreports.html )

“Acquaint the reader with the findings of others in the field and with the problem or question that the investigation addresses.” ( www.oncotarget.com )

“State the research objective of the study, or hypothesis tested” ( www.springer.com/biomed/human+physiology/journal/11517 )

In the first quote above, for example, “state the questions” could be addressed literally by listing (research) questions or less literally by stating the research objectives. Thus, journal guidelines seem to leave authors the flexibility to choose how to state their research purpose, even if suggesting that research questions or hypotheses are used. This also applies to the influential American Psychological Society guidelines, such as, “In empirical studies, [explaining your approach to solving the problem] usually involves stating your hypotheses or specific question” ( APA, 2009 , p. 28).

An important limitation of the methods is that the sample contains a small and biased subset of all open access research articles. For example, the open access publishers BMC, Hindawi, and MDPI have large journals in the data set. The small fields ( Table 3 ) can have unstable lines in the graphs because of a lack of data. Sharp changes between years for the same field are likely due to either small amounts of data or changes in the journals submitted to PubMed in those years, rather than changes in field norms. It is possible that the proportions discovered would be different for other collections. Another limitation is that although articles were searched with the text string “research question,” this may not always have signified research questions in the articles processed (e.g., if mentioned in a literature review or in a phrase such as “this research questions whether”). Although the corrections reported address this, they provide global correction figures rather than field-specific corrections. Conversely, a research question may just be described as a question (e.g., “the query of this research”) or phrased as a question without describing it as such (e.g., “To discover whether PGA implants are immunologically inert…”). Thus, the field-level results are only indicative.

RQ1: Only 23,282 (1.8%, 1.1% after correcting for irrelevant matches) out of 1,314,412 articles assessed in the current paper explicitly mentioned “research question(s),” with significant differences between fields. Although there has been a general trend for the increasing use of explicitly named research questions, they were employed in fewer than a quarter of articles in all fields. Research questions were mostly used by articles in Social Sciences, Philosophy & Theology, and ICTs, whereas they have been mentioned by under 2% of articles in engineering, physical, life, and medical sciences. Previous studies have shown that 73.3% of English articles in Physical Education ( Omidi & Farnia, 2016 ), 33% of Applied Linguistics articles ( Sheldon, 2011 ) and 32% of Computer Science articles ( Shehzad, 2011 ) included research questions or hypotheses. Studies focused on doctoral dissertations show that 97% of U.S. Applied Linguistics ( Lim, 2014 ), 90% of English Language Teaching ( Geçíklí, 2013 ), 70% of Education Management ( Cheung, 2012 ), and 50% of computing doctoral dissertations ( Soler-Monreal, Carbonell-Olivares, & Gil-Salom, 2011 ) listed research questions, a large difference.

The results also show that about 13% of Public Health and Health Services articles and 12% of Psychology and Cognitive Science articles use the term “research questions.” However, a study focused on Educational Psychology found that 35% of English-language papers listed research questions and 75% listed hypotheses ( Loi & Evans, 2010 ). Thus, the current results reveal a substantially lower overall prevalence than suggested by previous research.

RQ2: There has been a substantial increase in the use of the term “research questions” in some subjects, including ICTs, Social Sciences, and Public Health and Health Services ( Figure 2 ), as well as a general trend for increasing use of this term, but with most fields still rarely using it. This suggests that some disciplines are standardizing their terminology, either through author guidelines in journals (RQ4), formal training aided by frameworks such as Swales’ CARS model, or informal training or imitation. For example, the analysis of the “instructions for authors” given by 51 journals (online supplement doi.org/10.6084/m9.figshare.10274012 ) showed that the three biology journals, the three psychology journals, and two biomedical journals included in the analysis referred to both research questions and hypotheses in their author guidelines.

RQ3: Terminology for the purpose of an article seems to be quite widely used, including aims, objectives, and goals ( Figures 5 – 9 ). This is in line with a study examining the lexical bundles identified in research article introductions from several disciplines, which reported the terms “aim,” “objective,” and “purpose” as the main terms used to announce the research descriptively and/or purposefully, although no phrase related to research questions or hypotheses was identified ( Cortés, 2013 ), and with another study reporting similar terminology in medical articles ( Jalali & Moini, 2014 ). Related to this (RQ4), the analysis of the “instructions for authors” given by 51 journals (online supplement 10.6084/m9.figshare.10274012) showed that “purpose” is the term mostly mentioned in the Abstract guidelines and “aims” is the term mainly used in the body of the text (Introduction or Background) guidelines. The term “objective” also appears in some article body guidelines, whereas the term “goal” is not mentioned in them. After correcting for irrelevant matches (e.g., articles using the term “hypothesis” but not for their main research hypotheses) using the percentages reported with the figures above, no terminology was found in a majority of articles in any field. Thus, at least from the perspective of PMC Open Access publications, there is no standardization of research terminology in any broad field.

There are substantial disciplinary differences in the terminology used. Whereas the term “research question” is relevant in Social Sciences, Philosophy & Theology, and ICTs, the term “hypothesis” is important in Psychology and Cognitive Science, used in over 60% of articles. This is in line with a study focused on Educational Psychology, which found that the 75% out of 20 English papers introduced the hypotheses, whereas 35% of them introduced the research questions ( Loi & Evans, 2010 ). The three psychology journals with the highest frequency in the data set used for this study referred to hypotheses in their author guidelines (see online supplement 10.6084/m9.figshare.10274012).

The terms “aim,” “objective,” and “goal” are mainly used in Philosophy, Theology, ICTs, and Health. The term “aim” is also quite often used in health, mathematics, and psychological articles, whereas the term “objective” is also used in engineering and mathematics articles. The term “goal” is also used in psychology and biomedical articles. Although most articles in all fields include a term that could be used to specify the purpose of an article (question or questions, hypothesis, aim, objective, goal), they are relatively scarce in Chemistry and Physics & Astronomy. The use of purpose-related terms has also increased over time in most academic fields. This agrees with a study about Computer Science research articles that found an increasing use of outlining purpose or stating the nature of the research ( Shehzad, 2011 ).

An example article from Chemistry illustrates how a research purpose can be implicit. The paper, “Fluid catalytic cracking in a rotating fluidized bed in a static geometry: a CFD analysis accounting for the distribution of the catalyst coke content” has a purpose that is clear from its title but that is not described explicitly in the text. Its abstract starts by describing what the paper offers, but not why, “Computational Fluid Dynamics is used to evaluate the use of a rotating fluidized bed in a static geometry for the catalytic cracking of gas oil.” The first sentence of the last paragraph of the introduction performs a similar role, “The current paper presents CFD simulations of FCC in a RFB-SG using a model that accounts for a possible nonuniform temperature and catalyst coke content distribution in the reactor.” Both sentences could easily be rephrased to start with, “The purpose of this paper is to,” but it is apparently a stylistic feature of chemical research not to do this. Presumably purposes are clear enough in typical chemistry research that they do not need to be flagged linguistically, but this is untrue for much social science and health research, for example, partly due to nonstandard goals (i.e., task uncertainty: Whitley, 2000 ).

5.1. Possible Origins of the Differences Found

Broad epistemological: Fields work with knowledge in different ways and naturally use different terminology as a result. Arts and humanities research may have the goal to critique or analyze, or may be practice-based research rather than having a more specific knowledge purpose. For this, research questions would be inappropriate. Thus, terminology variation may partly reflect the extent to which a broad field typically attempts to create knowledge.

Narrow epistemological: Narrow fields that address similar problems may feel that they do not need to use research problem terminology to describe their work because the purpose of a paper is usually transparent from the description of the methods or outcome. For example, it would be unnecessary to formulate, “This paper investigates whether treatment x reduces death rates from disease y” as a named research question or even explain that it is the goal of a paper. This may also be relevant for fields that write short papers. It may be most relevant for papers that use statistical methods and have high standards of evidence requirement (e.g., medicine) and clearly defined problems. In contrast, many social sciences research projects are not intrinsically clearly demarcated and need an explanation to define the problem (as for the current article). Thus, describing what the problem is can be an important and nontrivial part of the research. This relates to “task uncertainty,” which varies substantially between fields ( Whitley, 2000 ) and affects scholarly communication ( Fry, 2006 ).

Field or audience homogeneity: Fields with homogeneous levels and types of expertise may avoid terminology that field members would be able to deduce from the context. For example, a mixed audience paper might need to specify statistical hypotheses, whereas a narrow audience paper might only need to specify the result, because the audience would understand the implicit null and alternative hypotheses.

Field cultures for term choice: Academic publishing relies to some extent on imitation and reaching a consensus about the ways in which research is presented (e.g., Becher & Trowler, 2001 ). It might therefore become a field norm to use one term in preference to a range of synonyms, such as “aims” instead of “objectives.”

Field cultures for term meaning: Following from the above, a field culture may evolve an informal convention that two synonyms have different specific uses. For example, “aims” could be used for wider goals and “objectives” for the narrower goals of a paper.

Guidelines: Fields or their core journals may adopt guidelines that specify terminology, presumably because they believe that this standardization will improve overall communication clarity.

The results suggest that the explicit use of research questions, in the sense that they are named as such, is almost completely absent in some research fields, and they are at best a substantial minority (under 20%) in most others (ignoring the fields that did not meet the inclusion threshold). Although the word search approach does not give conclusive findings, the results suggest that alternative terminologies for describing the purpose of a paper are more widespread in some fields, but no single terminology is used to describe research purposes in a majority of articles in any of the broad fields examined.

The lack of standardization for purpose terminology in most or all fields may cause problems for reviewers and readers expecting to see explicit statements. It is not clear whether guidelines to standardize terminology for journals or fields would be practical or helpful, however, but this should be explored in the future. Presumably any guidelines should allow exceptions for articles that make nonstandard contributions, although there are already successful journals with prescriptive guidelines, and the advantage of standardization through structured abstracts seems to be accepted ( Hartley, 2004 ).

The disciplinary differences found may cause problems for referees, authors, editors, and readers of interdisciplinary research or research from outside of their natural field if they fail to find an article’s purpose expressed in the terminology that they expect. This issue could not reasonably be resolved by standardizing across science because of the differing nature of research. Instead, evidence in the current article of the existence of valid disciplinary differences in style may help reviewers and editors of large interdisciplinary journals to accept stylistic differences in research problem formulations.

Mike Thelwall: Conceptualization, Investigation, Software, Writing—original draft. Amalia Mas-Bleda: Investigation, Writing—original draft.

The authors have no competing interests to declare.

This research received no funding.

The data behind the results are available at FigShare ( https://doi.org/10.6084/m9.figshare.10274012 ).

Author notes

Email alerts, affiliations.

• Online ISSN 2641-3337

A product of The MIT Press

Mit press direct.

• About MIT Press Direct

Information

• Accessibility
• For Authors
• For Customers
• For Librarians
• Direct to Open
• Open Access
• Media Inquiries
• Rights and Permissions
• For Advertisers
• About the MIT Press
• The MIT Press Reader
• MIT Press Blog
• Seasonal Catalogs
• MIT Press Home
• Give to the MIT Press
• Direct Service Desk
• Terms of Use
• Privacy Statement
• Crossref Member
• COUNTER Member  
• The MIT Press colophon is registered in the U.S. Patent and Trademark Office

This Feature Is Available To Subscribers Only

Sign In or Create an Account

• SpringerLink shop

Identifying your research question

Making informed decisions about what to study, and defining your research question, even within a predetermined field, is critical to a successful research career, and can be one of the hardest challenges for a scientist.

Being knowledgeable about the state of your field and up-to-date with recent developments can help you:

• Make decisions about  what to study within niche research areas
• Identify  top researchers  in your field whose work you can follow and potentially collaborate with
• Find  important journals to read regularly and publish in
• Explain to others  why your work is important by being able to recount the bigger picture

How can you identify a research question?

Reading regularly is the most common way of identifying a good research question. This enables you to keep up to date with recent advancements and identify certain issues or unsolved problems that keep appearing.

Begin by searching for and reading literature in your field. Start with  general interest  journals, but don’t limit yourself to journal publications only; you can also look for clues in the news or on research blogs. Once you have identified a few interesting topics, you should be reading the table of contents of journals and the abstracts of most articles in that subject area. Papers that are directly related to your research you should read in their entirety.

TIP Keep an eye out for  Review papers and special issues in your chosen subject area as they are very helpful in discovering new areas and hot topics.

TIP: you can sign up to receive table of contents or notifications when articles are published in your field from most journals or publishers.

TIP: Joining a journal club is a great way to read and dissect published papers in and around your subject area. Usually consisting of 5-10 people from the same research group or institute they meet to evaluate the good and bad points of the research presented in the paper. This not only helps you keep up to date with the field but helps you become familiar with what is necessary for a good paper which can help when you come to write your own.

If possible, communicate with some of the authors of these manuscripts via email or in person. Going to conferences if possible is a great way to meet some of these authors. Often,  talking with the author  of an important work in your research area will give you more ideas than just reading the manuscript would.

Back │ Next

Formulating a good research question: Pearls and pitfalls

Affiliation.

• 1 Guys' and St Thomas' Hospital National Health Service Foundation Trust, London, United Kingdom.
• PMID: 31462805
• PMCID: PMC6691636
• DOI: 10.4103/ija.IJA_198_19

The process of formulating a good research question can be challenging and frustrating. While a comprehensive literature review is compulsory, the researcher usually encounters methodological difficulties in the conduct of the study, particularly if the primary study question has not been adequately selected in accordance with the clinical dilemma that needs to be addressed. Therefore, optimising time and resources before embarking in the design of a clinical protocol can make an impact on the final results of the research project. Researchers have developed effective ways to convey the message of how to build a good research question that can be easily recalled under the acronyms of PICOT (population, intervention, comparator, outcome, and time frame) and FINER (feasible, interesting, novel, ethical, and relevant). In line with these concepts, this article highlights the main issues faced by clinicians, when developing a research question.

Keywords: Clinical protocols; medical education; medical writing; research design.

A business journal from the Wharton School of the University of Pennsylvania

AI and Innovation: A Question of Quantity vs. Quality

March 25, 2024 • 3 min read.

In a recent AI at Wharton webinar, scholars shared their research on how AI can enhance innovation and where the limitations are.

The following article was originally published by  AI at Wharton .

Generative artificial intelligence has a quantity-over-quality problem.

One of the biggest challenges in using large language models like ChatGPT is precision.

“Why do you need to buy the whole candy store if you just need a lollipop,” said Daniel Ringel , marketing professor at the University of Chapel Hill’s Kenan-Flagler Business School.

His latest study looks at how synthetic experts can refine results to give users exactly what they need. He created one for the study and compared it with the work of crowdsourcing amateurs, academic experts, and ChatGPT-4. The synthetic expert outdid them all, performing 66 times faster and 400 times cheaper than ChatGPT-4.

Ringel was one of four scholars who shared their work during the webinar, “AI and Innovation,” that streamed live on March 1. The webinar series is hosted by AI at Wharton . Wharton professor of operations, information and decisions, and co-director of AI at Wharton Kartik Hosanagar served as moderator.

“Clearly, AI is going to significantly transform work, the productivity of organizations, even how things work at the macroeconomic and societal level,” he said.

The webinar also featured research presentations from Dokyun Lee , information systems professor at Boston University’s Questrom School of Business; Rayna Xu , information systems and analytics professor at Miami University’s Farmer School of Business; and Léonard Boussioux , information systems and operations management professor at the University of Washington’s Foster School of Business.

Boussioux hailed AI’s ability to help humans solve the world’s most pressing problems — including the sustainable development goals set by the United Nations — if they can filter out the noise. In his co-authored paper on gen AI and crowdsourcing, he found humans and machines were equal in creating ideas for the circular economy. But humans were better at highly novel solutions, so the most valuable ideas came from a collaboration of both.

“It’s easy for AI to recombine ideas, but it’s harder to get moonshots.” — Léonard Boussioux

“This whole study is motivated by the statistical view of innovation. The best ideas are statistically rare,” Boussioux said. “It’s easy for AI to recombine ideas, but it’s harder to get moonshots.”

Xu’s co-authored paper looked at another aspect of AI’s quantity-over-quality problem by examining whether Google or ChatGPT was better at information retrieval. The study found ChatGPT excelled at pulling answers to questions but struggled with fact-checking. Depending on the task, integrating AI into traditional search engines may generate the most stable results, she said.

Xu pointed out ChatGPT’s accuracy is improving, with each iteration getting better at fact-checking.

Lee’s co-authored study introduced InnoVAET, an exploratory tool that enables interpretation, comparison, visualization, and augmented creation of multi-modal business objects. He said the tool can help business leaders understand the competitive landscape and filter good ideas for strategic action.

“Novelty is the easy part,” he said. “The challenging part is how do you make sure that things are valuable.”

More From Knowledge at Wharton

AI and the Workforce: How Gen AI Can Help Employees Flourish

AI and Machine Creativity: How Artistic Production Is Changing

How to Protect Intellectual Property in the Age of AI

Looking for more insights.

Sign up to stay informed about our latest article releases.

This paper is in the following e-collection/theme issue:

Published on 29.3.2024 in Vol 26 (2024)

Usability of Health Care Price Transparency Data in the United States: Mixed Methods Study

Authors of this article:

Original Paper

• Negar Maleki 1 , PhD   ; 
• Balaji Padmanabhan 2 , PhD   ; 
• Kaushik Dutta 1 , PhD  

1 School of Information Systems and Management, Muma College of Business, University of South Florida, Tampa, FL, United States

2 Decision, Operations & Information Technologies Department, Robert H. Smith School of Business, University of Maryland, College Park, MD, United States

Corresponding Author:

Negar Maleki, PhD

School of Information Systems and Management

Muma College of Business

University of South Florida

4202 E Fowler Avenue

Tampa, FL, 33620

United States

Phone: 1 8139742011

Email: [email protected]

Background: Increasing health care expenditure in the United States has put policy makers under enormous pressure to find ways to curtail costs. Starting January 1, 2021, hospitals operating in the United States were mandated to publish transparent, accessible pricing information online about the items and services in a consumer-friendly format within comprehensive machine-readable files on their websites.

Objective: The aims of this study are to analyze the available files on hospitals’ websites, answering the question—is price transparency (PT) information as provided usable for patients or for machines?—and to provide a solution.

Methods: We analyzed 39 main hospitals in Florida that have published machine-readable files on their website, including commercial carriers. We created an Excel (Microsoft) file that included those 39 hospitals along with the 4 most popular services—Current Procedural Terminology (CPT) 45380, 29827, and 70553 and Diagnosis-Related Group (DRG) 807—for the 4 most popular commercial carriers (Health Maintenance Organization [HMO] or Preferred Provider Organization [PPO] plans)—Aetna, Florida Blue, Cigna, and UnitedHealthcare. We conducted an A/B test using 67 MTurkers (randomly selected from US residents), investigating the level of awareness about PT legislation and the usability of available files. We also suggested format standardization, such as master field names using schema integration, to make machine-readable files consistent and usable for machines.

Results: The poor usability and inconsistent formats of the current PT information yielded no evidence of its usefulness for patients or its quality for machines. This indicates that the information does not meet the requirements for being consumer-friendly or machine readable as mandated by legislation. Based on the responses to the first part of the experiment (PT awareness), it was evident that participants need to be made aware of the PT legislation. However, they believe it is important to know the service price before receiving it. Based on the responses to the second part of the experiment (human usability of PT information), the average number of correct responses was not equal between the 2 groups, that is, the treatment group (mean 1.23, SD 1.30) found more correct answers than the control group (mean 2.76, SD 0.58; t 65 =6.46; P <.001; d =1.52).

Conclusions: Consistent machine-readable files across all health systems facilitate the development of tools for estimating customer out-of-pocket costs, aligning with the PT rule’s main objective—providing patients with valuable information and reducing health care expenditures.

Introduction

From 1970 to 2020, on a per capita basis, health care expenditures in the United States have increased sharply from US $353 per person to US $12,531 per person. In constant 2020 dollars, the increase was from US $1875 in 1970 to US $12,531 in 2020 [ 1 ]. The significant rise in health care expenses has put policy makers under enormous pressure to find ways to contain these expenditures. Price transparency (PT) in health care is 1 generally proposed strategy for addressing these problems [ 2 ] and has been debated for years [ 3 ]. Some economists believe that PT in health care will cut health care prices in the same way it has in other industries, while others argue that owing to the specific characteristics of the health care market, PT would not ameliorate rising health care costs. Price elasticity also does not typically apply in health care, since, if a problem gets severe, people will typically seek treatment regardless of cost, with the drawback that individuals learn of their health care costs after receiving treatment [ 4 ]. Complex billing processes, hidden insurer-provider contracts, the sheer quantity of third-party payers, and substantial quality differences in health care delivery are other unique aspects of health care that complicate the situation considerably.

The Centers for Medicare & Medicaid Services (CMS) mandated hospitals to post negotiated rates, including payer-specific negotiated costs, for 300 “shoppable services” beginning in January 2021. The list must include 70 CMS-specified services and an additional 230 services each hospital considers relevant to its patient population. Hospitals must include each third-party payer and their payer-specific fee when negotiating multiple rates for the same care. The data must be displayed simply, easily accessible (without requiring personal information from the patient), and saved in a machine-readable manner [ 5 ]. These efforts aim to facilitate informed patient decision-making, reduce out-of-pocket spending, and decrease health care expenditures. Former Secretary of Health and Human Services, Alex Azar, expressed a vision of hospital PT when declaring the new legislation “a patient-centered system that puts you in control and provides the affordability you need, the options and control you want, and the quality you deserve. Providing patients with clear, accessible information about the price of their care is a vital piece of delivering on that vision” [ 6 ].

Despite the legislation, it is not clear if people are actually engaging in using PT tools. For example, in 2007, New Hampshire’s HealthCost website was established, providing the negotiated price and out-of-pocket costs for 42 commonly used services by asking whether the patient is insured or their insurer and the zip code to post out-of-pocket costs in descending order. Mehrotra et al [ 7 ] examined this website over 3 years to understand how often and why these tools have mainly been used. Their analysis suggested that despite the growing interest in PT, approximately 1% of the state’s population used this tool. Low PT tool usage was also seen in other studies [ 8 - 10 ], suggesting that 3% to 12% of individuals who were offered the tool used it during the study period, and in all studies, the duration was at least 12 months. Thus, offering PT tools does not in itself lead to decreased total spending, since few people who have access to them use them to browse for lower-cost services [ 7 , 11 ].

In a recent paper, researchers addressed 1 possible reason for low engagement—lack of awareness. They implemented an extensive targeted online advertising campaign using Google Advertisements to increase awareness and assessed whether it increased New Hampshire’s PT website use. Their findings suggested that although lack of awareness is a possible reason for the low impact of PT tools in health care spending, structural factors might affect the use of health care information [ 12 ]. Individuals may not be able to exactly determine their out-of-pocket expenses from the information provided.

Surprisingly, there is little research on the awareness and usability of PT information after the current PT legislation went into effect. A recent study [ 13 ] highlighted the nonusability of existing machine-readable files for employers, policy makers, researchers, or consumers, and this paper adds to this literature by answering the question—is PT information as provided usable for patients or machines? Clearly, if it is of value to patients, it can be useful; the reason to take the perspective of machines was to examine whether this information as provided might also be useful for third-party programs that can extract information from the provided data (to subsequently help patients through other ways of presenting this information perhaps). We address this question through a combination of user experiments and data schema analysis. While there are recent papers that have also argued that PT data have deficiencies [ 13 , 14 ], ours is the first to combine user experiments with analysis of data schema from several hospitals in Florida to make a combined claim on value for patients and machines. We hope this can add to the discourse on PT and what needs to be done to extract value for patients and the health care system as a whole.

Impact of PT Tools

The impact of PT tools on consumers and health care facilities has been investigated in the literature. Some studies showed that consumers with access to PT tools are more likely to reduce forgone needed services over time. Moreover, consumers who use tools tend to find the lowest service prices [ 8 , 15 - 17 ]. A few studies investigated the impact of PT tools on the selection of health care facilities. They illustrated that some consumers tend to change health care facilities pursuing lower prices, while some others prefer to stay with expensive ones, although they are aware of some other facilities that offer lower prices [ 9 , 18 ]. Finally, some research studied the impact of PT tools on cost and showed that some consumers experienced no effect, while others experienced decreases in average consumer expenses [ 8 , 17 , 18 ]. However, the impact of PT tools on health care facilities is inconclusive, meaning different studies concluded different effects. Some stated that PT tools decrease the prices of imaging and laboratory services, while others said that although public charge disclosure lowers health care facility charges, the final prices remained unchanged [ 17 , 18 ].

Legislation Related Works

In a study, researchers considered 20 leading US hospitals to assess provided chargemasters to understand to what extent patients can obtain information from websites to determine the out-of-pocket costs [ 19 ]. Their findings showed that although all hospitals provided chargemasters on their websites, they rarely offered transparent information, making it hard for patients to determine out-of-pocket costs. Their analysis used advanced diagnostic imaging services to assess hospitals’ chargemasters since these are the most common services people look for. Mehrotra et al [ 7 ] also mentioned that the most common searches belonged to outpatient visits, magnetic resonance imaging (MRI), and emergency department visits. To this end, we used “MRI scan of the brain before and after contrast” as one of the shoppable services in our analysis. Another study examined imaging services in children’s hospitals (n=89), restricting the analysis to hospitals (n=35) that met PT requirements—published chargemaster rates, discounted cash prices, and payer-negotiated prices in a machine-readable file, and published costs for 300 common shoppable medical services in a consumer-friendly format. Their study revealed that, in addition to a broad range of imaging service charges, most hospitals lack the machine-readable file requirement [ 20 ].

Arvisais-Anhalt et al [ 21 ] identified 11 hospitals with available chargemasters in Dallas County to compare the prices of a wide range of available services. They observed significant variations for a laboratory test: partial thromboplastin time, a medication: 5 mg tablet of amlodipine, and a procedure: circumcision. Reddy et al [ 22 ] focus on New York State to assess the accessibility and usability of hospitals’ chargemasters from patients’ viewpoint. They found that 189 out of 202 hospitals had a locatable chargemaster on their home page. However, only 37 hospitals contain the Current Procedural Terminology (CPT) code, which makes those without the CPT code unusable due to the existence of many different descriptions for the same procedure; for example, an elective heart procedure had 34 entries. We add to this considerable literature by examining a subset of Florida hospitals.

In a competitive market, higher-quality goods and services require higher prices [ 23 ]. Based on this, Patel et al [ 24 ] examined the relationship between the Diagnosis-Related Group (DRG) chargemaster and quality measures. Although prior research found no convincing evidence that hospitals with greater costs also delivered better care [ 25 ], they discovered 2 important quality indicators that were linked to standard charges positively and substantially—mortality rate and readmission rates—which both are quality characteristics that are in line with economic theory. Moreover, Patel et al [ 24 ] studied the variety of one of the most commonly performed services (vaginal delivery) as a DRG code, which motivated us to select “Vaginal delivery without sterilization or D&C without CC/MCC” as another shoppable service in our analysis.

Ethical Considerations

All data used in this study, including the secondary data set obtained from hospitals’ websites and the data collected during the user experiment, underwent a thorough anonymization process. The study was conducted under protocols approved by the University of South Florida institutional review board (STUDY004145: “Effect of price transparency regulation (PTR) on the public decisions”) under HRP-502b(7) Social Behavioral Survey Consent. This approval encompassed the use of publicly available anonymized secondary data from hospitals’ websites, as well as a user experiment aimed at assessing awareness of the PT rule and the usability of hospitals’ files. No individual-specific data were collected during the experiment, which solely focused on capturing subjects’ awareness and opinions regarding the PT rule and associated files. At the onset of the experiment, participants were provided with a downloadable consent form and were allowed to withdraw their participation at any time. Survey participants were offered a US $2 reward, and their involvement was entirely anonymous.

Data Collection

According to CMS, “Starting January 1, 2021, each hospital operating in the United States will be required to provide clear, accessible pricing information online about the items and services they provide in two ways: 1- As a comprehensive machine-readable file with all items and services. 2- In a display of shoppable services in a consumer-friendly format.” As stated, files available on hospitals’ websites should be consumer-friendly, so the question of whether these files are for users arises. On the other hand, as stated, files should be machine-readable, so again the question of whether these files are for machines arises. Below we try to answer both questions in detail, respectively.

Value for Users: User Experiments

When a public announcement is disseminated, its efficacy relies on ensuring widespread awareness and facilitating practical use during times of necessity. Previous research on PT announcements has highlighted the challenges faced by patients in accurately estimating out-of-pocket expenses. However, a fundamental inquiry arises—are individuals adequately informed about the availability of tools that enable them to estimate their out-of-pocket costs for desired services? To address this, we conducted a survey to assess public awareness of PT legislation. The survey encompassed a range of yes or no and multiple-choice questions aimed at gauging participants’ familiarity with the PT rule in health care and their entitlement to obtain cost information prior to receiving a service. Additionally, we inquired about participants’ knowledge of resources for accessing pricing information and whether they were aware of the PT rule. Furthermore, we incorporated follow-up questions to ensure that the survey responses were not provided arbitrarily, thereby securing reliable and meaningful outcomes.

Moreover, considering the previously established evidence of subpar usability associated with the currently available files, we propose streamlining the existing files and developing a user-friendly and comprehensive document for conducting an A/B test. This test aims to evaluate which file better facilitates participants in accurately estimating their out-of-pocket costs. In collaboration with Florida Blue experts during biweekly meetings throughout the entire process outlined in this paper, the authors determined the optimal design for the summary table. This design, which presents prices in a more user-friendly format, enhancing overall participant comprehension, was used during the A/B testing. Participants were randomly assigned to either access the hospitals’ files or a meticulously constructed summary table, manually created in Excel, prominently displaying cost information (Please note that all files, including the hospitals’ files and our Excel file, are made available in the same format [Excel] on a cloud-based platform to eliminate any disparities in accessing the files. This ensures equitable ease of finding, downloading, and opening files, as accessing the hospitals’ files typically requires significant effort.). The experiment entailed presenting 3 distinct health-related scenarios and instructing participants to locate the price for the requested service. Subsequently, participants were asked to provide the hospital name, service price, insurer name, and insurance plan. Additionally, we sought feedback on the perceived difficulty of finding the requested service and their priority for selecting hospitals [ 26 ], followed by Likert scale questions to assess participants’ evaluation of the provided file’s efficacy in facilitating price retrieval.

The experiments were conducted to investigate the following questions: (1) Are the individuals aware of the PT legislation? and (2) Is the information provided usable for patients? To evaluate the usability of files found on websites, we selected 2 prevalent services based on existing literature and 2 other services recommended as high-demand ones by Florida Blue experts, Table 1 . Furthermore, meticulous efforts were made to ensure that both the control and treatment groups encountered identical circumstances, thus allowing for a systematic examination of the disparities solely attributable to variations in data representation.

a DRG: Diagnosis-Related Group.

b D&C: dilation and curettage.

c CC/MCC: complication or comorbidity/major complication or comorbidity.

d CPT: Current Procedural Terminology.

e MRI: magnetic resonance imaging.

Participants

A total of 67 adults (30 female individuals; mean 41.43, SD 12.39 years) were recruited on the Amazon Mechanical Turk platform, with no specific selection criteria other than being located in the United States.

We focused on 75 main hospitals (ie, the main hospital refers to distinguish a hospital from smaller clinics or specialized medical centers within the same health system) in the state of Florida. When we searched their websites for PT files (machine-readable files), only 89% (67/75) of hospitals included machine-readable files. According to the PT legislation, these files were supposed to contain information about 300 shoppable services. However, only 58% (39/67) of hospitals included information such as insurer prices in their files. Therefore, for the rest of the analysis, we only included the 39 hospitals that have the required information in their machine-readable files on their websites. We created an Excel file that included those 39 hospitals along with the 4 services—CPT 45380, 29827 and 70553 and DRG 807—mentioned in the literature ( Table 1 ) for 4 popular (suggested by Florida Blue experts) commercial carriers (Health Maintenance Organization [HMO] or Preferred Provider Organization [PPO] plans)—Aetna, Florida Blue, Cigna, and UnitedHealthcare.

Participants were recruited for the pilot and randomly assigned by the Qualtrics XM platform to answer multiple-choice questions and fill in blanks based on the given scenarios. First, participants responded to questions regarding the awareness of PT and then were divided into 2 groups randomly to answer questions regarding the usability of hospital-provided PT information. One group was assigned hospitals’ website links (control group), while the other group was given an Excel file with the same information provided in files on hospitals’ websites, but in a manner that was designed to allow easier comparison of prices across hospitals ( Multimedia Appendix 1 ). Participants were given 3 scenarios that asked them to find a procedure’s price based on their hospital and insurer selection to compare hospital-provided information with Excel. We provide some examples of hospitals’ files and our Excel file in Multimedia Appendix 1 and the survey experiment questions in Multimedia Appendix 2 .

Value for Machines: Schema Integration—Machine-Readable Files Representation

Through meticulous investigation of machine-readable files from 39 hospitals, we discovered that these files may vary in formats such as CSV or JSON, posing a challenge for machines to effectively manage the data within these files. Another significant obstacle arises from the lack of uniformity in data representation across these files, rendering them unsuitable for machine use without a cohesive system capable of processing them collectively. Our analysis revealed that hospitals within a single health system exhibit consistent data representation, although service prices may differ (we include both the same and different chargemaster prices in our study), while substantial disparities in data representation exist between hospitals affiliated with different health systems.

Moving forward, we will use the terms “data representation” and “schema” interchangeably, with “schema” denoting its database management context. In this context, a schema serves as a blueprint outlining the structure, organization, and relationships of data within a database system. It encompasses key details such as tables, fields, data types, and constraints that define the stored data. To systematically illustrate schema differences among hospitals associated with different health systems, we adopted the methodology outlined in reference [ 27 ] for schema integration, which offers a valid approach for comparing distinct data representations. The concept of schema integration encompasses four common categories: (1) identical: hospitals within the same health system adhere to this concept as their representations are identical; (2) equivalent: while hospitals in health system “A” may present different representations from those in health system “B,” they possess interchangeable columns; (3) compatible: in cases where hospitals across different health systems are neither identical nor equivalent, the modeling constructs, designer perception, and integrity constraints do not contradict one another; and (4) incompatible: in situations where hospitals within different health systems demonstrate contradictory representations, distinct columns exist for each health system due to specification incoherence.

Our analysis focused on health systems in Florida that encompassed a minimum of 4 main hospitals, using the most up-to-date data available on their respective websites. Within this scope, we identified 8 health systems with at least 4 main hospitals, of which 88% (7/8) of health systems had published machine-readable files on their websites. Consequently, our analysis included 65% (36/55) of hospitals that possessed machine-readable files available on their websites. To facilitate further investigation by interested researchers, we have made the analyzed data accessible on a cloud-based platform. During our analysis, we meticulously extracted the schema of each health system by closely scrutinizing the hospitals associated with each health system, capturing key details such as tables, fields, and data types. Subsequently, we compiled a comprehensive master field name table trying to have the same data type and field names that make it easier for machines to retrieve information. We elaborate on the master field names table in greater detail within the results section.

Value for Users

Question 1 (pt awareness).

Based on the responses, it is evident that participants need to be made aware of the PT legislation. Among the participants, 64% (49/76) reported that they had not heard about the legislation. However, they believe it is important to know the service price before receiving it—response charts are provided in Multimedia Appendix 3 .

Question 2 (Human Usability of PT Information)

Based on the responses to scenarios, the average number of correct responses is not equal between the 2 groups, that is, the treatment group (mean 1.23, SD 1.30) found more correct answers than the control group (mean 2.76, SD 0.58; t 65 =6.46; P <.049; d =1.52). The t tests (2-tailed) for the other questions in the experiment are in Multimedia Appendix 4 .

These suggest that current files on hospitals’ websites are not consumer-friendly, and participants find it challenging to estimate out-of-pocket costs for a desired service. For this reason, in addition to making the files easier to use, this information should also include thorough documentation that explains what each column represents, up to what amount an insurer covers for a specific service, or the stated price covers up to how many days of a particular service, that is, “contracting method.” For example, based on consulting with one of the senior network analysts of Florida Blue, some prices for a service like DRG 807 are presented as per diem costs, and based on the current information on these files, it cannot be recognizable without having comprehensive documentation for them.

Value for Machines

After carefully reviewing all machine-readable file schemas, we create a master field name table, including the available field names in machine-readable files ( Table 2 ). According to Table 2 , the first column represents master field names that we came up with, and the following columns each represent hospitals within a health system. The “✓” mark shows that hospitals within a health system have identical field names as we consider as master field names and the “written” cells show equivalent field names, meaning that hospitals within that health system use different field names—we write what they use in their representation—while the content is equivalent to what we select as the master field name. The “❋” mark means that although hospitals within health system #2 provide insurer names and plans in their field names, some codes make those columns unusable for machines to recognize them the same as master field names. We also include the type of field names for all representations in parentheses.

a As noted previously, since we focus on the health system level instead of the hospital level, our schema does not have hospital-level information; however, it would be beneficial to add hospital information to the table.

b ✓: it means the given master field name in that row appears on the given health system file in that column.

c str: shows “string” as the data type.

d int: shows “integer” as the data type.

e CPT: Current Procedural Terminology.

f HCPCS: Health care Common Procedure Coding System.

g Not applicable.

h Apr: all patients refined.

i DRG: Diagnosis-Related Group.

j Ms: Medicare severity.

k CDM: charge description master.

l UB: uniform billing.

m float: it shows “float” as the data type.

n ❋: it means that although hospitals within health system #2 provide insurer names and plans in their field names, some codes make those columns unusable for machines to recognize them the same as master field names.

We did reverse engineering and drew entity-relationship diagrams (ERDs) for each hospital based on their data representation. However, as hospitals within the same health system have the same ERDs, we only include 1 ERD for each health system ( Figure 1 ). According to Figure 1 , although hospitals have tried to follow an intuitive structure, we can still separate them into three groups: (1) group I: all hospitals within this group have several columns for different insurers. As shown in the ERDs, we decided to have a separate entity, called “Insurance” for this group; (2) group II: all hospitals within this group have many sheets, and each sheet belongs to a specific insurer with a specific plan. As shown in the ERDs, we decided to create an “Insurance_Name” entity for this group’s ERD to show the difference in data representation; and (3) group III: all hospitals within this system have a “payer” column which includes the names of insurers without their plans. As shown in the ERDs, we decided to put this column as an attribute in the “Service” entity, and do not have an “Insurance” entity for this group’s ERD.

In conclusion, although most hospitals have adopted group I logic for data representation, for full similarity, a standard representation with the same intuitive field names (like what we suggest as the master field name; Table 2 ) should be proposed so that it can cover all systems’ data representations and be used as machine-readable file, for at least machine benefits. Mainly, standardization in the format and semantics of the provided data can help substantially in making the data more machine friendly.

Comparison With New CMS Guidelines

Recently, CMS has published guidelines regarding the PT legislation [ 28 ]. The most recent CMS guideline is a step forward in ensuring standardization but is still only recommended and is not mandatory. These guidelines exhibit overlaps with our fields in Table 2 , with slight differences attributed to granularities. Our observation reveals that hospitals within the same health system adopt a uniform schema. Therefore, our suggested schema operates on the granularity of health systems rather than individual hospitals.

The recent CMS guidelines allocate 24% (6/25) of field names specifically to hospital information, encompassing details such as “Hospital Name,” “Hospital File Date,” “Version,” “Hospital Location,” “Hospital Financial Aid Policy,” and “Hospital Licensure Information.” These details, absent in current hospital files, are crucial for informed decision-making. As noted previously, since we focus on the health system level instead of the hospital level, our schema does not have hospital-level information; however, it would be beneficial to add hospital information to the tables.

Our analysis reveals that the 11 field names in Table 2 align with the field names in the new CMS guidelines, demonstrating a substantial overlap of 58% (11/19). The corresponding CMS field names (compatible with our schema) include “Item or Service Description (Description or CDM Service Description),” “Code (Code),” “Code Type (Code Type),” “Setting (Patient Class),” “Gross Charge (Gross Charge),” “Discounted Cash Price (Discounted Cash Price),” “Payer Name (Insurer Name),” “Plan Name (Insurer Plan),” “Payer Specific Negotiated Charge: Dollar Amount (Price),” “De-identified Minimum Negotiated Charge (Min Negotiated Rate),” and “De-identified Maximum Negotiated Charge (Max Negotiated Rate).” Additionally, both our schema and the new CMS guidelines propose data types for each field name.

In our schema, which represents current hospitals’ files, there are 5 field names absent in the new CMS guidelines “Revenue Description,” “Revenue Code,” “Package/Line Level,” “Procedure ID,” and “Self Pay.” Conversely, the new CMS guidelines introduce 8 additional field names “Billing Class,” “Drug Unit of Measurement,” “Drug Type of Measurement,” “Modifiers,” “Payer Specific Negotiated Charge: Percentage,” “Contracting Method,” “Additional Generic Notes,” and “Additional Payer-Specific Notes.” We regard these new field names as providing further detailed information and enhancing consumer decision-making. If hospitals within a health system adopt consistent formats and can map their formats to the new CMS guidelines clearly in a mapping document they also provide, this can be more useful than the current optional guideline that is suggested.

In summary, since our analysis is based on the current data schema that hospitals have in place, we believe the schema we put out is easier to implement with minimal change to what the hospitals are currently doing. However, given the recent CMS guidelines, we recommend adding 8 additional fields as well as hospital-specific information.

Implications

The PT legislation aims to enable informed decision-making, reduce out-of-pocket expenses, and decrease overall health care expenditures. This study investigates the usage of current files by individuals and machines. Our results, unfortunately, suggest that PT data—as currently reported—appear to be neither useful for patients nor machines, raising important questions as to what these appear to be achieving today. Moreover, the findings indicate that even individuals with basic computer knowledge struggle with the usability of these files, highlighting the need for significant revisions to make them consumer-friendly and accessible to individuals of all technical proficiency levels. Additionally, inconsistencies in data representation between hospitals affiliated with different health systems pose challenges for machines, necessitating schema design improvements and the implementation of a standardized data representation. By addressing these concerns, PT legislation can achieve consistency and enhance machine readability, thus improving its effectiveness in promoting informed decision-making and reducing health care costs.

Although the official announcement of PT legislation is recent, prior studies [ 15 - 17 ] have attempted to evaluate the usability of PT, while subsequent studies [ 19 - 22 ] have examined the effectiveness of PT tools following the announcement. However, despite the introduction of PT rules, it appears that the usability of these files has not undergone significant improvements, indicating the necessity for proactive measures from responsible executives to ensure the effectiveness of this legislation. Our analysis of this matter emphasizes 2 primary factors—a lack of awareness among stakeholders and the challenges associated with using files due to inconsistencies in their format and representation.

As of April 2023, the CMS has issued over 730 warning notices and 269 requests for Corrective Action Plans. A total of 4 hospitals have faced Civil Monetary Penalties for noncompliance, and these penalties are publicly disclosed on the CMS website. The remaining hospitals subjected to comprehensive compliance reviews have either rectified their deficiencies or are actively engaged in doing so. While we acknowledge these efforts to comply with PT rules, our research revealed a notable disparity in data representation among hospitals affiliated with different health systems. Consequently, we focused on schema design and proposed the implementation of a master field name that encompasses a comprehensive data representation derived from an analysis of 36 hospitals. Standardizing the data representation across all health systems’ machine-readable files will effectively address concerns about consistency. Therefore, significant modifications are required for the PT legislation to enhance machine readability and provide clearer guidance on the design and structure of the files’ schema. If the hospital-provided information is consistent and of high quality, PT tools provided by health insurers may be able to estimate an individual’s total expenses more accurately.

Limitations

Our objective was to have an equal number in both groups. However, in the case of the group tasked with obtaining information from the hospitals’ websites, most did not finish the task and dropped out without completing it. This occurred because the task of retrieving the cost from the hospitals’ websites in its current form is complex, as indicated by feedback from some participants. Only 19% (13/67) completed the task in that group (control group). Although this is a limitation of the study, it also highlights the complexity of obtaining cost information from hospitals’ websites in the current form. In the treatment group, 81% (54 out of 67) of participants completed the task of retrieving the data, and the completion percentage was much higher.

Conclusions

Due to the poor usability and inconsistency of the formats, we, unfortunately, did not find evidence that the PT rule as implemented currently is useful to consumers, researchers, or policy makers (despite the legislation’s goals that files are “consumer-friendly” and “machine-readable”). As 1 solution, we suggest a master field name for the data representation of machine-readable files to make them consistent, at least for the machines. Building tools that enable customers to estimate out-of-pocket costs is facilitated by having consistent machine-readable files across all health systems, which can be considered as future work for researchers and companies to help the PT rule reach its main goal, which is providing useful information for patients and reducing health care expenditures. In addition, another worthwhile approach to reducing some of the exorbitant health care costs in the United States would be to integrate clinical decision support tools into the providers’ workflow, triggered by orders for medications, diagnostic testing, and other billable services. In this regard, Bouayad et al [ 29 ] conducted experiments with physicians to demonstrate that PT, when included as part of the system they interact with, such as clinical decision support integrated into electronic health record systems, can significantly aid in cost reduction. This is a promising direction for practice but needs to be implemented carefully to avoid unanticipated consequences, such as scenarios where cost is incorrectly viewed as a proxy for quality, or where the use of this information introduces new biases for physicians and patients.

Conflicts of Interest

None declared.

Example of Excel format of hospitals’ files and our created Excel file.

Survey questions and experiment scenarios.

Participants’ responses chart regarding price transparency awareness.

The t test analysis regarding human usability of price transparency information based on participants’ responses.

• McGough M, Winger A, Rakshit S, Amin K. How has U.S. spending on healthcare changed over time? Health System Tracker. 2022. URL: https://www.healthsystemtracker.org/chart-collection/u-s-spending-healthcare-changed-time/ [accessed 2024-03-13]
• Christensen HB, Floyd E, Maffett M. The only prescription is transparency: the effect of charge-price-transparency regulation on healthcare prices. Manag Sci. 2020;66(7):2861-2882. [ CrossRef ]
• Muir MA, Alessi SA, King JS. Clarifying costs: can increased price transparency reduce healthcare spending? UC Hastings Research Paper No. 38 (SSRN). Feb 26, 2013.:319-367. [ FREE Full text ] [ CrossRef ]
• Reinhardt UE. Health care price transparency and economic theory. JAMA. 2014;312(16):1642-1643. [ CrossRef ] [ Medline ]
• CY 2020 hospital Outpatient Prospective Payment System (OPPS) policy changes: hospital price transparency requirements (CMS-1717-F2). CMS.gov. 2020. URL: https://tinyurl.com/mrafxtvd [accessed 2024-03-13]
• Secretary Azar statement on proposed rule for hospital price transparency. HHS.gov. 2020. URL: https://tinyurl.com/yc4dx2vx [accessed 2024-03-13]
• Mehrotra A, Brannen T, Sinaiko AD. Use patterns of a state health care price transparency web site: what do patients shop for? Inquiry. 2014;51:0046958014561496. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Desai S, Hatfield LA, Hicks AL, Sinaiko AD, Chernew ME, Cowling D, et al. Offering a price transparency tool did not reduce overall spending among California public employees and retirees. Health Aff (Millwood). 2017;36(8):1401-1407. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sinaiko AD, Joynt Maddox KE, Rosenthal MB. Association between viewing health care price information and choice of health care facility. JAMA Intern Med. 2016;176(12):1868-1870. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Desai S, Hatfield LA, Hicks AL, Chernew ME, Mehrotra A. Association between availability of a price transparency tool and outpatient spending. JAMA. 2016;315(17):1874-1881. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sinaiko AD, Rosenthal MB. Examining a health care price transparency tool: who uses it, and how they shop for care. Health Aff (Millwood). 2016;35(4):662-670. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Desai SM, Shambhu S, Mehrotra A. Online advertising increased New Hampshire residents' use of provider price tool but not use of lower-price providers. Health Aff (Millwood). 2021;40(3):521-528. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kona M, Corlette S. Hospital and insurer price transparency rules now in effect but compliance is still far away. Health Affairs Forefront. 2022. URL: hhttps://tinyurl.com/3x6ymxf2 [accessed 2024-03-13]
• Wheeler C, Taylor R. New year, new CMS price transparency rule for hospitals. Health Affairs Forefront. 2021. URL: https://www.healthaffairs.org/content/forefront/new-year-new-cms-price-transparency-rule-hospitals [accessed 2024-03-13]
• Chernew M, Cooper Z, Larsen-Hallock E, Morton FS. Are health care services shoppable? Evidence from the consumption of lower-limb MRI scans. National Bureau of Economic Research. 2021. URL: https://www.nber.org/papers/w24869 [accessed 2024-03-13]
• Mehrotra A, Dean KM, Sinaiko AD, Sood N. Americans support price shopping for health care, but few actually seek out price information. Health Aff (Millwood). 2017;36(8):1392-1400. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Brown ZY. Equilibrium effects of health care price information. Rev Econ Stat. 2019;101(4):699-712. [ FREE Full text ] [ CrossRef ]
• Wu SJ, Sylwestrzak G, Shah C, DeVries A. Price transparency for MRIs increased use of less costly providers and triggered provider competition. Health Aff (Millwood). 2014;33(8):1391-1398. [ CrossRef ] [ Medline ]
• Glover M, Whorms DS, Singh R, Almeida RR, Prabhakar AM, Saini S, et al. A radiology-focused analysis of transparency and usability of top U.S. hospitals' chargemasters. Acad Radiol. 2020;27(11):1603-1607. [ CrossRef ] [ Medline ]
• Hayatghaibi SE, Alves VV, Ayyala RS, Dillman JR, Trout AT. Transparency and variability in pricing for pediatric outpatient imaging in US children's hospitals. JAMA Netw Open. 2022;5(3):e220736. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Arvisais-Anhalt S, McDonald S, Park JY, Kapinos K, Lehmann CU, Basit M. Survey of hospital chargemaster transparency. Appl Clin Inform. 2021;12(2):391-398. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Reddy S, Daly G, Baban S, Kadesh A, Block AE, Grimes CL. Accessibility and usability of hospital chargemasters in New York state. J Gen Intern Med. 2022;37(8):2130-2131. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Robinson JC. Hospital quality competition and the economics of imperfect information. Milbank Q. 1988;66(3):465-481. [ Medline ]
• Patel KN, Mazurenko O, Ford E. Analysis of hospital quality measures and web-based chargemasters, 2019: cross-sectional study. JMIR Form Res. 2021;5(8):e26887. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Batty M, Ippolito B. Mystery of the chargemaster: examining the role of hospital list prices in what patients actually pay. Health Aff (Millwood). 2017;36(4):689-696. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Muhlestein DB, Wilks CEA, Richter JP. Limited use of price and quality advertising among American hospitals. J Med Internet Res. 2013;15(8):e185. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Batini C, Lenzerini M, Navathe SB. A comparative analysis of methodologies for database schema integration. ACM Comput Surv. 1986;18(4):323-364. [ FREE Full text ] [ CrossRef ]
• Voluntary hospital price transparency machine-readable file sample format data dictionary (version 1.1). CMS.gov. URL: https:/​/www.​cms.gov/​files/​document/​hospital-price-transparency-machine-readable-data-dictionary-tall.​pdf [accessed 2024-03-13]
• Bouayad L, Padmanabhan B, Chari K. Can recommender systems reduce healthcare costs? the role of time pressure and cost transparency in prescription choice. MIS Q. 2020;44(4):1859-1903. [ CrossRef ]

Abbreviations

Edited by S He; submitted 07.07.23; peer-reviewed by KN Patel, R Marshall, G Deckard; comments to author 03.12.23; revised version received 21.01.24; accepted 26.02.24; published 29.03.24.

©Negar Maleki, Balaji Padmanabhan, Kaushik Dutta. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 29.03.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

share this!

March 28, 2024

This article has been reviewed according to Science X's editorial process and policies . Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

peer-reviewed publication

trusted source

New research reveals full diversity of killer whales as two species come into view on Pacific Coast

by NOAA Headquarters

Scientists have resolved one of the outstanding questions about one of the world's most recognizable creatures, identifying two well-known killer whales in the North Pacific Ocean as separate species. The research is published in the journal Royal Society Open Science .

Killer whales are one of the most widespread animals on Earth. They have long been considered one worldwide species known scientifically as Orcinus orca, with different forms in various regions known as "ecotypes."

However, biologists have increasingly recognized the differences between resident and Bigg's killer whales. Resident killer whales maintain tight-knit family pods and prey on salmon and other marine fish. Bigg's killer whales roam in smaller groups, preying on other marine mammals such as seals and whales. (Killer whales actually belong to the dolphin family.) Bigg's killer whales, sometimes called transients, are named for Canadian scientist Michael Bigg, the first to describe telltale differences between the two types.

He noted in the 1970s that the two animals did not mix with each other even as they occupied many of the same coastal waters, which is often a sign of different species.

The finding recognizes the accuracy of the listing of Southern Resident killer whales as a Distinct Population Segment warranting protection under the Endangered Species Act in 2005. At the time, NOAA described the distinct population segment as part of an unnamed subspecies of resident killer whales in the North Pacific.

Now a team of scientists from NOAA Fisheries and universities have assembled genetic, physical, and behavioral evidence. The data distinguish two of the killer whale ecotypes of the North Pacific Coast—residents and Bigg's—as separate species.

"We started to ask this question 20 years ago, but we didn't have much data, and we did not have the tools that we do now," said Phil Morin, an evolutionary geneticist at NOAA Fisheries' Southwest Fisheries Science Center and lead author of the new paper. "Now we have more of both, and the weight of the evidence says these are different species."

Genetic data from previous studies revealed that the two species likely diverged more than 300,000 years ago and come from opposite ends of the killer whale family tree. That makes them about as genetically different as any killer whale ecotypes around the globe. Subsequent studies of genomic data confirm that they have evolved as genetically and culturally distinct groups, which occupy different niches in the same Northwest marine ecosystem.

"They're the most different killer whales in the world, and they live right next to each other and see each other all the time," said Barbara Taylor, a former NOAA Fisheries marine mammal biologist who was part of the science panel that assessed the status of Southern Residents. "They just do not mix."

Recognizing new species

The Taxonomy Committee of the Society of Marine Mammalogy will determine whether to recognize the new species in its official list of marine mammal species . The committee will likely determine whether to accept the new designations at its next annual review this summer.

The scientists proposed scientific names for the new species based on their earliest published descriptions in the 1800s. Neither will keep the ubiquitous worldwide moniker, orca. The team proposed to call resident killer whales Orcinus ater, a Latin reference to their dominant black coloring. Bigg's killer whales would be called Orcinus rectipinnus, a combination of Latin words for erect wing, probably referring to their tall, sharp dorsal fin.

Both species names were originally published in 1869 by Edward Drinker Cope, a Pennsylvania scientist known more for unearthing dinosaurs than studying marine mammals. He was working from a manuscript that California whaling captain Charles Melville Scammon had sent to the Smithsonian Institution describing West Coast marine mammals, including the two killer whales. While Cope credited Scammon for the descriptions, Scammon took issue with Cope for editing and publishing Scammon's work without telling him. (See accompanying story.)

The Smithsonian Institution had shared Scammon's work with Cope, and a Smithsonian official later apologized to Scammon for what he called "Cope's absurd blunder."

Species reflect ecosystem

The contested question of whether Southern Residents were distinct enough to merit endangered species protections initially drove much of the research that helped differentiate the two species, said Eric Archer, who leads the Marine Mammal Genetics Program at the Southwest Fisheries Science Center and is a co-author of the new research paper.

The increasing processing power of computers has made it possible to examine killer whale DNA in ever finer detail. He said the findings not only validate protection for the animals themselves, but also help reveal different components of the marine ecosystems the whales depend on.

"As we better understand what makes these species special, we learn more about how they use the ecosystems they inhabit and what makes those environments special, too," he said.

The new research synthesizes the earliest accounts of killer whales on the Pacific Coast with modern data on physical characteristics. They also use aerial imaging (called photogrammetry), and measurement and genetic testing of museum specimens at the Smithsonian and elsewhere.

While the two species look similar to the untrained eye, the evidence demonstrates they are very different species. They use different ecological niches, such as specializing in different prey, said Kim Parsons, a geneticist at the NOAA Fisheries Northwest Fisheries Science Center in Seattle and co-author of the new research.

Recent research with drones and precise aerial photos has helped differentiate Bigg's killer whales as longer and larger. This might better equip them to go after large marine mammal prey. The smaller size of residents is likely better suited to deep dives after their salmon prey, said John Durban, an associate professor at Oregon State University's Marine Mammal Institute. He leads killer whale drone research with Holly Fearnbach, a researcher at SR³.

The different prey of the two species may also help explain their different trajectories. Southern Residents are listed as endangered in part because of the scarcity of their salmon prey. Bigg's killer whales, by contrast, have multiplied while feeding on plentiful marine mammals, including California sea lions.

While killer whales represent some of the most efficient predators the world has ever seen, Durban said science is still unraveling the diversity among them. The identification of additional killer whale species is likely to follow. One leading candidate may be "Type D" killer whales identified in the Southern Ocean around Antarctica.

Other killer whales in Antarctica also look very different from the best-known black and white killer whales. This reflects a wider diversity within the species, said Durban, who has used drones to study killer whales around the world. "The more we learn," he said, "the clearer it becomes to me that at least some of these types will be recognized as different species in due course."

Journal information: Royal Society Open Science

Provided by NOAA Headquarters

Explore further

Feedback to editors

Romania center explores world's most powerful laser

15 hours ago

A cosmic 'speed camera' just revealed the staggering speed of neutron star jets in a world first

Mar 30, 2024

Saturday Citations: 100-year-old milk, hot qubits and another banger from the Event Horizon Telescope project

Curiosity rover searches for new clues about Mars' ancient water

Study says since 1979 climate change has made heat waves last longer, spike hotter, hurt more people

Scientist taps into lobsters' unusual habits to conquer the more than 120-year quest to farm them

Mar 29, 2024

Blind people can hear and feel April's total solar eclipse with new technology

Mapping the best route for a spacecraft traveling beyond the sun's sphere of influence

Researchers outline new approach in search for dark matter through future DUNE research project

Researchers reveal evolutionary path of important proteins

Relevant physicsforums posts, what do large moles on the body indicate, avian flu - a new study led by a team from the university of maryland.

Mar 27, 2024

Are all biological catabolic reactions exergonic?

Mar 20, 2024

A First of Its Kind: A Calcium-based signal in the Human Brain

Mar 18, 2024

Biological culture and cultural biology

Mar 17, 2024

Potentially fatal dog parasite found in the Colorado River

Mar 15, 2024

More from Biology and Medical

Related Stories

Killer whales use specialized hunting techniques to catch marine mammals in the open ocean

Shark-bitten orcas in the Northeastern Pacific could be a new population of killer whale

Orcas covered in scars left by 'cookiecutter sharks' may be new population, study says

Mar 23, 2024

As chinook salmon get thinner and fewer, southern resident killer whales struggle to find enough food

Sep 21, 2022

Why are killer whales harassing and killing porpoises without eating them?

Sep 28, 2023

Study shows links between Australia's false killer whales and endangered groups from Hawaii

Mar 16, 2023

Recommended for you

Tiny orchid flowers pollinated by tiny flies

Chickadees have unique neural 'barcodes' for memories of stashing away food

Discovering van Gogh in the wild: Scientists unveil a new gecko species

Mar 28, 2024

Supergene research solves the mystery of tiny ant queens

Small birds spice up the already diverse diet of spotted hyenas in Namibia

Let us know if there is a problem with our content.

Use this form if you have come across a typo, inaccuracy or would like to send an edit request for the content on this page. For general inquiries, please use our contact form . For general feedback, use the public comments section below (please adhere to guidelines ).

Please select the most appropriate category to facilitate processing of your request

Thank you for taking time to provide your feedback to the editors.

Your feedback is important to us. However, we do not guarantee individual replies due to the high volume of messages.

E-mail the story

Your email address is used only to let the recipient know who sent the email. Neither your address nor the recipient's address will be used for any other purpose. The information you enter will appear in your e-mail message and is not retained by Phys.org in any form.

Newsletter sign up

Get weekly and/or daily updates delivered to your inbox. You can unsubscribe at any time and we'll never share your details to third parties.

More information Privacy policy

Donate and enjoy an ad-free experience

We keep our content available to everyone. Consider supporting Science X's mission by getting a premium account.

E-mail newsletter

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Bras Pneumol
• v.42(6); Nov-Dec 2016

Developing research questions that make a difference

Desenvolvendo perguntas do estudo que fazem a diferença, cecilia maria patino.

1 Department of Preventive Medicine, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.

2 Methods in Epidemiologic, Clinical and Operations Research (MECOR) Program, American Thoracic Society, New York, NY, USA, and Asociación Latinoamericana de Tórax, Montevideo, Uruguay.

Juliana Carvalho Ferreira

3 Divisão de Pneumologia, Instituto do Coração - InCor - Hospital das Clínicas, Faculdade de Medicina, Universidade de São Paulo, São Paulo, Brasil.

A clinical research question is defined as an uncertainty about a health problem that points to the need for meaningful understanding and deliberate investigation. 1 For clinicians interested in conducting high-quality clinical research, it is essential to recognize the fact that the research process starts with developing a question about a specific health-related area of interest. This is important because once the research question is defined, it has an impact on every remaining component of the research process, including generating the hypothesis and defining the appropriate study design, as well as the study population, study variables, and statistical approach. However, conceiving a sound research question is not an easy task; it requires having a particular set of personal skills and utilizing structured approaches.

DEVELOPING AND WRITING A RESEARCH QUESTION

Developing a research question starts by identifying a clinical problem that is important to patients, being related to managing and ultimately improving their health. The process requires clinician scientists to be curious about and attentive to day-to-day practice outcomes, as well as to be avid readers of the scientific literature, to participate in scientific activities (e.g., journal clubs), and to have access to a scientific mentor or collaborators interested in clinical research.

The research question itself should meet certain criteria, as summarized by the acronym FINGER, which stands for Feasible, Interesting, Novel, Good (for your career), Ethical, and Relevant ( Chart 1 ). 1 We recommend going through the FINGER criteria systematically and discussing all issues with a mentor or colleague before writing the study protocol and conducting a study that will answer the proposed research question.

Once the research question has been defined, it should be written out in such a way that the answer can be expressed as either a number, typical of descriptive research questions (e.g., a prevalence related to disease burden, such as "What is the prevalence of asthma among favela residents in Brazil?"), or as a yes or no, typical of studies about associations between exposures and outcomes (e.g., "Is living in a favela in Brazil associated with increased mortality among adults with asthma?"). In addition, if the researcher has a hypothesis about the answer to the research question, 1 it is important that it be written out using a comprehensive approach, as summarized by the acronym PICOT, which stands for Population (the population to be included in the study), Intervention (treatment applied to participants in the treatment arm), Comparison (treatment applied to the control group), Outcome (the primary outcome variable), and Time (follow-up time to measure the outcome). 2

INVESTING THE TIME AND EFFORT TO COME UP WITH A HIGH-QUALITY, WELL-WRITTEN RESEARCH QUESTION IS WORTH IT!

As clinician scientists who train clinicians to become successful researchers, we cannot emphasize enough the importance of investing one's time wisely to develop a high-quality research question. Researchers who conceive and clearly state a research question about an important health-related problem are at an advantage because they are more likely to convince key individuals to provide them with the necessary resources and support to carry out the study, as well as to increase the reporting quality of the paper to be published. 3

  Nigerian School Library Journal Journal / Nigerian School Library Journal / Vol. 22 No. 2 (2023) / Articles (function() { function async_load(){ var s = document.createElement('script'); s.type = 'text/javascript'; s.async = true; var theUrl = 'https://www.journalquality.info/journalquality/ratings/2404-www-ajol-info-nslj'; s.src = theUrl + ( theUrl.indexOf("?") >= 0 ? "&" : "?") + 'ref=' + encodeURIComponent(window.location.href); var embedder = document.getElementById('jpps-embedder-ajol-nslj'); embedder.parentNode.insertBefore(s, embedder); } if (window.attachEvent) window.attachEvent('onload', async_load); else window.addEventListener('load', async_load, false); })();

Article sidebar, article details, main article content, knowledge management and work performance of academic librarians in selected academic institutions in kogi state, nigeria, olaronke o. fagbola, kudirat ize.

The study investigated the impact of knowledge management on the work performance of academic librarians in selected institutions in Kogi State, Nigeria, revealing that knowledge management positively influences librarians' performance in various aspects such as library routines, communication skills, resourcefulness, creativity, and enthusiasm for learning. The study adopted descriptive survey research design. Purposive sampling technique was used to select five (5) academic institutions in Kogi State, Nigeria. The population for the study was sixty-seven (67) academic librarians in the five (5) selected academic Institutions in Kogi State, Nigeria. Total enumeration technique was. Questionnaire was used for the data collection. Five (5) research questions were answered; data were analyzed using Statistical Package for Social Sciences (SPSS) version 25 Software. The findings highlight the significance of promoting knowledge management practices and creating a supportive environment within academic libraries to motivate librarians' engagement in knowledge sharing and application. 

AJOL is a Non Profit Organisation that cannot function without donations. AJOL and the millions of African and international researchers who rely on our free services are deeply grateful for your contribution. AJOL is annually audited and was also independently assessed in 2019 by E&Y.

Your donation is guaranteed to directly contribute to Africans sharing their research output with a global readership.

• For annual AJOL Supporter contributions, please view our Supporters page.

Journal Identifiers