spanish translation for research paper

• My Invoices
• Coupons/Discounts
• Referral Rewards
• English Language Editing
• Scientific Editing

Academic Translation

• Manuscript Formatting
• Research Data Support
• For Organizations
• Get a Price Quote
• Get Started

This Academic Translation service is brought to you by American Journal Experts (AJE). Get academic documents translated into English from Simplified Chinese, Portuguese, and Spanish. Upload your document for Academic Translation

Translation into polished English is available for documents in Simplified Chinese, Portuguese, and Spanish. You are matched with translators who are experts in your field of study and who will accurately translate field-specific terms. Once your document has been translated, it will be edited by AJE English Editing to ensure that the final document is polished and sounds natural in English.

Testimonials

I am very satisfied with AJE translation service. It is quick and efficient and return a high quality revision. Referee´s complains ended since I begun the revision of my papers with AJE.

Horacio Dottori, Brazil

American Journal Experts are very quick, have a very good price and do an excellent job. I have proved many companies of translation and this is one of the best. I think the journals reviewers will never say us more that the English of our manuscripts should be revised for a native!

David González-Cutre, Spain

Our Guarantee

All complete manuscripts submitted for the Academic Translation service are eligible to receive an AJE Editing Certificate.

Academic Translation includes unlimited free re-editing of your translated text and one additional free translation of up to 1,500 new words related to the original document. AJE guarantees that if you are not satisfied with the translation, or if a journal says that the English in your paper needs improvement, your document will be re-edited for free until English is no longer a barrier to publication.

Do you have translators in my field?

We have translators in every major field. If you don’t see your area of study on our list , please contact us, and we will let you know if any qualified translators are available.

How are AJE translation services different from traditional translators?

Unlike traditional translation companies, we use translators who are active researchers in your area of study. This ensures that they know the jargon of your field in both English and your native language, guaranteeing the most accurate translation possible. AJE also includes a quality review for every translation by one of our experienced Academic Translation Advisors. We believe this additional step is critical to providing the most accurate translation of your important research. Moreover, our translation service includes English language editing by fluent speakers in your field, so you can be assured of the quality of the language in your final paper. Finally, our translation service includes unlimited English re-editing, so if you need to make changes to your paper later, we will re-edit it for free.

How long will it take to have my paper re-edited?

Because a Translation re-edit may involve a variety of steps (e.g., Translation, Editing, and/or Formatting), we cannot provide a specific timetable that applies to all re-edits. Once you submit your paper for re-editing, we will determine what steps are needed and contact you with your actual deadline within 2 business days.

See All Frequently Asked Questions

• Nature China
• Nature Japan
• Nature Korea
• Nature India
• Nature Middle East
• Nature Research
• Palgrave Macmillan

Voice speed

Text translation, source text, translation results, document translation, drag and drop.

Website translation

Enter a URL

Image translation

We use cookies on this site to enhance your experience

By clicking any link on this page you are giving your consent for us to set cookies.

A link to reset your password has been sent to your email.

Back to login

We need additional information from you. Please complete your profile first before placing your order.

Thank you. payment completed., you will receive an email from us to confirm your registration, please click the link in the email to activate your account., there was error during payment, orcid profile found in public registry, download history.

Professional translation services for a wide range of content types Getting your content professionally translated is essential to ensuring your work can be understood in other languages. Automated or "machine" translations often lack comprehension of technical language or context, which can result in your work being difficult to understand. Our translation services are provided by specialists with qualifications in your subject or discipline, ensuring that your research, academic paper or publication maintains a consistent, accurate tone of voice, assisted by powerful semantic technologies that ensure consistency throughout.

Translation into English

A high-quality, professional translation service by specialist translators into native English, with a wide range of content types and languages.

Translation & Editing

A combination of our professional translation service with editing to meet your target journal's requirements.This includes a thorough check of spelling & grammar. A separate quality control check is also carried out before returning your assignment to you.

Pricing Structure

We supply an instant quote for each piece of work, starting at  $100  for translating  a 500 word article  into English.

Translation

A combination of our professional translation service with editing to meet your target journal's requirements - creating a thorough review of your work to maximise chances of publication.

This includes a thorough check of spelling & grammar. A separate quality control check is also carried out before returning your assignment to you.

Our Content Specialities

Our Language Specialities We can translate from:

Contact us to find out more about our translation services.                                

Translating Your Text To And From Spanish

Academic translation of spanish articles.

Academic Language Experts is the top academic translation and editing company for scholars. We translate, edit, and format all academic texts including books, articles, research papers and more . We understand that translating and editing academic studies is different than working on other generic texts and requires a specific knowledge and level of understanding of the specific field of expertise.

Our team of top language professionals from around the world is here to assist you with publishing your research . Each one of our translators and editors have their own area of specialization and we will match your project to the translator or editor who has the best in-depth understanding and knowledge of your subject matter.

Our team of native Spanish translators and editors are available to work on your research today . With hundreds of millions of Spanish speakers worldwide , Spanish translation will expose your work to an entirely new audience.

Translate your journal article into Spanish

Academic translation of your article and prepare it for submission in the Spanish journal of your choice. Whether your field is history or engineering, we have the right language expert to help you publish your article and progress in your career.

Translate your books into Spanish

Do you want to translate your book into Spanish? Our team of expert translators can help you prepare your text and make it sound as if it was written by a native. When it comes to academia nothing less will suffice.

Translate your research proposal into Spanish

If you would like to create scientific cooperation and collaboration with the Spanish academic community, make sure you sound like a native by translating or editing your research proposal with ALE.

• Our Mission and Values
• Journals and Publishers
• Research Authorities
• Foundations and Museums
• Language Experts
• Translation
• Academic Writing Coaching
• Academic Review
• Book Proposal Assistance
• Journal Finder
• Table and Graph Formatting
• Manuscript Preparation
• Grant Services
• Post-Publication Services
• Grant Proposal Assistance
• Publication Support
• Our Seven Step Recipe For Success
• Pricing & Turnaround
• Success Story- Grant Proposal Assistance
• Success Story- Translation
• Success Story: Editing
• Success Story: Publication Support
• Success Story- Junior Scholar
• 'Publication Success' Interview Series: Previous Recordings
• FAQ: People
• FAQ: Projects
• FAQ: Timeline and Pricing
• Author Resources
• Newsletter Archive
• Upcoming Live Events

• Open access
• Published: 24 May 2018

Translating research into action: an international study of the role of research funders

• Robert K. D. McLean   ORCID: orcid.org/0000-0001-8084-4817 1 , 2 ,
• Ian D. Graham 3 , 4 ,
• Jacqueline M. Tetroe 5 &
• Jimmy A. Volmink 1  

Health Research Policy and Systems volume  16 , Article number:  44 ( 2018 ) Cite this article

10k Accesses

39 Citations

37 Altmetric

Metrics details

It is widely accepted that research can lead to improved health outcomes. However, translating research into meaningful impacts in peoples’ lives requires actions that stretch well beyond those traditionally associated with knowledge creation. The research reported in this manuscript provides an international review of health research funders’ efforts to encourage this process of research uptake, application and scaling, often referred to as knowledge translation.

We conducted web-site review, document review and key informant interviews to investigate knowledge translation at 26 research funding agencies. The sample comprises the regions of Australia, Europe and North America, and a diverse range of funder types, including biomedical, clinical, multi-health domain, philanthropic, public and private organisations. The data builds on a 2008 study by the authors with the same international sample, which permitted longitudinal trend analysis.

Knowledge translation is an objective of growing significance for funders across each region studied. However, there is no clear international consensus or standard on how funders might support knowledge translation. We found that approaches and mechanisms vary across region and funder type. Strategically tailored funding opportunities (grants) are the most prevalent modality of support. The most common funder-driven strategy for knowledge translation within these grants is the linking of researchers to research users. Funders could not to provide empirical evidence to support the majority of the knowledge translation activities they encourage or undertake.

Conclusions

Knowledge translation at a research funder relies on context. Accordingly, we suggest that the diversity of approaches uncovered in our research is fitting. We argue that evaluation of funding agency efforts to promote and/or support knowledge translation should be prioritised and actioned. It is paradoxical that funders’ efforts to get evidence into practice are not themselves evidence based.

Peer Review reports

It is widely accepted that research can lead to improved health outcomes. However, uncovering health knowledge through research can be challenging, costly and time-consuming, and may fail to produce meaningful advances. Still, the expected positive benefits associated with knowledge creation have led governments, foundations and private institutions across the globe to dedicate envelopes of their tax-base or endowment to this endeavour. Around the world, recent estimates suggest that over US$ 100 billion is spent on biomedical research alone in a single year [ 1 ].

That being said, the benefits of knowledge are rarely achieved by its creation alone. Knowing what to do with health-related knowledge – how to access it, appraise it, tailor it for context, apply it in the practical world and know when it is not appropriate for practical application – is an entirely different challenge. Evidence indicates that health knowledge continues to be converted into practical applications in a slow and inconsistent way [ 2 , 3 ]. The rush of publicly funded organisations to address this issue has been well documented in primary research and systematic reviews [ 4 , 5 ]. Further, research funding agencies have also answered the call. Effectively, research funders have financed another charge for unlocking the power of knowledge, by increasing their focus on the conversion of knowledge into action. In this paper, we refer to this as knowledge translation (KT). Simply put, KT is the process of turning the knowledge that is generated in research studies into use in the real-world. For example, an improved product or device, a new policy or practice guideline, or a more accessible and thus equitable programme. The most broadly accepted definition of this process is offered by the Canadian Institutes of Health Research (CIHR), which uses the term KT to articulate to their Canadian constituency a support of:

“ A dynamic and iterative process that includes synthesis, dissemination, exchange, and ethically-sound application of knowledge to improve the health of Canadians, provide more effective health services and products, and strengthen the health care system. ” [ 2 ]

The primary objective of the research reported in this manuscript is to provide an international overview of the state of health research funders’ efforts to promote and support KT. To address this objective, the research takes a broad view, looking across an international sample of 26 funding agencies to identify trends and themes, rather than taking a deep dive into the particular practices of any single agency. In addition, the data collected and analysed in this manuscript builds on our previous work [ 6 ], which allows for longitudinal trend analysis. In our 2008 project, 33 health research funding agencies from around the world were reviewed to learn more about their KT roles and activities [ 6 ]. In this follow-up, we have replicated the study design from 2008, and have been able to include 26 of these agencies. A full account of our methods, findings and conclusions are provided in the following sections of this paper.

We believe the results of this study hold immediate practical value. The primary intended users of this study are health research funders. We hope this international stock-taking will facilitate evidence-informed reflection and debate amongst funders. Secondary users of this work will include health and science policy-makers, as well as researchers interested in KT (both those interested in the study of KT, but also those wanting to learn more about how they are being supported by funders to do KT).

Why focus on funders?

Health research funders are just one of many actors in the pursuit of translating knowledge into action. Many actors, including but not limited to researchers, governments, patients, activists and taxpayers, have grown increasingly concerned with research being solution-oriented, and each plays an important role in ensuring an effective balance is struck between the creation of new knowledge and its application to health needs.

Research funders are rarely in the business of research implementation. However, we argue that the role of the research funder in KT is particularly pertinent. Funders control access to resources and therefore hold a position of power. This power can be used to stimulate and incentivise action on the part of the research community, including researchers, brokers and research users.

Evidence shows that funders contributing to KT have potential for impact. In spite of identified roadblocks in the knowledge-to-action pathway, studies examining the return on investment of public research have shown ample evidence of value [ 7 , 8 ]. In simpler terms, research is a good investment in the public good. For example, one study performed by the Council of Economic Advisors to the President of the United States [ 9 ] demonstrated that the average social return on government investment in research and development was above 50%, a figure that was at the time, and still is, far greater than other areas of investment, including private sector research and development [ 9 ]. As impact assessment models that better capture the hard to measure ‘social impacts’ of research continue to evolve, we expect to see investments in research appearing even more attractive. Understanding how KT is supported by funders will help to ensure these investments generate the greatest return for people and society.

At the same time, the two primary stakeholders of the health research funder (the public and the academic community) have voiced clear interest in improved application and coordination of KT interventions by funders. The first stakeholder group, the public (often the research funder’s funder, and always the intended long-term beneficiary), has asserted a concern that governments pay greater attention to ensuring the practical utility of research they fund [ 10 ]. The second group, the academic community (the research funder’s primary beneficiary) has increasingly turned to funders to support their desire and build their capacity to meet this growing public call for knowledge translation [ 11 , 12 ].

Study design

This research was undertaken with the intent of providing a comprehensive account of the international state of KT support offered by health research funding agencies in high-income countries. We adopted a longitudinal study design, wherein data from the same cohort was gathered across subsequent intervals in time. To do this, we purposefully grounded our study design in the work of Tetroe et al. [ 6 ] and McLean et al. [ 13 ].

Theoretical frameworks

Two theoretical frameworks were employed to support data analysis. One to categorise KT activities and a second to classify evaluation activities at funding agencies.

The first, in which we categorise, describe and provide our assessments of funding agency support for KT, is derived from the work of Lavis et al. [ 14 ]. In their 2006 paper, Lavis and colleagues propose a framework for assessing country-level efforts to link research to action with a view to “ inform country-level dialogues about the options for linking research to action ” [ 14 ]. This framework was applied in our data analysis as it provided an internationally recognisable system for KT activity classification and interpretation. The framework outlines three components into which we classify funding agency KT activities, these are (1) push activities, (2) pull activities, and (3) linkage and exchange activities ( Box 1 ). For the reasons above, we believe this is a useful and rigorous means of analysing the KT activities of funding agencies. We caution that this framework does not facilitate a specific review of ‘priority-setting activities’ or ‘responsive grant-making’, i.e. when funding agencies strategically prioritise topics, disciplines or objectives for the research they fund. The KT activity classification framework employed here includes funding that has been ‘strategically prioritised’, but it does not examine the act of priority-setting separately.

To conduct an analysis of evaluation activities, we developed a new framework for funding agency evaluation activity classification. The framework is derived from theory and guidance on best practice in evaluation [ 15 , 16 , 17 , 18 , 19 ], as well as the work of Mintzberg on organisational strategy [ 20 ], and that of the Canada’s International Development Research Centre on how strategy and evaluation interact [ 21 , 22 ]. Mintzberg argues that organisational strategy is not just what we say we do, it is also what we do [ 20 , 22 ]. Because evaluation is used by organisations to understand what has been done, in essence, evaluation activities can be used to shine light on a picture of organisational strategy. In this research, we aimed to utilise this dynamic conceptualisation of organisational learning and strategic management by developing a framework for the analysis of funders’ KT actions.

Box 2 below provides further explanation of the framework that was developed and used for data analysis Footnote 1 . In short, it outlined three areas of strategy, namely (1) the ‘the intended strategy’ (what was planned), (2) ‘the realised strategy’ (what actually occurred), and (3) ‘the emergent strategy’ (the lessons learned and adopted into practice) of the funding organisation.

Longitudinal design and sampling frame

Our research was designed to provide a follow-up on the work of Tetroe et al. [ 6 ]. We drew our sample of funding agencies from the sample contacted in this 2008 study. With this approach we were able to undertake analysis of the same cohort at two discrete intervals in time. In the Tetroe et al. [ 6 ] study, a judgement sampling approach was employed to select funding agencies based on particular criteria of interest to the research team undertaking that study [ 23 ]. These criteria were (1) nationally scoped agencies and other disease-specific voluntary health organisations and (2) agencies that represented a continuum in or contrast in their KT support activities.

Data collection protocol

Website reviews and agency templates.

A data collection template was developed to gather information from the website and accessible publications of each funding agency. These templates were based on the data collected in the Tetroe et al. [ 6 ] study (keeping to our objective of conducting longitudinal analysis) and the theoretical categorisation of KT activities provided by Lavis et al. [ 14 ]. Templates were populated with information such as mandate, annual budget, types of KT support activities and KT evaluation activities. Following this initial web-based documenting process, the templates were sent via email to senior representatives of each agency for validation, updating and addition of data that were not available on the agency website or in accessible publications. The agencies were then asked to return the completed template to our study team, at which point a telephone interview was scheduled between the study research team and the agency.

Semi-structured qualitative interviews

We aimed to conduct telephone interviews with at least two representatives of each agency, including one senior representative of the KT function and one senior representative of the evaluation function. It should be noted that, in some cases, the two senior officials were the same individual, in some agencies an evaluation function did not exist, and in others a larger group of representatives desired to take part in the interview process and we agreed to this. The interview protocol was based on the completion of the agency template and a series of follow-up questions based on the flow of the discussion and emergent data of interest. This approach allowed us to complete the deductive learning exercise driven by the predefined template, but also to complement these data with new information on why and how any KT activities were being implemented in the eyes of the funding agency, and to allow the interviewer to probe further on issues of particular interest [ 24 ].

All interviews, except the CIHR interview, were conducted by telephone in either French or English. The CIHR interview was conducted in-person in the CIHR Ottawa, Canada, office. On average, interviews lasted between 30 and 90 minutes. Notes from any French language interview were translated into English by two bilingual members of the research team independently, and independent translations were cross-checked for consistency and accuracy. To minimise the threat of description or interpretation bias following the interview, the notes and the completed agency templates were returned to each agency for validation.

Research ethics

Ethics approval for this project was granted by the Ottawa Hospital Research Institute, in Ottawa, Ontario, Canada.

The following section of the paper reports on the results of our research and preliminary interpretation. The section is divided into three parts. In the first, we present an overview of the 26 agencies included in the research. In the second, we explore the role of KT at the funding agency, in other words, how funding agencies have positioned KT for their agency in a qualitative and quantitative sense. In the third, we outline the KT initiatives being offered and undertaken at the funding agency, taking stock of and analysing actual KT strategies, programmes, funding mechanisms and evaluations of these efforts.

Agency overview (region, focus, funding source, budget)

Table 1 provides a brief overview of the regions and agencies included in the study. It also displays those which were not in the study, but were involved in the t1 period research (from here forward the Tetroe et al. [ 6 ] project will be referred to as t1 and this project as t2). The seven agencies not included in this report were removed from the sample due to an inability to schedule an interview or an interview request being declined. Although it would have been possible to proceed with using publically available data, it was deemed that this would lead to questions about accuracy. Further, a brief description of the focus of each agency is included as well as the agencies’ overall annual budget at the time of contact.

The position of KT at the funding agency

KT can be supported by a funding agency in a multitude of ways; however, the way that KT is positioned within the funding agency – explicit or implicit – is an important indicator of the level of significance it holds and the impact it may eventually have. We have used five measures to allow us to identify and investigate the intended role for KT at each of our sampled funding agencies. Then, by aggregating data, we can examine regional, global and longitudinal trends. Our five measures are (1) language used to describe KT, (2) mandating KT, (3) a senior agency members’ priority rating of KT, (4) human resources devoted to KT, and (5) financial resources devoted to KT. Results related to each of these measures are reported in turn in the following sections of the manuscript.

KT terminology

As a follow-up to the t1 study, we looked to identify terms that funding agencies were using to describe the concept of translating research into action – what we refer to in this paper as KT. Through agency website scans and follow-up interviews with agency staff, we identified a total of 38 terms in use by the 26 agencies studied.

In the t1 study, 29 terms were identified, and therefore, over time, there has been an increase in the number of terms used to describe the KT concept within funding agencies, even though the number of funders studied decreased by seven between the two study periods (Table 2 ). One might interpret this growth in terminology over time in a number of ways. On the one hand, it could be seen as a popularisation of the concept, as further definitions are being used. On the other, it could be interpreted as a lack of coordination and consistency in KT conceptualisation from one agency to another. We discuss our understanding vis-à-vis other findings of our research in the Discussion and Conclusions section of this paper.

Mandating KT

Perhaps the most palpable indicator of KT’s intended prominence at a funding agency is whether or not the agency includes the concept in its mandate. A mandate is the publically stated raison d’etre of an organisation, and is often legislated by an external body such as a Parliament or Board of Directors. To assess this, we scanned agency mandates to look for terms or concepts describing KT – not specifically the term knowledge translation. We validated our result for each agency in the follow-up interview with the funder. Of the 26 funding agencies involved, we identified that 20 (77%) included the concept of KT directly in their agency mandate. This result indicates that the majority of research funders in our sample publically declare that KT is a part of their core mission. However, data also indicates that the inclusion of KT in the mandate is an emerging trend. As Fig.  1 shows, the number of agencies including KT in their mandate has increased from t1 to t2.

Change in knowledge translation inclusion in agency mandate over time

Every global region studied demonstrates an increase in the inclusion of KT in funding agency mandates, save Australia, where mandate inclusion did not decrease but remained unchanged. At the individual agency level, we found that none of the agencies that included KT at t1 had removed the concept at t2.

Stemming from terminology and mandate reviews, we aimed to collect data on additional indicators of the intended role of KT at the health research funding agency. Table  3 displays the results for the three additional measures of KT positioning at funding agencies, namely (3) self-prioritisation of KT, (4) human resources devoted to KT, and (5) financial resources devoted to KT.

KT prioritisation

KT prioritisation is unique to this period of data collection. The intent of introducing this data point was to capture, and subsequently compare, a challenge which arose quite frequently in the qualitative aspects of the t1 research. The difficulty we observed was that agency representatives reported in the interview stage that KT was of a certain priority at their agency, although they did not have precise written policy, budgetary or other ‘hard’ documented evidence to support this claim. Given that our interviews were performed with senior officials of each agency (in many cases up to the VP level), and that the vision of a leader may potentially be used to judge the importance of an idea, we designed a simple categorical tool for the collection and classification of these claims.

In interviews, we discussed why and how they reached the agency score they did. We found that, when respondents were given the opportunity to explain the numeric rating given to their agency, they frequently asserted that KT was becoming an increasingly important global objective and that the interviewee’s individual funding agency was well-attuned to this trend and following suit. This finding is aligned to the data presented earlier demonstrating the growing trend of embedding the KT concept within the agency’s mandate. The generally high scores of KT prioritisation across agencies and regions indicate a trend of increasing KT importance within our cohort. However, when compared to other proxy measures of the ‘KT role’ at an agency (staff and budget), we did not see any particularly compelling patterns emerge.

Human resources devoted to KT

In examining human resources devoted to KT, each agency was asked to self-interpret and self-classify who was considered KT staff. Through semi-structured interviews, we then aimed to identify underlying reasons for these classifications. Interestingly, there is a divergence in who is defined as KT staff across agencies. To give an example, in the United States agencies, definitions of staff varied substantially. The Robert Wood Johnson Foundation held a broad view, including its nine evaluation staff as KT staff, suggesting that a focus on learning and programme improvement are both an evaluative duty and a part of the agency’s KT approach. In contrast, the National Institutes of Health National Cancer Institute reported its dedicated Implementation Science Team, a group that works directly on issues of KT conceptualisation and programming with the organisation and with its research community. Many agencies included communications groups in their calculation of KT staff. Further, the Agency for Healthcare Research and Quality suggested that the embedded nature of the KT work at their organisation meant that all employees should be counted as KT staff. This variation in interpretation of who constitutes KT staff was not restricted to the United States. We did not discern any trends in types of staff (e.g. communications, evaluation, considering all staff as KT) being classified as KT in one region but not in another.

Although not indicative of any generalisable difference in resource support (given our purposeful sampling approach and differences in regional sample characteristics), the data illustrates that the United States currently devotes the largest human and financial resource contributions to the KT objective at the funding agency level.

As human resources devoted to KT was not a variable collected in the t1 study, we were unable to perform any comparative analyses across time.

The over-arching finding of this line of analysis is that there is no generally accepted view of who constitutes KT staff at a research funding agency.

Financial resources devoted to KT

Given that financial resources devoted to KT were measured in both the t1 and t2 studies, we performed various comparative analyses on KT budget data received from each agency. However, none of these proved, in our view, to provide enough insight into KT spending trends at funding agencies to warrant further demonstration and/or data tables in this manuscript. Furthermore, we concluded that there was limited value in presenting changes in KT spending per region or per agency given multiple, significant confounding factors that would limit the ability to interpret such analysis (e.g. changes in total agency budgets versus KT budgets, currency inflation, regional variation in inflation, currency conversation and exchange rate fluctuations over time). That being said, one trend did emerge, namely that, across all regions, the number of agencies that could provide a precise budget figure for KT did not change significantly. In other words, the number of agencies earmarking KT funds remained generally the same across time.

For a closer look at this issue, we unpacked the more recent t2 data further. In summary, less than half of all agencies interviewed (11 of 26) were able to identify a specific amount devoted to KT. Ten of 26 reported that the KT spending of the organisation could not or should not be seen as an independent budget line, but instead that KT was embedded-in across the organisation’s expenditures. Seven of 26 agencies were unable to provide any funding details related to KT, which was in contrast with the fact that only one of 26 was unable to publically disclose any budget information. In sum, these data indicate that earmarking KT financial resources is not the norm across any region or the sample at large. To better understand the return on KT activities, this may be a useful area of data for agencies to track more closely in the future.

KT initiatives

In this section, we turn to the specific programmes, mechanisms, modalities, activities, etc., that funding agencies were using to support KT.

Figure  2 presents the classification of agency initiatives across the three parts of our analytic framework, namely push, pull, and linkage and exchange (see Box 1 in the Methods section of this paper for a full description). Given the sampling approach employed, we caution against advanced quantitative comparative interpretation. We consider these data as categorical.

Number of push, pull, linkage and exchange programmes by country

Most agencies favour linkage and exchange (or integrated KT (IKT)) and push efforts over pull efforts. There were a substantial number of IKT programmes supplementing the funders’ support for traditional programmes of curiosity-driven research. Qualitative interview data did not provide any clear conclusion on why these trends toward push and linkage and exchange efforts existed.

Although any regional analysis should be made with caution, a pattern does emerge with regards to programme distribution across category, namely that agencies and regions offer a general mix of programming, which is consistent with what is considered by Lavis et al. [ 14 ] to be a favourable approach.

At t1, more attention was paid to the activities the funding agency required of researchers vis-a-vis the activities, either planned or unplanned, of the funding agency in support of KT. To address this, we restructured our classification of programmes in Fig.  3 by grants, awards and fellowships. Note that a single programme – the base unit in our above analysis (Fig.  2 ) – could include a series of grants or awards or fellowships. It should also be noted that Fig.  3 showcases programmes that were strategically designed for KT and does not include grants, awards or fellowships that were not designed specifically for KT, but may support KT due to the independent decision of a grantee to undertake KT.

Number of knowledge translation (KT) grants, awards and fellowships by country

The primary finding of this analysis is that the funding agencies are more involved in KT grant dispersion than other forms of KT support activity like awards or fellowships. To facilitate more precise interpretation, a comparison of how this trend in KT support varies from other areas within the health sciences would be a valuable area of additional study, e.g. investigating how the balance of grant, award and fellowship opportunities for KT compare to the balance of opportunities available for clinical trials, laboratory science, vaccinology and health systems research.

The final area of findings reported in this manuscript describes an investigation of the evaluation of KT being conducted at funding agencies. Evaluation is selected as a focus area for two distinct reasons. First, KT evaluation has been identified as a significant gap in published expert opinion, theoretical research and empirical research [ 6 , 14 , 25 , 26 , 27 , 28 , 29 ]. Second, in the t1 research, there were no evaluations identified in any of the 33 agencies studied; however, nearly all 33 agencies articulated that plans and designs for evaluations were underway. As a result, a specific follow-up on progress with evaluation was prioritised for t2. In other words, an objective for t2 was to provide more than a stock-taking of programmes and practice at funding agencies, it was also to dig into the evidence guiding these efforts.

Given our specific focus for this study – KT activities/support at the funding agency – we purposefully reviewed evaluation undertaken at the funding agency level only, that is to say, an evaluation that focused on the KT programmes and activities of the funding agency. We did not include any evaluation being done by funded researchers in their own projects or the health interventions of others, even if this evaluation was funded by an agency in our sample (e.g. a large body of work being performed through the National Health Services – Service Delivery and Organization, e.g. [ 30 , 31 ]) 1 . Our aim was to learn about funding agency programmes and practice specifically. Table  4 illustrates evaluation activities being conducted of funding agencies’ KT programmes and practice; it utilises the Intended, Realised, Emergent (IRE) framework articulated in Box 1 of this manuscript.

Data indicates that funding agencies are putting considerable effort and resources into thinking through KT theories and objectives but much less into carrying out critical evaluations of these efforts/resources. Indeed, 23/26 funding agencies had a defined and planned KT strategy to some extent (recall 20/26 are currently including the concept in their agency mandate), yet only 7/26 had evaluated KT efforts and only 1/26 could demonstrate that evaluation results had been used to guide KT programmes or practice (i.e. to support evidence-based decision-making). In other words, a commitment to KT is evident, but learning-focused programming of KT was rare.

A deeper dive into the three components of the IRE framework helps to further understand the agencies’ KT strategy. See Box 2 in the Methods section of the manuscript for a full description of the IRE conceptual framework.

Intended strategy

In terms of ‘intended strategy’ there is a strong base of activity and effort in our sample of funding agencies. The majority of this effort was in setting a KT definition and outlining a series of KT goals. A minority of agencies had derived implementation theories (e.g. a theory of change) to describe the intended process and results of their KT efforts. One of these was the National Institutes of Health National Cancer Institute’s Implementation Science team. This agency has worked to articulate a theory of KT implementation, integrated a research translation continuum, and developed a set of contingencies and considerations into their KT support processes. Another example was CIHR, who had articulated a KT Funding Program logic model, and initiated an evaluation of the organisations strategic intentions for KT, by using this model in the evaluation design to outline expected KT results and critical assumptions.

Realised strategy

In terms of ‘realised strategy’, we have included all evaluation activities related to the assessment of realised organisational KT strategy. Table  4 indicates a decline in activities as we move from ‘intended’ to ‘realised’ strategy. Some insight into why this was the case was uncovered in the qualitative interviews. While the vast majority of agencies asserted that they deemed the evaluation of their KT funding to be a paramount endeavour, they also informed the research team that they did not have a firm grounding in how to undertake this task. Generally, it was suggested by agency representatives that research evaluation was a difficult undertaking; however, evaluating the translation of research into action was the most difficult component of this problem.

Emergent strategy

‘Emergent strategy’ is not surprisingly deficient when considered in relation to the trend of decreasing activity moving from ‘intended’ to ‘realised’ strategy documentation. At the time of data collection, only one agency (Alberta Innovates, a Canadian public Provincial funder) was using KT-specific evaluation results to inform decision-making and action.

As health policy, practice and programming continues to lag behind research-generated knowledge, KT remains a crucial objective within the health system. As has been argued in the past [ 32 ], the funding agency role in supporting KT has merit for a number of reasons. In this manuscript, we premise this argument on the position that incentive-setting power funders occupy, given the control they hold over financial resources. Playing the role of financier to the research enterprise places funders in an influential position to stimulate action around a particular topic such as KT. As Nobel Prize winning economist Joseph Stiglitz has recognised:

“ …the scientists whose research and ideas have transformed our lives in the past two hundred years have, for the most part, not been motivated by the pursuit of wealth. This is fortunate, for if they had, they would have become bankers and not scientists. It is the pursuit of truth, the pleasure of using their minds, the sense of achievement from discovery – and the recognition of their peers – that matters most. Of course, that doesn’t mean they will turn down money if it is given to them. ” [ 33 ]

Positioning KT with funders

The purpose of this research was to take stock of how various funders are supporting KT and how well they are doing. In general, the 26 funding agencies whom we engaged demonstrated that KT is a high and a still growing priority. As mandates are changed (and maintained) to include the concept of KT, we interpret this to mean that governments and other health research funders are concerned with making research useful and actionable.

Previous studies on the role of the funding agency in KT (e.g. [ 6 , 25 , 34 ]) have argued that a common definition and/or classification of KT would be beneficial, and some suggest that a systematic framework to knowledge translation would contribute to conceptual clarity in the field [ 25 ]. We do not find evidence to disprove this hypothesis, but the findings of our research give us limited reason for concern. We suggest the diversity of experience across funders, by country, region, agency size and by agency type, is a representation of the diversity of context in which these organisations operate. We see no reason to conclude this is problematic. In our opinion, it is more than likely beneficial that programmes and strategies are contextually grounded.

At the same time, some trending in funder practice is evident. This study has identified further divergence in terminology over time (since 2008). It also uncovers a convergence of KT initiatives which funders are using to further their KT agendas. The prevalence of push efforts and linkage and exchange (or IKT) efforts, and the preference towards grants to support these, appears as a trend across our global sample. The emergence of IKT programming in particular represents a notable shift from traditional research funding approaches that have tended to favour the researcher over research users. Without robust evaluation data we cannot examine the evidence base for why these programmes and mechanisms are favoured, or appraise their effectiveness. However, we can offer some interpretation. First, and perhaps most likely, the similarity in programming and grant making activities might represent the emergence of an accepted framework for KT support based on informal exchanges between agencies. If this is the case, we note some concern that, in the absence of evaluations, the convergence of KT support activities may represent an emerging groupthink process rather than the co-development of a set of proven good practices. Secondly, although we would argue unlikely, these international patterns could be coincidental. Again, further evaluation would help to shed better light on the issue.

KT evaluation – still, an area for action

In the 2008 t1 study, we identified a significant gap in funders’ execution and ability to execute evaluations of KT efforts. As a result, in this t2 research, we set an intentional focus on the issue of KT evaluation in order to learn how evaluation practice had evolved over time and to collect complete evaluations in order to investigate the possibility of synthesis and meta-evaluation.

We aimed to better understand and analyse evaluation activities by developing and applying the IRE framework ( Box 2 ). This tool allowed us to identify, and subsequently investigate, evaluative and strategic strengths and weaknesses across agencies. In essence, the IRE framework enabled us to look beyond a simple count of evaluation reports or the potential undertaking of a synthesis of evaluation findings. Using the framework facilitated meaningful conclusions about the evaluation and strategy process. We would encourage those agencies who wish to improve their evaluation and strategy functions to consider a conceptual framework such as this, in particular for identifying areas for organisational improvement and/or cross-agency exchange, and we suggest the results of this research may provide some starting points. For example, applying the IRE framework highlighted how ‘intended KT strategy’ is relatively well developed across our sample of funders. Therefore, ‘intended strategy’ activities such as designing KT funding programmes, setting an organisational KT strategy, or developing KT theories of change, would be an immediately actionable and data-rich area of cross-agency learning and exchange.

Our data shows a substantial base of intended KT evaluation activities across the agencies and global regions in the sample. However, findings also highlight a significant lack of progress in undertaking targeted evaluations of KT, communicating results of these evaluations, and using findings of these evaluations to inform funder practice and policy. At the time of data collection, only one of the agencies in our sample had completed a targeted evaluation of KT. Though several agencies had evaluations that were underway or planned, it is important to recall the same was found in the t1 study (when nearly all agencies reported plans for KT evaluation). A clear conclusion is that evaluative data is not being used to measure progress against the objectives set out in earlier stages of KT programme design and planning. Given that the underlying objective of KT lies in moving evidence into action, it is paradoxical that the funders of KT do not employ this philosophy in their own work.

At the same time, this research has uncovered a lack of methodological know-how for evaluating KT as a major stumbling block for agencies who generally indicate a genuine interest in improving KT practice. As such, we suggest this is an area ripe for researcher (not just funder) focus. We also learned that funding agencies, themselves faced with budget austerity, do not always have the ability to make evaluations a priority, and especially the more challenging evaluations such as one focused on KT. Although we can understand this predicament, we do not agree that underfunding critical reflection is a sustainable cost-savings approach. We hope that identifying the persistent lack of KT-focused evaluation at funding agencies, both globally and across type of agency, will assist in kick-starting evaluative work. In our interviews with funding agencies, we heard great interest and genuine intention to undertake evaluations should the technical know-how be advanced and financial resources be available. We hope this research is used to support the cause.

Study limitations

Although we are confident that our methodological approach allowed us to capture an accurate snapshot of KT activities at each agency, we caution that, because we did not interview all departments or branches of each agency, we cannot claim with absolute certainty that all KT activities have been identified.

The research reported in this manuscript has focused on the intentional efforts of funders to support KT. Focusing on the intentional may not have captured all KT activities supported. KT activities may go unreported when they have occurred as a part of a funding agency programme that is not intentionally supporting KT, particularly when they occur by decision of a grantee or awardee. For example, a researcher may decide to use a portion of a research grant to organise a meeting with hospital managers to discuss their findings; this activity has been technically supported by the funding agency with the grant, but it may not have been directed or recorded by the funding agency as KT support.

The longitudinal nature of the study design is weakened by the project team’s inability to include seven of the agencies included in the t1 study [ 6 ]. An inability for the research team to establish a contact at a particular funding agency meant we removed the agency from the sample. We did not want to rely on data collected from the web and document reviews alone. We have no reason to believe this has introduced any bias into the study, neither could we identify any characteristic or quality that removed agencies have eliminated or re-weighted in this sample vis-à-vis the 2008 sample.

A significant limitation stems from our sample of funding agencies being from high-income countries and focused on funding research that addresses high-income country needs. A review which includes low- and middle-income country funders was undertaken by Cordero et al. [ 25 ] as a companion to Tetroe et al. [ 6 ]. The follow-up to Cordero et al. [ 25 ] will be critical to understand the global story of funding agency support for KT Footnote 2 .

A limitation in our study design relates to where we have focused attention in data collection and analysis. We have purposefully taken a broad view with this research, engaging 26 funding agencies from around the world in the study, allowing the identification of broad trends and themes for KT practice at research funding agencies. However, it does not facilitate the deep investigation of a particular funding agency’s experience with KT.

We note, for reader interpretation, the timing between data collection and publication – data were collected from funding agencies for this project in 2012/13 (9 to 10 years after the data collection in t1). Readers’ should interpret the findings accordingly.

In summary, our research confirms that KT is an objective of growing significance for the health research funders across the high-income regions of Europe, Australia and North America. The findings demonstrate that there is no clear-cut standard or practice for implementing KT at a funding agency. We suggest that KT is an idiosyncratic matter that relies on the many contextual factors presented to a particular research funder. There is very likely no viable one-size-fits-all solution. We suggest that the diversity of experience this research has uncovered indicates that any sweeping conclusions or directives for KT at funding agencies should be handled with caution, and also calls for evaluation of KT in these different funder contexts to learn what works, for what type of funder and why.

We suggest that the critical evaluation of KT should be prioritised and actioned so that evidence-based decision-making becomes not only the objective of KT programmes, but also a part of how these programmes operate and evolve. These evaluations should take into consideration the particular context of the agency that undertakes the evaluation, and should make this context clear in order to facilitate other agencies’ interpretation of the results. To kickstart and advance high-quality evaluation, we suggest funders support KT evaluation experimentation, innovation and collaboration among each other on the topic. Funders should not feel alone, this effort may well embrace the researcher community interested in doing and improving KT. As the significance of KT grows for funders, so must the evidence base to guide it.

Box 1 Knowledge translation activity classification framework

Push – activities and programmes targeted at the ‘pushing’ of research-produced knowledge into the hands of appropriate knowledge users – users who may not have otherwise been aware of the research and its implications. Examples include research communications, funding opportunities or funder activities, or typical end-of-grant funds that an agency may provide to a researcher to encourage the dissemination of findings such a publication in an Open Access journal, or the creation of a plain language findings brief.

Pull – activities and programmes that facilitate knowledge users’ access to research results. For example, a forum where researchers are brought to discuss an issue of importance with identified knowledge users.

Linkage and exchange – activities and programmes that support the establishment of partnerships between researchers and knowledge users through multiple parts of the process of research design, execution and/or dissemination. Linkage and exchange is also referred to as integrated knowledge translation and co-creation/co-production [ 35 , 36 , 37 ]. An example would be a research grant that required both a researcher and a knowledge user to apply in partnership for funding, representing a break with the traditional researcher curiosity-driven approach to science. This more participatory approach may also involve non-researchers (e.g. patients, clinicians, managers, etc.) as reviewers in the peer-review process.

Note: This is a very brief description of these well-developed concepts of promoting research use. Further reading on the subject could include Lomas [ 38 ] and Lavis et al. [ 39 ].

Box 2 Intended → Realised → Emergent (IRE) framework for strategy classification

Intended strategy (the planned knowledge translation (KT) strategy) – Includes actions such as defining KT, setting clear KT objectives or goals, mapping these objectives within internal and external structures, mechanisms and constraints (in a realist or classical empiricist way), defining stakeholders (intended and unintended), drawing identified factors into implementation theories for KT programmes.

Evaluator’s use ‘intended strategy’ to first understand programme purpose and to then construct measures for programme or organisational assessment. Robust evaluations will consider intended strategy during methodological design. Typically, these intentions are developed by evaluators in constructs such as ‘theories of change’, ‘logic models’, ‘logframes’, etc. All of these tools hold the same general purpose of articulating a programme’s intended strategy and results in more depth and detail than a stated objective.

Realised strategy (the KT strategy that was executed) – Includes the actual KT programmes, initiatives and activities of the funding agency.

Evaluators use these elements as the ‘object of assessment’ or ‘evaluand’. Evaluation activities related to this will include actions such as designing evaluation studies, monitoring and collecting data, analysis and interpretation of data and communicating findings. Here, evaluations would identify KT support that was working as intended and that which was not as well as unpacking the mechanisms, contexts and systems that govern success.

Emergent strategy (KT strategy evolving from evaluation use) – Includes the broad range of actions related to using evaluation findings in KT programme refinement, development, overhaul or cessation (i.e. evidence-based decision-making). In other words, the evidence-based direction that an agency embarks upon.

Evaluators produce knowledge related to a realised strategy that, through the complex process of uptake and implementation, is built into the re-thinking of strategy (e.g. confirming status quo, course correction or complete cessation). In the evaluation literature, this is referred to as ‘evaluation use’ and is manifested in evidence-based action.

There were many evaluations conducted by the research community through the CLARHC programme of the National Health Services – Service Delivery and Organization, we reference two examples. None of these evaluations focused on the activity of the funding agency in support of KT.

This study is currently in design by authors: RKDM, JAV, IDG.

Abbreviations

Canadian dollar

Canadian Institutes of Health Research

Integrated Knowledge Translation

Intended, Realised, Emergent

Knowledge Translation

Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374:86–9.

Article   PubMed   Google Scholar  

McLean R, Tucker J. Evaluation of CIHR’s Knowledge Translation Funding Program. Ottawa. 2013. http://www.cihr-irsc.gc.ca/e/47332.html . Accessed 1 June 2014.

Agency for Health Research and Quality. Translating Research into Practice (TRIP)-II. Rockville. 2001. http://www.ahrq.gov/research/findings/factsheets/translating/tripfac/index.html . Accessed 1 June 2014.

Mitton C, Adair CE, Mckenzie E, et al. Knowledge transfer and exchange: review and synthesis of the literature. Milbank Q. 2007;85(4):729–68.

Article   PubMed   PubMed Central   Google Scholar  

Contandriopoulos D, Lemire D, Denis J-L, et al. Knowledge exchange processes in organizations and policy arenas: a narrative systematic review of the literature. Milbank Q. 2010;88(4):444–83.

Tetroe JM, Graham ID, Foy R, et al. Health research funding agencies’ support and promotion of knowledge translation: an international study. Milibank Q. 2008;86(1):125–55.

Article   Google Scholar  

Sampat BN, Lichtenberg F. What are the respective roles of the public and private sectors in pharmaceutical innovation? Health Aff. 2011;30(2):332–9.

Dalziel M, Roswell J, Tahmina TN, et al. Impact of government investments in research & innovation: review of academic investigations. J Public Sect Manag. 2012;42(2):19–32. Optimum Online

Google Scholar  

Council of Economic Advisers, President of the United States of America. Supporting Research and Development to Promote Economic Growth. Washington, DC: The Federal Government’s Role; 1995.

Sutherland WJ, Bellingan L, Bellingham JR, et al. A collaboratively-derived science-policy research agenda. PLoS One. 2012;7:e31824.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Wilson P, Petticrew M, Calnan M, et al. Does dissemination extend beyond publication: a survey of a cross section of public funded research in the UK. Implement Sci. 2010;5:61.

Corluka A, Hyder AA, Segura E, Winch P, McLean RKD. Survey of Argentine health researchers on the use of evidence in policymaking. PLoS One. 2015;10(4):e0125711.

McLean RKD, Graham ID, Bosompra K, et al. Understanding the performance and impact of public knowledge translation funding interventions: Protocol for an evaluation of Canadian Institutes of Health Research knowledge translation funding programs. Implement Sci. 2012;7:57.

Lavis JN, Lomas J, Hamid M, Sewankambo NK. Assessing country-level efforts to link research to action. Bull World Health Organ. 2006;84:620–8.

Pawson R, Tilley N. An introduction to scientific realist evaluation. In: Chelimsky E, Shadish, William R, editors. Evaluation for the 21st Century: A Handbook. Thousand Oaks, CA: Sage Publications; 1997. p. 405–18.

Chapter   Google Scholar  

Pawson R. Evidence-based Policy: A Realist Perspective. Thousand Oaks, CA: Sage Publications; 2006.

Book   Google Scholar  

McDavid JC, Hawthorne LRL. Program Evaluation and Performance Measurement: An Introduction to Practice. Thousand Oaks, CA: Sage Publications; 2006.

Patton MQ. Utilization-focused Evaluation. Thousand Oaks, CA: Sage Publications; 2008.

Funnell S, Rogers P. Purposeful Program Theory: Effective Use of Theories of Change and Logic Models. San Francisco, CA: Jossey Bass; 2011.

Mintzberg H. Tracking Strategies. Oxford: Oxford University Press; 2007.

Wind T, Carden F. Strategy evaluation: Experience at the International Development Research Centre. N Dir Eval. 2010;128:29–46.

Patrizi P, Patton MQ. Learning from Doing: Reflections on IDRC’s Strategy in Action. Ottawa. 2009. https://idl-bnc-idrc.dspacedirect.org/handle/10625/47305.

Kuzel A. Sampling in Qualitative Inquiry. In: Crabtree B, Milner W, editors. Doing Qualitative Research. London: Sage; 1992. p. 31–44.

Crabtree B, Miller W. Doing Qualitative Research. Newbury Park: Sage Publications; 1992.

Cordero C, Delino R, Jeyaseelan L, et al. Funding agencies in low- and middle-income countries: support for knowledge translation. Bull World Health Organ. 2008;86(7):524–34.

Patton M. Search of Impact: An Analysis of the Utilization of Health Evaluation Research. In: Weiss CH, editor. Using Social Research in Public Policy Making. Lexington: Lexington Books; 1977. p. 141–63.

Lavis JN. Research, public policy, and knowledge-translation processes: Canadian efforts to build bridges. J Contin Educ Health Prof. 2006;26(1):37–45.

Ettelt S, Mays N. Health services research in Europe and its use for informing policy. J Health Serv Res Policy. 2011;16:48–60.

Holmes B, Scarrow G, Schellenberg M. Translating evidence into practice: the role of health research funders. Implement Sci. 2012;7:39.

Evans S, Scarbrough H. Supporting knowledge translation through collaborative translational research initiatives: ‘Bridging’ versus ‘blurring’ boundary-spanning approaches in the UK CLAHRC initiative. Soc Sci Med. 2014;106:119–27.

Rycroft-Malone J, Wilkinson J, et al. Collaborative action around implementation Collaboration for Leadership in Applied Health Research and Care: towards a program theory. J Health Serv Res Policy. 2013;18(Suppl. 3):13–26.

Graham ID, Tetroe JM. Getting evidence into policy and practice: perspective of a health research funder. J Can Acad Child Adolesc Psychiatry. 2009;18(1):46–50.

PubMed   PubMed Central   Google Scholar  

Stiglitz J. The Price of Inequality. New York: Norton; 2013.

Smits P, Denis J-L. How research funding agencies support science integration into policy and practice: an international overview. Implement Sci. 2014;9:28.

Canadian Institutes of Health Research. Knowledge Translation at CIHR. http://www.cihr-irsc.gc.ca/e/29418.html . Accessed 15 Nov 2017.

Bowen S, Graham ID. Integrated knowledge translation (Chapter 1.2, p14–23). In Knowledge Translation in Health Care. Moving From Evidence to Practice. Oxford: Wiley; 2013.

Graham I, Tetroe J, Pearson A. Turning Knowledge into Action: Practical Guidance on How to Do Integrated Knowledge Translation. Adelaide: Wolters Kluwer/MR Books; 2014. 

Lomas J. Using “linkage and exchange” to move research into policy at a Canadian foundation. Health Aff. 2000;19:236.

Article   CAS   Google Scholar  

Lavis JN, Robertson D, Woodside JM, et al. How can research organizations more effectively transfer research knowledge to decision makers? Milbank Q. 2003;81(2):221–48.

Graham ID, Kothari A, McCutcheon C, et al. Moving knowledge into action for more effective practice, programmes and policy: protocol for a research programme on integrated knowledge translation. Implement Sci. 2018;13:22.

Download references

Acknowledgements

The authors are grateful to the representatives of each funding agency that participated in this study, who took time to provide quantitative and qualitative data, and encouraged study completion and sharing of the results. The authors would also like to thank Dr. Barbara Davies, Professor in Nursing at the University of Ottawa, for her careful review and thoughtful criticisms of the manuscript and for allowing the authors to discuss parts of the research with students of her graduate course in Knowledge Mobilisation. This discussion was a deeply insightful experience.

IDG is a recipient of a CIHR Foundation Grant FDN #143237 [ 40 ]. RKDM is a trainee on this grant. JMT is a member of the grant advisory committee. JAV is an expert advisor for the grant.

Availability of data and materials

All data generated or analysed during this study are included in this published article.

Author information

Authors and affiliations.

Faculty of Medicine and Health Sciences, Stellenbosch University, Tygerberg, South Africa

Robert K. D. McLean & Jimmy A. Volmink

Policy and Evaluation Division, International Development Research Centre, Ottawa, Canada

Robert K. D. McLean

School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada

Ian D. Graham

Centre for Practice Changing Research, Ottawa Hospital Research Institute, Ottawa, Canada

Retired health researcher, Ottawa, Canada

Jacqueline M. Tetroe

You can also search for this author in PubMed   Google Scholar

Contributions

Conceived and designed the study: RKDM, IDG, JMT, JAV. Performed the study: RKDM, IDG, JMT. Analysed data and drafted the manuscript: RKDM. Wrote and revised the final manuscript: RKDM, IDG, JMT, JAV. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Robert K. D. McLean .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval for this study was granted by the Ottawa Hospital Research Institute. All study participants consented to participation.

Competing interests

Data collection for this study was undertaken while RKDM, IDG and JMT were affiliated to CIHR. All opinions presented in this manuscript belong to the authors alone, and not any institution to which they are or were affiliated. The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

McLean, R.K.D., Graham, I.D., Tetroe, J.M. et al. Translating research into action: an international study of the role of research funders. Health Res Policy Sys 16 , 44 (2018). https://doi.org/10.1186/s12961-018-0316-y

Download citation

Received : 06 December 2017

Accepted : 20 April 2018

Published : 24 May 2018

DOI : https://doi.org/10.1186/s12961-018-0316-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Knowledge translation
• Knowledge mobilisation
• Integrated knowledge translation
• Research use
• Research funding
• Research evaluation
• Research impact

Health Research Policy and Systems

ISSN: 1478-4505

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• FSG Selective Deposit

Creating accessible Spanish language materials for Clinical Sequencing Evidence-Generating Research consortium genomic projects: challenges and lessons learned

Nangel m lindberg.

1 Kaiser Permanente Northwest Center for Health Research, 3800 N. Interstate Ave, Portland, OR 97227, USA

Amanda M Gutierrez

2 Baylor College of Medicine Center for Medical Ethics & Health Policy, One Baylor Plaza, Suite 310D, Houston, TX 77030, USA

Kathleen F Mittendorf

Michelle a ramos.

3 Department of Population Health Science & Policy Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, Box 1077, New York, NY 10029, USA

Beatriz Anguiano

4 University of California, San Francisco (UCSF) Program in Bioethics, 3333 California Ave (suite 340), San Francisco, CA 94606, USA

Frank Angelo

5 CSER Coordinating Center, University of Washington, Division of Medical Genetics, Health Sciences Building, K-253 Box 357720 Seattle, WA 98195, USA

Galen Joseph

6 University of California San Francisco Department of Humanities & Social Sciences, 1450 3rd Street, Rm. 551 San Francisco, CA 94143, USA

To increase Spanish speakers' representation in genomics research, accessible study materials on genetic topics must be made available in Spanish.

Materials & methods:

The Clinical Sequencing Evidence-Generating Research consortium is evaluating genome sequencing for underserved populations. All sites needed Spanish translation of recruitment materials, surveys and return of results.

We describe our process for translating site-specific materials, as well as shared measures across sites, to inform future efforts to engage Spanish speakers in research.

Conclusion:

In translating and adapting study materials for roughly 1000 Spanish speakers across the USA, and harmonizing translated measures across diverse sites, we overcame numerous challenges. Translation should be performed by professionals. Studies must allocate sufficient time, effort and budget to translate and adapt participant materials.

Lay abstract

To encourage Spanish speakers to join research studies, researchers need to give them written study materials they can easily read and understand. Our study of genome sequencing adapted and translated study materials for use by Spanish speakers across the USA. We describe our process and share our lessons to help others engage Spanish speakers in research. Studies that want to reach Spanish speakers must plan to spend time, effort and money to produce consistent, accurate Spanish-language study materials.

The need to enhance diversity in genomic research is widely recognized. In many cases, diagnosis of an actionable genetic condition can improve clinical outcomes [ 1 ]. However, historically underserved populations, including Hispanic individuals, receive less frequent clinical genetic counseling and testing [ 2–5 ]. Because this disparity also exists in research settings, knowledge about genetic variants is overwhelmingly based on individuals of European ancestry [ 6 , 7 ]. Over 80% of participants in genomic databases are of European ancestry, with only 0.5% of Hispanic ancestry [ 6 , 8 ]. The European bias in genomic studies is likely the result of methodological, systemic, historical and sociocultural factors [ 7 , 9 ]. To reduce disparities, we must find new strategies to include individuals from historically underrepresented groups in genomic research [ 7 , 10 ]. Consortia-based team science is important to leverage the populations needed to accomplish this work.

The Office of Management and Budget (OMB) and the US Census Bureau define Hispanic or Latino as “ a person of Cuban, Mexican, Puerto Rican, South or Central American, or other Spanish culture regardless of race. ” [ 11 ] The terms Hispanic and Latino have different meanings in the USA and abroad. Place of residence, ancestry group or immigration generation may influence individuals' preference for the term Hispanic, Latino, both or neither [ 12 ]. Is this paper, the term Hispanic refers to persons of Spanish or Latin American descent who live in the USA; that is, those who self-identify or trace their roots to Spain or countries in the Americas where Spanish is the predominant language.

The US Census Bureau estimates that there are 58.8 million Hispanics living in the USA – nearly one-fifth (18.1%) of the US population [ 13 ]. The Hispanic category encompasses highly diverse populations with different socioeconomic profiles, migration histories and linguistic characteristics [ 12 ]. At 63%, people of Mexican origin comprise the largest Hispanic subpopulation, followed by mainland Puerto Ricans and Central Americans (each at 9.5%) [ 14 ]. Despite efforts to eliminate disparities [ 15 ], Hispanics lag behind other racial and ethnic groups in access to healthcare and remain overrepresented in the prevalence of diabetes, hypertension, [ 16 , 17 ] advanced-stage cancer [ 18 ] and inadequate cancer screening [ 19–22 ].

Language is a well-recognized barrier to accessing health-related services and participation in research [ 23 ]. Spanish is the most common non-English language spoken in the USA. Among the 41.4 million Spanish speakers in the USA, nearly 40% (or 16.2 million) are of limited English proficiency [ 24 ]. With more than 13% of USA residents speaking Spanish at home [ 25 ], language constitutes a major barrier to Spanish speakers' access to genomic research. To increase Spanish speakers' representation in this research area, accessible, socioculturally coherent study materials on complex genetic topics must be made available in Spanish. A historical mistrust of scientific research can also prevent Hispanics from participating in and receiving the benefits of genetic research [ 26–31 ]. To be fully transparent and address historical mistrust, materials must be written in plain language, clearly stating the process, costs, duration and potential benefits and risks of research participation.

The CSER consortium

The Clinical Sequencing Evidence-Generating Research (CSER) consortium seeks to address the underrepresentation of minority populations in genomics research. This national multi-site research program funded by the National Human Genome Research Institute, the National Cancer Institute and the National Institute on Minority Health and Health Disparities is evaluating the integration of genome sequencing into the clinical care of diverse and medically underserved populations [ 32 ]. Goals include measuring the clinical and personal utility of sequencing and analyzing patient and familial responses to genomic testing in different clinical settings. The consortium model was chosen to meet the need for very large sample sizes to investigate questions of clinical utility, and explore ethical, legal and social implications of genomic sequencing in diverse populations.

The six CSER sites host six separate studies targeting different populations (some adult and some pediatric) and addressing different research questions, but with the common goal of returning genetic findings that may inform treatment decisions and impact clinical care. Each site under the consortium umbrella designed its own set of survey questions and participant-facing materials, although some survey measures (‘harmonized’ measures, Table 1 ) were used by most if not all sites, in order to look at research questions about genetics studies across different subgroups. A detailed description of the CSER sites has been published elsewhere [ 32 ]. All sites set recruitment goals of at least 60% participants from underserved populations, of which a significant portion would be Spanish speakers. Toward this goal [ 33 ], the CSER consortium planned to include Spanish versions of all participant-facing documents for each site, including individual sites' Spanish versions of recruitment materials, surveys and return of results materials, as well as ‘harmonized’ measures that would be used by most CSER sites. Table 2 presents a list of the documents and measures translated by individual sites, along with translation strategies, cost and time spent on translation by each site, and barriers encountered by each site during their translation work.

Table 1. 

Table 2. .

CHARM: Cancer Health Assessments Reaching Many; CSER: Clinical Sequencing Evidence-Generating Research; P3EGS: Program in Prenatal & Pediatric Genomic Sequencing; ROR: Return of Results; UCSF: University of California, San Francisco.

Previous genetic and genomic multisite, multilingual consortia have used various approaches in the production of shared foreign language materials across sites [ 48–50 ], with some consortia having sites translate shared measures independently and some using different surveys already in existence in the foreign language. This approach requires using statistical methods to account for the resulting shared variance across sites. The CSER consortium decided to have each site undertake the Spanish translation of its site-specific measures, and to have a single independent translation of all harmonized measures that would be shared across sites. This paper describes the process followed by four CSER sites for translating their site-specific materials, as well as the process followed in the translation of the harmonized measures that were used across sites. We hope sharing our experience may support future efforts to conduct complex translation work for research consortiums, with a goal of increasing the participation of individuals of limited English proficient (LEP) in genomics research.

Cancer Health Assessments Reaching Many

Cancer Health Assessments Reaching Many (CHARM) is recruiting racially, ethnically and socioeconomically diverse adult primary care patients for risk assessment and genetic testing for hereditary cancer syndromes. CHARM will compare how exome sequencing impacts care utilization and health outcomes for 880 patients versus patients receiving usual care. All study materials are available in print or electronic format in both English and Spanish and bilingual recruitment staff are available. Data collection surveys and telephone interviews are conducted in Spanish or English.

Interested patients complete two validated risk assessment tools for hereditary cancer syndromes (B-RST ™ 3.0 and PREMM 5 ™ , respectively) [ 51 , 52 ] and/or an assessment for limited family structure or family knowledge [ 53 , 54 ]. Patients receive a plain-language summary of their risk results; at risk patients are offered clinical exome sequencing. Using an illustrated, plain-language web tool, patients receive pretest genetic education, consent to genetic testing and research use of information and select categories for secondary findings they want to receive.

Result disclosure is conducted by genetic counselors in English or via a professional Spanish-language interpreter. All participants who receive genetic testing complete surveys administered electronically containing unique-to-CHARM and harmonized measures. Some patients are selected for qualitative interviews (conducted in English or Spanish). Eligible participants who decline genetic testing are offered a survey containing harmonized measures.

CHARM is recruiting participants at Kaiser Permanente Northwest (KPNW) and Denver Health (DH). KPNW is an integrated healthcare system serving over 600,000 members in Northwest Oregon and Southwest Washington. Members are demographically representative of the coverage area. Approximately 30% are non-White, 9% self-identify as Hispanic and nearly 10% are Medicaid recipients. DH is an integrated healthcare system that includes a network of federally qualified health centers. DH serves approximately 150,000 patients. More than 75% of DH patients are racial/ethnic minorities (56% Hispanic, 16% African American), 98% live at or below 200% of the federal poverty level, 15% are uninsured and approximately 70% receive Medicaid or Medicare [ 55 ]. Less than 1% of KPNW patients and 21% of the DH primary care population have a documented need for Spanish interpretation. Targeted recruitment at both sites is used to enrich Spanish-speakers in the study population.

CHARM: site-specific translations

Although the CHARM study planned to enroll Spanish-speaking participants, no specific plans were developed or and the budget allocated for the translation process was limited (US$3040). One of the study co-investigators with expertise in adapting materials for populations of limited literacy (Dr Lindberg) is also a certified and experienced translator; she conducted the translation work. Prior to the Spanish translation, she led a literacy adaptation workgroup that evaluated and edited all CHARM materials for readability in English.

Budget constraints made it impossible to conduct a series of forward and back translations, so CHARM opted for a functionalist-collaborative approach. English to Spanish translations typically raise the literacy level of the text, making it more difficult to read. Our goal was to create a translation that was accurate, as easy to read or more readable than the original and culturally coherent. We created an interdisciplinary translation review team composed of three native Spanish-speaking healthcare providers and ten Spanish-speaking individuals with demographic characteristics mirroring those of anticipated study participants. This team was tasked with examining text that was more difficult or that included English colloquialisms or phrases – like ‘flipping a coin’ – that might involve Spanish regionalisms. Disagreements were resolved by consensus. We sought to produce a translation at a fifth grade-level, verified by the Inflesz program [ 56 ]. Because US Spanish speakers frequently use Anglicized terms (Spanglish), if Anglicisms were used in the translations, alternate Spanish terms were also included. To ensure consistency across CHARM surveys and documents, Dr Lindberg developed a lexicon on Spanish terms used in the Spanish translation of the study documents, as well as the terms used in the response options.

CHARM materials translated in this way ( Table 2 ) included recruitment materials (e.g., postcards, brochures, emails), consent for hereditary cancer risk assessment, a hereditary cancer risk assessment tool, risk assessment results, information about genetic testing for eligible patients, consent for genetic testing and research use of information, genetic testing results letters, letters informing family members about positive genetic findings, participant survey questions unique to CHARM and qualitative interview guides.

Texas KidsCanSeq Study

The Texas KidsCanSeq Study seeks to integrate genomic sequence information into the care of childhood cancer patients with high-risk solid tumors and brain tumors. It aims to enroll pediatric cancer patients and their parents, as well as oncologists from six sites across Texas. In conjunction with Texas Children's Cancer Center and BCM's Genome Laboratory, KidsCanSeq assesses the utility of exome sequence testing compared with more targeted methods in pediatric cancer patients.

Approximately half of the population served by this study's sites is Hispanic, with the majority (80%) of patients being of Mexican origin. Approximately one third of the parents enrolled in the study are Spanish speakers, and most prefer to speak Spanish with their child's doctor. Nearly half of study families live at or below 200% of the federal poverty level. About one third of the parents in the study are uninsured. Nearly half (45%) of the study's pediatric patients are insured through Medicaid and 10% are insured by the Children's Health Insurance Plan.

Spanish-speaking participants are recruited by bilingual study staff. During enrollment, participants choose their preferred language for receiving study communications. Parents watch videos that explain the informed consent process and all aspects of the study. Parents complete surveys at three time points: at enrollment, immediately after results disclosure and 6 months after results disclosure.

Texas KidsCanSeq: site-specific translations

The Texas KidsCanSeq Study has a research assistant (AG) of Nicaraguan and Honduran descent who is bilingual in English and Spanish. She is not a trained, professional, or certified translator but has over 7 years of experience working with Spanish-speaking communities. She manages quality control for all Spanish-language surveys. Recruitment documents, consent forms and site-specific survey measures were translated by a professional company. Enrollment videos scripts were translated into Spanish by a video development team. Table 2 shows documents translated for KidsCanSeq.

Translation strategy: Following completion of translation by professional translation companies, the individual designated as site translator (AG) reviewed translated survey measures for accuracy and further simplified complex language by consulting other Spanish-speaking study staff, and using online Spanish translation resources such as WordReference or Linguee as needed. After participants at one clinical site notified study staff that completion of surveys took up to 2 h, rather than the estimated 30 min, because of language complexity and unclear skip logic on paper surveys, the team decided to informally pilot-test the survey measures with a nonstudy Spanish-speaking population and, following review by bilingual study staff members, feedback on survey wording was incorporated to improve participant experience. Most problems were due to direct translation that did not include adaptations to increase readability.

NYCKidSeq is a New York City-based study recruiting from two large health systems, Mount Sinai Health System and Montefiore Medical Center. The study has four broad goals: to evaluate the clinical utility and diagnostic yield of genomic testing in a diverse population; to improve the delivery of genomic information through a novel communication tool; to engage stakeholders to facilitate implementation of genomic medicine; and to utilize novel electronic health record-based resources to enhance comprehension of genomic results. NYCKidSeq will compare the diagnostic yield of whole genome sequencing with targeted gene panels for 1130 children and young adults with neurologic disorders, primary immunodeficiencies and cardiovascular disorders with suspected genetic etiologies.

NYCKidSeq focuses on pediatric patients (up to 21 years) from predominantly low-income and minority communities in Harlem and the Bronx. Household poverty in the target recruitment areas of East and Central Harlem and the Bronx, ranged from 23.5 to 28% in 2017, significantly higher than the NYC average of 17.9% [ 57 ]. Parents complete questionnaires, in English or Spanish, about themselves and their child. The study estimated that approximately two thirds of participants would be of Black/African or Hispanic ancestry, with some being Spanish-speakers who would require study materials in Spanish and Spanish-speaking staff.

NYCKidSeq: site-specific translations

Translation strategy: For NYCKidSeq materials that were translated ( Table 2 ), six US-born bilingual staff of Latin American descent translated recruitment and retention materials (website information, brochures, hard to reach letters), informed consent forms and survey items specific to NYCKidSeq. None were professional or certified translators. All grew up in exclusively or mostly Spanish-speaking homes, completed Spanish coursework in high school or college and have worked on research projects that recruited Spanish-speaking participants of a variety of ages, countries of origin and literacy levels. All had assisted with translation and administration of study materials for prior projects. To translate patient-facing site materials, one research coordinator would translate a document, another would back-translate it and then the Program Manager (MAR), a native Spanish speaker with a degree in Spanish literature, would review it for accuracy and handle any discrepancies or questions by consulting several online Spanish translation resources such as WordReference or Linguee.

The translated site-specific survey measures and result disclosure communication tool were piloted with a group of NYCKidSeq parents to obtain their feedback, including understandability of the translated survey items and information for Spanish speakers. Their feedback was recorded and provided to MAR who discussed it with the translation team, revised survey items and communicated modifications. If relevant to the harmonized measures, changes were communicated to the harmonized measures translation group for consideration and consortium-wide adoption.

Program in Prenatal & Pediatric Genomic Sequencing

The Program in Prenatal & Pediatric Genomic Sequencing (P3EGS) study is based at the University of California, San Francisco and aims to enroll 200 prenatal and 700 pediatric families to undergo exome sequencing as duos and trios. The prenatal arm is recruiting pregnant women with fetal anomalies detected by ultrasound. The pediatric arm is enrolling patients up to age 25 who present with intellectual disability, metabolic disease, epilepsy, or multiple congenital anomalies. P3EGS is focused on evaluating the clinical utility of exome sequencings as well as addressing the ethical, social and economic issues surrounding genomic testing through consented observations and in-depth interviews. Families who decline exome sequencing are also asked to complete a brief demographic survey as well as an optional interview. Interviews are conducted in-person or over the phone in both English and Spanish.

The study is recruiting from four sites around the San Francisco Bay Area and one site in Fresno. 38% of P3EGS families live below 200% of the federal poverty level, compared with an estimated 25.5% of the Bay Area population overall [ 58 ]. 88% of pediatric participants and 9% of prenatal families are uninsured or enrolled in Medi-Cal/Medicaid. Based on parental self-report 42.1% of P3EGS families identify as Hispanic. Approximately 24% of families utilized a Spanish-speaking medical interpreter and 28% of those asked reported Spanish as the primary language they spoke most often at home [ 59 ]. This reflects the statistics indicating that 28.9% of Californians speak Spanish at home [ 60 ] and demonstrating a clear need to provide Spanish-language study materials for the P3EGS project.

P3EGS: site-specific translations

The P3EGS study has one bilingual research assistant (BA) who is a fluent Spanish speaker of Mexican descent. She is not a certified translator but has over 5 years of experience working with Spanish-speaking communities and has completed 40 h of healthcare interpreter training. She serves as the lead team member in data collection of Spanish surveys, translating research materials and integrating the CSER harmonized survey measures into P3EGS workflows. Translation strategy: Table 2 shows materials translated for P3EGS. The designated site translator (BA) translated the P3EGS interview guides, informational sheets, brochures and integrated the CSER harmonized measures. For all materials, BA completed an initial translation, using the online translation resource SpanishDict as a reference, then a Spanish-proficient study co-investigator reviewed the materials for quality control and consolidation of differences in meaning and word choices, and BA then finalized the materials. The study consent forms were translated into Spanish by a professional company. The prenatal recruitment brochure was reviewed for quality control by additional native Spanish speakers, including a clinician and a clinical research coordinator. The pediatric recruitment brochure was tested with study participants in the pediatric clinic. BA also pretested site-specific survey measures with patients by administering the surveys and assessing their readability based on participant feedback.

Overall translation & adaptation of harmonized consortium measures

Further information about the selection process for measures harmonized across CSER sites can be found in the CSER website [ 61 ]. Preference was given to well established and psychometrically validated measures. In cases where no surveys were available, Consortium investigators (including geneticists, health economists, health service researchers and genetic counselors) developed survey measures which were then translated into Spanish.

Because the CSER consortium focused on medically underserved populations, it was important for patient-facing surveys to be accessible and consistent. While the CSER consortium intended to recruit a significant number of Spanish speakers across sites, there were no specific plans for translating the harmonized measures and no budget was allocated a priori for this work. Given her credentials, translation expertise and experience adapting materials for readability and translating materials for the CHARM study, Dr Lindberg was also tasked with the translation and adaptation of the harmonized measures ( Table 1 ). To facilitate communication between the lead translator and the CSER steering committee as well as the various site translators, a translation coordinator (FA) was appointed. Despite the expectation that harmonized measures would be translated, the CSER Coordinating Center did not provide any specific funding for the translation work.

The linguistic adaptation of the surveys began with assessing readability, using the Flesch-Kincaid grade level formula [ 62 ] to establish a Reading Ease Score. Dr Lindberg then examined the text, modifying it as needed using plain language, familiar terms [ 63 ], concrete terms, avoiding superfluous words and using transition words [ 64 ] (e.g., providing examples, restating, contrasting, or sequencing ideas). A review of the translation work that was completed has yielded a total of approximately 180 pages of English text (~37,000 words) which would have required approximately 90–180 h of work and cost between US$8000–12,000 if done by a professional translator, excluding updates to modified text, or adaptation of text or format for individuals of limited literacy.

Considerations for the harmonized measures

Hispanic individuals of various national origins are united by a common language and by some shared cultural traditions and values. Yet, there are important regional and national linguistic differences across Spanish speakers. We aimed to produce a translation that used a neutral Spanish, without regional characteristics specific to any country. This was of paramount importance because CSER harmonized measures would be administered to Spanish speakers of diverse national origins across the country, from highly acculturated Spanish-speaking US-born Hispanic individuals, to recent immigrants from Mexico, Central and South America and Spain.

To ensure comprehension across different national origins, for Spanish terms that do not have a ‘universal’ equivalent, we provided several terms. To ensure the surveys were culturally accessible, texts were also modified for sociocultural congruency. For example, if a survey of physical activity offered an example involving “ playing golf or skiing ,” we substituted more culturally congruent activities, such as “ bailar o jugar fútbol” ’ (“ dancing or playing soccer” ).

In Spanish, the second person singular formal voice (“ usted ”) creates potential confusion regarding the target of the question ( you vs  he/she ). This was of concern, particularly because some surveys were to be completed by a parent of a child participating in the study. Adding clarifying words would greatly increase survey length and participant burden. Thus, the informal voice (“ tú” ) was used throughout the harmonized measures. To ensure consistency across the translation of harmonized measures, Dr Lindberg developed a lexicon on Spanish terms used in the translated documents, as well as the wording and formats used in the response options of the harmonized measures. Translated harmonized measures from the CSER Consortium are available for public use from: https://cser-consortium.org/cser-research-materials

Review & feedback by site translators

Budget and time constraints prevented multiple translations and back translations. Instead, we followed a similar approach to that used for the CHARM site. After Dr Lindberg translated the harmonized survey measures, each survey was reviewed by three native Spanish speakers (AMG, BA and MAR). Each was experienced with Spanish-speaking patients and had collaborated in the Spanish translation work of their respective CSER sites. The reviewers proofread the translation, provided feedback on wording (both in terms of readability and regional use) and helped ensure consistent terminology across surveys. Proposed edits were discussed in bi-weekly web meetings between the translator (Dr Lindberg), the three reviewers AMG, BA, MAR and the CSER translation coordinator (FA). Changes were made by consensus.

Post-translation adaptation of Spanish text

Following consensus, surveys were again reviewed by the translator and the translation coordinator to improve accessibility. There is a cultural bias associated with the use of Likert-style scales among Hispanics [ 65 ], particularly those with limited formal education, who often have difficulties understanding the graded response format [ 66 , 67 ]. To improve the quality of the resulting data, we modified the wording of some choices to increase clarity. For example, providing statements for each point of the Likert-style scale rather than providing statements for only the extreme anchor points. Similarly, we modified the wording of some response choices where the resulting Spanish terms presented some ambiguity. For instance, for the term “ uncertain ," one Spanish term (“ inseguro” ) may denote feeling unsafe, while another (“ indeciso” ) suggests capriciousness. In those cases, we opted for a more familiar term (“ I am not sure”  – “ no estoy seguro” ). Additional modifications included limiting sentence length, providing clear and concise instructions and using socioculturally appropriate examples.

Survey version control posed a repeated challenge. Multiple updates to the original English-language surveys necessitated corresponding changes to the translated versions, and many versions of the translated surveys were generated prior to reaching final consensus. Then, even after consensus was reached, additional translation work was required as minor changes to the harmonized measures were agreed upon by the wider consortium. The largest change resulted from the decision not to implement a translated survey in any site. While this may represent lost or unnecessary work, harmonizing survey data among several large research projects is a fluid process that requires iteration. Ongoing translation work is to be expected until all measures are complete.

The final harmonized Spanish language measures were posted to the CSER website for the individual consortia sites to download. In some cases, minor modifications were implemented by individual sites where staff did not feel comfortable with some wording, such as the use of the informal ‘ tú ’ that was used in the official translation, and instead used the formal voice.

Discussion & recommendations

In translating and adapting a large volume of complex, genetic-themed material for roughly 1000 Spanish-speaking study participants across the USA, and harmonizing translated measures across diverse study sites, we encountered numerous challenges. Along with the list of documents translated by each CSER site, Table 2 presents available data from the CSER Consortium and four participating sites on budget allocated for the translation work, estimated hours spent on the translation, barriers encountered during this translation process and some solutions that were implemented to address them. In light of our experience, we recommend the following for studies with participants of limited English proficiency:

Translation work must be acknowledged, planned for & prioritized

• Translation may be the most important factor impacting recruitment and data quality. Translation work, like other core elements of a research study, should be performed only by qualified and experienced professionals;
• Studies must include in the original grant specific plans for translation of patient-facing materials, and allocate sufficient time, staff and funding to translate, pilot test and administer materials;
• Because English-language surveys on genetics tend to be written at a high reading level, and translation into Spanish generally increases reading level, we recommend that materials first be adapted for readability [ 68 ] in English, then translated, and then reviewed by experienced bilingual/bicultural study staff, experts in culturally appropriate language, and members of the target population, who then provide feedback to the translator;
• Studies, especially across consortia, must implement training and practice for the standardized administration of surveys. This will allow translators to explain to those administering the surveys the reasoning behind specific wording and format choices and allow survey administrators to provide feedback on the translations.

Carefully select the materials to be translated

• The confusing and imprecise language and inappropriate literacy level of some survey measures were major challenges. Responses can be influenced by the wording, order of items and response options. A feedback loop between scientist, survey developer, translator and cultural expert would improve data quality;
• Given the complexity of genomic information and jargon, it was difficult to balance making a translation accessible for a population of limited literacy and avoiding over-simplification of complex terms. A close partnership between translator, genetic specialists and readability experts could improve this process;
• Many Spanish-language surveys used in the United States have been validated with samples (e.g., Spanish college students) whose country of origin and literacy level differ significantly from those of most Spanish-speaking target populations in the United States. Ideally, projects should use, if available, translated measures that have been validated with populations of similar sociocultural background to the target population. Otherwise, pilot-testing validated measures with members of the target population would improve data quality.

If current trends continue, the Hispanic population in the USA is projected to grow to over 21% of the population, and the number of Spanish speakers is projected to increase to well over 50 million. This suggests that providing services in Spanish to this population will become more critical in the next decade. Particularly in healthcare, accurate and culturally sensitive translation and adaptation of communications will likely become a cornerstone of culturally competent care. This will be especially important as genomic services move into day-to-day clinical care. We hope that the next decade will bring the establishment of guidelines for accurate Spanish translations that faithfully reflect the content and tone of original materials. Standards, increased professionalism and guidelines for translations may improve understanding of health-related information and reduce disparities in the healthcare and health of Hispanic populations.

Summary points

• To reduce disparities in genomics research, we need to include historically underrepresented groups, such as Hispanic Americans.
• Six Clinical Sequencing Evidence-Generating Research (CSER) consortium sites across the USA are enrolling adults and children in genomic research focused on returning findings that may inform clinical care.
• We adapted and translated English-language study materials for Spanish-speaking study participants with low literacy. In translating and adapting materials for roughly 1000 Spanish-speakers across the USA, and harmonizing translated measures across sites, we encountered numerous challenges.
• We provide a detailed account of how we overcame challenges at each study site. We describe our process for translating site-specific materials, as well as for translating shared measures across sites.
• Our experience and the processes we used can inform future efforts to engage Spanish speakers in research.
• Translation may be the most important factor impacting recruitment and data quality. Translation work, like other core elements of a research study, should be performed by qualified and experienced professionals;
• English-language genetics surveys tend to be written at a high reading level, and translation into Spanish generally increases reading level. We recommend that materials first be adapted for readability in English, then translated, and then reviewed by experienced bilingual study staff, experts in culturally appropriate language, and members of the target population, who provide feedback to the translator;
• Studies must standardize survey administration. This will allow translators to explain to survey administrators the reasoning behind wording and format choices and allow survey administrators to provide feedback on the translations;
• The confusing language and inappropriate literacy level of some survey measures were major challenges. A feedback loop between scientist, survey developer, translator and cultural expert would improve data quality;
• It was difficult to balance making a translation accessible for limited literacy and avoiding over-simplifying complex terms. A close partnership between translator, genetic specialists and readability experts could improve this process;
• Many Spanish-language surveys have been validated with samples that differ significantly from those of target populations. Ideally, projects should use translated measures validated with populations of similar background to the target population. Otherwise, pilot-test validated measures with the target population to improve data quality.

Acknowledgments

CHARM: The authors acknowledge J Pope and C Angus of the Kaiser Permanente Center for Health Research for editing and administrative assistance, respectively. KidsCanSeq Study: The authors would like to thank ML Jibaja-Weiss and GS Chauca for their work developing the study educational videos in English and Spanish, AM Recinos for help with translating study materials into Spanish and JO Robinson for assistance reviewing and editing this paper. NYCKidSeq Study: The authors would like to thank J Lopez and E Maria (Albert Einstein College of Medicine/Montefiore Medical Center) and J Rodriguez, N Yelton and KL Aguiñiga (Icahn School of Medicine at Mount Sinai) for their valuable contributions to the translation of NYCKidSeq study materials.

Author contributions

AM Guttierrez, KF Mittendorf, MA Ramos, B Anguiano and F Angelo contributed to the conceptualization of the manuscript. Conducted the translation and/or adaptation work as described in the manuscript. Individually drafted the presented work and revised it critically for content and style, and contributed to the integration of all portions of the manuscript. Provided final approval to all portions of the manuscript to be published. Agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. G Joseph, contributed to the conceptualization of the manuscript. Edited the presented work and revised it critically for content and style, and contributed to the integration of all portions of the manuscript. Provided final approval to all portions of the manuscript to be published. Agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Financial & competing interests disclosure

This work was funded as part of the Clinical Sequencing Evidence-Generating Research (CSER) consortium funded by the National Human Genome Research Institute (NHGRI) (UM1HG007292), with co-funding from the National Institute on Minority Health and Health Disparities (NIMHD) and the National Cancer Institute (NCI), supported by U01HG009610 (Mount Sinai), U01HG009599 (UCSF), U01HG006485 (Baylor College of Medicine), UM1HG007292 (KPNW). The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

No writing assistance was utilized in the production of this manuscript.

Ethical conduct of research

The translation and adaptation work described in this paper preceded subject recruitment and did not involve human subjects. However, the CSER consortium, including the Coordinating Center and all participating sites, obtained IRB approval for all aspects of the research, including final versions of produced materials.

Papers of special note have been highlighted as: • of interest

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

General Translation Strategies

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

This resource provides information on strategies that the students can use when incorporating languages other than English in their academic texts.

You will have to decide whether you need to keep the text in original, translate, or present the readers with both. The decision about the strategy you use for incorporating the non-English materials in your writing should be based on a number of considerations, including:

The familiarity of the language and culture that you expect from your audience

A research paper in your Spanish literature class might draw more heavily on Spanish language, because most of your readers will know some of it.

The attention that you put on the specific vocabulary that you are bringing into your writing

When an author used a particular term in their language and this term has many equivalents in your language. For example, Martin Heidegger coined the German term Dasein , which is often translated into English as “being there” or “presence.” If you substituted discussing the term Dasein with the word presence , the readers might come to the conclusion that it is a term that has no relation to Dasein. This might lead them to believe that you are using it in the original meaning of presence that has no relation to the Heideggerian definition of Dasein .

The effect you want to have on your audience

You can shape your audience’s reading experience and expectations by considering what effect using an untranslated text will have on the readers, in relationship to the purpose you set for your writing. The reason why parts a text might be left untranslated can vary between writers. For instance, you might require the audience to take a more active part in decoding the text and working on the translation on their own. Another reason might be offering the audience the experience of attempting to read a text in a language they have not learned before, in order to challenge them and provide them with that experience. Other reasons that you set for the audience you are writing for are valid too.

• Conjugation
• Pronunciation

THE BEST SPANISH-ENGLISH DICTIONARY

Get more than a translation, written by experts, translate with confidence, spanish and english example sentences, examples for everything, regional translations, say it like a local.

Making educational experiences better for everyone.

Immersive learning for 25 languages

Marketplace for millions of educator-created resources

Fast, easy, reliable language certification

Fun educational games for kids

Comprehensive K-12 personalized learning

Trusted tutors for 300+ subjects

35,000+ worksheets, games, and lesson plans

Adaptive learning for English vocabulary

• PDF to English
• PDF into Spanish
• Translate 100+ languages
• Financial and Banking
• Manufacturing
• Legal Translations
• Retail E-commerce
• Human Resources
• eLearning and Online Learning
• Government and Defense
• Patent Translation
• InDesign (.IDML) to PDF
• Convert PDF to DOCX
• XLSX to PDF
• PPTX to PDF
• TXT to XLSX
• JPEG to PDF
• DOCX to TXT
• Word Document (.DOCX)
• Excel File (.XLSX)
• PowerPoint (.PPT) Slides
• PowerPoint (PPTX) Translation
• InDesign File (.IDML)
• Translate TXT Files
• Translate CSV Files
• Translate JSON Files
• InDesign Word Count
• .DOCX Word Counter
• Excel File Count
• PowerPoint Word Count
• Google Translate PDF
• Translate Large PDF
• PDF Word Count Online
• Translate PDF
• Best Document Translator
• Document Translator
• Free Document Translation Software
• PDF Translator Software
• PDF Language Translator
• AI PDF Summarizer
• AI PDF Traslator
• AI Traslator Tool
• Spanish to English
• French to English
• Portuguese to English
• German to English
• Italian to English
• Russian to English
• Japanese to English
• Chinese to English
• Korean to English
• Turkish to English
• Indonesian to English
• Hindi to English
• Urdu to English
• English to Arabic
• Sign Up Free

Use your Google Account to sign in to DocTranslator

No more passwords to remember. Signing in is fast, simple and secure.

Paper Translator - Free Tool to Translate Paper Online

World’s Best AI Powered Translator At Your Service

Leading organizations around the world trust Doc Translator

How to do Paper Translation?

DocTranslator is a free online tool that allows you to translate documents from one language to another. Here’s a step-by-step guide on how to use DocTranslator for paper translation:

1. Access DocTranslator: – Open your web browser and go to the DocTranslator website (https://dashboard.doctranslator.com/register).

2. Upload Your Document: – Click on the “Choose file” button or drag and drop your paper document onto the webpage. DocTranslator supports various document formats, including PDF, DOC, DOCX, ODT, and TXT.

3. Select Source and Target Languages: – In the dropdown menus provided, select the source language of your document (the language it’s currently in) and the target language (the language you want to translate it into).

4. Optional Settings (if needed): – DocTranslator offers some additional settings you can customize, such as specifying whether the document contains sensitive content or preserving the document’s layout. Adjust these settings according to your preferences.

5. Start Translation: – Click the “Translate” button to begin the translation process. DocTranslator will use its translation engine to convert the text from the source language to the target language.

6. Wait for Translation: – The time it takes for the translation to complete may vary depending on the length and complexity of your document. DocTranslator will display a progress bar, and you can wait until the translation is finished.

7. Download the Translated Document: – Once the translation is complete, DocTranslator will provide a link to download the translated document. Click on the link to save the translated paper to your device.

8. Review and Edit (if necessary): – After downloading the translated document, it’s a good practice to review it for accuracy. Automated translation tools can have limitations, and you may need to make manual corrections or adjustments to ensure the quality of the translation.

Please keep in mind that while automated translation tools like DocTranslator can be helpful, they may not always provide perfect translations, especially for complex or nuanced content. It’s a good idea to double-check and refine the translation as necessary, especially for important documents or academic papers.

What is Paper Translator ?

Paper language translator services have revolutionized the way we break down language barriers and facilitate global communication. These services employ cutting-edge artificial intelligence algorithms to swiftly and accurately translate text from one language to another. Leveraging massive datasets and neural networks, AI translation services can handle a wide range of content, from everyday conversations to complex technical documents.

One of the key advantages of Translate a Paper services is their speed and scalability. They can process large volumes of text in a matter of seconds, making them invaluable for businesses and individuals seeking rapid translation solutions. Furthermore, these services are available 24/7, ensuring access to translation assistance at any time.

While AI translation services offer remarkable convenience, it’s essential to recognize that they may not always capture the nuances and cultural context of language as effectively as human translators. For critical or culturally sensitive content, human involvement may still be necessary to ensure the highest quality and accuracy. Nonetheless, AI translation services continue to evolve and play an integral role in bridging linguistic gaps in our interconnected world.

What is the difference between Translate Paper ?

“Translate Paper” and “Translate Document” are two terms that are often used interchangeably, but they can have slightly different connotations depending on the context. Here’s a breakdown of the key differences:

1. Translate Paper: – “Translate Paper” typically refers to the process of translating a physical document, such as a printed article, essay, or research paper, from one language to another. This involves manually reading the content of the paper in its original language and then providing a translated version in the target language. – “Translate Paper” is a more traditional and manual approach to translation. It may involve the expertise of a human translator who is fluent in both the source and target languages to ensure accurate and contextually appropriate translation.

2. Translate Document: – “Translate Document” can have a broader meaning. It can refer to the translation of various types of documents, including physical papers, digital documents (such as PDFs, Word files, or text files), web pages, emails, and more. – “Translate Document” can encompass both manual translation by human translators and automated translation using tools or software. It often implies a digital format, making it easier to use automated translation tools or online services.

In summary, the main difference lies in the specificity of the term “Translate Paper,” which suggests a physical document, while “Translate Document” has a broader scope and can refer to various types of documents, including both physical and digital formats. The choice between them depends on the nature of the content you want to translate and the tools or methods you intend to use for the translation process.

People already use this power of Paper Translator

How to translate a paper?

Writing academic papers in a language other than your native language can be a daunting task. Fortunately, there are many tools available to help you translate your paper into a different language. Here are some tips on how to translate a paper:

1. Use a Translation Software. There are many translation software products available, such as Google Translate, that can help you quickly and accurately translate your paper into the desired language. Some of these software products are free, while others may require a subscription or purchase. 2. Hire a Professional Translator. If you want a more accurate translation, it is best to hire a professional translator. This is a good option if you are looking to have your paper translated into a language you are unfamiliar with. Professional translators are experienced in translating academic papers and can provide accurate translations. 3. Use a Free Online Translator. If you don’t want to pay for a professional translator, there are some free online translators that can help you translate your paper. These translators may not be as accurate as a professional translator, but they can still get the job done if you are looking for a quick translation. 4. Proofread the Translation. Once you have your paper translated, it is important to proofread the translation to make sure it is accurate. This is especially important if you are using a free online translator, as these translations may not be as accurate as a professional translator.

5. Use DocTranslator tool. DocTranslator is a free online tool that enables users to accurately translate documents from one language to another. The tool is powered by Google Translate and uses machine learning algorithms to deliver accurate translations. DocTranslator offers a variety of document formats such as PDF, Word, Excel, PowerPoint, HTML, and more. It also supports more than 100 languages and can translate to and from any language. To use DocTranslator , users simply upload the document they wish to translate. The tool will then automatically detect the language of the document and the user can select the language they want the document translated into. DocTranslator is a great tool for anyone who needs to quickly and accurately translate documents. It is especially useful for businesses who need to translate documents such as contracts, legal documents, and reports. The tool is easy to use and delivers high quality translations quickly and accurately.

By following these tips, you should be able to easily and accurately translate your paper into a different language. Good luck!

Get 1000 Words Free!

We use cookies to enhance your browsing experience, show personalized advertising or content, and analyze our traffic. By clicking "Accept All" you agree to our use of cookies.

We use cookies to help you navigate effectively and perform certain functions. Detailed information about all the cookies in each consent category can be found below. Cookies categorized as "Necessary" are stored in your browser as they are essential for the functioning of the website's basic features. We also use third-party cookies that help us analyze how you use this website, store your preferences, and provide content and advertising relevant to you. These cookies will be stored in your browser only with your prior consent. You may enable or disable some or all of these cookies, but disabling some of them may affect your online experience.

Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.

Advertisement cookies are used to provide visitors with customized advertisements based on the pages you visited previously and to analyze the effectiveness of the ad campaigns.

Communicate your research to the world

Increase your chances of manuscript acceptance. let us help you..

• Editing Services
• All Services
• Submit Paper

Research communications and research paper editing services var quads_screen_width = document.body.clientWidth; if ( quads_screen_width >= 1140 ) {document.write(' '); (adsbygoogle = window.adsbygoogle || []).push({}); }if ( quads_screen_width >= 1024 && quads_screen_width '); (adsbygoogle = window.adsbygoogle || []).push({}); }if ( quads_screen_width >= 768 && quads_screen_width '); (adsbygoogle = window.adsbygoogle || []).push({}); }if ( quads_screen_width '); (adsbygoogle = window.adsbygoogle || []).push({}); }

Our range of publication services, standard editing.

We check language, sentence construction, punctuation, grammar and ensure that correct technical terminology is used. Learn more

Advanced Editing

We ensure journal requirements are met, check language, syntax, grammar, technical terminology and format your manuscript. Learn more

Premium Editing

We critically evaluate the content of your manuscript, and check language, technical terms, journal requirements and format it. Learn more

Technical Translation

Translation from 11 languages to English using technical translators to ensure your message is conveyed accurately. Learn more

Journal Cover Letter

We can write an effective cover letter to catch the journal editor's attention Learn more

Images and Illustrations

We can create beautiful illustrations or format your images to meet journal requirements. Learn more

Word count reduction

We can reduce the word count of your manuscript by up to 10% to meet journal requirements. Learn more

Why ASK Scientific?

Other services var quads_screen_width = document.body.clientwidth; if ( quads_screen_width >= 1140 ) {document.write(' '); (adsbygoogle = window.adsbygoogle || []).push({}); }if ( quads_screen_width >= 1024 && quads_screen_width '); (adsbygoogle = window.adsbygoogle || []).push({}); }if ( quads_screen_width >= 768 && quads_screen_width '); (adsbygoogle = window.adsbygoogle || []).push({}); }if ( quads_screen_width '); (adsbygoogle = window.adsbygoogle || []).push({}); }, technical writing, preclinical documentation, services for organisations, scientific writing training, some organisations we have worked with.

What research paper editing services does your company offer?

My research paper is not in english, can you translate it, do you comment on a manuscript's content in your research paper editing services, can you convert my manuscript to latex format, articles from our blog.

Norovirus (winter vomiting bug): life cycle, pathogenicity and prevention

ASK Scientific 2018-07-17T18:25:45+01:00 November 13th, 2017 | Comments Off on Norovirus (winter vomiting bug): life cycle, pathogenicity and prevention

Prevention, treatment, prognosis, diagnosis, complications of Zika Virus and known measures to restrict further spread and recurrence

ASK Scientific 2018-07-17T15:08:58+01:00 July 7th, 2016 | Comments Off on Prevention, treatment, prognosis, diagnosis, complications of Zika Virus and known measures to restrict further spread and recurrence

ASK Scientific’s new Technical Translator Tool

ASK Scientific 2017-05-17T11:30:35+01:00 July 1st, 2016 | Comments Off on ASK Scientific’s new Technical Translator Tool

Zika Virus: Life cycle and pathogenicity in humans

ASK Scientific Ltd St John's Innovation Centre Cowley Road Cambridge CB4 0WS United Kingdom

Company no.: 08660128 VAT no.: GB182934091

[table id=21 /] [table id=2 /]

Quill And Fox

Best Design Studio

Amber Robertson

Amber Robertson is the founder of Quill and Fox. A creative writing studio that helps people find their voice and share their stories. Amber is also a published author, with her first book slated for release in 2020. She loves spending time with her family and friends, reading, writing, and traveling. When Amber was younger, she loved to write short stories and plays. But somewhere along the way, she lost touch with her creativity. It wasn’t until she became a mom that she realized how important it was to share her stories—both the good and the bad—with the people she loves most. That’s when Quill and Fox was born. Amber is passionate about helping others find their voice and share their stories. She believes that every person has a story worth telling, and it’s her mission to help them tell it in a way that is authentic and true to themselves

Research Paper In Spanish

Table of Contents

If you are looking to communicate in Spanish using the word “research”, it is essential to understand the correct translations and how to pronounce them in the right context. In this article, we will discuss how to say “research” in Spanish, what is “research” in Spanish, and the meaning of “research” in Spanish, among other things.

Fast track your vocabulary with the 10.000 most common Spanish words!

What is “Research” in Spanish?

The Spanish word for research is “investigación” (IPA: /in.ves.ti.ɣa.ˈθjon/). This word is derived from the verb investigar (IPA: / in.bestiˈɣaɾ /), which means “to investigate”. The stress in this word is on the second to the last syllable, “ga”.

Let us start with the basic question: what is “research” in Spanish? The Spanish word for research is “investigación”, and it is used in Spain, as well as in most Latin American countries. However, it is important to note that there are some regional variations in the Spanish language. For example, in some South American countries such as Argentina, Colombia, and Uruguay, the term “investigación” may be replaced with the term indagación (IPA: /indaɣaˈθjon/).

Now, let us take a closer look at the meaning of “research” in Spanish. The word “investigación” refers to the process of searching, exploring, and analyzing information to discover new facts or validate existing ones. This term can be used in a variety of contexts, such as scientific research, market research, or academic research.

If you need to translate “research” to Spanish, the correct term is “investigación”. However, there are some other translations that you might come across, such as estudio (/esˈtu.djo/), análisis (/aˈna.li.sis/), or investigaciones (/in.be̞s.ti.ɣaˈθjo.nes/). While these translations are not incorrect, “investigación” is the most common and accurate translation for the term “research” in Spanish.

Meaning of “ Spanish Translation ” in Spanish

The meaning of “research” in Spanish is very similar to its English counterpart. It refers to the process of gathering information and analyzing it to find new knowledge or solutions to problems. “Investigación” can be used in academic, scientific, or business contexts. It can also refer to a particular study or project that involves research.

4 eBooks of the Spanish Frequency Dictionaries series by MostUsedWords

Regional Differences

There are some regional differences in the use of the word “investigación” in Spanish. In Spain, for example, the word “estudio” is sometimes used instead of “investigación” to refer to research, particularly in the social sciences. However, this usage is less common in Latin American countries.

In some Latin American countries, particularly Mexico and some Central American countries, the word “investigación” can also have a negative connotation, particularly when used in the context of law enforcement. This is because “investigación” can also mean “investigation” in the sense of criminal investigations.

You can find the paperbacks on Amazon (we have frequency dictionaries for beginners , intermediates , advanced and near-fluent students ), or get the eBooks directly from us here. (They are affiliate links. That means we might get a small commission if you make a purchase after clicking these links, at no extra cost to you.)

How to Say “ Research ” in Spanish: Sample Sentences

Here are five sample sentences you can use to say “research” in Spanish:

• Necesito hacer algo de investigación para mi ensayo.

(I need to do some research for my paper.)

• Realizamos un estudio de investigación sobre los efectos del cambio climático.

(We conducted a research study on the effects of climate change.)

• The company invested in research and development to create a new product.

(La empresa invirtió en investigación y desarrollo para crear un nuevo producto.)

• Ella es una experta en métodos de investigación.

(She is an expert in research methods.)

• El profesor nos asignó un proyecto de investigación sobre la historia de la región.

(The professor assigned us a research project on the history of the region.)

In conclusion, “investigación” is the most common and accurate translation of “research” in Spanish. Although there are some regional differences in the use of this word, it is widely understood throughout the Spanish-speaking world. Knowing how to say research in Spanish can be useful for students, researchers, and business professionals who work with Spanish-speaking colleagues or clients.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 26 March 2024

Predicting and improving complex beer flavor through machine learning

• Michiel Schreurs   ORCID: orcid.org/0000-0002-9449-5619 1 , 2 , 3   na1 ,
• Supinya Piampongsant 1 , 2 , 3   na1 ,
• Miguel Roncoroni   ORCID: orcid.org/0000-0001-7461-1427 1 , 2 , 3   na1 ,
• Lloyd Cool   ORCID: orcid.org/0000-0001-9936-3124 1 , 2 , 3 , 4 ,
• Beatriz Herrera-Malaver   ORCID: orcid.org/0000-0002-5096-9974 1 , 2 , 3 ,
• Christophe Vanderaa   ORCID: orcid.org/0000-0001-7443-5427 4 ,
• Florian A. Theßeling 1 , 2 , 3 ,
• Łukasz Kreft   ORCID: orcid.org/0000-0001-7620-4657 5 ,
• Alexander Botzki   ORCID: orcid.org/0000-0001-6691-4233 5 ,
• Philippe Malcorps 6 ,
• Luk Daenen 6 ,
• Tom Wenseleers   ORCID: orcid.org/0000-0002-1434-861X 4 &
• Kevin J. Verstrepen   ORCID: orcid.org/0000-0002-3077-6219 1 , 2 , 3  

Nature Communications volume  15 , Article number:  2368 ( 2024 ) Cite this article

27k Accesses

720 Altmetric

Metrics details

• Chemical engineering
• Gas chromatography
• Machine learning
• Metabolomics
• Taste receptors

The perception and appreciation of food flavor depends on many interacting chemical compounds and external factors, and therefore proves challenging to understand and predict. Here, we combine extensive chemical and sensory analyses of 250 different beers to train machine learning models that allow predicting flavor and consumer appreciation. For each beer, we measure over 200 chemical properties, perform quantitative descriptive sensory analysis with a trained tasting panel and map data from over 180,000 consumer reviews to train 10 different machine learning models. The best-performing algorithm, Gradient Boosting, yields models that significantly outperform predictions based on conventional statistics and accurately predict complex food features and consumer appreciation from chemical profiles. Model dissection allows identifying specific and unexpected compounds as drivers of beer flavor and appreciation. Adding these compounds results in variants of commercial alcoholic and non-alcoholic beers with improved consumer appreciation. Together, our study reveals how big data and machine learning uncover complex links between food chemistry, flavor and consumer perception, and lays the foundation to develop novel, tailored foods with superior flavors.

Similar content being viewed by others

High-speed and large-scale intrinsically stretchable integrated circuits

Donglai Zhong, Can Wu, … Zhenan Bao

Edible mycelium bioengineered for enhanced nutritional value and sensory appeal using a modular synthetic biology toolkit

Vayu Maini Rekdal, Casper R. B. van der Luijt, … Jay D. Keasling

The environmental price of fast fashion

Kirsi Niinimäki, Greg Peters, … Alison Gwilt

Introduction

Predicting and understanding food perception and appreciation is one of the major challenges in food science. Accurate modeling of food flavor and appreciation could yield important opportunities for both producers and consumers, including quality control, product fingerprinting, counterfeit detection, spoilage detection, and the development of new products and product combinations (food pairing) 1 , 2 , 3 , 4 , 5 , 6 . Accurate models for flavor and consumer appreciation would contribute greatly to our scientific understanding of how humans perceive and appreciate flavor. Moreover, accurate predictive models would also facilitate and standardize existing food assessment methods and could supplement or replace assessments by trained and consumer tasting panels, which are variable, expensive and time-consuming 7 , 8 , 9 . Lastly, apart from providing objective, quantitative, accurate and contextual information that can help producers, models can also guide consumers in understanding their personal preferences 10 .

Despite the myriad of applications, predicting food flavor and appreciation from its chemical properties remains a largely elusive goal in sensory science, especially for complex food and beverages 11 , 12 . A key obstacle is the immense number of flavor-active chemicals underlying food flavor. Flavor compounds can vary widely in chemical structure and concentration, making them technically challenging and labor-intensive to quantify, even in the face of innovations in metabolomics, such as non-targeted metabolic fingerprinting 13 , 14 . Moreover, sensory analysis is perhaps even more complicated. Flavor perception is highly complex, resulting from hundreds of different molecules interacting at the physiochemical and sensorial level. Sensory perception is often non-linear, characterized by complex and concentration-dependent synergistic and antagonistic effects 15 , 16 , 17 , 18 , 19 , 20 , 21 that are further convoluted by the genetics, environment, culture and psychology of consumers 22 , 23 , 24 . Perceived flavor is therefore difficult to measure, with problems of sensitivity, accuracy, and reproducibility that can only be resolved by gathering sufficiently large datasets 25 . Trained tasting panels are considered the prime source of quality sensory data, but require meticulous training, are low throughput and high cost. Public databases containing consumer reviews of food products could provide a valuable alternative, especially for studying appreciation scores, which do not require formal training 25 . Public databases offer the advantage of amassing large amounts of data, increasing the statistical power to identify potential drivers of appreciation. However, public datasets suffer from biases, including a bias in the volunteers that contribute to the database, as well as confounding factors such as price, cult status and psychological conformity towards previous ratings of the product.

Classical multivariate statistics and machine learning methods have been used to predict flavor of specific compounds by, for example, linking structural properties of a compound to its potential biological activities or linking concentrations of specific compounds to sensory profiles 1 , 26 . Importantly, most previous studies focused on predicting organoleptic properties of single compounds (often based on their chemical structure) 27 , 28 , 29 , 30 , 31 , 32 , 33 , thus ignoring the fact that these compounds are present in a complex matrix in food or beverages and excluding complex interactions between compounds. Moreover, the classical statistics commonly used in sensory science 34 , 35 , 36 , 37 , 38 , 39 require a large sample size and sufficient variance amongst predictors to create accurate models. They are not fit for studying an extensive set of hundreds of interacting flavor compounds, since they are sensitive to outliers, have a high tendency to overfit and are less suited for non-linear and discontinuous relationships 40 .

In this study, we combine extensive chemical analyses and sensory data of a set of different commercial beers with machine learning approaches to develop models that predict taste, smell, mouthfeel and appreciation from compound concentrations. Beer is particularly suited to model the relationship between chemistry, flavor and appreciation. First, beer is a complex product, consisting of thousands of flavor compounds that partake in complex sensory interactions 41 , 42 , 43 . This chemical diversity arises from the raw materials (malt, yeast, hops, water and spices) and biochemical conversions during the brewing process (kilning, mashing, boiling, fermentation, maturation and aging) 44 , 45 . Second, the advent of the internet saw beer consumers embrace online review platforms, such as RateBeer (ZX Ventures, Anheuser-Busch InBev SA/NV) and BeerAdvocate (Next Glass, inc.). In this way, the beer community provides massive data sets of beer flavor and appreciation scores, creating extraordinarily large sensory databases to complement the analyses of our professional sensory panel. Specifically, we characterize over 200 chemical properties of 250 commercial beers, spread across 22 beer styles, and link these to the descriptive sensory profiling data of a 16-person in-house trained tasting panel and data acquired from over 180,000 public consumer reviews. These unique and extensive datasets enable us to train a suite of machine learning models to predict flavor and appreciation from a beer’s chemical profile. Dissection of the best-performing models allows us to pinpoint specific compounds as potential drivers of beer flavor and appreciation. Follow-up experiments confirm the importance of these compounds and ultimately allow us to significantly improve the flavor and appreciation of selected commercial beers. Together, our study represents a significant step towards understanding complex flavors and reinforces the value of machine learning to develop and refine complex foods. In this way, it represents a stepping stone for further computer-aided food engineering applications 46 .

To generate a comprehensive dataset on beer flavor, we selected 250 commercial Belgian beers across 22 different beer styles (Supplementary Fig.  S1 ). Beers with ≤ 4.2% alcohol by volume (ABV) were classified as non-alcoholic and low-alcoholic. Blonds and Tripels constitute a significant portion of the dataset (12.4% and 11.2%, respectively) reflecting their presence on the Belgian beer market and the heterogeneity of beers within these styles. By contrast, lager beers are less diverse and dominated by a handful of brands. Rare styles such as Brut or Faro make up only a small fraction of the dataset (2% and 1%, respectively) because fewer of these beers are produced and because they are dominated by distinct characteristics in terms of flavor and chemical composition.

Extensive analysis identifies relationships between chemical compounds in beer

For each beer, we measured 226 different chemical properties, including common brewing parameters such as alcohol content, iso-alpha acids, pH, sugar concentration 47 , and over 200 flavor compounds (Methods, Supplementary Table  S1 ). A large portion (37.2%) are terpenoids arising from hopping, responsible for herbal and fruity flavors 16 , 48 . A second major category are yeast metabolites, such as esters and alcohols, that result in fruity and solvent notes 48 , 49 , 50 . Other measured compounds are primarily derived from malt, or other microbes such as non- Saccharomyces yeasts and bacteria (‘wild flora’). Compounds that arise from spices or staling are labeled under ‘Others’. Five attributes (caloric value, total acids and total ester, hop aroma and sulfur compounds) are calculated from multiple individually measured compounds.

As a first step in identifying relationships between chemical properties, we determined correlations between the concentrations of the compounds (Fig.  1 , upper panel, Supplementary Data  1 and 2 , and Supplementary Fig.  S2 . For the sake of clarity, only a subset of the measured compounds is shown in Fig.  1 ). Compounds of the same origin typically show a positive correlation, while absence of correlation hints at parameters varying independently. For example, the hop aroma compounds citronellol, and alpha-terpineol show moderate correlations with each other (Spearman’s rho=0.39 and 0.57), but not with the bittering hop component iso-alpha acids (Spearman’s rho=0.16 and −0.07). This illustrates how brewers can independently modify hop aroma and bitterness by selecting hop varieties and dosage time. If hops are added early in the boiling phase, chemical conversions increase bitterness while aromas evaporate, conversely, late addition of hops preserves aroma but limits bitterness 51 . Similarly, hop-derived iso-alpha acids show a strong anti-correlation with lactic acid and acetic acid, likely reflecting growth inhibition of lactic acid and acetic acid bacteria, or the consequent use of fewer hops in sour beer styles, such as West Flanders ales and Fruit beers, that rely on these bacteria for their distinct flavors 52 . Finally, yeast-derived esters (ethyl acetate, ethyl decanoate, ethyl hexanoate, ethyl octanoate) and alcohols (ethanol, isoamyl alcohol, isobutanol, and glycerol), correlate with Spearman coefficients above 0.5, suggesting that these secondary metabolites are correlated with the yeast genetic background and/or fermentation parameters and may be difficult to influence individually, although the choice of yeast strain may offer some control 53 .

Spearman rank correlations are shown. Descriptors are grouped according to their origin (malt (blue), hops (green), yeast (red), wild flora (yellow), Others (black)), and sensory aspect (aroma, taste, palate, and overall appreciation). Please note that for the chemical compounds, for the sake of clarity, only a subset of the total number of measured compounds is shown, with an emphasis on the key compounds for each source. For more details, see the main text and Methods section. Chemical data can be found in Supplementary Data  1 , correlations between all chemical compounds are depicted in Supplementary Fig.  S2 and correlation values can be found in Supplementary Data  2 . See Supplementary Data  4 for sensory panel assessments and Supplementary Data  5 for correlation values between all sensory descriptors.

Interestingly, different beer styles show distinct patterns for some flavor compounds (Supplementary Fig.  S3 ). These observations agree with expectations for key beer styles, and serve as a control for our measurements. For instance, Stouts generally show high values for color (darker), while hoppy beers contain elevated levels of iso-alpha acids, compounds associated with bitter hop taste. Acetic and lactic acid are not prevalent in most beers, with notable exceptions such as Kriek, Lambic, Faro, West Flanders ales and Flanders Old Brown, which use acid-producing bacteria ( Lactobacillus and Pediococcus ) or unconventional yeast ( Brettanomyces ) 54 , 55 . Glycerol, ethanol and esters show similar distributions across all beer styles, reflecting their common origin as products of yeast metabolism during fermentation 45 , 53 . Finally, low/no-alcohol beers contain low concentrations of glycerol and esters. This is in line with the production process for most of the low/no-alcohol beers in our dataset, which are produced through limiting fermentation or by stripping away alcohol via evaporation or dialysis, with both methods having the unintended side-effect of reducing the amount of flavor compounds in the final beer 56 , 57 .

Besides expected associations, our data also reveals less trivial associations between beer styles and specific parameters. For example, geraniol and citronellol, two monoterpenoids responsible for citrus, floral and rose flavors and characteristic of Citra hops, are found in relatively high amounts in Christmas, Saison, and Brett/co-fermented beers, where they may originate from terpenoid-rich spices such as coriander seeds instead of hops 58 .

Tasting panel assessments reveal sensorial relationships in beer

To assess the sensory profile of each beer, a trained tasting panel evaluated each of the 250 beers for 50 sensory attributes, including different hop, malt and yeast flavors, off-flavors and spices. Panelists used a tasting sheet (Supplementary Data  3 ) to score the different attributes. Panel consistency was evaluated by repeating 12 samples across different sessions and performing ANOVA. In 95% of cases no significant difference was found across sessions ( p  > 0.05), indicating good panel consistency (Supplementary Table  S2 ).

Aroma and taste perception reported by the trained panel are often linked (Fig.  1 , bottom left panel and Supplementary Data  4 and 5 ), with high correlations between hops aroma and taste (Spearman’s rho=0.83). Bitter taste was found to correlate with hop aroma and taste in general (Spearman’s rho=0.80 and 0.69), and particularly with “grassy” noble hops (Spearman’s rho=0.75). Barnyard flavor, most often associated with sour beers, is identified together with stale hops (Spearman’s rho=0.97) that are used in these beers. Lactic and acetic acid, which often co-occur, are correlated (Spearman’s rho=0.66). Interestingly, sweetness and bitterness are anti-correlated (Spearman’s rho = −0.48), confirming the hypothesis that they mask each other 59 , 60 . Beer body is highly correlated with alcohol (Spearman’s rho = 0.79), and overall appreciation is found to correlate with multiple aspects that describe beer mouthfeel (alcohol, carbonation; Spearman’s rho= 0.32, 0.39), as well as with hop and ester aroma intensity (Spearman’s rho=0.39 and 0.35).

Similar to the chemical analyses, sensorial analyses confirmed typical features of specific beer styles (Supplementary Fig.  S4 ). For example, sour beers (Faro, Flanders Old Brown, Fruit beer, Kriek, Lambic, West Flanders ale) were rated acidic, with flavors of both acetic and lactic acid. Hoppy beers were found to be bitter and showed hop-associated aromas like citrus and tropical fruit. Malt taste is most detected among scotch, stout/porters, and strong ales, while low/no-alcohol beers, which often have a reputation for being ‘worty’ (reminiscent of unfermented, sweet malt extract) appear in the middle. Unsurprisingly, hop aromas are most strongly detected among hoppy beers. Like its chemical counterpart (Supplementary Fig.  S3 ), acidity shows a right-skewed distribution, with the most acidic beers being Krieks, Lambics, and West Flanders ales.

Tasting panel assessments of specific flavors correlate with chemical composition

We find that the concentrations of several chemical compounds strongly correlate with specific aroma or taste, as evaluated by the tasting panel (Fig.  2 , Supplementary Fig.  S5 , Supplementary Data  6 ). In some cases, these correlations confirm expectations and serve as a useful control for data quality. For example, iso-alpha acids, the bittering compounds in hops, strongly correlate with bitterness (Spearman’s rho=0.68), while ethanol and glycerol correlate with tasters’ perceptions of alcohol and body, the mouthfeel sensation of fullness (Spearman’s rho=0.82/0.62 and 0.72/0.57 respectively) and darker color from roasted malts is a good indication of malt perception (Spearman’s rho=0.54).

Heatmap colors indicate Spearman’s Rho. Axes are organized according to sensory categories (aroma, taste, mouthfeel, overall), chemical categories and chemical sources in beer (malt (blue), hops (green), yeast (red), wild flora (yellow), Others (black)). See Supplementary Data  6 for all correlation values.

Interestingly, for some relationships between chemical compounds and perceived flavor, correlations are weaker than expected. For example, the rose-smelling phenethyl acetate only weakly correlates with floral aroma. This hints at more complex relationships and interactions between compounds and suggests a need for a more complex model than simple correlations. Lastly, we uncovered unexpected correlations. For instance, the esters ethyl decanoate and ethyl octanoate appear to correlate slightly with hop perception and bitterness, possibly due to their fruity flavor. Iron is anti-correlated with hop aromas and bitterness, most likely because it is also anti-correlated with iso-alpha acids. This could be a sign of metal chelation of hop acids 61 , given that our analyses measure unbound hop acids and total iron content, or could result from the higher iron content in dark and Fruit beers, which typically have less hoppy and bitter flavors 62 .

Public consumer reviews complement expert panel data

To complement and expand the sensory data of our trained tasting panel, we collected 180,000 reviews of our 250 beers from the online consumer review platform RateBeer. This provided numerical scores for beer appearance, aroma, taste, palate, overall quality as well as the average overall score.

Public datasets are known to suffer from biases, such as price, cult status and psychological conformity towards previous ratings of a product. For example, prices correlate with appreciation scores for these online consumer reviews (rho=0.49, Supplementary Fig.  S6 ), but not for our trained tasting panel (rho=0.19). This suggests that prices affect consumer appreciation, which has been reported in wine 63 , while blind tastings are unaffected. Moreover, we observe that some beer styles, like lagers and non-alcoholic beers, generally receive lower scores, reflecting that online reviewers are mostly beer aficionados with a preference for specialty beers over lager beers. In general, we find a modest correlation between our trained panel’s overall appreciation score and the online consumer appreciation scores (Fig.  3 , rho=0.29). Apart from the aforementioned biases in the online datasets, serving temperature, sample freshness and surroundings, which are all tightly controlled during the tasting panel sessions, can vary tremendously across online consumers and can further contribute to (among others, appreciation) differences between the two categories of tasters. Importantly, in contrast to the overall appreciation scores, for many sensory aspects the results from the professional panel correlated well with results obtained from RateBeer reviews. Correlations were highest for features that are relatively easy to recognize even for untrained tasters, like bitterness, sweetness, alcohol and malt aroma (Fig.  3 and below).

RateBeer text mining results can be found in Supplementary Data  7 . Rho values shown are Spearman correlation values, with asterisks indicating significant correlations ( p  < 0.05, two-sided). All p values were smaller than 0.001, except for Esters aroma (0.0553), Esters taste (0.3275), Esters aroma—banana (0.0019), Coriander (0.0508) and Diacetyl (0.0134).

Besides collecting consumer appreciation from these online reviews, we developed automated text analysis tools to gather additional data from review texts (Supplementary Data  7 ). Processing review texts on the RateBeer database yielded comparable results to the scores given by the trained panel for many common sensory aspects, including acidity, bitterness, sweetness, alcohol, malt, and hop tastes (Fig.  3 ). This is in line with what would be expected, since these attributes require less training for accurate assessment and are less influenced by environmental factors such as temperature, serving glass and odors in the environment. Consumer reviews also correlate well with our trained panel for 4-vinyl guaiacol, a compound associated with a very characteristic aroma. By contrast, correlations for more specific aromas like ester, coriander or diacetyl are underrepresented in the online reviews, underscoring the importance of using a trained tasting panel and standardized tasting sheets with explicit factors to be scored for evaluating specific aspects of a beer. Taken together, our results suggest that public reviews are trustworthy for some, but not all, flavor features and can complement or substitute taste panel data for these sensory aspects.

Models can predict beer sensory profiles from chemical data

The rich datasets of chemical analyses, tasting panel assessments and public reviews gathered in the first part of this study provided us with a unique opportunity to develop predictive models that link chemical data to sensorial features. Given the complexity of beer flavor, basic statistical tools such as correlations or linear regression may not always be the most suitable for making accurate predictions. Instead, we applied different machine learning models that can model both simple linear and complex interactive relationships. Specifically, we constructed a set of regression models to predict (a) trained panel scores for beer flavor and quality and (b) public reviews’ appreciation scores from beer chemical profiles. We trained and tested 10 different models (Methods), 3 linear regression-based models (simple linear regression with first-order interactions (LR), lasso regression with first-order interactions (Lasso), partial least squares regressor (PLSR)), 5 decision tree models (AdaBoost regressor (ABR), extra trees (ET), gradient boosting regressor (GBR), random forest (RF) and XGBoost regressor (XGBR)), 1 support vector regression (SVR), and 1 artificial neural network (ANN) model.

To compare the performance of our machine learning models, the dataset was randomly split into a training and test set, stratified by beer style. After a model was trained on data in the training set, its performance was evaluated on its ability to predict the test dataset obtained from multi-output models (based on the coefficient of determination, see Methods). Additionally, individual-attribute models were ranked per descriptor and the average rank was calculated, as proposed by Korneva et al. 64 . Importantly, both ways of evaluating the models’ performance agreed in general. Performance of the different models varied (Table  1 ). It should be noted that all models perform better at predicting RateBeer results than results from our trained tasting panel. One reason could be that sensory data is inherently variable, and this variability is averaged out with the large number of public reviews from RateBeer. Additionally, all tree-based models perform better at predicting taste than aroma. Linear models (LR) performed particularly poorly, with negative R 2 values, due to severe overfitting (training set R 2  = 1). Overfitting is a common issue in linear models with many parameters and limited samples, especially with interaction terms further amplifying the number of parameters. L1 regularization (Lasso) successfully overcomes this overfitting, out-competing multiple tree-based models on the RateBeer dataset. Similarly, the dimensionality reduction of PLSR avoids overfitting and improves performance, to some extent. Still, tree-based models (ABR, ET, GBR, RF and XGBR) show the best performance, out-competing the linear models (LR, Lasso, PLSR) commonly used in sensory science 65 .

GBR models showed the best overall performance in predicting sensory responses from chemical information, with R 2 values up to 0.75 depending on the predicted sensory feature (Supplementary Table  S4 ). The GBR models predict consumer appreciation (RateBeer) better than our trained panel’s appreciation (R 2 value of 0.67 compared to R 2 value of 0.09) (Supplementary Table  S3 and Supplementary Table  S4 ). ANN models showed intermediate performance, likely because neural networks typically perform best with larger datasets 66 . The SVR shows intermediate performance, mostly due to the weak predictions of specific attributes that lower the overall performance (Supplementary Table  S4 ).

Model dissection identifies specific, unexpected compounds as drivers of consumer appreciation

Next, we leveraged our models to infer important contributors to sensory perception and consumer appreciation. Consumer preference is a crucial sensory aspects, because a product that shows low consumer appreciation scores often does not succeed commercially 25 . Additionally, the requirement for a large number of representative evaluators makes consumer trials one of the more costly and time-consuming aspects of product development. Hence, a model for predicting chemical drivers of overall appreciation would be a welcome addition to the available toolbox for food development and optimization.

Since GBR models on our RateBeer dataset showed the best overall performance, we focused on these models. Specifically, we used two approaches to identify important contributors. First, rankings of the most important predictors for each sensorial trait in the GBR models were obtained based on impurity-based feature importance (mean decrease in impurity). High-ranked parameters were hypothesized to be either the true causal chemical properties underlying the trait, to correlate with the actual causal properties, or to take part in sensory interactions affecting the trait 67 (Fig.  4A ). In a second approach, we used SHAP 68 to determine which parameters contributed most to the model for making predictions of consumer appreciation (Fig.  4B ). SHAP calculates parameter contributions to model predictions on a per-sample basis, which can be aggregated into an importance score.

A The impurity-based feature importance (mean deviance in impurity, MDI) calculated from the Gradient Boosting Regression (GBR) model predicting RateBeer appreciation scores. The top 15 highest ranked chemical properties are shown. B SHAP summary plot for the top 15 parameters contributing to our GBR model. Each point on the graph represents a sample from our dataset. The color represents the concentration of that parameter, with bluer colors representing low values and redder colors representing higher values. Greater absolute values on the horizontal axis indicate a higher impact of the parameter on the prediction of the model. C Spearman correlations between the 15 most important chemical properties and consumer overall appreciation. Numbers indicate the Spearman Rho correlation coefficient, and the rank of this correlation compared to all other correlations. The top 15 important compounds were determined using SHAP (panel B).

Both approaches identified ethyl acetate as the most predictive parameter for beer appreciation (Fig.  4 ). Ethyl acetate is the most abundant ester in beer with a typical ‘fruity’, ‘solvent’ and ‘alcoholic’ flavor, but is often considered less important than other esters like isoamyl acetate. The second most important parameter identified by SHAP is ethanol, the most abundant beer compound after water. Apart from directly contributing to beer flavor and mouthfeel, ethanol drastically influences the physical properties of beer, dictating how easily volatile compounds escape the beer matrix to contribute to beer aroma 69 . Importantly, it should also be noted that the importance of ethanol for appreciation is likely inflated by the very low appreciation scores of non-alcoholic beers (Supplementary Fig.  S4 ). Despite not often being considered a driver of beer appreciation, protein level also ranks highly in both approaches, possibly due to its effect on mouthfeel and body 70 . Lactic acid, which contributes to the tart taste of sour beers, is the fourth most important parameter identified by SHAP, possibly due to the generally high appreciation of sour beers in our dataset.

Interestingly, some of the most important predictive parameters for our model are not well-established as beer flavors or are even commonly regarded as being negative for beer quality. For example, our models identify methanethiol and ethyl phenyl acetate, an ester commonly linked to beer staling 71 , as a key factor contributing to beer appreciation. Although there is no doubt that high concentrations of these compounds are considered unpleasant, the positive effects of modest concentrations are not yet known 72 , 73 .

To compare our approach to conventional statistics, we evaluated how well the 15 most important SHAP-derived parameters correlate with consumer appreciation (Fig.  4C ). Interestingly, only 6 of the properties derived by SHAP rank amongst the top 15 most correlated parameters. For some chemical compounds, the correlations are so low that they would have likely been considered unimportant. For example, lactic acid, the fourth most important parameter, shows a bimodal distribution for appreciation, with sour beers forming a separate cluster, that is missed entirely by the Spearman correlation. Additionally, the correlation plots reveal outliers, emphasizing the need for robust analysis tools. Together, this highlights the need for alternative models, like the Gradient Boosting model, that better grasp the complexity of (beer) flavor.

Finally, to observe the relationships between these chemical properties and their predicted targets, partial dependence plots were constructed for the six most important predictors of consumer appreciation 74 , 75 , 76 (Supplementary Fig.  S7 ). One-way partial dependence plots show how a change in concentration affects the predicted appreciation. These plots reveal an important limitation of our models: appreciation predictions remain constant at ever-increasing concentrations. This implies that once a threshold concentration is reached, further increasing the concentration does not affect appreciation. This is false, as it is well-documented that certain compounds become unpleasant at high concentrations, including ethyl acetate (‘nail polish’) 77 and methanethiol (‘sulfury’ and ‘rotten cabbage’) 78 . The inability of our models to grasp that flavor compounds have optimal levels, above which they become negative, is a consequence of working with commercial beer brands where (off-)flavors are rarely too high to negatively impact the product. The two-way partial dependence plots show how changing the concentration of two compounds influences predicted appreciation, visualizing their interactions (Supplementary Fig.  S7 ). In our case, the top 5 parameters are dominated by additive or synergistic interactions, with high concentrations for both compounds resulting in the highest predicted appreciation.

To assess the robustness of our best-performing models and model predictions, we performed 100 iterations of the GBR, RF and ET models. In general, all iterations of the models yielded similar performance (Supplementary Fig.  S8 ). Moreover, the main predictors (including the top predictors ethanol and ethyl acetate) remained virtually the same, especially for GBR and RF. For the iterations of the ET model, we did observe more variation in the top predictors, which is likely a consequence of the model’s inherent random architecture in combination with co-correlations between certain predictors. However, even in this case, several of the top predictors (ethanol and ethyl acetate) remain unchanged, although their rank in importance changes (Supplementary Fig.  S8 ).

Next, we investigated if a combination of RateBeer and trained panel data into one consolidated dataset would lead to stronger models, under the hypothesis that such a model would suffer less from bias in the datasets. A GBR model was trained to predict appreciation on the combined dataset. This model underperformed compared to the RateBeer model, both in the native case and when including a dataset identifier (R 2  = 0.67, 0.26 and 0.42 respectively). For the latter, the dataset identifier is the most important feature (Supplementary Fig.  S9 ), while most of the feature importance remains unchanged, with ethyl acetate and ethanol ranking highest, like in the original model trained only on RateBeer data. It seems that the large variation in the panel dataset introduces noise, weakening the models’ performances and reliability. In addition, it seems reasonable to assume that both datasets are fundamentally different, with the panel dataset obtained by blind tastings by a trained professional panel.

Lastly, we evaluated whether beer style identifiers would further enhance the model’s performance. A GBR model was trained with parameters that explicitly encoded the styles of the samples. This did not improve model performance (R2 = 0.66 with style information vs R2 = 0.67). The most important chemical features are consistent with the model trained without style information (eg. ethanol and ethyl acetate), and with the exception of the most preferred (strong ale) and least preferred (low/no-alcohol) styles, none of the styles were among the most important features (Supplementary Fig.  S9 , Supplementary Table  S5 and S6 ). This is likely due to a combination of style-specific chemical signatures, such as iso-alpha acids and lactic acid, that implicitly convey style information to the original models, as well as the low number of samples belonging to some styles, making it difficult for the model to learn style-specific patterns. Moreover, beer styles are not rigorously defined, with some styles overlapping in features and some beers being misattributed to a specific style, all of which leads to more noise in models that use style parameters.

Model validation

To test if our predictive models give insight into beer appreciation, we set up experiments aimed at improving existing commercial beers. We specifically selected overall appreciation as the trait to be examined because of its complexity and commercial relevance. Beer flavor comprises a complex bouquet rather than single aromas and tastes 53 . Hence, adding a single compound to the extent that a difference is noticeable may lead to an unbalanced, artificial flavor. Therefore, we evaluated the effect of combinations of compounds. Because Blond beers represent the most extensive style in our dataset, we selected a beer from this style as the starting material for these experiments (Beer 64 in Supplementary Data  1 ).

In the first set of experiments, we adjusted the concentrations of compounds that made up the most important predictors of overall appreciation (ethyl acetate, ethanol, lactic acid, ethyl phenyl acetate) together with correlated compounds (ethyl hexanoate, isoamyl acetate, glycerol), bringing them up to 95 th percentile ethanol-normalized concentrations (Methods) within the Blond group (‘Spiked’ concentration in Fig.  5A ). Compared to controls, the spiked beers were found to have significantly improved overall appreciation among trained panelists, with panelist noting increased intensity of ester flavors, sweetness, alcohol, and body fullness (Fig.  5B ). To disentangle the contribution of ethanol to these results, a second experiment was performed without the addition of ethanol. This resulted in a similar outcome, including increased perception of alcohol and overall appreciation.

Adding the top chemical compounds, identified as best predictors of appreciation by our model, into poorly appreciated beers results in increased appreciation from our trained panel. Results of sensory tests between base beers and those spiked with compounds identified as the best predictors by the model. A Blond and Non/Low-alcohol (0.0% ABV) base beers were brought up to 95th-percentile ethanol-normalized concentrations within each style. B For each sensory attribute, tasters indicated the more intense sample and selected the sample they preferred. The numbers above the bars correspond to the p values that indicate significant changes in perceived flavor (two-sided binomial test: alpha 0.05, n  = 20 or 13).

In a last experiment, we tested whether using the model’s predictions can boost the appreciation of a non-alcoholic beer (beer 223 in Supplementary Data  1 ). Again, the addition of a mixture of predicted compounds (omitting ethanol, in this case) resulted in a significant increase in appreciation, body, ester flavor and sweetness.

Predicting flavor and consumer appreciation from chemical composition is one of the ultimate goals of sensory science. A reliable, systematic and unbiased way to link chemical profiles to flavor and food appreciation would be a significant asset to the food and beverage industry. Such tools would substantially aid in quality control and recipe development, offer an efficient and cost-effective alternative to pilot studies and consumer trials and would ultimately allow food manufacturers to produce superior, tailor-made products that better meet the demands of specific consumer groups more efficiently.

A limited set of studies have previously tried, to varying degrees of success, to predict beer flavor and beer popularity based on (a limited set of) chemical compounds and flavors 79 , 80 . Current sensitive, high-throughput technologies allow measuring an unprecedented number of chemical compounds and properties in a large set of samples, yielding a dataset that can train models that help close the gaps between chemistry and flavor, even for a complex natural product like beer. To our knowledge, no previous research gathered data at this scale (250 samples, 226 chemical parameters, 50 sensory attributes and 5 consumer scores) to disentangle and validate the chemical aspects driving beer preference using various machine-learning techniques. We find that modern machine learning models outperform conventional statistical tools, such as correlations and linear models, and can successfully predict flavor appreciation from chemical composition. This could be attributed to the natural incorporation of interactions and non-linear or discontinuous effects in machine learning models, which are not easily grasped by the linear model architecture. While linear models and partial least squares regression represent the most widespread statistical approaches in sensory science, in part because they allow interpretation 65 , 81 , 82 , modern machine learning methods allow for building better predictive models while preserving the possibility to dissect and exploit the underlying patterns. Of the 10 different models we trained, tree-based models, such as our best performing GBR, showed the best overall performance in predicting sensory responses from chemical information, outcompeting artificial neural networks. This agrees with previous reports for models trained on tabular data 83 . Our results are in line with the findings of Colantonio et al. who also identified the gradient boosting architecture as performing best at predicting appreciation and flavor (of tomatoes and blueberries, in their specific study) 26 . Importantly, besides our larger experimental scale, we were able to directly confirm our models’ predictions in vivo.

Our study confirms that flavor compound concentration does not always correlate with perception, suggesting complex interactions that are often missed by more conventional statistics and simple models. Specifically, we find that tree-based algorithms may perform best in developing models that link complex food chemistry with aroma. Furthermore, we show that massive datasets of untrained consumer reviews provide a valuable source of data, that can complement or even replace trained tasting panels, especially for appreciation and basic flavors, such as sweetness and bitterness. This holds despite biases that are known to occur in such datasets, such as price or conformity bias. Moreover, GBR models predict taste better than aroma. This is likely because taste (e.g. bitterness) often directly relates to the corresponding chemical measurements (e.g., iso-alpha acids), whereas such a link is less clear for aromas, which often result from the interplay between multiple volatile compounds. We also find that our models are best at predicting acidity and alcohol, likely because there is a direct relation between the measured chemical compounds (acids and ethanol) and the corresponding perceived sensorial attribute (acidity and alcohol), and because even untrained consumers are generally able to recognize these flavors and aromas.

The predictions of our final models, trained on review data, hold even for blind tastings with small groups of trained tasters, as demonstrated by our ability to validate specific compounds as drivers of beer flavor and appreciation. Since adding a single compound to the extent of a noticeable difference may result in an unbalanced flavor profile, we specifically tested our identified key drivers as a combination of compounds. While this approach does not allow us to validate if a particular single compound would affect flavor and/or appreciation, our experiments do show that this combination of compounds increases consumer appreciation.

It is important to stress that, while it represents an important step forward, our approach still has several major limitations. A key weakness of the GBR model architecture is that amongst co-correlating variables, the largest main effect is consistently preferred for model building. As a result, co-correlating variables often have artificially low importance scores, both for impurity and SHAP-based methods, like we observed in the comparison to the more randomized Extra Trees models. This implies that chemicals identified as key drivers of a specific sensory feature by GBR might not be the true causative compounds, but rather co-correlate with the actual causative chemical. For example, the high importance of ethyl acetate could be (partially) attributed to the total ester content, ethanol or ethyl hexanoate (rho=0.77, rho=0.72 and rho=0.68), while ethyl phenylacetate could hide the importance of prenyl isobutyrate and ethyl benzoate (rho=0.77 and rho=0.76). Expanding our GBR model to include beer style as a parameter did not yield additional power or insight. This is likely due to style-specific chemical signatures, such as iso-alpha acids and lactic acid, that implicitly convey style information to the original model, as well as the smaller sample size per style, limiting the power to uncover style-specific patterns. This can be partly attributed to the curse of dimensionality, where the high number of parameters results in the models mainly incorporating single parameter effects, rather than complex interactions such as style-dependent effects 67 . A larger number of samples may overcome some of these limitations and offer more insight into style-specific effects. On the other hand, beer style is not a rigid scientific classification, and beers within one style often differ a lot, which further complicates the analysis of style as a model factor.

Our study is limited to beers from Belgian breweries. Although these beers cover a large portion of the beer styles available globally, some beer styles and consumer patterns may be missing, while other features might be overrepresented. For example, many Belgian ales exhibit yeast-driven flavor profiles, which is reflected in the chemical drivers of appreciation discovered by this study. In future work, expanding the scope to include diverse markets and beer styles could lead to the identification of even more drivers of appreciation and better models for special niche products that were not present in our beer set.

In addition to inherent limitations of GBR models, there are also some limitations associated with studying food aroma. Even if our chemical analyses measured most of the known aroma compounds, the total number of flavor compounds in complex foods like beer is still larger than the subset we were able to measure in this study. For example, hop-derived thiols, that influence flavor at very low concentrations, are notoriously difficult to measure in a high-throughput experiment. Moreover, consumer perception remains subjective and prone to biases that are difficult to avoid. It is also important to stress that the models are still immature and that more extensive datasets will be crucial for developing more complete models in the future. Besides more samples and parameters, our dataset does not include any demographic information about the tasters. Including such data could lead to better models that grasp external factors like age and culture. Another limitation is that our set of beers consists of high-quality end-products and lacks beers that are unfit for sale, which limits the current model in accurately predicting products that are appreciated very badly. Finally, while models could be readily applied in quality control, their use in sensory science and product development is restrained by their inability to discern causal relationships. Given that the models cannot distinguish compounds that genuinely drive consumer perception from those that merely correlate, validation experiments are essential to identify true causative compounds.

Despite the inherent limitations, dissection of our models enabled us to pinpoint specific molecules as potential drivers of beer aroma and consumer appreciation, including compounds that were unexpected and would not have been identified using standard approaches. Important drivers of beer appreciation uncovered by our models include protein levels, ethyl acetate, ethyl phenyl acetate and lactic acid. Currently, many brewers already use lactic acid to acidify their brewing water and ensure optimal pH for enzymatic activity during the mashing process. Our results suggest that adding lactic acid can also improve beer appreciation, although its individual effect remains to be tested. Interestingly, ethanol appears to be unnecessary to improve beer appreciation, both for blond beer and alcohol-free beer. Given the growing consumer interest in alcohol-free beer, with a predicted annual market growth of >7% 84 , it is relevant for brewers to know what compounds can further increase consumer appreciation of these beers. Hence, our model may readily provide avenues to further improve the flavor and consumer appreciation of both alcoholic and non-alcoholic beers, which is generally considered one of the key challenges for future beer production.

Whereas we see a direct implementation of our results for the development of superior alcohol-free beverages and other food products, our study can also serve as a stepping stone for the development of novel alcohol-containing beverages. We want to echo the growing body of scientific evidence for the negative effects of alcohol consumption, both on the individual level by the mutagenic, teratogenic and carcinogenic effects of ethanol 85 , 86 , as well as the burden on society caused by alcohol abuse and addiction. We encourage the use of our results for the production of healthier, tastier products, including novel and improved beverages with lower alcohol contents. Furthermore, we strongly discourage the use of these technologies to improve the appreciation or addictive properties of harmful substances.

The present work demonstrates that despite some important remaining hurdles, combining the latest developments in chemical analyses, sensory analysis and modern machine learning methods offers exciting avenues for food chemistry and engineering. Soon, these tools may provide solutions in quality control and recipe development, as well as new approaches to sensory science and flavor research.

Beer selection

250 commercial Belgian beers were selected to cover the broad diversity of beer styles and corresponding diversity in chemical composition and aroma. See Supplementary Fig.  S1 .

Chemical dataset

Sample preparation.

Beers within their expiration date were purchased from commercial retailers. Samples were prepared in biological duplicates at room temperature, unless explicitly stated otherwise. Bottle pressure was measured with a manual pressure device (Steinfurth Mess-Systeme GmbH) and used to calculate CO 2 concentration. The beer was poured through two filter papers (Macherey-Nagel, 500713032 MN 713 ¼) to remove carbon dioxide and prevent spontaneous foaming. Samples were then prepared for measurements by targeted Headspace-Gas Chromatography-Flame Ionization Detector/Flame Photometric Detector (HS-GC-FID/FPD), Headspace-Solid Phase Microextraction-Gas Chromatography-Mass Spectrometry (HS-SPME-GC-MS), colorimetric analysis, enzymatic analysis, Near-Infrared (NIR) analysis, as described in the sections below. The mean values of biological duplicates are reported for each compound.

HS-GC-FID/FPD

HS-GC-FID/FPD (Shimadzu GC 2010 Plus) was used to measure higher alcohols, acetaldehyde, esters, 4-vinyl guaicol, and sulfur compounds. Each measurement comprised 5 ml of sample pipetted into a 20 ml glass vial containing 1.75 g NaCl (VWR, 27810.295). 100 µl of 2-heptanol (Sigma-Aldrich, H3003) (internal standard) solution in ethanol (Fisher Chemical, E/0650DF/C17) was added for a final concentration of 2.44 mg/L. Samples were flushed with nitrogen for 10 s, sealed with a silicone septum, stored at −80 °C and analyzed in batches of 20.

The GC was equipped with a DB-WAXetr column (length, 30 m; internal diameter, 0.32 mm; layer thickness, 0.50 µm; Agilent Technologies, Santa Clara, CA, USA) to the FID and an HP-5 column (length, 30 m; internal diameter, 0.25 mm; layer thickness, 0.25 µm; Agilent Technologies, Santa Clara, CA, USA) to the FPD. N 2 was used as the carrier gas. Samples were incubated for 20 min at 70 °C in the headspace autosampler (Flow rate, 35 cm/s; Injection volume, 1000 µL; Injection mode, split; Combi PAL autosampler, CTC analytics, Switzerland). The injector, FID and FPD temperatures were kept at 250 °C. The GC oven temperature was first held at 50 °C for 5 min and then allowed to rise to 80 °C at a rate of 5 °C/min, followed by a second ramp of 4 °C/min until 200 °C kept for 3 min and a final ramp of (4 °C/min) until 230 °C for 1 min. Results were analyzed with the GCSolution software version 2.4 (Shimadzu, Kyoto, Japan). The GC was calibrated with a 5% EtOH solution (VWR International) containing the volatiles under study (Supplementary Table  S7 ).

HS-SPME-GC-MS

HS-SPME-GC-MS (Shimadzu GCMS-QP-2010 Ultra) was used to measure additional volatile compounds, mainly comprising terpenoids and esters. Samples were analyzed by HS-SPME using a triphase DVB/Carboxen/PDMS 50/30 μm SPME fiber (Supelco Co., Bellefonte, PA, USA) followed by gas chromatography (Thermo Fisher Scientific Trace 1300 series, USA) coupled to a mass spectrometer (Thermo Fisher Scientific ISQ series MS) equipped with a TriPlus RSH autosampler. 5 ml of degassed beer sample was placed in 20 ml vials containing 1.75 g NaCl (VWR, 27810.295). 5 µl internal standard mix was added, containing 2-heptanol (1 g/L) (Sigma-Aldrich, H3003), 4-fluorobenzaldehyde (1 g/L) (Sigma-Aldrich, 128376), 2,3-hexanedione (1 g/L) (Sigma-Aldrich, 144169) and guaiacol (1 g/L) (Sigma-Aldrich, W253200) in ethanol (Fisher Chemical, E/0650DF/C17). Each sample was incubated at 60 °C in the autosampler oven with constant agitation. After 5 min equilibration, the SPME fiber was exposed to the sample headspace for 30 min. The compounds trapped on the fiber were thermally desorbed in the injection port of the chromatograph by heating the fiber for 15 min at 270 °C.

The GC-MS was equipped with a low polarity RXi-5Sil MS column (length, 20 m; internal diameter, 0.18 mm; layer thickness, 0.18 µm; Restek, Bellefonte, PA, USA). Injection was performed in splitless mode at 320 °C, a split flow of 9 ml/min, a purge flow of 5 ml/min and an open valve time of 3 min. To obtain a pulsed injection, a programmed gas flow was used whereby the helium gas flow was set at 2.7 mL/min for 0.1 min, followed by a decrease in flow of 20 ml/min to the normal 0.9 mL/min. The temperature was first held at 30 °C for 3 min and then allowed to rise to 80 °C at a rate of 7 °C/min, followed by a second ramp of 2 °C/min till 125 °C and a final ramp of 8 °C/min with a final temperature of 270 °C.

Mass acquisition range was 33 to 550 amu at a scan rate of 5 scans/s. Electron impact ionization energy was 70 eV. The interface and ion source were kept at 275 °C and 250 °C, respectively. A mix of linear n-alkanes (from C7 to C40, Supelco Co.) was injected into the GC-MS under identical conditions to serve as external retention index markers. Identification and quantification of the compounds were performed using an in-house developed R script as described in Goelen et al. and Reher et al. 87 , 88 (for package information, see Supplementary Table  S8 ). Briefly, chromatograms were analyzed using AMDIS (v2.71) 89 to separate overlapping peaks and obtain pure compound spectra. The NIST MS Search software (v2.0 g) in combination with the NIST2017, FFNSC3 and Adams4 libraries were used to manually identify the empirical spectra, taking into account the expected retention time. After background subtraction and correcting for retention time shifts between samples run on different days based on alkane ladders, compound elution profiles were extracted and integrated using a file with 284 target compounds of interest, which were either recovered in our identified AMDIS list of spectra or were known to occur in beer. Compound elution profiles were estimated for every peak in every chromatogram over a time-restricted window using weighted non-negative least square analysis after which peak areas were integrated 87 , 88 . Batch effect correction was performed by normalizing against the most stable internal standard compound, 4-fluorobenzaldehyde. Out of all 284 target compounds that were analyzed, 167 were visually judged to have reliable elution profiles and were used for final analysis.

Discrete photometric and enzymatic analysis

Discrete photometric and enzymatic analysis (Thermo Scientific TM Gallery TM Plus Beermaster Discrete Analyzer) was used to measure acetic acid, ammonia, beta-glucan, iso-alpha acids, color, sugars, glycerol, iron, pH, protein, and sulfite. 2 ml of sample volume was used for the analyses. Information regarding the reagents and standard solutions used for analyses and calibrations is included in Supplementary Table  S7 and Supplementary Table  S9 .

NIR analyses

NIR analysis (Anton Paar Alcolyzer Beer ME System) was used to measure ethanol. Measurements comprised 50 ml of sample, and a 10% EtOH solution was used for calibration.

Correlation calculations

Pairwise Spearman Rank correlations were calculated between all chemical properties.

Sensory dataset

Trained panel.

Our trained tasting panel consisted of volunteers who gave prior verbal informed consent. All compounds used for the validation experiment were of food-grade quality. The tasting sessions were approved by the Social and Societal Ethics Committee of the KU Leuven (G-2022-5677-R2(MAR)). All online reviewers agreed to the Terms and Conditions of the RateBeer website.

Sensory analysis was performed according to the American Society of Brewing Chemists (ASBC) Sensory Analysis Methods 90 . 30 volunteers were screened through a series of triangle tests. The sixteen most sensitive and consistent tasters were retained as taste panel members. The resulting panel was diverse in age [22–42, mean: 29], sex [56% male] and nationality [7 different countries]. The panel developed a consensus vocabulary to describe beer aroma, taste and mouthfeel. Panelists were trained to identify and score 50 different attributes, using a 7-point scale to rate attributes’ intensity. The scoring sheet is included as Supplementary Data  3 . Sensory assessments took place between 10–12 a.m. The beers were served in black-colored glasses. Per session, between 5 and 12 beers of the same style were tasted at 12 °C to 16 °C. Two reference beers were added to each set and indicated as ‘Reference 1 & 2’, allowing panel members to calibrate their ratings. Not all panelists were present at every tasting. Scores were scaled by standard deviation and mean-centered per taster. Values are represented as z-scores and clustered by Euclidean distance. Pairwise Spearman correlations were calculated between taste and aroma sensory attributes. Panel consistency was evaluated by repeating samples on different sessions and performing ANOVA to identify differences, using the ‘stats’ package (v4.2.2) in R (for package information, see Supplementary Table  S8 ).

Online reviews from a public database

The ‘scrapy’ package in Python (v3.6) (for package information, see Supplementary Table  S8 ). was used to collect 232,288 online reviews (mean=922, min=6, max=5343) from RateBeer, an online beer review database. Each review entry comprised 5 numerical scores (appearance, aroma, taste, palate and overall quality) and an optional review text. The total number of reviews per reviewer was collected separately. Numerical scores were scaled and centered per rater, and mean scores were calculated per beer.

For the review texts, the language was estimated using the packages ‘langdetect’ and ‘langid’ in Python. Reviews that were classified as English by both packages were kept. Reviewers with fewer than 100 entries overall were discarded. 181,025 reviews from >6000 reviewers from >40 countries remained. Text processing was done using the ‘nltk’ package in Python. Texts were corrected for slang and misspellings; proper nouns and rare words that are relevant to the beer context were specified and kept as-is (‘Chimay’,’Lambic’, etc.). A dictionary of semantically similar sensorial terms, for example ‘floral’ and ‘flower’, was created and collapsed together into one term. Words were stemmed and lemmatized to avoid identifying words such as ‘acid’ and ‘acidity’ as separate terms. Numbers and punctuation were removed.

Sentences from up to 50 randomly chosen reviews per beer were manually categorized according to the aspect of beer they describe (appearance, aroma, taste, palate, overall quality—not to be confused with the 5 numerical scores described above) or flagged as irrelevant if they contained no useful information. If a beer contained fewer than 50 reviews, all reviews were manually classified. This labeled data set was used to train a model that classified the rest of the sentences for all beers 91 . Sentences describing taste and aroma were extracted, and term frequency–inverse document frequency (TFIDF) was implemented to calculate enrichment scores for sensorial words per beer.

The sex of the tasting subject was not considered when building our sensory database. Instead, results from different panelists were averaged, both for our trained panel (56% male, 44% female) and the RateBeer reviews (70% male, 30% female for RateBeer as a whole).

Beer price collection and processing

Beer prices were collected from the following stores: Colruyt, Delhaize, Total Wine, BeerHawk, The Belgian Beer Shop, The Belgian Shop, and Beer of Belgium. Where applicable, prices were converted to Euros and normalized per liter. Spearman correlations were calculated between these prices and mean overall appreciation scores from RateBeer and the taste panel, respectively.

Pairwise Spearman Rank correlations were calculated between all sensory properties.

Machine learning models

Predictive modeling of sensory profiles from chemical data.

Regression models were constructed to predict (a) trained panel scores for beer flavors and quality from beer chemical profiles and (b) public reviews’ appreciation scores from beer chemical profiles. Z-scores were used to represent sensory attributes in both data sets. Chemical properties with log-normal distributions (Shapiro-Wilk test, p  <  0.05 ) were log-transformed. Missing chemical measurements (0.1% of all data) were replaced with mean values per attribute. Observations from 250 beers were randomly separated into a training set (70%, 175 beers) and a test set (30%, 75 beers), stratified per beer style. Chemical measurements (p = 231) were normalized based on the training set average and standard deviation. In total, three linear regression-based models: linear regression with first-order interaction terms (LR), lasso regression with first-order interaction terms (Lasso) and partial least squares regression (PLSR); five decision tree models, Adaboost regressor (ABR), Extra Trees (ET), Gradient Boosting regressor (GBR), Random Forest (RF) and XGBoost regressor (XGBR); one support vector machine model (SVR) and one artificial neural network model (ANN) were trained. The models were implemented using the ‘scikit-learn’ package (v1.2.2) and ‘xgboost’ package (v1.7.3) in Python (v3.9.16). Models were trained, and hyperparameters optimized, using five-fold cross-validated grid search with the coefficient of determination (R 2 ) as the evaluation metric. The ANN (scikit-learn’s MLPRegressor) was optimized using Bayesian Tree-Structured Parzen Estimator optimization with the ‘Optuna’ Python package (v3.2.0). Individual models were trained per attribute, and a multi-output model was trained on all attributes simultaneously.

Model dissection

GBR was found to outperform other methods, resulting in models with the highest average R 2 values in both trained panel and public review data sets. Impurity-based rankings of the most important predictors for each predicted sensorial trait were obtained using the ‘scikit-learn’ package. To observe the relationships between these chemical properties and their predicted targets, partial dependence plots (PDP) were constructed for the six most important predictors of consumer appreciation 74 , 75 .

The ‘SHAP’ package in Python (v0.41.0) was implemented to provide an alternative ranking of predictor importance and to visualize the predictors’ effects as a function of their concentration 68 .

Validation of causal chemical properties

To validate the effects of the most important model features on predicted sensory attributes, beers were spiked with the chemical compounds identified by the models and descriptive sensory analyses were carried out according to the American Society of Brewing Chemists (ASBC) protocol 90 .

Compound spiking was done 30 min before tasting. Compounds were spiked into fresh beer bottles, that were immediately resealed and inverted three times. Fresh bottles of beer were opened for the same duration, resealed, and inverted thrice, to serve as controls. Pairs of spiked samples and controls were served simultaneously, chilled and in dark glasses as outlined in the Trained panel section above. Tasters were instructed to select the glass with the higher flavor intensity for each attribute (directional difference test 92 ) and to select the glass they prefer.

The final concentration after spiking was equal to the within-style average, after normalizing by ethanol concentration. This was done to ensure balanced flavor profiles in the final spiked beer. The same methods were applied to improve a non-alcoholic beer. Compounds were the following: ethyl acetate (Merck KGaA, W241415), ethyl hexanoate (Merck KGaA, W243906), isoamyl acetate (Merck KGaA, W205508), phenethyl acetate (Merck KGaA, W285706), ethanol (96%, Colruyt), glycerol (Merck KGaA, W252506), lactic acid (Merck KGaA, 261106).

Significant differences in preference or perceived intensity were determined by performing the two-sided binomial test on each attribute.

Reporting summary

Further information on research design is available in the  Nature Portfolio Reporting Summary linked to this article.

Data availability

The data that support the findings of this work are available in the Supplementary Data files and have been deposited to Zenodo under accession code 10653704 93 . The RateBeer scores data are under restricted access, they are not publicly available as they are property of RateBeer (ZX Ventures, USA). Access can be obtained from the authors upon reasonable request and with permission of RateBeer (ZX Ventures, USA).  Source data are provided with this paper.

Code availability

The code for training the machine learning models, analyzing the models, and generating the figures has been deposited to Zenodo under accession code 10653704 93 .

Tieman, D. et al. A chemical genetic roadmap to improved tomato flavor. Science 355 , 391–394 (2017).

Article   ADS   CAS   PubMed   Google Scholar  

Plutowska, B. & Wardencki, W. Application of gas chromatography–olfactometry (GC–O) in analysis and quality assessment of alcoholic beverages – A review. Food Chem. 107 , 449–463 (2008).

Article   CAS   Google Scholar  

Legin, A., Rudnitskaya, A., Seleznev, B. & Vlasov, Y. Electronic tongue for quality assessment of ethanol, vodka and eau-de-vie. Anal. Chim. Acta 534 , 129–135 (2005).

Loutfi, A., Coradeschi, S., Mani, G. K., Shankar, P. & Rayappan, J. B. B. Electronic noses for food quality: A review. J. Food Eng. 144 , 103–111 (2015).

Ahn, Y.-Y., Ahnert, S. E., Bagrow, J. P. & Barabási, A.-L. Flavor network and the principles of food pairing. Sci. Rep. 1 , 196 (2011).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Bartoshuk, L. M. & Klee, H. J. Better fruits and vegetables through sensory analysis. Curr. Biol. 23 , R374–R378 (2013).

Article   CAS   PubMed   Google Scholar  

Piggott, J. R. Design questions in sensory and consumer science. Food Qual. Prefer. 3293 , 217–220 (1995).

Article   Google Scholar  

Kermit, M. & Lengard, V. Assessing the performance of a sensory panel-panellist monitoring and tracking. J. Chemom. 19 , 154–161 (2005).

Cook, D. J., Hollowood, T. A., Linforth, R. S. T. & Taylor, A. J. Correlating instrumental measurements of texture and flavour release with human perception. Int. J. Food Sci. Technol. 40 , 631–641 (2005).

Chinchanachokchai, S., Thontirawong, P. & Chinchanachokchai, P. A tale of two recommender systems: The moderating role of consumer expertise on artificial intelligence based product recommendations. J. Retail. Consum. Serv. 61 , 1–12 (2021).

Ross, C. F. Sensory science at the human-machine interface. Trends Food Sci. Technol. 20 , 63–72 (2009).

Chambers, E. IV & Koppel, K. Associations of volatile compounds with sensory aroma and flavor: The complex nature of flavor. Molecules 18 , 4887–4905 (2013).

Pinu, F. R. Metabolomics—The new frontier in food safety and quality research. Food Res. Int. 72 , 80–81 (2015).

Danezis, G. P., Tsagkaris, A. S., Brusic, V. & Georgiou, C. A. Food authentication: state of the art and prospects. Curr. Opin. Food Sci. 10 , 22–31 (2016).

Shepherd, G. M. Smell images and the flavour system in the human brain. Nature 444 , 316–321 (2006).

Meilgaard, M. C. Prediction of flavor differences between beers from their chemical composition. J. Agric. Food Chem. 30 , 1009–1017 (1982).

Xu, L. et al. Widespread receptor-driven modulation in peripheral olfactory coding. Science 368 , eaaz5390 (2020).

Kupferschmidt, K. Following the flavor. Science 340 , 808–809 (2013).

Billesbølle, C. B. et al. Structural basis of odorant recognition by a human odorant receptor. Nature 615 , 742–749 (2023).

Article   ADS   PubMed   PubMed Central   Google Scholar  

Smith, B. Perspective: Complexities of flavour. Nature 486 , S6–S6 (2012).

Pfister, P. et al. Odorant receptor inhibition is fundamental to odor encoding. Curr. Biol. 30 , 2574–2587 (2020).

Moskowitz, H. W., Kumaraiah, V., Sharma, K. N., Jacobs, H. L. & Sharma, S. D. Cross-cultural differences in simple taste preferences. Science 190 , 1217–1218 (1975).

Eriksson, N. et al. A genetic variant near olfactory receptor genes influences cilantro preference. Flavour 1 , 22 (2012).

Ferdenzi, C. et al. Variability of affective responses to odors: Culture, gender, and olfactory knowledge. Chem. Senses 38 , 175–186 (2013).

Article   PubMed   Google Scholar  

Lawless, H. T. & Heymann, H. Sensory evaluation of food: Principles and practices. (Springer, New York, NY). https://doi.org/10.1007/978-1-4419-6488-5 (2010).

Colantonio, V. et al. Metabolomic selection for enhanced fruit flavor. Proc. Natl. Acad. Sci. 119 , e2115865119 (2022).

Fritz, F., Preissner, R. & Banerjee, P. VirtualTaste: a web server for the prediction of organoleptic properties of chemical compounds. Nucleic Acids Res 49 , W679–W684 (2021).

Tuwani, R., Wadhwa, S. & Bagler, G. BitterSweet: Building machine learning models for predicting the bitter and sweet taste of small molecules. Sci. Rep. 9 , 1–13 (2019).

Dagan-Wiener, A. et al. Bitter or not? BitterPredict, a tool for predicting taste from chemical structure. Sci. Rep. 7 , 1–13 (2017).

Pallante, L. et al. Toward a general and interpretable umami taste predictor using a multi-objective machine learning approach. Sci. Rep. 12 , 1–11 (2022).

Malavolta, M. et al. A survey on computational taste predictors. Eur. Food Res. Technol. 248 , 2215–2235 (2022).

Lee, B. K. et al. A principal odor map unifies diverse tasks in olfactory perception. Science 381 , 999–1006 (2023).

Mayhew, E. J. et al. Transport features predict if a molecule is odorous. Proc. Natl. Acad. Sci. 119 , e2116576119 (2022).

Niu, Y. et al. Sensory evaluation of the synergism among ester odorants in light aroma-type liquor by odor threshold, aroma intensity and flash GC electronic nose. Food Res. Int. 113 , 102–114 (2018).

Yu, P., Low, M. Y. & Zhou, W. Design of experiments and regression modelling in food flavour and sensory analysis: A review. Trends Food Sci. Technol. 71 , 202–215 (2018).

Oladokun, O. et al. The impact of hop bitter acid and polyphenol profiles on the perceived bitterness of beer. Food Chem. 205 , 212–220 (2016).

Linforth, R., Cabannes, M., Hewson, L., Yang, N. & Taylor, A. Effect of fat content on flavor delivery during consumption: An in vivo model. J. Agric. Food Chem. 58 , 6905–6911 (2010).

Guo, S., Na Jom, K. & Ge, Y. Influence of roasting condition on flavor profile of sunflower seeds: A flavoromics approach. Sci. Rep. 9 , 11295 (2019).

Ren, Q. et al. The changes of microbial community and flavor compound in the fermentation process of Chinese rice wine using Fagopyrum tataricum grain as feedstock. Sci. Rep. 9 , 3365 (2019).

Hastie, T., Friedman, J. & Tibshirani, R. The Elements of Statistical Learning. (Springer, New York, NY). https://doi.org/10.1007/978-0-387-21606-5 (2001).

Dietz, C., Cook, D., Huismann, M., Wilson, C. & Ford, R. The multisensory perception of hop essential oil: a review. J. Inst. Brew. 126 , 320–342 (2020).

CAS   Google Scholar  

Roncoroni, Miguel & Verstrepen, Kevin Joan. Belgian Beer: Tested and Tasted. (Lannoo, 2018).

Meilgaard, M. Flavor chemistry of beer: Part II: Flavor and threshold of 239 aroma volatiles. in (1975).

Bokulich, N. A. & Bamforth, C. W. The microbiology of malting and brewing. Microbiol. Mol. Biol. Rev. MMBR 77 , 157–172 (2013).

Dzialo, M. C., Park, R., Steensels, J., Lievens, B. & Verstrepen, K. J. Physiology, ecology and industrial applications of aroma formation in yeast. FEMS Microbiol. Rev. 41 , S95–S128 (2017).

Article   PubMed   PubMed Central   Google Scholar  

Datta, A. et al. Computer-aided food engineering. Nat. Food 3 , 894–904 (2022).

American Society of Brewing Chemists. Beer Methods. (American Society of Brewing Chemists, St. Paul, MN, U.S.A.).

Olaniran, A. O., Hiralal, L., Mokoena, M. P. & Pillay, B. Flavour-active volatile compounds in beer: production, regulation and control. J. Inst. Brew. 123 , 13–23 (2017).

Verstrepen, K. J. et al. Flavor-active esters: Adding fruitiness to beer. J. Biosci. Bioeng. 96 , 110–118 (2003).

Meilgaard, M. C. Flavour chemistry of beer. part I: flavour interaction between principal volatiles. Master Brew. Assoc. Am. Tech. Q 12 , 107–117 (1975).

Briggs, D. E., Boulton, C. A., Brookes, P. A. & Stevens, R. Brewing 227–254. (Woodhead Publishing). https://doi.org/10.1533/9781855739062.227 (2004).

Bossaert, S., Crauwels, S., De Rouck, G. & Lievens, B. The power of sour - A review: Old traditions, new opportunities. BrewingScience 72 , 78–88 (2019).

Google Scholar  

Verstrepen, K. J. et al. Flavor active esters: Adding fruitiness to beer. J. Biosci. Bioeng. 96 , 110–118 (2003).

Snauwaert, I. et al. Microbial diversity and metabolite composition of Belgian red-brown acidic ales. Int. J. Food Microbiol. 221 , 1–11 (2016).

Spitaels, F. et al. The microbial diversity of traditional spontaneously fermented lambic beer. PLoS ONE 9 , e95384 (2014).

Blanco, C. A., Andrés-Iglesias, C. & Montero, O. Low-alcohol Beers: Flavor Compounds, Defects, and Improvement Strategies. Crit. Rev. Food Sci. Nutr. 56 , 1379–1388 (2016).

Jackowski, M. & Trusek, A. Non-Alcohol. beer Prod. – Overv. 20 , 32–38 (2018).

Takoi, K. et al. The contribution of geraniol metabolism to the citrus flavour of beer: Synergy of geraniol and β-citronellol under coexistence with excess linalool. J. Inst. Brew. 116 , 251–260 (2010).

Kroeze, J. H. & Bartoshuk, L. M. Bitterness suppression as revealed by split-tongue taste stimulation in humans. Physiol. Behav. 35 , 779–783 (1985).

Mennella, J. A. et al. A spoonful of sugar helps the medicine go down”: Bitter masking bysucrose among children and adults. Chem. Senses 40 , 17–25 (2015).

Wietstock, P., Kunz, T., Perreira, F. & Methner, F.-J. Metal chelation behavior of hop acids in buffered model systems. BrewingScience 69 , 56–63 (2016).

Sancho, D., Blanco, C. A., Caballero, I. & Pascual, A. Free iron in pale, dark and alcohol-free commercial lager beers. J. Sci. Food Agric. 91 , 1142–1147 (2011).

Rodrigues, H. & Parr, W. V. Contribution of cross-cultural studies to understanding wine appreciation: A review. Food Res. Int. 115 , 251–258 (2019).

Korneva, E. & Blockeel, H. Towards better evaluation of multi-target regression models. in ECML PKDD 2020 Workshops (eds. Koprinska, I. et al.) 353–362 (Springer International Publishing, Cham, 2020). https://doi.org/10.1007/978-3-030-65965-3_23 .

Gastón Ares. Mathematical and Statistical Methods in Food Science and Technology. (Wiley, 2013).

Grinsztajn, L., Oyallon, E. & Varoquaux, G. Why do tree-based models still outperform deep learning on tabular data? Preprint at http://arxiv.org/abs/2207.08815 (2022).

Gries, S. T. Statistics for Linguistics with R: A Practical Introduction. in Statistics for Linguistics with R (De Gruyter Mouton, 2021). https://doi.org/10.1515/9783110718256 .

Lundberg, S. M. et al. From local explanations to global understanding with explainable AI for trees. Nat. Mach. Intell. 2 , 56–67 (2020).

Ickes, C. M. & Cadwallader, K. R. Effects of ethanol on flavor perception in alcoholic beverages. Chemosens. Percept. 10 , 119–134 (2017).

Kato, M. et al. Influence of high molecular weight polypeptides on the mouthfeel of commercial beer. J. Inst. Brew. 127 , 27–40 (2021).

Wauters, R. et al. Novel Saccharomyces cerevisiae variants slow down the accumulation of staling aldehydes and improve beer shelf-life. Food Chem. 398 , 1–11 (2023).

Li, H., Jia, S. & Zhang, W. Rapid determination of low-level sulfur compounds in beer by headspace gas chromatography with a pulsed flame photometric detector. J. Am. Soc. Brew. Chem. 66 , 188–191 (2008).

Dercksen, A., Laurens, J., Torline, P., Axcell, B. C. & Rohwer, E. Quantitative analysis of volatile sulfur compounds in beer using a membrane extraction interface. J. Am. Soc. Brew. Chem. 54 , 228–233 (1996).

Molnar, C. Interpretable Machine Learning: A Guide for Making Black-Box Models Interpretable. (2020).

Zhao, Q. & Hastie, T. Causal interpretations of black-box models. J. Bus. Econ. Stat. Publ. Am. Stat. Assoc. 39 , 272–281 (2019).

Article   MathSciNet   Google Scholar  

Hastie, T., Tibshirani, R. & Friedman, J. The Elements of Statistical Learning. (Springer, 2019).

Labrado, D. et al. Identification by NMR of key compounds present in beer distillates and residual phases after dealcoholization by vacuum distillation. J. Sci. Food Agric. 100 , 3971–3978 (2020).

Lusk, L. T., Kay, S. B., Porubcan, A. & Ryder, D. S. Key olfactory cues for beer oxidation. J. Am. Soc. Brew. Chem. 70 , 257–261 (2012).

Gonzalez Viejo, C., Torrico, D. D., Dunshea, F. R. & Fuentes, S. Development of artificial neural network models to assess beer acceptability based on sensory properties using a robotic pourer: A comparative model approach to achieve an artificial intelligence system. Beverages 5 , 33 (2019).

Gonzalez Viejo, C., Fuentes, S., Torrico, D. D., Godbole, A. & Dunshea, F. R. Chemical characterization of aromas in beer and their effect on consumers liking. Food Chem. 293 , 479–485 (2019).

Gilbert, J. L. et al. Identifying breeding priorities for blueberry flavor using biochemical, sensory, and genotype by environment analyses. PLOS ONE 10 , 1–21 (2015).

Goulet, C. et al. Role of an esterase in flavor volatile variation within the tomato clade. Proc. Natl. Acad. Sci. 109 , 19009–19014 (2012).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Borisov, V. et al. Deep Neural Networks and Tabular Data: A Survey. IEEE Trans. Neural Netw. Learn. Syst. 1–21 https://doi.org/10.1109/TNNLS.2022.3229161 (2022).

Statista. Statista Consumer Market Outlook: Beer - Worldwide.

Seitz, H. K. & Stickel, F. Molecular mechanisms of alcoholmediated carcinogenesis. Nat. Rev. Cancer 7 , 599–612 (2007).

Voordeckers, K. et al. Ethanol exposure increases mutation rate through error-prone polymerases. Nat. Commun. 11 , 3664 (2020).

Goelen, T. et al. Bacterial phylogeny predicts volatile organic compound composition and olfactory response of an aphid parasitoid. Oikos 129 , 1415–1428 (2020).

Article   ADS   Google Scholar  

Reher, T. et al. Evaluation of hop (Humulus lupulus) as a repellent for the management of Drosophila suzukii. Crop Prot. 124 , 104839 (2019).

Stein, S. E. An integrated method for spectrum extraction and compound identification from gas chromatography/mass spectrometry data. J. Am. Soc. Mass Spectrom. 10 , 770–781 (1999).

American Society of Brewing Chemists. Sensory Analysis Methods. (American Society of Brewing Chemists, St. Paul, MN, U.S.A., 1992).

McAuley, J., Leskovec, J. & Jurafsky, D. Learning Attitudes and Attributes from Multi-Aspect Reviews. Preprint at https://doi.org/10.48550/arXiv.1210.3926 (2012).

Meilgaard, M. C., Carr, B. T. & Carr, B. T. Sensory Evaluation Techniques. (CRC Press, Boca Raton). https://doi.org/10.1201/b16452 (2014).

Schreurs, M. et al. Data from: Predicting and improving complex beer flavor through machine learning. Zenodo https://doi.org/10.5281/zenodo.10653704 (2024).

Download references

Acknowledgements

We thank all lab members for their discussions and thank all tasting panel members for their contributions. Special thanks go out to Dr. Karin Voordeckers for her tremendous help in proofreading and improving the manuscript. M.S. was supported by a Baillet-Latour fellowship, L.C. acknowledges financial support from KU Leuven (C16/17/006), F.A.T. was supported by a PhD fellowship from FWO (1S08821N). Research in the lab of K.J.V. is supported by KU Leuven, FWO, VIB, VLAIO and the Brewing Science Serves Health Fund. Research in the lab of T.W. is supported by FWO (G.0A51.15) and KU Leuven (C16/17/006).

Author information

These authors contributed equally: Michiel Schreurs, Supinya Piampongsant, Miguel Roncoroni.

Authors and Affiliations

VIB—KU Leuven Center for Microbiology, Gaston Geenslaan 1, B-3001, Leuven, Belgium

Michiel Schreurs, Supinya Piampongsant, Miguel Roncoroni, Lloyd Cool, Beatriz Herrera-Malaver, Florian A. Theßeling & Kevin J. Verstrepen

CMPG Laboratory of Genetics and Genomics, KU Leuven, Gaston Geenslaan 1, B-3001, Leuven, Belgium

Leuven Institute for Beer Research (LIBR), Gaston Geenslaan 1, B-3001, Leuven, Belgium

Laboratory of Socioecology and Social Evolution, KU Leuven, Naamsestraat 59, B-3000, Leuven, Belgium

Lloyd Cool, Christophe Vanderaa & Tom Wenseleers

VIB Bioinformatics Core, VIB, Rijvisschestraat 120, B-9052, Ghent, Belgium

Łukasz Kreft & Alexander Botzki

AB InBev SA/NV, Brouwerijplein 1, B-3000, Leuven, Belgium

Philippe Malcorps & Luk Daenen

You can also search for this author in PubMed   Google Scholar

Contributions

S.P., M.S. and K.J.V. conceived the experiments. S.P., M.S. and K.J.V. designed the experiments. S.P., M.S., M.R., B.H. and F.A.T. performed the experiments. S.P., M.S., L.C., C.V., L.K., A.B., P.M., L.D., T.W. and K.J.V. contributed analysis ideas. S.P., M.S., L.C., C.V., T.W. and K.J.V. analyzed the data. All authors contributed to writing the manuscript.

Corresponding author

Correspondence to Kevin J. Verstrepen .

Ethics declarations

Competing interests.

K.J.V. is affiliated with bar.on. The other authors declare no competing interests.

Peer review

Peer review information.

Nature Communications thanks Florian Bauer, Andrew John Macintosh and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, peer review file, description of additional supplementary files, supplementary data 1, supplementary data 2, supplementary data 3, supplementary data 4, supplementary data 5, supplementary data 6, supplementary data 7, reporting summary, source data, source data, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Schreurs, M., Piampongsant, S., Roncoroni, M. et al. Predicting and improving complex beer flavor through machine learning. Nat Commun 15 , 2368 (2024). https://doi.org/10.1038/s41467-024-46346-0

Download citation

Received : 30 October 2023

Accepted : 21 February 2024

Published : 26 March 2024

DOI : https://doi.org/10.1038/s41467-024-46346-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: Translational Research newsletter — top stories in biotechnology, drug discovery and pharma.

Help | Advanced Search

Computer Science > Computer Vision and Pattern Recognition

Title: sv3d: novel multi-view synthesis and 3d generation from a single image using latent video diffusion.

Abstract: We present Stable Video 3D (SV3D) -- a latent video diffusion model for high-resolution, image-to-multi-view generation of orbital videos around a 3D object. Recent work on 3D generation propose techniques to adapt 2D generative models for novel view synthesis (NVS) and 3D optimization. However, these methods have several disadvantages due to either limited views or inconsistent NVS, thereby affecting the performance of 3D object generation. In this work, we propose SV3D that adapts image-to-video diffusion model for novel multi-view synthesis and 3D generation, thereby leveraging the generalization and multi-view consistency of the video models, while further adding explicit camera control for NVS. We also propose improved 3D optimization techniques to use SV3D and its NVS outputs for image-to-3D generation. Extensive experimental results on multiple datasets with 2D and 3D metrics as well as user study demonstrate SV3D's state-of-the-art performance on NVS as well as 3D reconstruction compared to prior works.

Submission history

Access paper:.

• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

• Search for: Toggle Search

‘You Transformed the World,’ NVIDIA CEO Tells Researchers Behind Landmark AI Paper

Of GTC ’s 900+ sessions, the most wildly popular was a conversation hosted by NVIDIA founder and CEO Jensen Huang with seven of the authors of the legendary research paper that introduced the aptly named transformer — a neural network architecture that went on to change the deep learning landscape and enable today’s era of generative AI.

“Everything that we’re enjoying today can be traced back to that moment,” Huang said to a packed room with hundreds of attendees, who heard him speak with the authors of “ Attention Is All You Need .”

Sharing the stage for the first time, the research luminaries reflected on the factors that led to their original paper, which has been cited more than 100,000 times since it was first published and presented at the NeurIPS AI conference. They also discussed their latest projects and offered insights into future directions for the field of generative AI.

While they started as Google researchers, the collaborators are now spread across the industry, most as founders of their own AI companies.

“We have a whole industry that is grateful for the work that you guys did,” Huang said.

Origins of the Transformer Model

The research team initially sought to overcome the limitations of recurrent neural networks , or RNNs, which were then the state of the art for processing language data.

Noam Shazeer, cofounder and CEO of Character.AI, compared RNNs to the steam engine and transformers to the improved efficiency of internal combustion.

“We could have done the industrial revolution on the steam engine, but it would just have been a pain,” he said. “Things went way, way better with internal combustion.”

“Now we’re just waiting for the fusion,” quipped Illia Polosukhin, cofounder of blockchain company NEAR Protocol.

The paper’s title came from a realization that attention mechanisms — an element of neural networks that enable them to determine the relationship between different parts of input data — were the most critical component of their model’s performance.

“We had very recently started throwing bits of the model away, just to see how much worse it would get. And to our surprise it started getting better,” said Llion Jones, cofounder and chief technology officer at Sakana AI.

Having a name as general as “transformers” spoke to the team’s ambitions to build AI models that could process and transform every data type — including text, images, audio, tensors and biological data.

“That North Star, it was there on day zero, and so it’s been really exciting and gratifying to watch that come to fruition,” said Aidan Gomez, cofounder and CEO of Cohere. “We’re actually seeing it happen now.”

Envisioning the Road Ahead 

Adaptive computation, where a model adjusts how much computing power is used based on the complexity of a given problem, is a key factor the researchers see improving in future AI models.

“It’s really about spending the right amount of effort and ultimately energy on a given problem,” said Jakob Uszkoreit, cofounder and CEO of biological software company Inceptive. “You don’t want to spend too much on a problem that’s easy or too little on a problem that’s hard.”

A math problem like two plus two, for example, shouldn’t be run through a trillion-parameter transformer model — it should run on a basic calculator, the group agreed.

They’re also looking forward to the next generation of AI models.

“I think the world needs something better than the transformer,” said Gomez. “I think all of us here hope it gets succeeded by something that will carry us to a new plateau of performance.”

“You don’t want to miss these next 10 years,” Huang said. “Unbelievable new capabilities will be invented.”

The conversation concluded with Huang presenting each researcher with a framed cover plate of the NVIDIA DGX-1 AI supercomputer, signed with the message, “You transformed the world.”

There’s still time to catch the session replay by registering for a virtual GTC pass — it’s free.

To discover the latest in generative AI, watch Huang’s GTC keynote address:

NVIDIA websites use cookies to deliver and improve the website experience. See our cookie policy for further details on how we use cookies and how to change your cookie settings.

Share on Mastodon

• Share full article

Advertisement

Supported by

More Studies by Columbia Cancer Researchers Are Retracted

The studies, pulled because of copied data, illustrate the sluggishness of scientific publishers to address serious errors, experts said.

By Benjamin Mueller

Scientists in a prominent cancer lab at Columbia University have now had four studies retracted and a stern note added to a fifth accusing it of “severe abuse of the scientific publishing system,” the latest fallout from research misconduct allegations recently leveled against several leading cancer scientists.

A scientific sleuth in Britain last year uncovered discrepancies in data published by the Columbia lab, including the reuse of photos and other images across different papers. The New York Times reported last month that a medical journal in 2022 had quietly taken down a stomach cancer study by the researchers after an internal inquiry by the journal found ethics violations.

Despite that study’s removal, the researchers — Dr. Sam Yoon, chief of a cancer surgery division at Columbia University’s medical center, and Changhwan Yoon, a more junior biologist there — continued publishing studies with suspicious data. Since 2008, the two scientists have collaborated with other researchers on 26 articles that the sleuth, Sholto David, publicly flagged for misrepresenting experiments’ results.

One of those articles was retracted last month after The Times asked publishers about the allegations. In recent weeks, medical journals have retracted three additional studies, which described new strategies for treating cancers of the stomach, head and neck. Other labs had cited the articles in roughly 90 papers.

A major scientific publisher also appended a blunt note to the article that it had originally taken down without explanation in 2022. “This reuse (and in part, misrepresentation) of data without appropriate attribution represents a severe abuse of the scientific publishing system,” it said .

Still, those measures addressed only a small fraction of the lab’s suspect papers. Experts said the episode illustrated not only the extent of unreliable research by top labs, but also the tendency of scientific publishers to respond slowly, if at all, to significant problems once they are detected. As a result, other labs keep relying on questionable work as they pour federal research money into studies, allowing errors to accumulate in the scientific record.

“For every one paper that is retracted, there are probably 10 that should be,” said Dr. Ivan Oransky, co-founder of Retraction Watch, which keeps a database of 47,000-plus retracted studies. “Journals are not particularly interested in correcting the record.”

Columbia’s medical center declined to comment on allegations facing Dr. Yoon’s lab. It said the two scientists remained at Columbia and the hospital “is fully committed to upholding the highest standards of ethics and to rigorously maintaining the integrity of our research.”

The lab’s web page was recently taken offline. Columbia declined to say why. Neither Dr. Yoon nor Changhwan Yoon could be reached for comment. (They are not related.)

Memorial Sloan Kettering Cancer Center, where the scientists worked when much of the research was done, is investigating their work.

The Columbia scientists’ retractions come amid growing attention to the suspicious data that undergirds some medical research. Since late February, medical journals have retracted seven papers by scientists at Harvard’s Dana-Farber Cancer Institute . That followed investigations into data problems publicized by Dr. David , an independent molecular biologist who looks for irregularities in published images of cells, tumors and mice, sometimes with help from A.I. software.

The spate of misconduct allegations has drawn attention to the pressures on academic scientists — even those, like Dr. Yoon, who also work as doctors — to produce heaps of research.

Strong images of experiments’ results are often needed for those studies. Publishing them helps scientists win prestigious academic appointments and attract federal research grants that can pay dividends for themselves and their universities.

Dr. Yoon, a robotic surgery specialist noted for his treatment of stomach cancers, has helped bring in nearly $5 million in federal research money over his career.

The latest retractions from his lab included articles from 2020 and 2021 that Dr. David said contained glaring irregularities . Their results appeared to include identical images of tumor-stricken mice, despite those mice supposedly having been subjected to different experiments involving separate treatments and types of cancer cells.

The medical journal Cell Death & Disease retracted two of the latest studies, and Oncogene retracted the third. The journals found that the studies had also reused other images, like identical pictures of constellations of cancer cells.

The studies Dr. David flagged as containing image problems were largely overseen by the more senior Dr. Yoon. Changhwan Yoon, an associate research scientist who has worked alongside Dr. Yoon for a decade, was often a first author, which generally designates the scientist who ran the bulk of the experiments.

Kun Huang, a scientist in China who oversaw one of the recently retracted studies, a 2020 paper that did not include the more senior Dr. Yoon, attributed that study’s problematic sections to Changhwan Yoon. Dr. Huang, who made those comments this month on PubPeer, a website where scientists post about studies, did not respond to an email seeking comment.

But the more senior Dr. Yoon has long been made aware of problems in research he published alongside Changhwan Yoon: The two scientists were notified of the removal in January 2022 of their stomach cancer study that was found to have violated ethics guidelines.

Research misconduct is often pinned on the more junior researchers who conduct experiments. Other scientists, though, assign greater responsibility to the senior researchers who run labs and oversee studies, even as they juggle jobs as doctors or administrators.

“The research world’s coming to realize that with great power comes great responsibility and, in fact, you are responsible not just for what one of your direct reports in the lab has done, but for the environment you create,” Dr. Oransky said.

In their latest public retraction notices, medical journals said that they had lost faith in the results and conclusions. Imaging experts said some irregularities identified by Dr. David bore signs of deliberate manipulation, like flipped or rotated images, while others could have been sloppy copy-and-paste errors.

The little-noticed removal by a journal of the stomach cancer study in January 2022 highlighted some scientific publishers’ policy of not disclosing the reasons for withdrawing papers as long as they have not yet formally appeared in print. That study had appeared only online.

Roland Herzog, the editor of the journal Molecular Therapy, said that editors had drafted an explanation that they intended to publish at the time of the article’s removal. But Elsevier, the journal’s parent publisher, advised them that such a note was unnecessary, he said.

Only after the Times article last month did Elsevier agree to explain the article’s removal publicly with the stern note. In an editorial this week , the Molecular Therapy editors said that in the future, they would explain the removal of any articles that had been published only online.

But Elsevier said in a statement that it did not consider online articles “to be the final published articles of record.” As a result, company policy continues to advise that such articles be removed without an explanation when they are found to contain problems. The company said it allowed editors to provide additional information where needed.

Elsevier, which publishes nearly 3,000 journals and generates billions of dollars in annual revenue , has long been criticized for its opaque removals of online articles.

Articles by the Columbia scientists with data discrepancies that remain unaddressed were largely distributed by three major publishers: Elsevier, Springer Nature and the American Association for Cancer Research. Dr. David alerted many journals to the data discrepancies in October.

Each publisher said it was investigating the concerns. Springer Nature said investigations take time because they can involve consulting experts, waiting for author responses and analyzing raw data.

Dr. David has also raised concerns about studies published independently by scientists who collaborated with the Columbia researchers on some of their recently retracted papers. For example, Sandra Ryeom, an associate professor of surgical sciences at Columbia, published an article in 2003 while at Harvard that Dr. David said contained a duplicated image . As of 2021, she was married to the more senior Dr. Yoon, according to a mortgage document from that year.

A medical journal appended a formal notice to the article last week saying “appropriate editorial action will be taken” once data concerns had been resolved. Dr. Ryeom said in a statement that she was working with the paper’s senior author on “correcting the error.”

Columbia has sought to reinforce the importance of sound research practices. Hours after the Times article appeared last month, Dr. Michael Shelanski, the medical school’s senior vice dean for research, sent an email to faculty members titled “Research Fraud Accusations — How to Protect Yourself.” It warned that such allegations, whatever their merits, could take a toll on the university.

“In the months that it can take to investigate an allegation,” Dr. Shelanski wrote, “funding can be suspended, and donors can feel that their trust has been betrayed.”

Benjamin Mueller reports on health and medicine. He was previously a U.K. correspondent in London and a police reporter in New York. More about Benjamin Mueller

To revisit this article, visit My Profile, then View saved stories .

• Backchannel
• Newsletters
• WIRED Insider
• WIRED Consulting

Will Knight

Apple’s MM1 AI Model Shows a Sleeping Giant Is Waking Up

While the tech industry went gaga for generative artificial intelligence , one giant has held back: Apple. The company has yet to introduce so much as an AI-generated emoji, and according to a New York Times report today and earlier reporting from Bloomberg, it is in preliminary talks with Google about adding the search company’s Gemini AI model to iPhones .

Yet a research paper quietly posted online last Friday by Apple engineers suggests that the company is making significant new investments into AI that are already bearing fruit. It details the development of a new generative AI model called MM1 capable of working with text and images. The researchers show it answering questions about photos and displaying the kind of general knowledge skills shown by chatbots like ChatGPT. The model’s name is not explained but could stand for MultiModal 1. MM1 appears to be similar in design and sophistication to a variety of recent AI models from other tech giants, including Meta’s open source Llama 2 and Google’s Gemini . Work by Apple’s rivals and academics shows that models of this type can be used to power capable chatbots or build “agents” that can solve tasks by writing code and taking actions such as using computer interfaces or websites. That suggests MM1 could yet find its way into Apple’s products.

“The fact that they’re doing this, it shows they have the ability to understand how to train and how to build these models,” says Ruslan Salakhutdinov , a professor at Carnegie Mellon who led AI research at Apple several years ago. “It requires a certain amount of expertise.”

MM1 is a multimodal large language model, or MLLM, meaning it is trained on images as well as text. This allows the model to respond to text prompts and also answer complex questions about particular images.

One example in the Apple research paper shows what happened when MM1 was provided with a photo of a sun-dappled restaurant table with a couple of beer bottles and also an image of the menu. When asked how much someone would expect to pay for “all the beer on the table,” the model correctly reads off the correct price and tallies up the cost.

When ChatGPT launched in November 2022, it could only ingest and generate text, but more recently its creator OpenAI and others have worked to expand the underlying large language model technology to work with other kinds of data. When Google launched Gemini (the model that now powers its answer to ChatGPT ) last December, the company touted its multimodal nature as beginning an important new direction in AI. “After the rise of LLMs, MLLMs are emerging as the next frontier in foundation models,” Apple’s paper says.

MM1 is a relatively small model as measured by its number of “parameters,” or the internal variables that get adjusted as a model is trained. Kate Saenko , a professor at Boston University who specializes in computer vision and machine learning, says this could make it easier for Apple’s engineers to experiment with different training methods and refinements before scaling up when they hit on something promising.

Saenko says the MM1 paper provides a surprising amount of detail on how the model was trained for a corporate publication. For instance, the engineers behind MM1 describe tricks for improving the performance of the model including increasing the resolution of images and mixing text and image data. Apple is famed for its secrecy, but it has previously shown unusual openness about AI research as it has sought to lure the talent needed to compete in the crucial technology.

Aarian Marshall

Saenko says it’s hard to draw too many conclusions about Apple’s plans from the research paper. Multimodal models have proven adaptable to many different use cases. But she suggests that MM1 could perhaps be a step toward building “some type of multimodal assistant that can describe photos, documents, or charts and answer questions about them.”

Apple’s flagship product, the iPhone, already has an AI assistant—Siri. The rise of ChatGPT and its rivals has quickly made the once revolutionary helper look increasingly limited and out-dated. Amazon and Google have said they are integrating LLM technology into their own assistants, Alexa and Google Assistant. Google allows users of Android phones to replace the Assistant with Gemini. Reports from The New York Times and Bloomberg that Apple may add Google’s Gemini to iPhones suggest Apple is considering expanding the strategy it has used for search on mobile devices to generative AI. Rather than develop web search technology in-house, the iPhone maker leans on Google, which reportedly pays more than $18 billion to make its search engine the iPhone default. Apple has also shown it can build its own alternatives to outside services, even when it starts from behind. Google Maps used to be the default on iPhones but in 2012 Apple replaced it with its own maps app .

Apple CEO Tim Cook has promised investors that the company will reveal more of its generative AI plans this year. The company faces pressure to keep up with rival smartphone makers, including Samsung and Google, that have introduced a raft of generative AI tools for their devices.

Apple could end up tapping both Google and its own, in-house AI, perhaps by introducing Gemini as a replacement for conventional Google Search while also building new generative AI tools on top of MM1 and other homegrown models. Last September, several of the researchers behind MM1 published details of MGIE , a tool that uses generative AI to manipulate images based on a text prompt.

Salakhutdinov believes his former employer may focus on developing LLMs that can be installed and run securely on Apple devices. That would fit with the company’s past emphasis on using “on-device” algorithms to safeguard sensitive data and avoid sharing it with other companies. A number of recent AI research papers from Apple concern machine-learning methods designed to preserve user privacy. “I think that's probably what Apple is going to do,” he says.

When it comes to tailoring generative AI to devices, Salakhutdinov says, Apple may yet turn out to have a distinct advantage because of its control over the entire software-hardware stack. The company has included a custom “neural engine” in the chips that power its mobile devices since 2017, with the debut of the iPhone X. “Apple is definitely working in that space, and I think at some point they will be in the front, because they have phones, the distribution.”

In a thread on X, Apple researcher Brandon McKinzie, lead author of the MM1 paper wrote : “This is just the beginning. The team is already hard at work on the next generation of models.”

You Might Also Like …

In your inbox: Will Knight's Fast Forward explores advances in AI

This shadowy firm enables businesses to operate in near-total secrecy

Scientists are inching closer to bringing back the woolly mammoth

The first rule of the Extreme Dishwasher Loading Facebook group is …

Phones for every budget: These devices stood up to WIRED’s testing

Morgan Meaker

Makena Kelly

Vittoria Elliott