thesis recommendation for future researchers

• Cookies & Privacy
• GETTING STARTED
• Introduction
• FUNDAMENTALS
• Acknowledgements
• Research questions & hypotheses
• Concepts, constructs & variables
• Research limitations
• Getting started
• Sampling Strategy
• Research Quality
• Research Ethics
• Data Analysis

FUTURE RESEARCH

Types of future research suggestion.

The Future Research section of your dissertation is often combined with the Research Limitations section of your final, Conclusions chapter. This is because your future research suggestions generally arise out of the research limitations you have identified in your own dissertation. In this article, we discuss six types of future research suggestion. These include: (1) building on a particular finding in your research; (2) addressing a flaw in your research; examining (or testing) a theory (framework or model) either (3) for the first time or (4) in a new context, location and/or culture; (5) re-evaluating and (6) expanding a theory (framework or model). The goal of the article is to help you think about the potential types of future research suggestion that you may want to include in your dissertation.

Before we discuss each of these types of future research suggestion, we should explain why we use the word examining and then put or testing in brackets. This is simply because the word examining may be considered more appropriate when students use a qualitative research design; whereas the word testing fits better with dissertations drawing on a quantitative research design. We also put the words framework or model in brackets after the word theory . We do this because a theory , framework and model are not the same things. In the sections that follow, we discuss six types of future research suggestion.

Addressing research limitations in your dissertation

Building on a particular finding or aspect of your research, examining a conceptual framework (or testing a theoretical model) for the first time, examining a conceptual framework (or testing a theoretical model) in a new context, location and/or culture.

• Expanding a conceptual framework (or testing a theoretical model)

Re-evaluating a conceptual framework (or theoretical model)

In the Research Limitations section of your Conclusions chapter, you will have inevitably detailed the potential flaws (i.e., research limitations) of your dissertation. These may include:

An inability to answer your research questions

Theoretical and conceptual problems

Limitations of your research strategy

Problems of research quality

Identifying what these research limitations were and proposing future research suggestions that address them is arguably the easiest and quickest ways to complete the Future Research section of your Conclusions chapter.

Often, the findings from your dissertation research will highlight a number of new avenues that could be explored in future studies. These can be grouped into two categories:

Your dissertation will inevitably lead to findings that you did not anticipate from the start. These are useful when making future research suggestions because they can lead to entirely new avenues to explore in future studies. If this was the case, it is worth (a) briefly describing what these unanticipated findings were and (b) suggesting a research strategy that could be used to explore such findings in future.

Sometimes, dissertations manage to address all aspects of the research questions that were set. However, this is seldom the case. Typically, there will be aspects of your research questions that could not be answered. This is not necessarily a flaw in your research strategy, but may simply reflect that fact that the findings did not provide all the answers you hoped for. If this was the case, it is worth (a) briefly describing what aspects of your research questions were not answered and (b) suggesting a research strategy that could be used to explore such aspects in future.

You may want to recommend that future research examines the conceptual framework (or tests the theoretical model) that you developed. This is based on the assumption that the primary goal of your dissertation was to set out a conceptual framework (or build a theoretical model). It is also based on the assumption that whilst such a conceptual framework (or theoretical model) was presented, your dissertation did not attempt to examine (or test) it in the field . The focus of your dissertations was most likely a review of the literature rather than something that involved you conducting primary research.

Whilst it is quite rare for dissertations at the undergraduate and master's level to be primarily theoretical in nature like this, it is not unknown. If this was the case, you should think about how the conceptual framework (or theoretical model) that you have presented could be best examined (or tested) in the field . In understanding the how , you should think about two factors in particular:

What is the context, location and/or culture that would best lend itself to my conceptual framework (or theoretical model) if it were to be examined (or tested) in the field?

What research strategy is most appropriate to examine my conceptual framework (or test my theoretical model)?

If the future research suggestion that you want to make is based on examining your conceptual framework (or testing your theoretical model) in the field , you need to suggest the best scenario for doing so.

More often than not, you will not only have set out a conceptual framework (or theoretical model), as described in the previous section, but you will also have examined (or tested) it in the field . When you do this, focus is typically placed on a specific context, location and/or culture.

If this is the case, the obvious future research suggestion that you could propose would be to examine your conceptual framework (or test the theoretical model) in a new context, location and/or culture. For example, perhaps you focused on consumers (rather than businesses), or Canada (rather than the United Kingdom), or a more individualistic culture like the United States (rather than a more collectivist culture like China).

When you propose a new context, location and/or culture as your future research suggestion, make sure you justify the choice that you make. For example, there may be little value in future studies looking at different cultures if culture is not an important component underlying your conceptual framework (or theoretical model). If you are not sure whether a new context, location or culture is more appropriate, or what new context, location or culture you should select, a review the literature will often help clarify where you focus should be.

Expanding a conceptual framework (or theoretical model)

Assuming that you have set out a conceptual framework (or theoretical model) and examined (or tested) it in the field , another series of future research suggestions comes out of expanding that conceptual framework (or theoretical model).

We talk about a series of future research suggestions because there are so many ways that you can expand on your conceptual framework (or theoretical model). For example, you can do this by:

Examining constructs (or variables) that were included in your conceptual framework (or theoretical model) but were not focused.

Looking at a particular relationship aspect of your conceptual framework (or theoretical model) further.

Adding new constructs (or variables) to the conceptual framework (or theoretical model) you set out (if justified by the literature).

It would be possible to include one or a number of these as future research suggestions. Again, make sure that any suggestions you make have are justified , either by your findings or the literature.

With the dissertation process at the undergraduate and master's level lasting between 3 and 9 months, a lot a can happen in between. For example, a specific event (e.g., 9/11, the economic crisis) or some new theory or evidence that undermines (or questions) the literature (theory) and assumptions underpinning your conceptual framework (or theoretical model). Clearly, there is little you can do about this. However, if this happens, reflecting on it and re-evaluating your conceptual framework (or theoretical model), as well as your findings, is an obvious source of future research suggestions.

Suggestions for Future Research

Your dissertation needs to include suggestions for future research. Depending on requirements of your university, suggestions for future research can be either integrated into Research Limitations section or it can be a separate section.

You will need to propose 4-5 suggestions for future studies and these can include the following:

1. Building upon findings of your research . These may relate to findings of your study that you did not anticipate. Moreover, you may suggest future research to address unanswered aspects of your research problem.

2. Addressing limitations of your research . Your research will not be free from limitations and these may relate to formulation of research aim and objectives, application of data collection method, sample size, scope of discussions and analysis etc. You can propose future research suggestions that address the limitations of your study.

3. Constructing the same research in a new context, location and/or culture . It is most likely that you have addressed your research problem within the settings of specific context, location and/or culture. Accordingly, you can propose future studies that can address the same research problem in a different settings, context, location and/or culture.

4. Re-assessing and expanding theory, framework or model you have addressed in your research . Future studies can address the effects of specific event, emergence of a new theory or evidence and/or other recent phenomenon on your research problem.

My e-book,  The Ultimate Guide to Writing a Dissertation in Business Studies: a step by step assistance  offers practical assistance to complete a dissertation with minimum or no stress. The e-book covers all stages of writing a dissertation starting from the selection to the research area to submitting the completed version of the work within the deadline. John Dudovskiy

Research Recommendations – Guiding policy-makers for evidence-based decision making

Research recommendations play a crucial role in guiding scholars and researchers toward fruitful avenues of exploration. In an era marked by rapid technological advancements and an ever-expanding knowledge base, refining the process of generating research recommendations becomes imperative.

But, what is a research recommendation?

Research recommendations are suggestions or advice provided to researchers to guide their study on a specific topic . They are typically given by experts in the field. Research recommendations are more action-oriented and provide specific guidance for decision-makers, unlike implications that are broader and focus on the broader significance and consequences of the research findings. However, both are crucial components of a research study.

Difference Between Research Recommendations and Implication

Although research recommendations and implications are distinct components of a research study, they are closely related. The differences between them are as follows:

Types of Research Recommendations

Recommendations in research can take various forms, which are as follows:

These recommendations aim to assist researchers in navigating the vast landscape of academic knowledge.

Let us dive deeper to know about its key components and the steps to write an impactful research recommendation.

Key Components of Research Recommendations

The key components of research recommendations include defining the research question or objective, specifying research methods, outlining data collection and analysis processes, presenting results and conclusions, addressing limitations, and suggesting areas for future research. Here are some characteristics of research recommendations:

Research recommendations offer various advantages and play a crucial role in ensuring that research findings contribute to positive outcomes in various fields. However, they also have few limitations which highlights the significance of a well-crafted research recommendation in offering the promised advantages.

The importance of research recommendations ranges in various fields, influencing policy-making, program development, product development, marketing strategies, medical practice, and scientific research. Their purpose is to transfer knowledge from researchers to practitioners, policymakers, or stakeholders, facilitating informed decision-making and improving outcomes in different domains.

How to Write Research Recommendations?

Research recommendations can be generated through various means, including algorithmic approaches, expert opinions, or collaborative filtering techniques. Here is a step-wise guide to build your understanding on the development of research recommendations.

1. Understand the Research Question:

Understand the research question and objectives before writing recommendations. Also, ensure that your recommendations are relevant and directly address the goals of the study.

2. Review Existing Literature:

Familiarize yourself with relevant existing literature to help you identify gaps , and offer informed recommendations that contribute to the existing body of research.

3. Consider Research Methods:

Evaluate the appropriateness of different research methods in addressing the research question. Also, consider the nature of the data, the study design, and the specific objectives.

4. Identify Data Collection Techniques:

Gather dataset from diverse authentic sources. Include information such as keywords, abstracts, authors, publication dates, and citation metrics to provide a rich foundation for analysis.

5. Propose Data Analysis Methods:

Suggest appropriate data analysis methods based on the type of data collected. Consider whether statistical analysis, qualitative analysis, or a mixed-methods approach is most suitable.

6. Consider Limitations and Ethical Considerations:

Acknowledge any limitations and potential ethical considerations of the study. Furthermore, address these limitations or mitigate ethical concerns to ensure responsible research.

7. Justify Recommendations:

Explain how your recommendation contributes to addressing the research question or objective. Provide a strong rationale to help researchers understand the importance of following your suggestions.

8. Summarize Recommendations:

Provide a concise summary at the end of the report to emphasize how following these recommendations will contribute to the overall success of the research project.

By following these steps, you can create research recommendations that are actionable and contribute meaningfully to the success of the research project.

Download now to unlock some tips to improve your journey of writing research recommendations.

Example of a Research Recommendation

Here is an example of a research recommendation based on a hypothetical research to improve your understanding.

Research Recommendation: Enhancing Student Learning through Integrated Learning Platforms

Background:

The research study investigated the impact of an integrated learning platform on student learning outcomes in high school mathematics classes. The findings revealed a statistically significant improvement in student performance and engagement when compared to traditional teaching methods.

Recommendation:

In light of the research findings, it is recommended that educational institutions consider adopting and integrating the identified learning platform into their mathematics curriculum. The following specific recommendations are provided:

• Implementation of the Integrated Learning Platform:

Schools are encouraged to adopt the integrated learning platform in mathematics classrooms, ensuring proper training for teachers on its effective utilization.

• Professional Development for Educators:

Develop and implement professional programs to train educators in the effective use of the integrated learning platform to address any challenges teachers may face during the transition.

• Monitoring and Evaluation:

Establish a monitoring and evaluation system to track the impact of the integrated learning platform on student performance over time.

• Resource Allocation:

Allocate sufficient resources, both financial and technical, to support the widespread implementation of the integrated learning platform.

By implementing these recommendations, educational institutions can harness the potential of the integrated learning platform and enhance student learning experiences and academic achievements in mathematics.

This example covers the components of a research recommendation, providing specific actions based on the research findings, identifying the target audience, and outlining practical steps for implementation.

Using AI in Research Recommendation Writing

Enhancing research recommendations is an ongoing endeavor that requires the integration of cutting-edge technologies, collaborative efforts, and ethical considerations. By embracing data-driven approaches and leveraging advanced technologies, the research community can create more effective and personalized recommendation systems. However, it is accompanied by several limitations. Therefore, it is essential to approach the use of AI in research with a critical mindset, and complement its capabilities with human expertise and judgment.

Here are some limitations of integrating AI in writing research recommendation and some ways on how to counter them.

1. Data Bias

AI systems rely heavily on data for training. If the training data is biased or incomplete, the AI model may produce biased results or recommendations.

How to tackle: Audit regularly the model’s performance to identify any discrepancies and adjust the training data and algorithms accordingly.

2. Lack of Understanding of Context:

AI models may struggle to understand the nuanced context of a particular research problem. They may misinterpret information, leading to inaccurate recommendations.

How to tackle: Use AI to characterize research articles and topics. Employ them to extract features like keywords, authorship patterns and content-based details.

3. Ethical Considerations:

AI models might stereotype certain concepts or generate recommendations that could have negative consequences for certain individuals or groups.

How to tackle: Incorporate user feedback mechanisms to reduce redundancies. Establish an ethics review process for AI models in research recommendation writing.

4. Lack of Creativity and Intuition:

AI may struggle with tasks that require a deep understanding of the underlying principles or the ability to think outside the box.

How to tackle: Hybrid approaches can be employed by integrating AI in data analysis and identifying patterns for accelerating the data interpretation process.

5. Interpretability:

Many AI models, especially complex deep learning models, lack transparency on how the model arrived at a particular recommendation.

How to tackle: Implement models like decision trees or linear models. Provide clear explanation of the model architecture, training process, and decision-making criteria.

6. Dynamic Nature of Research:

Research fields are dynamic, and new information is constantly emerging. AI models may struggle to keep up with the rapidly changing landscape and may not be able to adapt to new developments.

How to tackle: Establish a feedback loop for continuous improvement. Regularly update the recommendation system based on user feedback and emerging research trends.

The integration of AI in research recommendation writing holds great promise for advancing knowledge and streamlining the research process. However, navigating these concerns is pivotal in ensuring the responsible deployment of these technologies. Researchers need to understand the use of responsible use of AI in research and must be aware of the ethical considerations.

Exploring research recommendations plays a critical role in shaping the trajectory of scientific inquiry. It serves as a compass, guiding researchers toward more robust methodologies, collaborative endeavors, and innovative approaches. Embracing these suggestions not only enhances the quality of individual studies but also contributes to the collective advancement of human understanding.

Frequently Asked Questions

The purpose of recommendations in research is to provide practical and actionable suggestions based on the study's findings, guiding future actions, policies, or interventions in a specific field or context. Recommendations bridges the gap between research outcomes and their real-world application.

To make a research recommendation, analyze your findings, identify key insights, and propose specific, evidence-based actions. Include the relevance of the recommendations to the study's objectives and provide practical steps for implementation.

Begin a recommendation by succinctly summarizing the key findings of the research. Clearly state the purpose of the recommendation and its intended impact. Use a direct and actionable language to convey the suggested course of action.

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• Diversity and Inclusion
• Industry News

6 Reasons Why There is a Decline in Higher Education Enrollment: Action plan to overcome this crisis

Over the past decade, colleges and universities across the globe have witnessed a concerning trend…

• Reporting Research

Academic Essay Writing Made Simple: 4 types and tips

The pen is mightier than the sword, they say, and nowhere is this more evident…

Ensuring Academic Integrity and Transparency in Academic Research: A comprehensive checklist for researchers

Academic integrity is the foundation upon which the credibility and value of scientific findings are…

• Publishing News

Unified AI Guidelines Crucial as Academic Writing Embraces Generative Tools

As generative artificial intelligence (AI) tools like ChatGPT are advancing at an accelerating pace, their…

How to Effectively Cite a PDF (APA, MLA, AMA, and Chicago Style)

The pressure to “publish or perish” is a well-known reality for academics, striking fear into…

How to Optimize Your Research Process: A step-by-step guide

Digital Citations: A comprehensive guide to citing of websites in APA, MLA, and CMOS…

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

As a researcher, what do you consider most when choosing an image manipulation detector?

Research Implications & Recommendations

A Plain-Language Explainer With Examples + FREE Template

By: Derek Jansen (MBA) | Expert Reviewer: Dr Eunice Rautenbach | May 2024

The research implications and recommendations are closely related but distinctly different concepts that often trip students up. Here, we’ll unpack them using plain language and loads of examples , so that you can approach your project with confidence.

Overview: Implications & Recommendations

• What are research implications ?
• What are research recommendations ?
• Examples of implications and recommendations
• The “ Big 3 ” categories
• How to write the implications and recommendations
• Template sentences for both sections
• Key takeaways

Implications & Recommendations 101

Let’s start with the basics and define our terms.

At the simplest level, research implications refer to the possible effects or outcomes of a study’s findings. More specifically, they answer the question, “ What do these findings mean?” . In other words, the implications section is where you discuss the broader impact of your study’s findings on theory, practice and future research.

This discussion leads us to the recommendations section , which is where you’ll propose specific actions based on your study’s findings and answer the question, “ What should be done next?” . In other words, the recommendations are practical steps that stakeholders can take to address the key issues identified by your study.

In a nutshell, then, the research implications discuss the broader impact and significance of a study’s findings, while recommendations provide specific actions to take, based on those findings. So, while both of these components are deeply rooted in the findings of the study, they serve different functions within the write up.

Need a helping hand?

Examples: Implications & Recommendations

The distinction between research implications and research recommendations might still feel a bit conceptual, so let’s look at one or two practical examples:

Let’s assume that your study finds that interactive learning methods significantly improve student engagement compared to traditional lectures. In this case, one of your recommendations could be that schools incorporate more interactive learning techniques into their curriculums to enhance student engagement.

Let’s imagine that your study finds that patients who receive personalised care plans have better health outcomes than those with standard care plans. One of your recommendations might be that healthcare providers develop and implement personalised care plans for their patients.

Now, these are admittedly quite simplistic examples, but they demonstrate the difference (and connection ) between the research implications and the recommendations. Simply put, the implications are about the impact of the findings, while the recommendations are about proposed actions, based on the findings.

The “Big 3” Categories

Now that we’ve defined our terms, let’s dig a little deeper into the implications – specifically, the different types or categories of research implications that exist.

Broadly speaking, implications can be divided into three categories – theoretical implications, practical implications and implications for future research .

Theoretical implications relate to how your study’s findings contribute to or challenge existing theories. For example, if a study on social behaviour uncovers new patterns, it might suggest that modifications to current psychological theories are necessary.

Practical implications , on the other hand, focus on how your study’s findings can be applied in real-world settings. For example, if your study demonstrated the effectiveness of a new teaching method, this would imply that educators should consider adopting this method to improve learning outcomes.

Practical implications can also involve policy reconsiderations . For example, if a study reveals significant health benefits from a particular diet, an implication might be that public health guidelines be re-evaluated.

Last but not least, there are the implications for future research . As the name suggests, this category of implications highlights the research gaps or new questions raised by your study. For example, if your study finds mixed results regarding a relationship between two variables, it might imply the need for further investigation to clarify these findings.

To recap then, the three types of implications are the theoretical, the practical and the implications on future research. Regardless of the category, these implications feed into and shape the recommendations , laying the foundation for the actions you’ll propose.

How To Write The  Sections

Now that we’ve laid the foundations, it’s time to explore how to write up the implications and recommendations sections respectively.

Let’s start with the “ where ” before digging into the “ how ”. Typically, the implications will feature in the discussion section of your document, while the recommendations will be located in the conclusion . That said, layouts can vary between disciplines and institutions, so be sure to check with your university what their preferences are.

For the implications section, a common approach is to structure the write-up based on the three categories we looked at earlier – theoretical, practical and future research implications. In practical terms, this discussion will usually follow a fairly formulaic sentence structure – for example:

This research provides new insights into [theoretical aspect], indicating that…

The study’s outcomes highlight the potential benefits of adopting [specific practice] in..

This study raises several questions that warrant further investigation, such as…

Moving onto the recommendations section, you could again structure your recommendations using the three categories. Alternatively, you could structure the discussion per stakeholder group – for example, policymakers, organisations, researchers, etc.

Again, you’ll likely use a fairly formulaic sentence structure for this section. Here are some examples for your inspiration: 

Based on the findings, [specific group] should consider adopting [new method] to improve…

To address the issues identified, it is recommended that legislation should be introduced to…

Researchers should consider examining [specific variable] to build on the current study’s findings.

Remember, you can grab a copy of our tried and tested templates for both the discussion and conclusion sections over on the Grad Coach blog. You can find the links to those, as well as loads of other free resources, in the description 🙂

FAQs: Implications & Recommendations

How do i determine the implications of my study.

To do this, you’ll need to consider how your findings address gaps in the existing literature, how they could influence theory, practice, or policy, and the potential societal or economic impacts.

When thinking about your findings, it’s also a good idea to revisit your introduction chapter, where you would have discussed the potential significance of your study more broadly. This section can help spark some additional ideas about what your findings mean in relation to your original research aims. 

Should I discuss both positive and negative implications?

Absolutely. You’ll need to discuss both the positive and negative implications to provide a balanced view of how your findings affect the field and any limitations or potential downsides.

Can my research implications be speculative?

Yes and no. While implications are somewhat more speculative than recommendations and can suggest potential future outcomes, they should be grounded in your data and analysis. So, be careful to avoid overly speculative claims.

How do I formulate recommendations?

Ideally, you should base your recommendations on the limitations and implications of your study’s findings. So, consider what further research is needed, how policies could be adapted, or how practices could be improved – and make proposals in this respect.

How specific should my recommendations be?

Your recommendations should be as specific as possible, providing clear guidance on what actions or research should be taken next. As mentioned earlier, the implications can be relatively broad, but the recommendations should be very specific and actionable. Ideally, you should apply the SMART framework to your recommendations.

Can I recommend future research in my recommendations?

Absolutely. Highlighting areas where further research is needed is a key aspect of the recommendations section. Naturally, these recommendations should link to the respective section of your implications (i.e., implications for future research).

Wrapping Up: Key Takeaways

We’ve covered quite a bit of ground here, so let’s quickly recap.

• Research implications refer to the possible effects or outcomes of a study’s findings.
• The recommendations section, on the other hand, is where you’ll propose specific actions based on those findings.
• You can structure your implications section based on the three overarching categories – theoretical, practical and future research implications.
• You can carry this structure through to the recommendations as well, or you can group your recommendations by stakeholder.

Remember to grab a copy of our tried and tested free dissertation template, which covers both the implications and recommendations sections. If you’d like 1:1 help with your research project, be sure to check out our private coaching service, where we hold your hand throughout the research journey, step by step.

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

You Might Also Like:

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Writing Effective Conclusions and Recommendations in a Thesis

Table of Contents

Have you ever experienced the triumphant moment of finishing a jigsaw puzzle , only to find that the final piece doesn’t quite fit? Such is the challenge when writing the conclusion and findings .">recommendations in your thesis . This final section isn’t just an endnote; it’s your opportunity to make your research resonate. Let’s explore how to craft a compelling conclusion that not only summarizes your findings but also sets the stage for future inquiry.

Understanding the weight of conclusions and recommendations

In the grand scheme of your thesis, the conclusion is your final act, your closing argument. It is here that you revisit your research objectives to demonstrate how your findings address them. But it’s not just a rehash of your research; it’s your chance to interpret the implications of your work and persuade your audience of its significance.

The art of summarizing findings

Begin by revisiting the questions you posed at the start of your journey. Summarize the answers you’ve uncovered, concisely distilling the essence of your research. This isn’t a place for exhaustive detail—that’s what the body of your thesis is for. Instead, think of it as the highlight reel, showcasing the most significant insights.

Drawing insightful conclusions

Conclusions are the thoughtful deductions you’ve drawn from your research. Linking back to your objectives, ask yourself: What have I learned? How have my findings contributed to the field? This is where you can shine a light on the larger implications of your work, the ‘so what?’ of your thesis.

Outlining clear and actionable recommendations

Your recommendations are your legacy, the path you lay for future research ers or practitioners. They should be clear, specific, and grounded in your findings. Think of them as a call to action—what should be done in light of your research? Whether it’s suggesting areas for further study or recommending changes to practice, your recommendations should inspire action.

Linking conclusions to research objectives

To ensure your conclusions are meaningful, they must be tethered to your research objectives. This alignment reinforces the relevance of your work and ensures you’re answering the questions you set out to explore.

Reflecting on research questions

Revisit your research questions or hypotheses . How have your findings addressed them? This reflection ensures your conclusions are directly tied to your initial inquiry, creating a cohesive narrative from start to finish.

Emphasizing the impact of your findings

Highlight the impact of your research. Have you filled a gap in knowledge? Challenged existing theories? Applied your findings to practice? Your conclusions should underscore the value of your work and its ripple effect in your field.

Formulating recommendations with purpose

Recommendations should never be an afterthought. They are your opportunity to influence the future, to suggest how the knowledge you’ve generated can be used to better understand or improve upon a particular issue.

Grounding recommendations in evidence

Ensure your recommendations are evidence\-based . They should flow naturally from the conclusions you’ve drawn, each one supported by the data you’ve collected.

Keeping recommendations achievable

Make your recommendations achievable. Pie-in-the-sky ideas might be inspirational, but they’re less helpful than practical, attainable suggestions that can be realistically implemented.

Setting the stage for future research

Use your recommendations to set the stage for future research. What questions have emerged from your study? Where are the knowledge gaps ? Propel the academic conversation forward by identifying the next steps.

Writing with clarity and conviction

The best conclusions and recommendations are those written with clarity and conviction. You’ve spent countless hours on your research—now’s the time to confidently present your findings and their implications.

Using precise language

Be precise in your language. Avoid ambiguity and make every word count. This precision not only makes your writing clearer but also reinforces the authority of your conclusions and recommendations.

Ensuring cohesiveness

Your conclusion should feel like a natural culmination of your thesis, not a separate entity. Ensure that it flows logically from the body of your work, with each part reinforcing the other.

Engaging the reader’s imagination

Engage your reader’s imagination by painting a picture of what could be. Use your recommendations to inspire visions of future possibilities, grounded in the solid foundation of your research.

In conclusion, remember that the final sections of your thesis are not mere formalities but the bridge between your research and its potential impact on the world. The conclusions and recommendations you write are your scholarly legacy , a testament to the hard work you’ve poured into your research. Approach them with the same rigor , creativity, and dedication as the rest of your thesis, and you’ll leave a lasting mark on your field.

What do you think? How do you approach writing conclusions and recommendations, and what strategies do you find most effective? Are there particular challenges you’ve faced in this part of the thesis-writing process?

How useful was this post?

Click on a star to rate it!

Average rating / 5. Vote count:

No votes so far! Be the first to rate this post.

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Submit Comment

Research Methodology

1 Introduction to Research in General

• Research in General
• Research Circle
• Tools of Research
• Methods: Quantitative or Qualitative
• The Product: Research Report or Papers

2 Original Unity of Philosophy and Science

• Myth Philosophy and Science: Original Unity
• The Myth: A Spiritual Metaphor
• Myth Philosophy and Science
• The Greek Quest for Unity
• The Ionian School
• Towards a Grand Unification Theory or Theory of Everything
• Einstein’s Perennial Quest for Unity

3 Evolution of the Distinct Methods of Science

• Definition of Scientific Method
• The Evolution of Scientific Methods
• Theory-Dependence of Observation
• Scope of Science and Scientific Methods
• Prevalent Mistakes in Applying the Scientific Method

4 Relation of Scientific and Philosophical Methods

• Definitions of Scientific and Philosophical method
• Philosophical method
• Scientific method
• The relation
• The Importance of Philosophical and scientific methods

5 Dialectical Method

• Introduction and a Brief Survey of the Method
• Types of Dialectics
• Dialectics in Classical Philosophy
• Dialectics in Modern Philosophy
• Critique of Dialectical Method

6 Rational Method

• Understanding Rationalism
• Rational Method of Investigation
• Descartes’ Rational Method
• Leibniz’ Aim of Philosophy
• Spinoza’ Aim of Philosophy

7 Empirical Method

• Common Features of Philosophical Method
• Empirical Method
• Exposition of Empiricism
• Locke’s Empirical Method
• Berkeley’s Empirical Method
• David Hume’s Empirical Method

8 Critical Method

• Basic Features of Critical Theory
• On Instrumental Reason
• Conception of Society
• Human History as Dialectic of Enlightenment
• Substantive Reason
• Habermasian Critical Theory
• Habermas’ Theory of Society
• Habermas’ Critique of Scientism
• Theory of Communicative Action
• Discourse Ethics of Habermas

9 Phenomenological Method (Western and Indian)

• Phenomenology in Philosophy
• Phenomenology as a Method
• Phenomenological Analysis of Knowledge
• Phenomenological Reduction
• Husserl’s Triad: Ego Cogito Cogitata
• Intentionality
• Understanding ‘Consciousness’
• Phenomenological Method in Indian Tradition
• Phenomenological Method in Religion

10 Analytical Method (Western and Indian)

• Analysis in History of Philosophy
• Conceptual Analysis
• Analysis as a Method
• Analysis in Logical Atomism and Logical Positivism
• Analytic Method in Ethics
• Language Analysis
• Quine’s Analytical Method
• Analysis in Indian Traditions

11 Hermeneutical Method (Western and Indian)

• The Power (Sakti) to Convey Meaning
• Three Meanings
• Pre-understanding
• The Semantic Autonomy of the Text
• Towards a Fusion of Horizons
• The Hermeneutical Circle
• The True Scandal of the Text
• Literary Forms

12 Deconstructive Method

• The Seminal Idea of Deconstruction in Heidegger
• Deconstruction in Derrida
• Structuralism and Post-structuralism
• Sign Signifier and Signified
• Writing and Trace
• Deconstruction as a Strategic Reading
• The Logic of Supplement
• No Outside-text

13 Method of Bibliography

• Preparing to Write
• Writing a Paper
• The Main Divisions of a Paper
• Writing Bibliography in Turabian and APA
• Sample Bibliography

14 Method of Footnotes

• Citations and Notes
• General Hints for Footnotes
• Writing Footnotes
• Examples of Footnote or Endnote
• Example of a Research Article

15 Method of Notes Taking

• Methods of Note-taking
• Note Book Style
• Note taking in a Computer
• Types of Note-taking
• Notes from Field Research
• Errors to be Avoided

16 Method of Thesis Proposal and Presentation

• Preliminary Section
• Presenting the Problem of the Thesis
• Design of the Study
• Main Body of the Thesis
• Conclusion Summary and Recommendations
• Reference Material

Share on Mastodon

• Privacy Policy

Home » Future Research – Thesis Guide

Future Research – Thesis Guide

Table of Contents

Future Research

Definition:

Future research refers to investigations and studies that are yet to be conducted, and are aimed at expanding our understanding of a particular subject or area of interest. Future research is typically based on the current state of knowledge and seeks to address unanswered questions, gaps in knowledge, and new areas of inquiry.

How to Write Future Research in Thesis

Here are some steps to help you write effectively about future research in your thesis :

• Identify a research gap: Before you start writing about future research, identify the areas that need further investigation. Look for research gaps and inconsistencies in the literature , and note them down.
• Specify research questions : Once you have identified a research gap, create a list of research questions that you would like to explore in future research. These research questions should be specific, measurable, and relevant to your thesis.
• Discuss limitations: Be sure to discuss any limitations of your research that may require further exploration. This will help to highlight the need for future research and provide a basis for further investigation.
• Suggest methodologies: Provide suggestions for methodologies that could be used to explore the research questions you have identified. Discuss the pros and cons of each methodology and how they would be suitable for your research.
• Explain significance: Explain the significance of the research you have proposed, and how it will contribute to the field. This will help to justify the need for future research and provide a basis for further investigation.
• Provide a timeline : Provide a timeline for the proposed research , indicating when each stage of the research would be conducted. This will help to give a sense of the practicalities involved in conducting the research.
• Conclusion : Summarize the key points you have made about future research and emphasize the importance of exploring the research questions you have identified.

Examples of Future Research in Thesis

SomeExamples of Future Research in Thesis are as follows:

Future Research:

Although this study provides valuable insights into the effects of social media on self-esteem, there are several avenues for future research that could build upon our findings. Firstly, our sample consisted solely of college students, so it would be beneficial to extend this research to other age groups and demographics. Additionally, our study focused only on the impact of social media use on self-esteem, but there are likely other factors that influence how social media affects individuals, such as personality traits and social support. Future research could examine these factors in greater depth. Lastly, while our study looked at the short-term effects of social media use on self-esteem, it would be interesting to explore the long-term effects over time. This could involve conducting longitudinal studies that follow individuals over a period of several years to assess changes in self-esteem and social media use.

While this study provides important insights into the relationship between sleep patterns and academic performance among college students, there are several avenues for future research that could further advance our understanding of this topic.

• This study relied on self-reported sleep patterns, which may be subject to reporting biases. Future research could benefit from using objective measures of sleep, such as actigraphy or polysomnography, to more accurately assess sleep duration and quality.
• This study focused on academic performance as the outcome variable, but there may be other important outcomes to consider, such as mental health or well-being. Future research could explore the relationship between sleep patterns and these other outcomes.
• This study only included college students, and it is unclear if these findings generalize to other populations, such as high school students or working adults. Future research could investigate whether the relationship between sleep patterns and academic performance varies across different populations.
• Fourth, this study did not explore the potential mechanisms underlying the relationship between sleep patterns and academic performance. Future research could investigate the role of factors such as cognitive functioning, motivation, and stress in this relationship.

Overall, there is a need for continued research on the relationship between sleep patterns and academic performance, as this has important implications for the health and well-being of students.

Further research could investigate the long-term effects of mindfulness-based interventions on mental health outcomes among individuals with chronic pain. A longitudinal study could be conducted to examine the sustainability of mindfulness practices in reducing pain-related distress and improving psychological well-being over time. The study could also explore the potential mediating and moderating factors that influence the relationship between mindfulness and mental health outcomes, such as emotional regulation, pain catastrophizing, and social support.

Purpose of Future Research in Thesis

Here are some general purposes of future research that you might consider including in your thesis:

• To address limitations: Your research may have limitations or unanswered questions that could be addressed by future studies. Identify these limitations and suggest potential areas for further research.
• To extend the research : You may have found interesting results in your research, but future studies could help to extend or replicate your findings. Identify these areas where future research could help to build on your work.
• To explore related topics : Your research may have uncovered related topics that were outside the scope of your study. Suggest areas where future research could explore these related topics in more depth.
• To compare different approaches : Your research may have used a particular methodology or approach, but there may be other approaches that could be compared to your approach. Identify these other approaches and suggest areas where future research could compare and contrast them.
• To test hypotheses : Your research may have generated hypotheses that could be tested in future studies. Identify these hypotheses and suggest areas where future research could test them.
• To address practical implications : Your research may have practical implications that could be explored in future studies. Identify these practical implications and suggest areas where future research could investigate how to apply them in practice.

Applications of Future Research

Some examples of applications of future research that you could include in your thesis are:

• Development of new technologies or methods: If your research involves the development of new technologies or methods, you could discuss potential applications of these innovations in future research or practical settings. For example, if you have developed a new drug delivery system, you could speculate about how it might be used in the treatment of other diseases or conditions.
• Extension of your research: If your research only scratches the surface of a particular topic, you could suggest potential avenues for future research that could build upon your findings. For example, if you have studied the effects of a particular drug on a specific population, you could suggest future research that explores the drug’s effects on different populations or in combination with other treatments.
• Investigation of related topics: If your research is part of a larger field or area of inquiry, you could suggest potential research topics that are related to your work. For example, if you have studied the effects of climate change on a particular species, you could suggest future research that explores the impacts of climate change on other species or ecosystems.
• Testing of hypotheses: If your research has generated hypotheses or theories, you could suggest potential experiments or studies that could test these hypotheses in future research. For example, if you have proposed a new theory about the mechanisms of a particular disease, you could suggest experiments that could test this theory in other populations or in different disease contexts.

Advantage of Future Research

Including future research in a thesis has several advantages:

• Demonstrates critical thinking: Including future research shows that the author has thought deeply about the topic and recognizes its limitations. It also demonstrates that the author is interested in advancing the field and is not satisfied with only providing a narrow analysis of the issue at hand.
• Provides a roadmap for future research : Including future research can help guide researchers in the field by suggesting areas that require further investigation. This can help to prevent researchers from repeating the same work and can lead to more efficient use of resources.
• Shows engagement with the field : By including future research, the author demonstrates their engagement with the field and their understanding of ongoing debates and discussions. This can be especially important for students who are just entering the field and want to show their commitment to ongoing research.
• I ncreases the impact of the thesis : Including future research can help to increase the impact of the thesis by highlighting its potential implications for future research and practical applications. This can help to generate interest in the work and attract attention from researchers and practitioners in the field.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Survey Instruments – List and Their Uses

Research Report – Example, Writing Guide and...

Research Paper Introduction – Writing Guide and...

Delimitations in Research – Types, Examples and...

APA Table of Contents – Format and Example

Research Paper Conclusion – Writing Guide and...

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Raine R, Atkin W, von Wagner C, et al. Testing innovative strategies to reduce the social gradient in the uptake of bowel cancer screening: a programme of four qualitatively enhanced randomised controlled trials. Southampton (UK): NIHR Journals Library; 2017 Mar. (Programme Grants for Applied Research, No. 5.8.)

Testing innovative strategies to reduce the social gradient in the uptake of bowel cancer screening: a programme of four qualitatively enhanced randomised controlled trials.

Chapter 12 conclusions and recommendations for future research.

Socioeconomic inequalities in health and health care are widespread internationally. Research is usually designed to improve access to, and uptake of, health care and health outcomes among socially disadvantaged groups. To our knowledge, this is the first national research programme to focus on the entire population in order to improve uptake across the whole socioeconomic gradient. By developing four theoretically based interventions that aimed to increase the uptake of bowel cancer screening across the socioeconomic gradient, we hope to ultimately reduce inequalities in cancer outcomes. Here we summarise our key findings and their implications for future research and practice. These results are equally applicable to the FIT , which the NHS BCSP have begun to introduce as a pilot in 2014.

• The national RCT of the ER (see Chapter 10 ) demonstrated a significant effect on both the social gradient in uptake and uptake overall. Given the extremely low marginal cost of this intervention, the findings suggest that implementation by the NHS BCSP would be beneficial.
• Our national trial of GPE significantly increased uptake overall (although it did not reduce the social gradient in uptake) (see Chapter 10 ). Given the exceptionally high level of agreement by general practices to endorse the NHS BCSP , the small one-off cost to modify the standard invitation letter within the NHS BCSP IT system (to adjust for any recent changes in practice name) and the zero marginal cost per person screened, the evidence suggests that the NHS BCSP could usefully consider adding GPE to the screening invitation letter.
• Enhanced and explicit involvement of familiar health-care settings and professionals should be further investigated, with an emphasis on tailoring NHS involvement to the specific needs of defined sociocultural groups. The impact of such initiatives on the social gradient and ethnic differences in uptake should be examined.
• The evidence suggests that policies aimed at reducing the gradient in bowel cancer screening participation could include strategies aimed at increasing acceptability and comprehension of the screening test among those with lower levels of literacy. This is supported by the strong relationship between health literacy and SECs. The pilot of the gist leaflet in workstream 2 demonstrated that improving the accessibility of screening information to individuals with low health literacy may improve their comprehension of the decision, but additional intervention may be required to have an impact on screening behaviour. This observation was supported by the null effect of the gist leaflet on screening behaviour in workstream 3. One alternative approach suggested from data in workstream 1 is patient navigation, a method involving a trained health professional offering one-to-one support to help address individual barriers to screening. A patient navigation trial has been carried out by members of our team to promote uptake of FS in the UK screening programme. 237 Effective interventions should also emphasise the social implications of screening and its benefit not just to the individual but to their family and friends.
• Our exploration of why non-participants subsequently undergo screening (see Chapter 3 ) found that talking increased awareness of screening uptake among peers and significant others and was key to overcoming objections and to subsequent screening participation. Our results suggest that initiatives such as the advertising campaign in Greater Manchester to normalise open discussions about bowel cancer screening should be designed and evaluated (Audrey Howarth Greater Manchester Bowel Cancer Screening Health Improvement Team, 2012, personal communication).
• Reasons for low CRC screening uptake shared across South Asian faith groups were limitations posed by the written word, low awareness of CRC and screening, and difficulties with handling faeces and gFOBt completion. In addition, we identified that written materials are particularly inappropriate for the Sylheti-speaking Bangladeshi Muslim community, because they are unlikely to be able to read either Bengali or English. Sikh participants described a social stigma surrounding cancer which may hinder engagement with screening. A preference for information to be delivered verbally within a familiar community setting was described across faith groups. To increase accessibility to CRC screening in South Asian communities, screening information should be delivered via local ethnic media and face to face within community groups and faith settings.
• The gist and narrative leaflets were designed to make the screening offer more visible to people with lower literacy skills on the basis of extensive evidence that social disadvantage is associated with lower literacy. The gist leaflet improved comprehension compared with standard information alone (see Chapter 5 ) and the narrative leaflet led to stronger screening intentions than the standard information (see Chapter 6 ). However, there was no significant increase in uptake when examined in combination with the nationally approved standard materials. This may be because the determinants of action differ from the determinants of intention, and we influenced intentions rather than actions. In addition, although the new leaflets were designed to provide simple information, they nonetheless increased the total mass of written material that participants received and this may have undermined the goal of making the screening offer more visible. The leaflets supplemented rather than replaced the existing NHS BCSP information booklet because this was a requirement of the NHS National Cancer Screening Programme. One way forward could be to allow RCT testing of the leaflets in comparison to existing materials. An alternative approach could be to supply the necessary screening information in smaller ‘bites’ by integrating additional ‘communication points’ into the screening pathway.
• We demonstrated that it is possible to successfully engage GPs in applied health research. Nationally, 80% of general practices agreed to participate in the RCT of GPE in the NHS BCSP (see Chapter 7 ). This exceptionally high response rate was achieved by adopting the following practices, all of which have implications for future research requiring GP collaboration: recruitment letters were sent by NHS BCSP hub directors rather than from the ASCEND research team, GPs had to do no more than agree that their practice name could be used on the screening invitation letters, and we sent up to three reminder letters at approximately monthly intervals.
• Our research programme ASCEND was a highly successful example of experts from academia and from the health service working effectively across traditional boundaries to create new knowledge to improve the delivery of health services. We also demonstrated that, although studies of health care delivery typically rely on observational and quasi-experimental methods, it is possible to use a randomised design without adding substantially to the cost or difficulty of the study. We have also provided an example of applying a randomised design without loss of external validity, thereby offering an unparalleled ability to provide credible evidence of the impact of ER and GPE and potentially accelerating the pace for their widespread implementation.
• In the future, further development of the randomised design should be considered, involving ‘realist’ RCTs. 238 These could first explore decision-making processes leading to change and then test how intervention effects vary with context and among different sociocultural groups. Such RCTs could be orientated towards building and validating theories about how interventions interact with context to produce outcomes. They would comprise an important extension of existing pragmatic methods and enhance our understanding of sociocultural variations in screening uptake.

Research recommendations are summarised in Box 2 . They are equally applicable to gFOBt and to FIT .

Recommendations for future research to reduce the social gradient and increase overall uptake of bowel cancer screening

Included under terms of UK Non-commercial Government License .

• Cite this Page Raine R, Atkin W, von Wagner C, et al. Testing innovative strategies to reduce the social gradient in the uptake of bowel cancer screening: a programme of four qualitatively enhanced randomised controlled trials. Southampton (UK): NIHR Journals Library; 2017 Mar. (Programme Grants for Applied Research, No. 5.8.) Chapter 12, Conclusions and recommendations for future research.
• PDF version of this title (125M)
• Disable Glossary Links

Other titles in this collection

• Programme Grants for Applied Research

Recent Activity

• Conclusions and recommendations for future research - Testing innovative strateg... Conclusions and recommendations for future research - Testing innovative strategies to reduce the social gradient in the uptake of bowel cancer screening: a programme of four qualitatively enhanced randomised controlled trials

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Implications or Recommendations in Research: What's the Difference?

• Peer Review

High-quality research articles that get many citations contain both implications and recommendations. Implications are the impact your research makes, whereas recommendations are specific actions that can then be taken based on your findings, such as for more research or for policymaking.

Updated on August 23, 2022

That seems clear enough, but the two are commonly confused.

This confusion is especially true if you come from a so-called high-context culture in which information is often implied based on the situation, as in many Asian cultures. High-context cultures are different from low-context cultures where information is more direct and explicit (as in North America and many European cultures).

Let's set these two straight in a low-context way; i.e., we'll be specific and direct! This is the best way to be in English academic writing because you're writing for the world.

Implications and recommendations in a research article

The standard format of STEM research articles is what's called IMRaD:

• Introduction
• Discussion/conclusions

Some journals call for a separate conclusions section, while others have the conclusions as the last part of the discussion. You'll write these four (or five) sections in the same sequence, though, no matter the journal.

The discussion section is typically where you restate your results and how well they confirmed your hypotheses. Give readers the answer to the questions for which they're looking to you for an answer.

At this point, many researchers assume their paper is finished. After all, aren't the results the most important part? As you might have guessed, no, you're not quite done yet.

The discussion/conclusions section is where to say what happened and what should now happen

The discussion/conclusions section of every good scientific article should contain the implications and recommendations.

The implications, first of all, are the impact your results have on your specific field. A high-impact, highly cited article will also broaden the scope here and provide implications to other fields. This is what makes research cross-disciplinary.

Recommendations, however, are suggestions to improve your field based on your results.

These two aspects help the reader understand your broader content: How and why your work is important to the world. They also tell the reader what can be changed in the future based on your results.

These aspects are what editors are looking for when selecting papers for peer review.

Implications and recommendations are, thus, written at the end of the discussion section, and before the concluding paragraph. They help to “wrap up” your paper. Once your reader understands what you found, the next logical step is what those results mean and what should come next.

Then they can take the baton, in the form of your work, and run with it. That gets you cited and extends your impact!

The order of implications and recommendations also matters. Both are written after you've summarized your main findings in the discussion section. Then, those results are interpreted based on ongoing work in the field. After this, the implications are stated, followed by the recommendations.

Writing an academic research paper is a bit like running a race. Finish strong, with your most important conclusion (recommendation) at the end. Leave readers with an understanding of your work's importance. Avoid generic, obvious phrases like "more research is needed to fully address this issue." Be specific.

The main differences between implications and recommendations (table)

Now let's dig a bit deeper into actually how to write these parts.

What are implications?

Research implications tell us how and why your results are important for the field at large. They help answer the question of “what does it mean?” Implications tell us how your work contributes to your field and what it adds to it. They're used when you want to tell your peers why your research is important for ongoing theory, practice, policymaking, and for future research.

Crucially, your implications must be evidence-based. This means they must be derived from the results in the paper.

Implications are written after you've summarized your main findings in the discussion section. They come before the recommendations and before the concluding paragraph. There is no specific section dedicated to implications. They must be integrated into your discussion so that the reader understands why the results are meaningful and what they add to the field.

A good strategy is to separate your implications into types. Implications can be social, political, technological, related to policies, or others, depending on your topic. The most frequently used types are theoretical and practical. Theoretical implications relate to how your findings connect to other theories or ideas in your field, while practical implications are related to what we can do with the results.

Key features of implications

• State the impact your research makes
• Helps us understand why your results are important
• Must be evidence-based
• Written in the discussion, before recommendations
• Can be theoretical, practical, or other (social, political, etc.)

Examples of implications

Let's take a look at some examples of research results below with their implications.

The result : one study found that learning items over time improves memory more than cramming material in a bunch of information at once .

The implications : This result suggests memory is better when studying is spread out over time, which could be due to memory consolidation processes.

The result : an intervention study found that mindfulness helps improve mental health if you have anxiety.

The implications : This result has implications for the role of executive functions on anxiety.

The result : a study found that musical learning helps language learning in children .

The implications : these findings suggest that language and music may work together to aid development.

What are recommendations?

As noted above, explaining how your results contribute to the real world is an important part of a successful article.

Likewise, stating how your findings can be used to improve something in future research is equally important. This brings us to the recommendations.

Research recommendations are suggestions and solutions you give for certain situations based on your results. Once the reader understands what your results mean with the implications, the next question they need to know is "what's next?"

Recommendations are calls to action on ways certain things in the field can be improved in the future based on your results. Recommendations are used when you want to convey that something different should be done based on what your analyses revealed.

Similar to implications, recommendations are also evidence-based. This means that your recommendations to the field must be drawn directly from your results.

The goal of the recommendations is to make clear, specific, and realistic suggestions to future researchers before they conduct a similar experiment. No matter what area your research is in, there will always be further research to do. Try to think about what would be helpful for other researchers to know before starting their work.

Recommendations are also written in the discussion section. They come after the implications and before the concluding paragraphs. Similar to the implications, there is usually no specific section dedicated to the recommendations. However, depending on how many solutions you want to suggest to the field, they may be written as a subsection.

Key features of recommendations

• Statements about what can be done differently in the field based on your findings
• Must be realistic and specific
• Written in the discussion, after implications and before conclusions
• Related to both your field and, preferably, a wider context to the research

Examples of recommendations

Here are some research results and their recommendations.

A meta-analysis found that actively recalling material from your memory is better than simply re-reading it .

• The recommendation: Based on these findings, teachers and other educators should encourage students to practice active recall strategies.

A medical intervention found that daily exercise helps prevent cardiovascular disease .

• The recommendation: Based on these results, physicians are recommended to encourage patients to exercise and walk regularly. Also recommended is to encourage more walking through public health offices in communities.

A study found that many research articles do not contain the sample sizes needed to statistically confirm their findings .

The recommendation: To improve the current state of the field, researchers should consider doing power analysis based on their experiment's design.

What else is important about implications and recommendations?

When writing recommendations and implications, be careful not to overstate the impact of your results. It can be tempting for researchers to inflate the importance of their findings and make grandiose statements about what their work means.

Remember that implications and recommendations must be coming directly from your results. Therefore, they must be straightforward, realistic, and plausible.

Another good thing to remember is to make sure the implications and recommendations are stated clearly and separately. Do not attach them to the endings of other paragraphs just to add them in. Use similar example phrases as those listed in the table when starting your sentences to clearly indicate when it's an implication and when it's a recommendation.

When your peers, or brand-new readers, read your paper, they shouldn't have to hunt through your discussion to find the implications and recommendations. They should be clear, visible, and understandable on their own.

That'll get you cited more, and you'll make a greater contribution to your area of science while extending the life and impact of your work.

The AJE Team

See our "Privacy Policy"

Improving the Nation's Water Security: Opportunities for Research (2007)

Chapter: 6 recommendations for future research directions, 6 recommendations for future research directions.

Progress has been made in the Environmental Protection Agency’s (EPA’s) water security research program (see Chapter 4 ), but many important research questions and technical support needs remain. In Chapter 3 , a framework is suggested for evaluating water security research initiatives that gives priority to research that improves response and recovery and/or develops risk reduction or consequence mitigation measures. The research should also produce tools with a reasonable likelihood of implementation and, where feasible, dual-use benefits. Based on this framework and the review of water security efforts already under way, two important water security research gaps are identified and discussed briefly in this chapter. In addition, short- and long-term water security research recommendations are made. The research recommendations are organized in this chapter according to the three long-term program objectives proposed in Chapter 5 emphasizing pre-incident, incident, and post-incident applications: (1) develop products to support more resilient design and operation of facilities and systems, (2) improve the ability of operators and responders to detect and assess incidents, and (3) improve response and recovery. Both drinking water and wastewater research priorities are addressed together within these three objectives to maximize research synergies that may exist.

KEY RESEARCH GAPS

The Water Security Research and Technical Support Action Plan (EPA, 2004a) set out a comprehensive guide for the EPA’s near-term research initiatives. Although the Action Plan was intended to provide a short-term (three- to four- year) research agenda, the previous National Research Council review (NRC, 2004) noted that several of the Action Plan projects represented long-term research questions not easily ad-

dressed in the original time frame. Therefore, the Action Plan provides a reasonable starting point for building the EPA’s future research program. Nevertheless, the short-term planning horizon of the Action Plan prevented consideration of two key subjects that are critical to a long-term water security research program: behavioral science and innovative system design. The committee recommends the EPA work in collaboration with other organizations to build research initiatives in these two areas.

Behavioral Science

The threat of bioterrorism presents new and different types of risks that are dynamic and pose difficult trade-offs, bringing about intellectual challenges and an emotional valence possibly more important than the risks themselves. Developing an effective communication strategy that meets the needs of the broad range of stakeholders (e.g., response organizations, water organizations and utilities, public health agencies, the public, the media) while addressing security concerns is clearly a high-priority research area. The EPA’s work on risk communication is focused primarily on the development of guidance, protocols, and training, and little emphasis has been devoted to interdisciplinary behavioral science research to better prepare stakeholders for water security incidents or to build confidence in their ability to respond. Behavioral science research could help address, for example, what the public’s beliefs, opinions, and knowledge about water security risks are; how risk perception and other psychological factors affect responses to water-related events; and how to communicate these risks with the public (Gray and Ropeik, 2002; Means, 2002; Roberson and Morely, 2005b). A better understanding of what short-term disruptions customers are prepared to tolerate may also guide response and recovery planning and the development of recovery technologies.

Previous experience with natural disasters and environmental risks provides a basis for investigating and predicting human behavior in risky situations (Fischoff, 2005). Existing models of human behavior during other kinds of crises, however, may not be adequate to forecast human behavior during bioterrorism or water security incidents (DiGiovanni et al., 2003).

Risk communicators consider empirical findings from psychology, cognitive science, communications, and other behavioral and social sciences to varying extents (Bostrom and Lofstedt, 2003). Although decision makers frequently predict panic and irrational behavior in times of

crisis, behavioral science researchers have found that people respond reasonably to such challenges (e.g., Fishoff, 2005). Given the urgency of terror risk communication, risk communicators are obliged to incorporate existing behavioral science research as it relates to water security risks.

The EPA should take advantage of existing behavioral science research that may be applicable to water security issues, but this requires knowledge and experience in behavioral science research. Where gaps exist, the EPA will need to engage in interdisciplinary, rigorous empirical research to obtain the necessary knowledge.

Innovative Designs for Secure and Resilient Water and Wastewater Systems

Innovative designs for water and wastewater infrastructure were not addressed in the EPA Action Plan, but the topic deserves a place in a long-term water security research program. The EPA’s research mission has traditionally included the development and testing of new concepts, technologies, and management structures for water and wastewater utilities to achieve practical objectives in public health, sustainability and cost-effectiveness. The addition of homeland security to its mission provides a unique opportunity to take a holistic view of current design and management of water and wastewater infrastructures. Innovation is needed to address the problem of aging infrastructures while making new water systems more resilient to natural hazards and malicious incidents. The EPA should, therefore, take a leadership role in providing guidance for the planning, design, and implementation of new, more sustainable and resilient water and wastewater facilities for the 21st century.

Disagreggation of large water and wastewater systems should be an overarching theme of innovation. Large and complex systems have developed in the United States following the pattern of urban and suburban sprawl. While there are clear economies of scale for large utilities in construction and system management, there are distinct disadvantages as well. The complexity of large systems makes security measures difficult to implement and complicates the response to an attack. For example, locating the source of incursion within the distribution system and isolating contaminated sections are more difficult in large and complex water systems. Long water residence times are also more likely to occur in large drinking water systems, and, as a result, disinfectant residual may be lacking in the extremities of the system because of the chemical and biological reactions that occur during transport. From a security perspec-

tive, inadequate disinfectant residual means less protection against intentional contamination by a microbial agent.

A breadth of possibilities exists for improving security through innovative infrastructure design. Satellite water treatment plants could boost water quality. Strategic placement of treatment devices (e.g., ultraviolet lamp arrays) within the distribution system could counter a bioterrorism attack. Wastewater treatment systems could be interconnected to provide more flexibility in case of attack, and diversion devices could be installed to isolate contaminants. Box 6-1 describes some of these concepts in greater detail, and specific research recommendations are suggested in the following section.

RESEARCH RECOMMENDATIONS: DEVELOP PRODUCTS TO SUPPORT MORE RESILIENT DESIGN AND OPERATION OF FACILITIES AND SYSTEMS

Specific research topics are suggested here in two areas to support development of more resilient water and wastewater systems: (1) innovative designs for water and wastewater and (2) improved methods for risk assessment, including processes for threat and consequence assessments.

Innovative Designs for Water and Wastewater Systems

Innovative changes to water infrastructure will require long-term investment in research. Existing systems have been in place for more than a century in older cities. Thus, bold new directions will understandably require intensive research at the outset to produce a defensible economic argument for change. On the other hand, the EPA has the opportunity to develop innovative approaches that can be implemented almost immediately in relatively new, as well as planned, urban and suburban areas. The first step in research would be to enumerate the opportunities for innovation, recognizing the constraints brought about by the size, age, and complexity of existing water and wastewater infrastructures. A broad-gauge, economic analysis should follow that would quantify the costs and multiple benefits of these innovative designs (e.g., increased security, improved drinking water quality, enhanced sustainability of water resources). In addition, there is an implicit need for EPA research-

ers to coordinate with the agency’s regulatory branch to validate the feasibility of the innovative concepts that are proposed.

Each of the infrastructure concepts illustrated in Box 6-1 require far more research to become feasible. The recommendations below outline specific research topics that, if addressed, could improve the safety and sustainability of water resources in the 21st century.

Disaggregation of Water and Wastewater Systems

The “distributed optimal technology network” (DOT-Net) concept (Norton and Weber, 2005; Weber, 2002; 2004) hinges upon the feasibility of distributed treatment via point-of-use (POU)/point-of-entry (POE) devices installed at the scale of individual buildings or perhaps small neighborhoods. The corollary premise is that installation of expensive advanced treatment technology at the centralized water treatment facility is unnecessary when only a fraction of the service area outside a “critical radius” requires additional protection. Only a broad economic analysis of this concept has been published thus far for a hypothetical urban center, but the assumptions need to be verified for actual systems, particularly because of the unique characteristics of individual cities. In addition, far more research is needed on the utility management required to ensure the reliability of POU/POE devices in widespread implementation.

Dual water systems have also been proposed to address aging infrastructure (see Box 6-1 ; Okun, 1997; 2005). As with the DOT-Net concept, long-term research is needed to determine the costs and benefits of constructing an entirely new paradigm for distribution system design. Research issues would include assessing the acceptability of reclaimed water for progressively more intense levels of nonpotable use (e.g., irrigation, toilet flushing, laundering), the acceptability and management demands of decentralized wastewater treatment facilities, and the net benefits to water security.

In-Pipe Interventions to Reduce Exposure

In-pipe engineering interventions (see Box 6-1 ) are deserving of research in a long-term water security research strategy. For example, research is needed to optimize the location of disinfection booster stations or to examine the effectiveness and feasibility of in situ ultraviolet (UV)

irradiation systems as a decontamination strategy. EPA research could also examine various pipe materials (e.g., stainless steel) and evaluate their benefits for security and sustainability relative to their costs.

Infrastructure Designs to Enable Isolation and Interconnection

Most large drinking water systems have the ability to isolate portions of their distribution systems during necessary system repairs, but security concerns provide a new impetus for rapid and effective isolation mechanisms. Research on innovative mechanisms to isolate or divert contaminated water in drinking water and wastewater systems would be useful. The EPA should identify these design options, research their costs and benefits (including dual-use benefits) and their feasibility both for existing systems and new infrastructure, and make this information available to system managers.

Improved Risk Assessments Procedures

A sound risk assessment process allows utilities to make better resource management decisions for enhancing their recovery capacity or security strategies to mitigate the consequences of an attack. The risk assessment process includes assessments of threat, consequences, and vulnerability. To date, most of the efforts to guide utilities in their own risk assessments have focused on vulnerabilities.

Threat Assessment

Water and wastewater utilities today are making resource management decisions related to security without adequate information about the nature and likelihood of threats to their systems. As discussed in Chapter 4 , the EPA has focused their efforts on identifying contaminant threats without conducting similarly detailed analyses of possible physical and cyber threats. Both the nature and likelihood of these threats are needed for efficient allocation of resources at the utility level and within the EPA’s research program. Improved threat assessment would require the EPA and/or a consortium of water experts to work closely with the intelligence community and local law enforcement agencies. Other national and federal laboratory expertise within the Department of Energy,

Department of Defense, and private-public community might be needed as well. Threat assessments for water and wastewater should be periodically reviewed to identify threat scenarios that should be added to the list and to remove those that are no longer a concern. The development of a threat assessment process for local water and wastewater utilities with current techniques used in other infrastructures would also be helpful, provided the threat information could be communicated to those who need it (ASME, 2004; Sandia National Laboratories, 2001).

Consequence Assessment

A consequence assessment should accompany the threat assessment within the risk assessment process. Consequence assessments would provide decision makers with information on the potential for fatalities, public health impacts, economic impacts, property damage, systems disruption, effects on other infrastructures, and loss of public confidence. Procedures for determining the expected consequences from an attack or natural disaster are not currently being systematically developed. As a result, water system managers do not have sufficient data to make decisions about the benefits of risk reduction relative to the costs. The development and application of a consequence assessment procedure would provide decision makers with information needed to decide whether to mitigate the consequences, upgrade with countermeasures, take steps to improve response and recovery capacity, and/or decide to accept the level of risk and take no further action. A fault tree analysis that includes, for example, options for redundant systems or contingency water supplies could provide vital information on whether to invest in security upgrades or less costly consequence mitigation strategies . Many of these approaches have already been developed for other infrastructures (e.g., Risk Assessment Methodology [RAM]-T for the high-voltage power transmission industry or RAM-D for dams, locks, and levees; see Sandia National Laboratories, 2001; 2002). A thorough review of other RAM methodologies could provide guidance for consequence assessment strategies that could be incorporated into the Risk Assessment Methodology for Water Utilities (RAM-W).

The EPA has worked to develop the AT Planner tool to assist utilities in assessing the consequences from physical attacks (see Chapter 4 ). While AT Planner has been validated against actual blast test data for nonwater systems, there remains significant uncertainty in the applicability of the modeling for water security because it has not been validated

against the structures specific to those systems. Therefore, the ongoing evaluation of AT Planner by the EPA and select water utility operators should include an assessment of the applicability of AT Planner for each of the critical and high-consequence components of a water system. The EPA and water utilities should then consider whether any additional validation testing is needed to determine specific failure modes of relevant water system components (e.g., actual storage tanks, pumps, water conduits, chlorine tanks) and possible countermeasures.

Summary of Research Priorities for Secure and Resilient Systems

Short-term priorities.

Develop an improved understanding of physical, cyber, and contaminant threats to water and wastewater systems, especially focusing on physical and cyber threats.

Communicate information on threats and consequences to water system managers through training and information exchange.

Develop an improved threat assessment procedure for water and wastewater utilities that will assist local utilities with their security and response planning.

Develop a process to assist local utilities in determining the consequences from physical, cyber, and contaminant attacks.

Update the risk assessment methodology for water systems to incorporate the latest approaches used in other industries, including developing credible threat descriptions and identifying cascading consequences.

Long-Term Priorities

Develop innovative design strategies for drinking water and wastewater systems that mitigate security risks and identify their costs and benefits in the context of public health, sustainability, cost-effectiveness, and homeland security. These designs might include:

In-pipe intervention strategies for drinking water systems,

Disaggregation of water and wastewater treatment facilities to achieve dual-use benefits, and

Designs that allow for interconnections and isolation.

Evaluate the need to validate AT Planner against structures specific to water systems.

Periodically review the EPA’s prioritized list of threats, contaminants, and threat scenarios to identify items that should be added to the list and remove items that are no longer a concern.

Continue development of technology transfer/training programs so that utilities understand the value of the EPA’s products for both homeland security incidents and natural disasters and know how to utilize the tools to their full extent.

Implementation of Priorities

Some of the research recommendations to support more resilient design and operation of drinking water and wastewater systems lie outside of the EPA’s traditional areas of expertise. To support the Action Plan efforts so far, the EPA has relied heavily on expert contractors to conduct this type of work. The EPA should continue to seek the relevant expertise of other federal agencies and national laboratories in these future efforts. However, the EPA will need to consider how best to balance intramural and extramural research funding to carry out this research, while maintaining appropriate oversight and input into the research activities (see also Chapter 5 ). Increasing staff expertise in some key areas, such as physical security, will be necessary to build a strong and well-rounded water security research program to support more resilient system design and operation.

RESEARCH RECOMMENDATIONS: IMPROVE THE ABILITY OF OPERATORS AND RESPONDERS TO DETECT AND ASSESS INCIDENTS

Suggestions are provided in this section for future research that should improve the ability of operators and responders to detect and assess water security incidents. Specific research suggestions in the areas of analytical methodologies and monitoring and distribution system modeling are discussed below.

Analytical Methodologies and Monitoring

Expanding existing analytical methods.

For some analytes of relevance to water security concerns, the available or approved detection methods are poor (e.g., some nonregulated analytes). More work needs to be done to expand existing methods to a broader range of analytes. For example, method 300.1 (EPA, 2000) covers only the common anions but could be extended to others, including toxic substances. The extension of existing methods to new analytes would allow a broader range of laboratories to expand their capabilities into the water security area.

Screening methods using conventional gas chromatography (GC) or high-performance liquid chromatography (HPLC) should also be investigated. Modern high-resolution chromatography combined with high-sensitivity detection (e.g., electron capture, fluorescence) is a powerful, yet accessible tool. Protocols should be developed to make the best use of these widely available capabilities. Software will have to be developed to facilitate the documentation of normal, background signals (fingerprint-type chromatograms). This background information can then be used to detect anomalies. Final protocols would have to be tested thoroughly against priority chemical contaminants. Chromatographic finger-prints have been used to monitor water supplies for nonintentional contamination, so this line of research would provide a dual benefit (D. Metz, Ohio River, personal communication, 2006; P. Schulhof, Seine River, personal communication, 2006).

Progress is being made with the protocol to concentrate samples and identify biological contaminants by polymerase chain reaction (PCR) analysis. Continued research, however, needs to be directed towards reducing the time and effort required to collect, process, and identify samples by automating portions of the protocol such as the concentration step. Such automated collection and sample processing systems would be especially valuable in response to security threats, when water samples could be channeled to existing or new detection technologies capable of onsite processing. The EPA should continue to expand the number of biothreat agents tested with the concentration/PCR protocol to include microbes other than spores, prioritizing test organisms that are both a threat to public health and resistant to chlorine (Morales-Morales, et al., 2003; Straub and Chandler, 2003). Continued testing of the concentration/PCR protocol should include various mixed suspensions of a target

microbe and background microbes to determine specificity of detection and various dilutions of the target microbe to determine sensitivity of detection. The protocol should also be tested on chloraminated water samples.

Developing New Monitoring Technologies

Chemical Detection. New chemical monitoring technologies for security-relevant analytes should be investigated. Examples include quartz crystal microbalance (QCM) sensors, microfluidic devices (lab-on-a-chip), ion-sensitive field-effect transistors (ISFETs), and larger-scale optrodes. Extramural agency and corporate partnerships developed by the EPA and longer-term research projects will help the evaluation and consideration of a broader range of detection platforms.

Biological Detection. Biological monitoring devices are essential to assess the type and extent of contamination in a suspected water security event. A broader range of innovative and developing detection technologies for biological agents, including methods that are field deployable and reagent-free, should be considered and evaluated. Innovative, field-deployable detection technologies (e.g., genetic fingerprinting, immunodetection, other technologies in development by universities, the Department of Defense, and industry) could reduce the time and effort for detection and enable earlier response efforts (Iqbal et al., 2000; Ivnitski et al., 2003; Lim et al., 2005; Monk and Walt, 2004; Yu and Bruno, 1996; Zhu et al., 2004). These new technologies might also increase the accuracy of detecting deliberate contamination events and reduce false alarms. Methods that can detect multiple biological agents and those with dual-use benefits should be emphasized over those methods limited to very specific agents (Peruski and Peruski, 2003; Rogers and Mulchandani, 1998). For example, DNA fingerprinting might be more useful than immunodetection systems dependent on a highly specific antibody for operation. The accuracy of these detection methods will depend on availability of quality reagents such as antibodies and primers; therefore, researchers will need to work closely with the Centers for Disease Control and Prevention (CDC) and other agencies that have access to such reagents.

Monitoring Devices for Wastewater Collection Systems . Contamination incidents have the potential to disrupt wastewater biological treat-

ment systems; thus, a long-term research program should also include research on monitoring technologies relevant to wastewater security concerns. Although a number of devices are available that can be used to monitor physical, chemical, and biological parameters, none of the currently available devices are robust or reliable enough when used in untreated wastewater to meet security requirements. The EPA should, therefore, encourage development of robust or reliable monitoring devices for wastewater infrastructure.

Syndromic Surveillance Tools. Syndromic surveillance tools may have the potential for detecting disease outbreaks and for investigating the possible role of water in such outbreaks (Berger et al., 2006). The EPA is already working to test two syndromic disease surveillance tools (RODS, ESSENCE) against prior water contamination outbreak data. There are substantive research needs that should be undertaken, however. Clearly, the improvement of existing syndromic surveillance tools is a long-term research objective. For syndromic surveillance to become worthwhile, it should achieve a favorable cost-benefit ratio considering the costs of false positives, and syndromic surveillance should also be adequately integrated into response plans. The implementation of syndromic surveillance systems on a large scale would require a more detailed linkage between disparate databases used in the public health sector and the water supply sector. Research to develop tools to allow local systems to readily fuse information from these disparate sources would be desirable. Such linkages would improve detection and response to waterborne disease outbreaks and more rapidly exclude water as a possible vehicle of disease. This would have important applications for both intentional and nonintentional water contamination events.

Real-Time Monitoring Systems

The development of a fully functional, easy-to-maintain, real-time monitoring system (RTMS) that could someday be used to prevent harm from deliberate attacks on the water system (“detect to prevent”), even with substantial research investments, is many years away. Therefore, the primary emphasis of future research on RTMSs, at least in the near term, should be on developing these technologies to assess the spread of contaminants, not to prevent exposure.

The committee also questions the likelihood of implementation of real-time monitoring devices for specific chemical or biological parame-

ters that are not useful in the day-to-day operation of a system (see Chapters 2 and 4 ). However, there are a few scenarios where implementation of continuous monitors for biological contaminants might be valuable, such as their use in certain water systems under heightened threat conditions (e.g., utilities for which specific intelligence information indicates they may be targeted). As discussed in Chapter 4 , deployment under these circumstances has a greater likelihood for success because the probability of an event is estimated to be much higher and the length of monitoring time is shortened. The use of highly sensitive and specific detection devices under such targeted circumstances would significantly lower the probability of false alarms and reduce the problem of poor positive predictive value (see Chapter 2 ) while also minimizing implementation and maintenance costs. Thus, improving monitoring systems for specific chemical or biological agents in drinking water is a valid long-term research goal. The EPA may find that longer-term research on more speculative sensor development could benefit from a further broadening of the circle of collaborators. Such speculative research may be more appropriately funded through the National Science Foundation or the Homeland Security Advanced Research Projects Agency, thus freeing up EPA resources for other purposes. To encourage such research, the EPA may wish to build its connections with the private sector on this technology.

Research on detection methods for RTMSs should proceed with careful consideration of the likelihood of implementation of the monitoring devices. In its near-term research plans, the EPA should adopt a first-stage approach to RTMSs, emphasizing generic sensors to detect intrusion or a system anomaly. The intrusion detection would then trigger more resource-intensive follow-up monitoring and analysis. Such an approach has significant dual-use benefits for routine contamination events that could outweigh the costs of implementing and operating these systems. Additional effort to develop cheaper, more accurate, and more easily deployable and maintainable sensors for routine water quality parameters would be useful both for anomaly detection and routine operation. Additional research is also needed, even in first-stage RTMSs, to understand normal water quality variations and distinguish variations that might be caused by a deliberate contamination attack. For example, continuous monitoring of chlorine residual at multiple points in the distribution system often reveals wide variations at different temporal scales due to changes in water demand that affect water residence time (e.g., operation of storage tanks). Although some work to understand inherent water quality variability in distribution systems is being conducted through the

Water Sentinel program, a significant amount of work is needed to translate the findings of this research into criteria for RTMSs to develop systems that have a reasonable likelihood of implementation.

An important component of RTMS research should include data fusion, whereby multiple anomalies must occur before an alarm signal is sent (see also Chapter 4 ). The private sector seems to be taking the lead on many types of multiparameter approaches to RTMSs and the processing of data, especially as described by contaminant or event signatures. It is important that the algorithms are open to peer review and can be accessed by all for development of new and refined approaches.

RTMS sensor research should consider a broader range of technologies, including full-spectrum UV and visible absorption, fluorescence excitation emission matrices, and ionization sensors (Alupoaei et al., 2004; Fenselau and Demirev, 2001; Lay, 2001). Many of these techniques are used as nonspecific chromatography detectors, and as such, they are highly sensitive. Most prototype RTMSs are composed of existing sensors that are designed to measure a specific contaminant, and some technologies have been excluded because they have not led to sensors with a high degree of selectivity. However, RTMSs need not be contaminant-specific; they only need to detect anomalies. Detection of an anomaly can then be followed by more specific contaminant analyses.

The problem of false positive signals from real-time contaminant-specific warning systems has been discussed in Chapter 2 . In essence, the problem is one of unfavorable arithmetic when the probability of a true positive is very small, as it would be for an intentional contamination attack on any particular water system of the tens of thousands of such systems. Therefore, most contaminant-specific alarm signals will be false positives. The EPA should consider the consequences of various rates of false positive signals for both large and small utilities and collect information on how alarms are currently handled by utilities. Workshops and structured surveys on this issue would provide valuable information on current practices, the extent to which positive signals are confirmed, the costs of false alarms, and the views of utility operators on their tolerance for various levels and types of false alarms. This research would provide useful guidance for the developers of water quality monitoring devices, for utilities that are considering implementing devices that are commercially available, and for local and state regulatory agencies who will need assistance interpreting alarm signals in light of the public health consequences.

Technology Testing

The EPA has developed a rigorous technology testing program to provide security product guidance to end users focusing on monitoring and decontamination technology. However, as noted in Chapter 4 , the number of relevant security technologies and agents of interest exceed the capacity and budget of the Technology Testing and Evaluation Program (TTEP). Therefore, developing a test-prioritization plan for TTEP seems especially important and is strongly recommended. Although the process of identifying technologies of interest has begun through the use of stakeholder meetings and advisory boards, activities to date have been weighted toward doing the easiest things first, and only some of these tests provided dual-use benefits. Balancing the homeland security benefits and the benefits to routine water system operations in TTEP will likely require additional strategic planning. One strategy has been to test equipment that is commercially available regardless of whether it addresses a high-risk agent. Instead, the EPA should look beyond the easy-to-identify commercially available equipment and make a greater effort to identify technologies in development that have the potential to address those agents identified as posing the greatest risk to water, considering the likelihood of the threat (including the ease of acquiring particular chemical or biological agents), the potential consequences, and the likelihood of implementing the technology. For a few of the highest-priority threats, the EPA may wish to consider providing technical support and/or funding to encourage more rapid development of a particularly promising technology that has a high likelihood of implementation and significant dual-use benefits, similar to the EPA Superfund Innovative Technology Evaluation (SITE) Emerging Technology Program.

Develop Laboratory Capability and Capacity

Adequate laboratory capacity is critical for responding to a terrorist incident affecting water supplies, and although this is not a research issue, the EPA has much to contribute from an applied perspective. The need for mobile analysis units capable of supplementing local laboratories and rapidly responding to geographical areas impacted by terrorist events should be considered. Such mobile laboratories could also address analytical needs that arise during natural catastrophes, such as Hurricane Katrina. Many states have begun to develop mobile laboratory

capabilities as part of their water security activities, and the EPA could glean information on their experiences to date.

The EPA is working with utilities and state and federal agencies to build a national laboratory response network for water sample analysis (i.e., the Water Laboratory Alliance). Some university laboratories may have capabilities that could merit inclusion in the nationwide network. Other laboratories may be stimulated to conduct additional research on improved analytical methods for toxic and biothreat agents if they were better informed of the current state of knowledge and had access to reference standards (access to some reference standards is currently limited due to security concerns). To be successful, a dual-use philosophy should be adopted whenever possible in the development of laboratory capacity (e.g., employing methods/instruments that can also be used for standard analytes).

Distribution System Modeling Tools

Distribution system models provide valuable tools for locating the source of contamination or assessing the spread if the source is known, estimating exposure, identifying locations for sampling, and developing decontamination strategies (see also Chapter 4 ). Distribution system models also have important dual-use applications to routine water quality concerns, and the EPA should continue to emphasize the dual-use value of its modeling tools. Specific recommendations are provided below to advance the capabilities and implementation of the Threat Ensemble Vulnerability Assessment (TEVA) and EPANET models.

Experimental Verification of Species Interaction Subcomponent Models

The final goal of producing a more flexible EPANET model through Multi-Species EPANET (MS-EPANET) is commendable. However, the new subcomponents are based upon developing better fundamental knowledge of reactions within the distribution system involving chemistry (e.g., disinfection kinetics, chemical partitioning), biology (e.g., development of biofilms, release and attachment of microbes), and materials science (e.g., corrosion of pipe materials and its relationship to disinfection efficacy). The large number of system constants in both MS-EPANET and TEVA necessitate significant investment in sensitivity

analysis research to quantify the accuracy of model predictions. The development and testing of all new features of MS-EPANET should be a long-term research goal. Until the validity of these subcomponents is verified and system constants can be assigned with more certainty, the water industry will be reluctant to use the full capability of MS-EPANET. Limitations in the accuracy of model predictions will need to be addressed in guidance to decision makers. A significant commitment will be needed in resources for experimental verification.

Alternate Approaches to Uncertainty Modeling

The Action Plan acknowledges correctly that the distribution system model simulations should incorporate an analysis of uncertainty because the point of attack is unknown. This has led to the use of the well-known Monte Carlo analysis to randomize the location of the attack and run repeated distribution system model simulations (1,000 or more) to generate a probability distribution to relate point of attack to human exposure impact. The focus on short-term results, however, has produced weaknesses in the current EPA approach to uncertainty research.

A broader discussion about how to incorporate uncertainty into the TEVA model should be invited. Approaches such as fuzzy logic (McKone and Deshpande, 2005) and Bayesian Maximum Entropy modeling (Serre and Christakos, 1999) are showing promise but have been applied mainly to homogenous space rather than to network domains. The EPA should encourage alternative ideas for handling uncertainty. If the expertise is not available within the agency, there needs to be a mechanism to expand extramural support for research, particularly within the university community.

Technology Transfer and Training in Use of the TEVA and EPANET Models

Advances in the TEVA model add significant complexity to the EPANET model, which may limit its widespread implementation. The EPA should work to communicate the capabilities of EPANET, MS-EPANET, and TEVA to utilities, emphasizing their value for routine water quality concerns, advanced homeland security planning, and contamination assessment and response activities. Until TEVA and MS-EPANET are further developed and widely available, the EPA should

consider an interim strategy to better inform water utilities on the value and use of existing distribution system models, such as EPANET. Progressive water utilities are already using EPANET to examine possible locations of attack and to track the concentration of contaminants within the distribution system.

Training in the use of MS-EPANET and the proposed TEVA model is also needed. Water utility managers need to be convinced that the costs for adapting a new model for their respective distribution systems are worthwhile, because many utilities have already invested heavily in development, verification, and calibration of existing models. The complexity of the TEVA model may increase these costs further, because many more implementation steps follow those for EPANET to adapt the TEVA “template” to the specifics of each water utility.

Summary of Research Priorities for Better Equipping Operators to Detect and Assess Incidents

Automate the concentration step of the concentration/PCR protocol.

Continue to test the concentration/PCR protocol:

Expand the number of biothreat agents tested to four or five organisms that include microbes other than spores, focusing on microbes that are both a threat to public health and resistant to chlorine.

Test the concentration/PCR protocol with chloraminated water samples.

Test the concentration/PCR protocol to determine sensitivity and specificity of detection.

Field-test RTMSs to determine false positive/false negative rates and maintenance requirements and develop basic criteria for the technology that might lead to a reasonable likelihood of implementation.

Continue research to develop a first-stage RTMS based on routine water quality sensors with dual-use applications.

Analyze the consequences of false positive signals from realtime monitoring systems, emphasizing current practices, the extent to which positive signals are confirmed, the costs of false alarms, and the tolerance of utility operators for false alarms.

Test standard chromatographic methods for their ability to screen for a broad range of toxic agents in routine laboratory testing.

Develop a test-prioritization strategy for TTEP to optimize the resources devoted to this effort.

Invite external peer review of the TEVA model before investing in field testing.

Long-term Priorities

Continue to develop portable, field-deployable systems that can be used to collect and process samples at event locations.

Formulate protocols and develop software for using GC- and HPLC-based fingerprinting to detect suspicious anomalies.

Stimulate research and ultimately development of new sensors for water security analytes based on innovative technologies, such as QCM, ISFETS, and microfluidics.

Evaluate and develop new field-deployable detection technologies for biological agents, including genetic fingerprinting, immunodetection, and reagentless technologies, that have the necessary sensitivity, specificity, and multiplex capabilities.

Develop improved, cheaper, and accurate RTMSs for routine water quality measurements.

Examine the use of nonspecific detection technologies for RTMSs.

Develop data fusion approaches for RTMSs that can minimize false positives.

Develop and test new monitoring technologies suitable for wastewater security applications.

Improve syndromic surveillance tools and develop a health surveillance network with appropriate linkages to water quality monitoring.

Continue to develop and refine the efficiency of a system-wide laboratory response network, including the development of mobile analysis units.

Continue fundamental research to understand the chemical and biological reactions that affect the fate and transport of contaminants in distribution systems to verify the constants used in MS-EPANET and TEVA.

Include alternative approaches to uncertainty design (e.g., fuzzy logic, Bayesian Maximum Entropy) in the TEVA model that are based more strongly upon stochastic than deterministic principles given that many of the input parameters to the current TEVA model are highly uncertain.

Develop projects for training water utilities in the value and use of EPANET, MS-EPANET, and TEVA.

Some of these research priorities may be more appropriately accomplished by universities, companies, or other agencies that have the necessary expertise, resources, and funding to successfully complete these tasks. The development of multiplex detection protocols and portable, field-deployable platforms are examples of tasks that might be better managed by some group other than the EPA. Work to determine the sensitivity and specificity of designated protocols for different biothreat agents could be conducted by university laboratories or private industry, with collaborative input from the EPA, considering their understanding of the needs of the water sector. Utilization of research resources outside the EPA would expand the variety of emerging, innovative analytical technologies that might be used to support the EPA’s efforts in enhancing the nation’s water security.

RESEARCH RECOMMENDATIONS: IMPROVED RESPONSE AND RECOVERY

Recommendations are provided in this section for future research that should improve response and recovery after a water security incident. Research suggestions related to tools and data for emergency planning and response, contingencies, risk communication and behavioral sciences, decontamination, and lessons learned from natural disasters are presented below.

Tools and Data for Emergency Planning and Response

Continued development of emergency response databases.

The EPA released preliminary versions of the Water Contamination Information Tool (WCIT) and the Consequence Assessment Tool (CAT) to provide data on contaminant properties, toxicity, and exposure threats (see Chapter 4 ), but the databases are still in their infancy, and numerous data gaps exist. The EPA will need to prioritize its continued efforts to further develop these response databases. Therefore, the EPA should develop strategic plans for WCIT and CAT, outlining the long-term goals for the databases and addressing questions such as:

What stakeholders will be served by the databases?

What categories of information do these stakeholders need?

How many contaminants should be included?

What linkages to other databases should be established?

The EPA will need to determine criteria for prioritizing what contaminants are added to the database and how to maintain and update the information. If WCIT and CAT are not continually revised to incorporate the latest scientific knowledge, the databases will become outdated. Expanding or even maintaining a database requires considerable resources, both intellectual and financial. If a commitment is not made initially for the necessary resources to update and maintain a database, spending the resources to create it becomes debatable. The EPA is currently facing similar issues maintaining its Integrated Risk Information System (IRIS) database.

The EPA should also clearly define the data quality objectives for WCIT/CAT and incorporate peer review of the data, as necessary, to meet these objectives. For example, the EPA may decide that some information about a contaminant is better than none, even if that information has limitations. This is a legitimate approach; however, the EPA should provide a mechanism that helps to ensure that individuals using the databases understand the data quality and their limitations. One mechanism for accomplishing this would be to add quality notations for each datum. Regardless of the approach taken, the EPA needs to describe the extent to which the data have been reviewed.

Evaluation and Improvement of Tools and Databases

With the forthcoming completion of at least the first stages of many tools and databases (e.g., WCIT, CAT), the EPA should consider the evaluation/improvement cycle. This will require the development of procedures to evaluate the utility and usability of these tools by potential constituencies. In addition, the EPA should take advantage of the tests afforded in response to “real-life” incidents. For example, some of the tools and databases were used (albeit in an early stage of their development) in the response to Hurricane Katrina. A formal assessment of knowledge gained from this experience could assist in the improvement and development of the tools.

Filling Data Gaps

The state of knowledge of the health risks from water contaminants that could be used in a malicious event is quite limited, as shown by the limited number of chemicals and even fewer biologicals in the WCIT/CAT databases and the many blank data fields in these databases. Important experimental and computational research is under way at the EPA to address some of these data gaps (see Chapter 4 , Section 3.6), but many gaps remain. There are two applications of toxicity/infectivity information that would be useful to the EPA for response and recovery efforts. The first is development of guidance for dissolved concentrations that would pose an immediate acute risk to exposed individuals, analogous to the inhalation immediate danger to life and health values of the National Institute for Occupational Safety and Health. The EPA is currently working on this problem by developing a database on acute and

chronic health effects associated with priority contaminants, although much work remains to be done. The second is guidance for determining the appropriate “acceptable” level remaining after cleanup/decontamination. This second aspect has not yet been strongly emphasized in the EPA research program. It is recommended that the EPA convene a working group to develop research and prioritization strategies for filling these data gaps and for ascertaining current gaps in knowledge with respect to rapid estimation of toxicity/infectivity in the absence of specific experimental information. Decisions for setting priorities for the data gathering efforts should be made with full consideration of dual-use benefits.

Contingencies for Water System Emergencies

Further study of water supply alternatives should be a high priority, considering their pivotal role in response and recovery and their dual-use applications for natural disasters or system failures. However, the subject of water supply contingencies seems to have been given a low priority in the EPA’s research program to date. Completion of the work in progress should be the first priority. The committee debated the value of investing significant resources in developing technologies that could supply drinking water for large communities over long-term disruptions because of the rarity of the need for such technologies. Nevertheless, the EPA should draw upon the research and development efforts of the Department of Defense in this area and work to test the application of these technologies to water security scenarios.

The EPA should consider including new research on contingencies for failures of the human subsystem in water system security. Such research could examine current practices for identifying back-up operators in the case of widespread incapacitation in both short-term and long-term scenarios. This research could also identify best practices, which could be incorporated into EPA guidance to water utilities for their emergency response planning.

Preliminary research suggests that geographic information systems (GIS) could be of significant value to utilities for identifying contingencies in the event of system failures. Therefore, further efforts may be needed to inform utilities about the value of GIS for emergency response and provide guidance for integrating GIS into their emergency planning procedures. National geodata standards may be needed to promote consistency and facilitate data exchange among users.

Behavioral Sciences and Risk Communication

The National Homeland Security Research Center (NHSRC) has made substantial progress in the development of risk communication guidance and training (see Chapter 4 ), but very little emphasis has been devoted to research on understanding how the public may respond to risk communication messages and how to improve communication of risks to the public. Terrorism presents risks that are new, evolving, and difficult to characterize; thus, water security poses communication challenges that should be addressed using scientifically rigorous research in the fields of risk communication and behavioral sciences. The EPA should continually reassess the role risk communication has in its overall risk management framework and fully integrate risk communication efforts into the overall risk management program. Behavioral science and associated risk communication research should be a high priority in the EPA’s future water security research plans. The following recommendations are targeted toward water-security events, but the proposed research has dual benefits for improving non-security-related communications with the public.

Analysis of Factors that Build Trust and Improve Communication

Research and experience prove that one of the most important keys to communication success is an organization’s ability to establish, maintain, and increase trust and credibility with key stakeholders, including employees, regulatory agencies, citizen groups, the public, and the media. To improve overall communication strategies in a water-related emergency, research is needed that analyzes factors that build trust and reduce fear (e.g., What types of concerns do people have related to public health emergencies, water security issues, or bioterrorism? How do utilities build trust and credibility with the public around water security incidents?). In addition, research is needed to analyze methods to counter and reduce the possibility of misinformation or false information being distributed to the public and key stakeholders.

Understanding Institutional Behavior

Building response and recovery capacity requires agencies that might be involved in a water security event to develop stronger working relationships. Although water utilities, public health agencies, law enforcement, emergency responders, and the media do not have a long history of collaborating and working together, several state drinking water programs have taken the lead in carrying out tabletop exercises as well as on-the-ground exercises to address this issue. These state programs have also undertaken measures to facilitate an understanding of the roles and responsibilities of the various potential players, including federal, state, and local law enforcement; state and local health agencies; state and local emergency response agencies; and water utilities. The EPA could glean useful information from these ongoing state and local activities. Nevertheless, additional research is needed to better understand the culture of the agencies that will be responding to events, how these agencies will interact in a water-related crisis, and what level of effort is needed to maintain collaboration in planning and preparedness. This research could identify barriers to more effective collaboration, and these findings could be used to create training scenarios that could improve coordination and resolve potential conflicts in advance. This research is a short-term priority given the importance of coordinated interaction during a crisis. The research could be performed relatively quickly because there is a wealth of experiences, particularly at the state level, related to agency interactions in water-related crises.

Investigate Applicability of Research in Behavioral Science

While some of the recommended research on risk communication and behavioral science may need to be managed by the EPA to address specific water security-related issues, the EPA should also take advantage of other behavioral science research currently being conducted through university-based partnerships, including those established by the Homeland Security Centers of Excellence program. For example, the University of Maryland’s National Consortium for the Study of Terrorism and Responses to Terror (START) is conducting original research on issues that are poorly understood, including risk perception and communication, household and community preparedness for terrorist attacks, likely behavioral responses by the public, social and psychological vulnerability to terrorism, and strategies for mitigating negative psychologi-

cal effects and enhancing resilience in the face of the terror threat. The START center is also synthesizing existing research findings in order to provide timely guidance for decision makers and the public, paying special attention to how diverse audiences react to and are affected by threats and preparedness efforts.

In addition, the CDC has developed a national network of 50 Centers for Public Health Preparedness (CPHP) to train the public health workforce to respond to threats to our nation's health, including bioterrorism. These centers work to strengthen terrorism preparedness and emergency public health response at the state and local level and to develop a network of academic-based programs contributing to national terrorism preparedness and emergency response capacity. Information from the CPHP may be relevant and useful to the water sector.

Pretesting Risk Communication Messages

Although the message mapping workshops are a good start to assist stakeholders in preparing messages that will be relevant in a water security incident, the messages have not been tested and evaluated. Therefore, the EPA should engage the research community in pretesting messages being developed by the Center for Risk Communication so that case studies and scenarios can be analyzed for effectiveness in reaching key audiences, and problems can be corrected in advance. Sophisticated evaluation techniques and standard research procedures are used by the CDC to pretest public messages. This evaluation research should be based on standard criteria established in the risk communication literature (e.g., Mailback and Parrott, 1995; National Cancer Institute, 2002; Witte et al., 2001).

Analysis of the Risks and Benefits of Releasing Security Information

The decision of when to release or withhold water security information is critical to the development of a risk communication strategy. Therefore, the EPA should analyze the risks and benefits of releasing water security information, considering input from its broad range of constituents, and develop transparent agency guidance on when to release information versus when to withhold it due to security concerns.

The committee considers this a priority because of the difficulty and importance of the information sharing problem.

Water-Related Risk Communication Training

As the lead U.S. agency in water system security, the EPA should assume the responsibility for developing a national training program on water-related risk communication planning and implementation for water managers. This should be done in collaboration with the water and wastewater organizations, state government agencies, public health officials, health care officials, and others engaged in communication of risks during water-related emergencies.

Decontamination

Decontamination research is critical to improving response and recovery, and the products are applicable to address unintentional contamination events from natural disasters (e.g., hurricanes, floods, earthquakes) and routine malfunctions (e.g., pipe breaks, negative pressures due to power losses). The EPA has numerous ongoing projects in this area that should be completed, but additional research topics are also suggested below.

Addressing Data Gaps

EPA decontamination research products released thus far have shown that fundamental physical, chemical, and/or biological characteristics of many threat agents of concern are not yet known. Therefore, additional laboratory research is needed related to the behavior of contaminants in water supply and wastewater systems and methods for decontaminating water infrastructure. For example, one research priority would be to develop inactivation rate data for all microbes of concern with both free and combined chlorine strategies, because both approaches are used in the water industry. Rate and equilibrium data for adsorption/desorption of contaminants on pipe walls is also needed, although the EPA could also take advantage of existing databases on structure-activity relationships to predict these behaviors. Long-term re-

search, perhaps in partnership with other Office of Research and Development units, could enhance our understanding of the fate, transport, and transformation of toxics in water and wastewater environments.

Decontamination Strategies

The EPA should build on its ongoing work in the area of decontamination and address gaps in the current knowledge base. For example, research is needed to examine readily available household inactivation methods for biological agents (including spore-formers), such as microwaving. The EPA should also work to further the development of innovative decontamination technologies that address important water security concerns. Research and development on new POU/POE technologies, such as superheated water devices, could help overcome operational disadvantages of the products currently on the market.

Prioritizing Future Surrogate Research

Surrogates are relevant to numerous water security research applications, including research on contaminant fate and transport, human exposure risks, and decontamination. Research is ongoing to identify surrogates or simulants for biological agents, to determine which surrogates are appropriate, and to determine the ability of typical drinking water disinfection practices (chlorination and chloramination) to inactivate those agents (see Chapter 4 , Section 3.2). Much of the research has focused on Bacillus anthracis and other bacterial agents, but the EPA should determine if surrogates for research on biotoxins and viruses are needed and whether additional surrogates are needed for other bacterial agents. A viral simulant or surrogate would be helpful to examine virus survival in fresh water, drinking water, and sewage, as well as virus susceptibility to water disinfectants. Research in this area has relevance to viral bioterrorism agents and also has strong dual-use research applications because viral surrogates could facilitate risk assessment studies on natural viruses (e.g., SARS, avian influenza).

Surrogate research is a laborious experimental process (see Box 4-1 ) that must be conducted in one of the few laboratories already authorized to keep and work with select agents. Considerable research is required to compare the select agent with candidate surrogates under the experimental conditions of interest. As discussed in Chapter 4 , surrogates need not

mimic in all respects the agents they stand in for. For some important security or decontamination uses, it may only be necessary that they provide an appropriate bound on the characteristic of interest in the target agent (e.g., persistence, disinfectant sensitivity). Therefore, the EPA should carefully consider and prioritize the agents and the research applications for which surrogates are needed. The prioritization process for surrogates should consider the following:

Which types of research could be greatly facilitated through the availability of surrogates?

Which types of research with surrogates might have “dual-use” applications (i.e., could the properties of certain surrogates also be usefully extrapolated to other common organisms)?

Which types of research should be done only with select agents?

How closely should the surrogate properties of interest match that of the target organism?

What are the costs and benefits to the research program associated with surrogate development versus use of the pathogenic agents?

The EPA should engage a limited number of individuals (e.g., federal partners, academics) who are involved in similar research in this prioritization process.

Lessons Learned from Natural Disasters

Midway through the committee’s work, NRC (2005; see Appendix A ) suggested the EPA take advantage of experience gained in the aftermath of Katrina so as to improve future response and recovery efforts for water security. While a hurricane caused this catastrophe, it is conceivable that a similar result might have occurred if the levees had been destroyed by terrorist explosives. Thus, New Orleans offered a living laboratory to study many aspects of the impacts of a disaster on water and wastewater systems of all sizes. Failure modes, infrastructure interdependencies, decontamination and service restoration strategies, the availability of alternative supplies, communication strategies, and the ability to service special institutions (e.g., hospitals) and special needs individuals could all have been examined in the immediate aftermath of the hurricane. To the best of the committee’s knowledge, however, the EPA has not attempted to compile a knowledge base from this experience. As

time passes, it will become increasingly difficult to reconstruct what transpired. Other natural or manmade disasters, such as the earthquakes in California in 1989 and 1994 or the “Great Flood of 1993” in the Mid-west, or natural contamination events, such as the Milwaukee C ryptosporidium outbreak, may also offer opportunities to mine important data about the failure or recovery of water and wastewater systems, but detailed information on these earlier occurrences may be lacking. In the future, the NHSRC should be poised to seize opportunities for learning about response and recovery after major natural or man-made disasters affecting water or wastewater systems.

Summary of Research Priorities for Improving Response and Recovery

Determine strategic plans for managing and maintaining the WCIT/CAT databases, considering the likely uses and long-term goals for the databases.

Develop and implement a strategy for evaluating the utility and usability of the response tools and databases, including stakeholder feedback and lessons learned during their use under “real-life” incidents.

Convene a working group to develop research strategies for filling the data gaps in WCIT/CAT and other planned emergency response databases.

Contingencies for Water Emergencies

Complete the work in progress on contingencies and infrastructure interdependencies under Section 3.5 of the Action Plan.

Test and evaluate the most promising innovative water supply technologies that enable or enhance the short- or long-term delivery of drinking water in the event of systemic failure of water systems. Analyze the positive features and those areas needing improvement prior to full-scale deployment.

Conduct research on potential contingencies for failures of the “human subsystem.”

Analyze factors that build trust, reduce fear, and prevent panic to improve overall communication strategies in a water-related emergency.

Investigate the behavioral science research being conducted by the Homeland Security University Centers of Excellence and other federal agencies for applicability to the water sector.

Pretest messages being developed by the Center for Risk Communication and analyze case studies and scenarios for effectiveness.

Analyze the risks and benefits of releasing security information to inform the EPA’s risk communication strategies and its practices on information sharing.

Fully integrate risk communication efforts into the overall risk management program and provide adequate resources that ensure these efforts remain a high priority in the EPA’s future water security research program.

Conduct research to better understand how agencies will interact in a water-related crisis situation and determine what strategies will be most effective in encouraging and maintaining collaboration in planning and preparedness.

Complete the many decontamination projects in progress under Section 3.4 of the Action Plan.

Develop predictive models or laboratory data for inactivation of bioterrorism agents in both free chlorine and chloramines that can be used in MS-EPANET and the TEVA model.

Explore development and testing of new POU/POE devices that may overcome the disadvantages of existing devices.

Examine readily available household inactivation methods for biological agents (including spore-forming agents), such as microwaving.

Determine the costs and benefits of further research to identify additional surrogates, considering which agents under which conditions or applications should be prioritized for surrogate development research.

Use the remaining data from the experience of Hurricane Katrina to analyze the optimal response and recovery techniques (e.g., water supply alternatives, contingency planning, and infrastructure interdependencies) that would also apply to water security events.

Integrate experience with decontamination of the distribution system in New Orleans after Hurricane Katrina to improve EPA guidance for water security decontamination.

Evaluate risk communication strategies related to Hurricane Katrina or other past disaster events to determine if communication strategies related to drinking water safety reached the most vulnerable populations.

Develop a post-event strategy for learning from future natural disasters affecting water systems. This strategy should support on-site assessments of impacts and interdependencies and evaluations of successes and failures during response and recovery.

Continue to develop and maintain the WCIT/CAT databases according to the objectives set forth in the strategic database management plan. Incorporate a mechanism to provide on-going peer review of the data to meet its data quality objectives.

Continue experimental and computational research to fill critical data gaps in WCIT/CAT, including research on the health effects of both acute and chronic exposure to priority contaminants.

Develop new, innovative technologies for supplying drinking water to affected customers over both short- and long-term water system failures.

Risk Communication and Behavioral Sciences

Develop a program of interdisciplinary empirical research in behavioral sciences to better understand how to prepare stakeholders for water security incidents. The EPA should support original research that will help address critical knowledge gaps. For example:

What are the public’s beliefs, opinions, and knowledge about water security risks?

How do risk perception and other psychological factors affect responses to water-related events?

How can these risks be communicated more effectively to the public?

Develop a national training program on water-related risk communication planning and implementation for water managers.

Continue laboratory research to fill the data gaps related to behavior of contaminants in water supply and wastewater systems and methods for decontaminating water infrastructure.

Continue surrogate research based on the research prioritization determined in collaboration with an interagency working group. The EPA should also explore ways that this surrogate research could assist in responding to everyday agents or to other routes of exposure (e.g., inhalation, inactivating agents on surfaces).

The EPA has historically been a lead federal agency in understanding the fate and transport of contaminants in the environment and has a clear understanding of the practical concerns of the water sector. Thus, the EPA remains the appropriate lead agency to develop the tools for emergency response and to prioritize the research needed to fill the remaining gaps, with input from key stakeholders. The EPA is also well suited to develop a national training program on water-related risk communication and to evaluate lessons learned from Hurricane Katrina and other past disaster events. However, innovative technology development research, such as the development of novel technologies for supplying water during system failures, should be conducted by other agencies,

university researchers, or firms with the greatest expertise. The EPA, instead, should focus its efforts on harvesting information on existing technologies, synthesizing this information for end users, and providing guidance to developers on unique technology needs for water security. Behavioral science research and evaluation research is more appropriately conducted by universities or other federal agencies (e.g., CDC) that have the necessary expertise to complete these tasks. However, the EPA still needs in-house behavioral science experts able to supervise and use this work to best advantage.

CONCLUSIONS AND RECOMMENDATIONS

In this chapter, recommendations are provided for future research directions in the area of water security. Two key water security research gaps—behavioral science and innovative future system design—that were not considered in the short-term planning horizon of the Action Plan are identified. In accordance with the committee’s charge (see Chapter 1 ), short- and long-term water security research priorities are presented in three areas: (1) developing products to support more resilient design and operation of facilities and systems, (2) improving the ability of operators and responders to detect and assess incidents, and (3) improving response and recovery.

The EPA should develop a program of interdisciplinary empiri cal research in behavioral science to better understand how to pre pare stakeholders for water security incidents. The risks of terrorism are dynamic and uncertain and involve complex behavioral phenomena. The EPA should take advantage of existing behavioral science research that could be applied to water security issues to improve response and recovery efforts. At the same time, when gaps exist, the EPA should support rigorous empirical research that will help address, for example, what the public’s beliefs, opinions, and knowledge about water security risks are; how risk perception and other psychological factors affect responses to water-related events; and how to communicate these risks effectively to the public.

The EPA should take a leadership role in providing guidance for the planning, design, and implementation of new, more sustainable and resilient water and wastewater facilities for the 21st century. Given the investments necessary to upgrade and sustain the country’s water and wastewater systems, research on innovative approaches to make the infrastructure more sustainable and resilient both to routine and

malicious incidents would provide substantial dual-use benefits. The EPA should help develop and test new concepts, technologies, and management structures for water and wastewater utilities to meet objectives of public health, sustainability, cost-effectiveness, and homeland security. Specific research topics related to drinking water and wastewater, such as decentralized systems and in-pipe interventions to reduce exposure from contaminants, are suggested.

Recommended research topics in the area of supporting more resilient design and operation of drinking water and wastewater systems include improved processes for threat and consequence assessments and innovative designs for water and wastewater. A thorough and balanced threat assessment encompassing physical, cyber, and contaminant threats is lacking. To date, the EPA has focused its threat assessments on contaminant threats, but physical and cyber threats deserve more attention and analysis because this information could influence the EPA’s future research priorities and utilities’ preparedness and response planning.

Research suggestions that improve the ability of operators and responders to detect and assess incidents build upon the EPA’s current research in the areas of analytical methodologies and monitoring and distribution system modeling. In the short term, the EPA should continue research to develop and refine a first-stage RTMS based on routine water quality parameters with dual-use applications. Long-term research recommendations include the development of innovative detection technologies and cheaper, more accurate RTMSs. To support the simulation models in development, a substantial amount of fundamental research is needed to improve understanding of the fate and transport of contaminants in distribution systems. Based on the number of emerging technologies and agents of interest, the EPA should develop a prioritization strategy for technology testing to optimize the resources devoted to this effort.

Recommendations for future research priorities to improve response and recovery emphasize the sustainability of tools for emergency planning and response (e.g., WCIT/CAT) and improving research on water security contingencies, behavioral sciences, and risk communication. The EPA should also evaluate the relative importance of future laboratory work on surrogate development and address data gaps in the knowledge of decontamination processes and behavior. So far, the EPA has not taken advantage of the many opportunities from Hurricane Katrina to harvest lessons learned related to response and recovery, and the window of opportunity is rapidly closing.

Some of the research recommendations provided in this chapter lie outside of the EPA’s traditional areas of expertise. The EPA will need to consider how best to balance intramural and extramural research funding to carry out this research, while maintaining appropriate oversight and input into the research activities. Increasing staff expertise in some key areas, such as physical security and behavioral sciences, will be necessary to build a strong and well-rounded water security research program.

This page intentionally left blank.

Concern over terrorist attacks since 2001 has directed attention to potential vulnerabilities of the nation's water and wastewater systems. The Environmental Protection Agency (EPA), which leads federal efforts to protect the water sector, initiated a research program in 2002 to address immediate research and technical support needs. This report, conducted at EPA's request, evaluates research progress and provides a long-term vision for EPA's research program. The report recommends that EPA develop a strategic research plan, address gaps in expertise among EPA program managers and researchers, and improve its approaches to information dissemination. The report recommends several high-priority research topics for EPA, including conducting empirical research in behavioral science to better understand how to prepare people for water security incidents.

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

Switch between the Original Pages , where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

Conclusion and Recommendations for Future Research

Cite this chapter.

• Siti Nuryanah 3 &
• Sardar M. N. Islam 4  

283 Accesses

This final chapter provides a summary of the research developed by the study. Initially, this chapter discusses briefly the background and literature that motivated this study. Next, how to achieve its main objective: developing sound financial management strategies through a financial optimisation model is reviewed. Section 8.4 presents how this book achieves its specific aims which are: (1) determining the GCG practices, business risks and regulatory environments that should be incorporated into the financial management strategies of a company; (2) quantifying the policy and mechanisms of GCG practices and identifying the right proxy for them; (3) designing an integrated multi-period financial optimisation model based on managerial and financial accounting perspectives; and (4) using a FCF approach, which is believed to be the best approach, to quantify the economic benefits of GCG practice.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Unable to display preview.  Download preview PDF.

Similar content being viewed by others

Concluding Observations

Alternative Performance Measures: A Structured Literature Review of Research in Academic and Professional Journals

ESG Issues as Strategic Components of Long-term Success of Financial Institutions: Are There Differences in Financial Performance and Firm Value?

Author information, authors and affiliations.

Department of Accounting, Faculty of Economics, University of Indonesia, Indonesia

Siti Nuryanah

VISES, College of Business, Victoria University, Australia

Sardar M. N. Islam

You can also search for this author in PubMed   Google Scholar

Copyright information

© 2015 Siti Nuryanah and Sardar M. N. Islam

About this chapter

Nuryanah, S., Islam, S.M.N. (2015). Conclusion and Recommendations for Future Research. In: Corporate Governance and Financial Management. Palgrave Macmillan, London. https://doi.org/10.1057/9781137435613_8

Download citation

DOI : https://doi.org/10.1057/9781137435613_8

Publisher Name : Palgrave Macmillan, London

Print ISBN : 978-1-349-49322-7

Online ISBN : 978-1-137-43561-3

eBook Packages : Palgrave Economics & Finance Collection Economics and Finance (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

How to write a PhD thesis: a step-by-step guide

A draft isn’t a perfect, finished product; it is your opportunity to start getting words down on paper, writes Kelly Louise Preece

Kelly Louise Preece

• More on this topic

Created in partnership with

You may also like

Popular resources

.css-1txxx8u{overflow:hidden;max-height:81px;text-indent:0px;} The secrets to success as a provost

Using non verbal cues to build rapport with students, emotionally challenging research and researcher well-being, augmenting the doctoral thesis in preparation for a viva, how hard can it be testing ai detection tools.

Congratulations; you’ve finished your research! Time to write your PhD thesis. This resource will take you through an eight-step plan for drafting your chapters and your thesis as a whole. 

Organise your material

Before you start, it’s important to get organised. Take a step back and look at the data you have, then reorganise your research. Which parts of it are central to your thesis and which bits need putting to one side? Label and organise everything using logical folders – make it easy for yourself! Academic and blogger Pat Thomson calls this  “Clean up to get clearer” . Thomson suggests these questions to ask yourself before you start writing:

• What data do you have? You might find it useful to write out a list of types of data (your supervisor will find this list useful too.) This list is also an audit document that can go in your thesis. Do you have any for the “cutting room floor”? Take a deep breath and put it in a separate non-thesis file. You can easily retrieve it if it turns out you need it.
• What do you have already written? What chunks of material have you written so far that could form the basis of pieces of the thesis text? They will most likely need to be revised but they are useful starting points. Do you have any holding text? That is material you already know has to be rewritten but contains information that will be the basis of a new piece of text.
• What have you read and what do you still need to read? Are there new texts that you need to consult now after your analysis? What readings can you now put to one side, knowing that they aren’t useful for this thesis – although they might be useful at another time?
• What goes with what? Can you create chunks or themes of materials that are going to form the basis of some chunks of your text, perhaps even chapters?

Once you have assessed and sorted what you have collected and generated you will be in much better shape to approach the big task of composing the dissertation. 

Decide on a key message

A key message is a summary of new information communicated in your thesis. You should have started to map this out already in the section on argument and contribution – an overarching argument with building blocks that you will flesh out in individual chapters.

You have already mapped your argument visually, now you need to begin writing it in prose. Following another of Pat Thomson’s exercises, write a “tiny text” thesis abstract. This doesn’t have to be elegant, or indeed the finished product, but it will help you articulate the argument you want your thesis to make. You create a tiny text using a five-paragraph structure:

• The first sentence addresses the broad context. This locates the study in a policy, practice or research field.
• The second sentence establishes a problem related to the broad context you have set out. It often starts with “But”, “Yet” or “However”.
• The third sentence says what specific research has been done. This often starts with “This research” or “I report…”
• The fourth sentence reports the results. Don’t try to be too tricky here, just start with something like: “This study shows,” or “Analysis of the data suggests that…”
• The fifth and final sentence addresses the “So What?” question and makes clear the claim to contribution.

Here’s an example that Thomson provides:

Secondary school arts are in trouble, as the fall in enrolments in arts subjects dramatically attests. However, there is patchy evidence about the benefits of studying arts subjects at school and this makes it hard to argue why the drop in arts enrolments matters. This thesis reports on research which attempts to provide some answers to this problem – a longitudinal study which followed two groups of senior secondary students, one group enrolled in arts subjects and the other not, for three years. The results of the study demonstrate the benefits of young people’s engagement in arts activities, both in and out of school, as well as the connections between the two. The study not only adds to what is known about the benefits of both formal and informal arts education but also provides robust evidence for policymakers and practitioners arguing for the benefits of the arts. You can  find out more about tiny texts and thesis abstracts on Thomson’s blog.

• Writing tips for higher education professionals
• Resource collection on academic writing
• What is your academic writing temperament?

Write a plan

You might not be a planner when it comes to writing. You might prefer to sit, type and think through ideas as you go. That’s OK. Everybody works differently. But one of the benefits of planning your writing is that your plan can help you when you get stuck. It can help with writer’s block (more on this shortly!) but also maintain clarity of intention and purpose in your writing.

You can do this by creating a  thesis skeleton or storyboard , planning the order of your chapters, thinking of potential titles (which may change at a later stage), noting down what each chapter/section will cover and considering how many words you will dedicate to each chapter (make sure the total doesn’t exceed the maximum word limit allowed).

Use your plan to help prompt your writing when you get stuck and to develop clarity in your writing.

Some starting points include:

• This chapter will argue that…
• This section illustrates that…
• This paragraph provides evidence that…

Of course, we wish it werethat easy. But you need to approach your first draft as exactly that: a draft. It isn’t a perfect, finished product; it is your opportunity to start getting words down on paper. Start with whichever chapter you feel you want to write first; you don’t necessarily have to write the introduction first. Depending on your research, you may find it easier to begin with your empirical/data chapters.

Vitae advocates for the “three draft approach” to help with this and to stop you from focusing on finding exactly the right word or transition as part of your first draft.

This resource originally appeared on Researcher Development .

Kelly Louse Preece is head of educator development at the University of Exeter.

If you would like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter .

The secrets to success as a provost

Emotions and learning: what role do emotions play in how and why students learn, the podcast: bringing an outsider’s eye to primary sources, a diy guide to starting your own journal, formative, summative or diagnostic assessment a guide, harnessing the power of data to drive student success.

Register for free

and unlock a host of features on the THE site

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Welcome to the Purdue Online Writing Lab

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

The Online Writing Lab at Purdue University houses writing resources and instructional material, and we provide these as a free service of the Writing Lab at Purdue. Students, members of the community, and users worldwide will find information to assist with many writing projects. Teachers and trainers may use this material for in-class and out-of-class instruction.

The Purdue On-Campus Writing Lab and Purdue Online Writing Lab assist clients in their development as writers—no matter what their skill level—with on-campus consultations, online participation, and community engagement. The Purdue Writing Lab serves the Purdue, West Lafayette, campus and coordinates with local literacy initiatives. The Purdue OWL offers global support through online reference materials and services.

A Message From the Assistant Director of Content Development 

The Purdue OWL® is committed to supporting  students, instructors, and writers by offering a wide range of resources that are developed and revised with them in mind. To do this, the OWL team is always exploring possibilties for a better design, allowing accessibility and user experience to guide our process. As the OWL undergoes some changes, we welcome your feedback and suggestions by email at any time.

Please don't hesitate to contact us via our contact page  if you have any questions or comments.

All the best,

Social Media

Facebook twitter.

Axos: Glaring Commercial Real Estate Loan Problems and Lax Underwriting Beneath This Priced-For-Perfection Bank

Get our latest reports delivered to your inbox.

• Axos Financial is a $3.1 billion market cap regional bank headquartered in San Diego, California that was incorporated in 1999 and went public in March 2005. 
• Axos trades at a 35% P/TBV premium compared to similarly sized community and regional banks, pricing in investor expectations of outsized growth, a low-risk loan book and multiple years of runway ahead.
• Its aggressive valuation premium implies that, despite operating in an environment that has devastated many peers, Axos has navigated it by (1) allocating to the lowest-risk asset classes (2) with superior underwriting standards, that have resulted in (3) a portfolio of high-performing projects.
• Our research, including industry analysis, interviews with 21 former employees, lease agents and industry experts, combined with an examination of Axos’ loan book derived through local property records, indicates a company exposed to the riskiest asset classes with lax underwriting standards and a loan book filled with multiple glaring problems.
• Contrary to many peers who backed away from the deteriorating commercial real estate market post-covid pandemic, Axos doubled down, increasing its total exposure from $5.5 billion in March 2021 to $9.9 billion in March 2024. Now, 53% of Axos’ total loan book is exposed to these segments.
• By comparison, a 2023 Moody’s study found that regional banks’ exposure to direct commercial real estate made up only 16.5% of their respective loan books, on average.
• The industry has begun to plummet: distressed U.S. commercial properties rose from $56.9 billion in 2022 to $85.8 billion in 2023. $2.2 trillion in commercial mortgages are set to mature before the end of 2027.
• Even AAA-rated bonds backed by CRE debt are now taking losses, and multifamily investments have plunged to four-year lows.
• “It’s a sh*t show,” one Axos deal partner told us about the office market in New York. 37.5% of Axos’ commercial real estate loans were in New York, where a CRE “bloodbath” has resulted in commercial realty foreclosures increasing 65% year over year.
• Beyond its dangerous allocation choices, Axos’ lax underwriting standards expose it to major added risk. A former regional leader told us Axos’ customer base in commercial and multifamily revolved around “borrowers who couldn’t get loans from other banks”.
• In New York alone, with over 125 competing mortgage lenders including top tier banks, Axos faces stiff competition. A former loan originator told us that Axos had to stretch its underwriting criteria to compete: “There had to be a problem for us to even have a shot at it”.
• Per one former employee on Axos’ embrace of borrowers with criminal histories: “ If the felony was explainable, we’ll bank them”. A second added that bad “credit scores…didn’t kill a deal”. A third said “I don’t recall there ever really being a…minimum net worth or minimum liquidity requirement”.
• A former Axos credit review officer detailed the practice of loan “evergreening,” or providing loans to non-performing or doubtful borrowers to avoid recognizing problems, per litigation records. Similar schemes, also known as “extend and pretend”, were described by former employees during our investigation.
• These underwriting standards have resulted in a portfolio filled with clear problem loans, based on our review of a cross-section of Axos’ loan book. On March 31, 2024, Axos reported it had provisions for credit losses in its CRE category of just $83 million, a metric that seems vastly understated.
• For example, Axos lent up to $97.5 million (4.8% of tangible book value), for an apartment construction project in Queens. The underlying borrower has been criminally indicted twice personally, including a case involving a construction kickback scheme with a mafia captain. His company was indicted in a third case.
• One reason most banks don’t lend up to $97.5 million to individuals with multiple indictments and documented mob ties is that even if things go well, it can be difficult to get your money back. This is made even more challenging when things go poorly, as seems to be the case with this property.
• The project was scheduled for completion in May 2024, but contrary to many buildings that pre-lease significant portions of a project, the leasing agent told us it has leased zero units, indicating it may already be in distress.
• In December 2020, Axos lent up to $48.2 million (2.4% of tangible book value) to build an apartment block in Brooklyn, according to local property records. Four years later, amidst zoning issues with the Department of City Planning in New York, there are no signs of construction activity and weeds are growing on the land, per our visit to the site last month.
• In 2021, Axos lent up to $34.7 million (1.7% of tangible book value) for the construction of a medical office building in Harlem at 2226 Third Avenue, according to local property records.
• By 2023, the borrower faced multiple foreclosures after allegations emerged that he had embezzled money from real estate projects and lost much of it to extravagant personal spending and gambling on stocks. The borrower is now under investigation by the DoJ and SEC over allegations of embezzling real estate funds, per local media.
• Construction on the medical office building was completed in February 2024. The whole building is vacant, and the owners are attempting to remarket it with a change in use, per our interview with the leasing agent.
• In 2022, Axos lent up to $35 million (1.7% of tangible book value) to a developer to construct a 12-story apartment building in Manhattan. Two years later, we found a seemingly derelict property with graffiti tags and little signs of construction.
• In November 2023, Axos took over a $105 million loan (5.2% of tangible book value) on an office building in Manhattan called “The Six.” Its owner, Savanna, has been “defaulting on debt, repeatedly extending loans, and reducing its ownership stake within individual properties,” per media reports. The building is currently less than 50% occupied, per CoStar.
• Axos has $1.4 billion of multifamily loans originated during or prior to 2020, when interest rates were close to zero. Former employees told us that, given a typical fixed period of around 5-6 years, many of these come due within the next year and risk being underwater.
• Axos has been extending loans or attempting to offer slight discounts to keep them afloat, but “in a lot of cases the loans, the properties didn’t cover”, per a former employee.
• Axos’ rosy-looking credit metrics show signs of manipulation or distortion. Disclosed loan to value (LTV) ratios in commercial real estate are 17% less than the median average of 9 of its peers.
• “the Bank routinely misrepresented the average loan to value ratio’s of its loans to investors,” per allegations by a former credit review officer documented in litigation records.
• Non-performing loans as a percent of its key commercial real estate category have stayed almost flat in the last 2 years, from 0.4% in 2021 to 0.43% in 2023. By contrast, large national banks like Wells Fargo and Bank of America have seen this ratio increase by 1.9%-2.0% in the same period.
• Provisions for future stress as a percent of the commercial real estate category (called “allowances”) have inexplicably decreased 0.5% since June 2021, from 1.8% to 1.3% at the end of December 2023.
• We estimate that at least $1.1 billion of CRE loans originated at lower interest rates will face renewal in the coming year, testing the sufficiency of Axos’ abnormally low provisions. Axos’ auditors identified this as a critical audit matter given “the judgmental and subjective nature” of the company’s forecasts.
• Banking is an industry as old as Mesopotamia. Over the last 4,000 years, the industry’s participants have evolved in tandem. All U.S. banks are subject to the same federal funds rate and interest rate dynamics, and similar regulations and requirements. Competitive edges, along with comparable ratios and margins, are generally razor thin.
• For as long as banks have been around, there have also been ‘too good to be true’ outliers that claim to have discovered a magical lending formula. In this case, we are meant to believe that Axos’ has selected only the best felons and troubled borrowers and properties to lend to, allowing it to waltz gracefully through the imploding commercial real estate market. We think otherwise.
• While we do not to call into question Axos’ liquidity position or its depositor base, history shows that these ‘outliers’ often take on undisclosed risk to fuel their optimistic numbers.
• Overall, Axos’ exposure to the riskiest asset classes, its lax underwriting standards, and glaring issues with its portfolio indicate that the company faces significant stress ahead.

Initial Disclosure: After extensive research, we have taken a short position in shares of Axos Financial, Inc. (NYSE:AX). This report represents our opinion, and we encourage every reader to do their own due diligence. Please see our full disclaimer at the bottom of the report.

Basics On The Business: A $3.1 Billion Digital Focused Bank, Headquartered in California

Axos Financial (“Axos”) is a $3.1 billion market cap regional bank headquartered in San Diego, California that was incorporated in 1999 and went public in March 2005 . [1]

The company describes itself as a “technology-driven financial services company that provides innovative banking products and services to customers nationwide”, per its website . It is led by Gregory Garrabrants, who has been CEO and President since 2007.

Axos has two key businesses: its “ banking business ” and its “ securities business ”.

Its banking business covers deposit and loan activities for a range of customers, from personal consumers to commercial and industrial customers, per Axos’ filings . Its banking business accounted for ~95% of Axos’ pre-tax income as of December 2023 . [2]  

Its securities business, comprising the rest of Axos’ pre-tax income, is aimed at financial market participants and includes brokerage, investment advisory and clearing.

The bank’s entire loan book was valued at $18.7 billion on March 31st, 2024 , comparable in size to many regional banks.

Bull Case: Asset-Light Business Generating Returns On Assets In The 90 th Percentile Of Its Peer Group, While Maintaining Prudent Underwriting

Axos has operated since inception as a digital bank, meaning it doesn’t have the fixed costs typically associated with legacy “brick and mortar” banks.

The bank has described its loan growth as “ prudent ” with “low LTVs” and has said in the past that its conservative underwriting and secured lending has well-positioned its portfolio for an economic downturn. [ Pg. 12 ]

Over the last 10 years, Axos has grown its total loan book at an average of 22% per year. Its profitability has regularly increased in tandem, with net income growing 23% per year on average.

In its May 2024 investor presentation, Axos reported earnings metrics that dwarfed most of its peers, with returns on equity, returns on average assets and net interest income all at, or above the top 94 th percentile of its peer group. [4]

Along with its strong growth and margin performance, Axos’ risk metrics remain better than industry averages, with non-performing loans and leases to total loans currently at just 0.63%, compared to a Q1 2024 industry average of 0.91%.

Fundamentals: Axos Trades At A 35% Premium Relative To Peers Based On Price/Tangible Book Value

Price to Tangible Book Value (P/ TBV ) is a baseline valuation metric commonly used to analyze bank stocks. It compares the company’s price to the equity available to stockholders, leaving out intangible assets such as goodwill or intellectual property.

Axos trades at a ~35% P/TBV premium compared to similarly sized community and regional banks, pricing in investor expectations of outsized growth, a low-risk loan book and multiple years of runway ahead.

On May 15 th , 2024, Axos shares hit all-time highs of $63, with the stock currently trading around 15% off those highs. 

Axos’ aggressive valuation premium implies that, despite operating in an environment that has devastated many of its peers, Axos has navigated it by (1) allocating to the lowest-risk asset classes and (2) accumulating a portfolio of extremely high-performing projects driven by (3) superior underwriting standards.

Part I: Axos Has Allocated To The Highest Risk Asset Classes—53% Of Axos’ Total Loan Book Is Exposed To The Sharply Deteriorating US Commercial Real Estate Market

Between commercial real estate and multifamily segments, axos’ total exposure is $9.91 billion as of march 2024.

Rather than allocating to the lowest-risk asset classes, Axos is highly exposed to the riskiest.

Commercial real estate (“CRE”) lending is commonly defined as loans originated on properties that are used for business purposes, such as offices or retail stores, as well as multi-unit residential buildings, called “ multifamily ”. [5]

Collectively we refer to Axos’ disclosed categories of “Commercial Real Estate” and “Multifamily & Commercial Mortgage”, found in its filings, as Axos’ “Total Commercial Real Estate Exposure”.   Over the last 3 years, between March 2021 and March 2024, pandemic lockdowns and the shift to working from home helped catalyze an exodus from commercial real estate by many industry participants. [6]

During that time, Axos doubled down. Its total commercial real estate exposure grew in absolute value by 81%. [ 1 , 2 ] Its CRE category grew 94%, during the same time period. [ 1 , 2 ]

At the end of March 2024, Axos reported a balance of $5.9 billion in its commercial real estate segment —32% of its entire loan book or 292% of tangible book value . It also reported $4 billion in its “Multifamily and Commercial Mortgage” category—21% of its entire loan book or 198% of tangible book value . [7] [ Slide 10 ]

As a result, Axos’ total commercial real estate exposure has ballooned to 53% of total net loans , or $9.91 billion.

By Comparison, A 2023 Moody’s Study Found That Regional Banks’ Exposure To Direct Commercial Real Estate Made Up Only 16.5% Of Their Respective Loan Books, On Average

A 2023 Moody’s study found that the average regional bank, defined as holding $10-160 billion in assets, had 16.5% of assets exposed to CRE, far less than Axos’ 53%.

The same study found that the highest direct total CRE exposure came from community banks at 24.3%, a number still less than half of Axos’ current exposure.

The Industry Has Begun To Plummet: Distressed U.S. Commercial Properties Rose From $56.9 Billion In 2022 to $85.8 Billion In 2023

“at some point, that avalanche is going to hit”: $2.2 trillion in commercial mortgages are set to mature before the end of 2027, aaa-rated bonds backed by cre debt are now taking losses and multifamily investments have plunged to four year lows.

Distressed U.S. commercial properties grew from $56.9 billion in 2022 to $85.8 billion by the end of 2023, according to MSCI , which noted that offices constituted 41% of distressed assets and that the “multifamily market constituted the largest pool of potentially distressed assets, with a value of USD 67.3 billion, ahead of office with USD 54.7 billion.” [8]

Cumulative distress has been on a steady rise since 2022 and has eclipsed its post-Covid highs.

The Federal Reserve and the International Monetary Fund (IMF) have both commented on the threat, with the latter noting in January 2024 that the US commercial real estate market has seen “one of the steepest price declines in at least a half century”.

Commercial real estate data firm Trepp predicts “distress will keep rising,” per a February 2024 Wall Street Journal report that states more than $2.2 trillion in commercial mortgages are set to mature before the end of 2027. Lonnie Hendry of Trepp told the Journal:

“At some point, that avalanche is going to hit.”

NBC reported on May 21, 2024 that multifamily rents in April fell 0.8% because “a massive amount of new supply entered the market, with still more in the pipeline”.

This supply glut is further exacerbated by weak investment appetite: data retrieved from Costar as of May 2024 shows that multifamily investments have fallen to four-year lows.

On May 23, 2024, Bloomberg reported that “for the first time since the financial crisis”, AAA rated bonds backed by commercial real estate debt were suffering losses.

The article detailed how buyers of the AAA portion of a $308 million note on the 1740 Broadway building in midtown Manhattan took a 26% loss after the loan was sold at a steep discount.

Lea Overby, a CMBS strategist at Barclays, told Bloomberg :

“Now that we’ve seen the first commercial mortgage backed securities get hit, other AAA bonds are bound to see losses…”

37.5% Of Axos’ Commercial Real Estate Loans Were In New York

“the worst market i’ve seen”: new york city cre “bloodbath” has resulted in commercial realty foreclosures increasing 65% year over year, “it’s a sh*t show,” one axos deal partner told us about the office market in new york.

Axos reported that as of June 2023, 37.5% of its commercial real estate loan category was in New York, per its latest 10-K. [ Pg. 6 ]

New York is a geography that warranted its own “Risk Factor” in Axos’ 10-K, where it disclosed that “declining real estate values, particularly in California and New York, could reduce the value of our loan and lease portfolio and impair our profitability and financial condition.”

Over the last year, numerous media reports and industry research reports detailed the unfolding disaster in New York’s CRE market. The Financial Times called it a “bloodbath” last year. Bisnow quoted a source calling it the worst market they’ve ever seen, and the New York Times wrote in March that prices were tumbling. [ 1 , 2 , 3 , 4 ]

One commonly tracked indicator is the number of foreclosures, measuring a key risk for when lenders are generally forced to sell the underlying property collateral.

In New York, the number of commercial property foreclosures increased 65% year on year in March 2024, according to Attom Data, a commercial real estate analytics firm.

After California, which accounts for another 7.1% of Axos’ CRE category, New York had the highest number of commercial foreclosures out of any state, per the data.

Our research found office buildings in New York financed by Axos between 2020-2024 that are facing stresses in commercial real estate markets, as detailed in Part II. We interviewed an Axos deal partner, asking about their assessment on New York office pricing. They told us “it’s a sh*t show”, further explaining:

“… every day you’re hearing about a different office that loses a very large tenant. And when you have these empty offices, you can’t fill them back up. Offices have a distinct issue where they’re huge amounts of blocks of space where the landlord pays to build them out. So you have to pay tenant improvement”.

In brief, Axos is highly exposed to the riskiest asset classes in the country that are in a state of collapse. Navigating this minefield unscathed, as its stock price suggests it has done, would require Axos to have nearly flawless loan selection.

Part II: A Closer Look At Axos’ Portfolio Shows A Loan Book Filled With Glaring Problems

On March 31, 2024, Axos reported it had provisions for credit losses in its CRE category of just $83 million. [ Slide 27 ]

Axos has reassured investors that it has largely been immune to broader market stress, saying “we’re not really seeing anything that we find concerning”, per the CFO’s comment on the Q3 2023 earnings call. [ 1 ]

Our findings show otherwise—that Axos’ embrace of troubled borrowers in the riskiest markets in the country has predictably resulted in major projects that appear to be either stalled or deeply underperforming. Others are nowhere near completion or barren and vacant.

Axos Lent Up To $97.5 Million (4.8% Of Tangible Book Value), For An Apartment Construction Project In Queens

The underlying borrower has been criminally indicted twice personally, including a case involving a construction kickback scheme with a mafia captain, his company was indicted in a 3 rd case.

In October 2023, Axos entered into an agreement to lend up to $97.5 million for an apartment construction project at 147-35 95 th Avenue, Jamaica, Queens per New York property records . [9] [10] [ Pg. 5 ] The property is slated to be a 521-unit apartment building, per Multi-Housing News .

By itself, the loan represents up to 4.8% of Axos’ tangible book value and could be one of Axos’ larger loans, even higher than the maximum loan values indicated on Axos’ website for construction lending.

The underlying borrower is Sutphin Boulevard Equities, owned by Solomon Feder, per state government records. [ Pg. 25 ] [11]

A basic Google search shows Solomon Feder is party to 3 criminal indictments for fraud, a kick-back scheme with a reported mobster and an off-the-books compensation scheme. [12]

#1: In February 2022, Solomon Feder was indicted over allegations relating to a “$20m off-the-books compensation scheme” involving “envelopes of cash” to avoid detection.

In February 2022, Solomon Feder was indicted in a scheme that enabled construction companies to under-report payrolls, per a press release by the Manhattan District Attorney . This involved workers being paid “ with envelopes of cash ”, to avoid detection.

#2: In January, 2023, Solomon Feder was indicted in a construction kick-back scheme involving a Gambino mobster “captain”.

A Statement of Facts, released by the Manhattan District Attorney, mentioned Feder and his company. [ Pg. 10 ] The indictment showed Solomon “Sol” Feder’s company had upped the price for contract work on 250 5 th Avenue at the behest of the key accused in the kickback scheme, Robert Baselice. [ Pgs. 9-10 ] The case is still ongoing, per court records.

At the time of the kickback scheme, the purported mastermind, Robert Baselice (aka Robert Basilice) was an executive with The Rinaldi Group , a contractor that has not been charged with any wrongdoing. A 2010 NY Post article referred to Baselice as a “Gambino crime-family associate” when he was indicted in a large scale gambling ring bust. [13]

#3: In May, 2023, Big Apple Designers, owned by Solomon Feder, was indicted over a scheme to fraudulently qualify as minority and/or women-owned business enterprises, per a New York government press release. [ Pg. 5 ]

The Project Run By The Axos Borrower Involved In 3 Indictments, With Documented Mob Ties, Is Scheduled For Completion In May 2024

It has leased zero units, per the lease agent, indicating it may already be in distress.

One of the reasons most banks don’t lend up to $97.5 million to individuals with multiple indictments and documented mob ties is that even if things go well, it can be difficult to get your money back.

This is made even more challenging when things go poorly, as seems to be the case with this property, slated for completion in May 2024, per CoStar.

Given the imminence of completion, we expected many of the 521 units to have already been rented, as is common with most successful projects. But when we called the lease agent, The LiscoGroup, in mid-May they told us: [14]

“ Nothing is leased yet… The building is still empty… It will finish in construction, it should be done in the next month or so.”

When we asked if it would be possible for us to rent 200-300 units they told us it “shouldn’t be an issue”.

In short, one of Axos’ large specialty loans appears to have zero apartments rented and is owned by an individual with multiple indictments and alleged ties to the mob, information found through a basic internet search.

Given our conversations with former employees, as described later in Part III, we suspect Axos agreed to lend to this individual knowing his history or was almost incomprehensibly negligent in missing these red flags.

In December 2020, Axos Lent Up To $48.2 Million (2.4% Of Tangible Book Value) To Build An Apartment Block In Brooklyn, According To Local Property Records 4 Years Later, There Are No Signs Of Construction Activity And Weeds Growing On The Land, Per Our Visit To The Site

In December 2020, Axos was listed as a lender on 962 Franklin Avenue, along with Fortress, a global asset manager, per New York property records . [ Pg. 3 ] The developer is Continuum , a boutique New York developer. New York City Department of Finance Records lists the loan as up to $48.2 million.

The project appears to be running into challenges with environmental and zoning permissions. In June 2023, the New York City Department of City Planning determined that the rezoning proposal may have a “significant adverse impact on the environment”.

Over 3 years after Axos’ entered into the lending agreement, the property zoning proposals still do not appear to have been approved, according to NYC planning records .

In late 2023, local media reported that the site was still an empty lot with overgrown weeds.

We visited the site in May 2024 and found that the situation remains unchanged.

Its neighboring property, 960 Franklin Avenue, has also faced opposition from local residents and construction has not commenced, per media reports . Just months ago, in April 2024, Axos loaned up to $91 million to the new stalled development on 960 Franklin Avenue, according to New York Property records . [ Pg. 5 ]

In 2021, Axos Lent Up To $34.7 Million (1.7% Of Tangible Book Value) For Construction Of A Medical Office Building In Harlem At 2226 Third Avenue, According To Local Property Records By 2023, The Borrower Faced Multiple Foreclosures After Allegations Emerged That He Had Embezzled Money From Real Estate Projects And Lost Much of It To Extravagant Personal Spending And Gambling On Stocks

The borrower is under investigation by the doj and sec over allegations of embezzling real estate funds, per local media.

In October, 2021 Axos lent up to $34.7 million for construction of a property at 2226 Third Avenue in Harlem per New York property records . [15] [ Pg. 5 ] The deal was recorded during the height of Covid, exemplifying Axos’ strategy of ‘doubling down’ on CRE when many others began to withdraw.

The ultimate owner of the development is Elie Schwartz, through his firm Nightingale Properties. [16] As of media articles from 2023, Schwartz and his firm are currently facing foreclosure on multiple properties due to allegations he embezzled real estate funds for personal use and to gamble on stocks, which he lost. Schwartz is in the process of unwinding his assets including his properties, personal residences, and jewelry as part of restitution efforts to harmed investors.

Schwartz is currently under investigation by the SEC and DOJ related to the allegations of embezzlement and real estate fraud.

Construction On The Medical Office Building Was Completed In February 2024

The whole building is vacant and the owners are attempting to remarket it with a change in use, per the agent.

Beyond the project sponsor’s extreme state of distress, the development Axos lent to appears to be failing.

Construction was completed on the office property in early February of this year, per local media . The development, called “The Labs on 121”, offers lab space that is “precisely developed to not only meet – but far surpass – today’s life science demands”, per its brochure. [ 1 , 2 ]

In April 2023, The Commercial Observer, a real estate news site, reported that “the 10-story, 193,000-square-foot building at 2226 Third Avenue hasn’t signed any leases yet”. A leading CRE data provider, CoStar, shows that the whole building is still vacant.

We spoke to the agent of the property, CBRE, in early May 2024. They told us that they were attempting to remarket it as a more general facility and not just life sciences:

“So the building was really built to be a life sciences building, but we’re sort of marketing it as a dual, like, we’re also looking at some schools and stuff like that.”

They went on to confirm that the entire building is still available:

“Yes. The whole building is available. It’s 200,000 [square] feet.”

For a such a large, completed construction to have zero tenants and face a major rebranding all while the owner battles foreclosure and potential criminal and civil fraud charges indicates that Axos will likely face distress on the property.

In 2022, Axos Lent Up To $35 Million (1.7% Of Tangible Book Value) To A Developer To Construct A 12 Story Apartment Building In Manhattan 2 Years Later, We Found A Seemingly Derelict Property With Graffiti Tags and Little Signs Of Construction

In March 2022, Axos lent up to $35 million to developer ZD Jasper on 3 parcels on 429 West 36 th and 430th & 434th West 37 th street in Manhattan to build a 12 story apartment block, per The Real Deal and New York Property Records . [ Pg. 3 ] ZD Jasper is a family firm that has around 12 employees, per a 2022 YouTube video. [ 0:21 ]

In March 2023, it was reported that development was being stalled by a neighboring property owner who has not signed off on the demolition of the existing property, per The Real Deal . In May 2024, nearly 2 years after the loan sanction, we visited the property. All we saw was a derelict building with graffiti tags and little sign of construction activity.

In short, the project has apparently not started construction ~2.5 years after Axos financed the transaction.

In November 2023, Axos Took Over A $105 Million Loan (5.2% Of Tangible Book Value) On An Office Building In Manhattan Called “The Six” Its Owner Has Been “Defaulting On Debt, Repeatedly Extending Loans And Reducing Its Ownership Stake Within Individual Properties” Per Media Reports

The building is currently less than 50% occupied, per costar.

As we noted in the introduction, the first AAA rated bonds backed by commercial real estate debt that were suffering losses were office buildings.

Axos and its CEO Garrabrants have often tried to reassure investors that they are not affected by major dislocations in the office category (as discussed earlier). In April 2023, Garrabrants implied that Axos has avoided the most stressed pockets of the office market:

  “I think office, clearly in many cities in places that we stayed away from for a long time is doing terribly” (Source: Axos April 2023 Investor Call )

In July 2023, Garrabrants further reassured investors:

“The majority of our commercial real estate secured by office properties are located in metropolitan areas that have not seen a meaningful negative impact from work-from-home and other dynamics.” (Source: Axos July 2023 Investor Call )

Despite this claim, we found signs of trouble in this segment. For example, in November 2023, Axos took over a $105 million loan on an office called “The Six”, located on West 56 th Street in Manhattan, per New York property records . [ Pg. 4 , Pg. 4 ]

The underlying owner of the building is the Savanna Group, a “vertically integrated real estate investment manager” per its website and CoStar.

The general contractor on this project was the aforementioned Rinaldi Group, where Robert Baselice, the purported “ ringleader ” of the mafia kickback scheme mentioned above had previously worked as a senior executive.

Baselice, representing the Rinaldi Group, was present for the topping out of the building in July, 2019.

The Savanna Group portfolio is reportedly in a state of distress, per The Real Deal .  It has been “defaulting on debt, repeatedly extending loans and reducing its ownership stake within individual properties”.

Data from CoStar shows the building is only 46.5% leased, 6 months after Axos’ loan agreement.

At the end of May 2024, we spoke to CBRE, the broker for the property, who confirmed about half of the building was still available:

“We leased by about a half a building. It’s a 90,000 square foot building. I say probably what’s left is close to that. In that 40 to 50k range”

The broker also offered that pricing may need to come down further to have a better chance at occupancy:

“ I think the pricing is pretty expensive.”

In terms of the collateral value, a recent transaction at 1740 Broadway is illustrative of the current stress in the Class A office space in New York. 1740 Broadway is a 640k square foot office tower located less than two blocks away from 106 W 56th Street. After a fire sale in mid-April 2024, investors in the AAA rated bonds, backed by 1740 Broadway, got less than three quarters of their return, per Bloomberg .

Assigning a similar square foot valuation of $290.46 would value The Six at a mere $25.4 million. [17] [ 1 ]

While Axos has claimed to be immune from the stress in the office market, in our view, the characteristics of one of its largest office loan transactions suggest otherwise and show hallmarks of stress.

Axos Has $1.4 Billion Of Multifamily Loans Originated Prior To 2020, When Interest Rates Were Close To Zero

Former employees told us that, given a typical fixed period of around 5-6 years, many of these come due within the next year and risk being underwater, axos has been extending loans or attempting to offer slight discounts in an effort to keep them afloat, according to a former employee.

Asked about potential loan stress last month, Axos’ CFO Derrick Walsh told investors:

“No, we’re not really seeing anything that we find concerning.” ( Q3 2023 Earnings Call )

Interest rates have risen from near zero to 5.5% since 2022. With the sharp rise in rates, many of Axos’ multifamily mortgages are now coming out of their fixed period and risk becoming delinquent.

Axos reports the vintage of its loans, showing that $1.4 billion of loans, or 36% of the multifamily book, was originated before June 2020.

An executive who left the bank in 2024 told us that many of these multifamily loans coming up for renewal were now underwater on the collateral value:

“We would find out that a loan was in trouble… [What] really exacerbated it [was] when the rates started going up and skyrocketed last year. And a lot of the hybrid loans started to come due… Because you [the borrower] get into this, you’ve had this loan for 3 or 5 or 7 years and it’s been fixed and everything’s been hunky dory and you forget about it. You forget that time frame is coming to a close. And now you’re going to have to address these higher rates. And in a lot of cases the loans, the properties didn’t cover.”

They went on to opine that some of Axos strategies to cure these loans were “kind of a joke” and ultimately ineffective given the magnitude of interest rate increases:

“I think what they were really trying to do is give the borrowers adequate time to try to refinance somewhere else. Sometimes they would maybe negotiate a little bit smaller, initial bump instead of being nine and a quarter. Maybe they’d go to, which I thought was kind of a joke, eight and three quarters or something like that. I mean, if it doesn’t work at nine and a quarter, it’s probably not going to work at eight and three quarters.”

A Call For Transparency: If Axos Is Truly Confident In Its Underwriting, It Should Disclose Details About Its Commercial Lending Portfolio

There is significant opacity around Axos’ loan book. We were only able to infer loan amounts and counterparties through local property records, scattered media reports, and litigation records, without full details on each deal.

If Axos is confident in its loan quality, as it suggests, it should be happy to provide more details to investors on its commercial loan portfolio such as deal terms, disbursed amounts and counterparties, in order to fully assess its exposures and risks.

Part III: Axos’ Lax Underwriting Standards Expose It To Major Risk

Our findings show that the issues with Axos’ portfolio stem from its lax underwriting standards.

Axos lends in the most competitive metro regions of the country, including New York and California, which account for 66.5% of its real estate loans. [ Pg. 26 ]

In these states, borrowers have hundreds of options competing for their business, including well-established banks like J.P Morgan, Citi, and many others.

A New York commercial real estate broker and former investment bank director we interviewed told us:

“[New York] It’s very competitive from a lending perspective, which makes it possible to do deals here, but also makes it very cutthroat in a lot of ways. A lot of banks will bend over backwards to do deals in New York… From a competition standpoint, I don’t think that there’s a market that’s tougher .” “ In New York, the active subset is probably about 250 [lenders] …” [18]

Axos has touted its “conservative” underwriting. The company’s ‘commercial lending’ website touts “streamlined processing and common sense underwriting.” [ Pg. 12 ]

Our research revealed precisely the opposite. A former Axos loan originator told us that Axos had to stretch its underwriting criteria to compete:

“There had to be a problem for us to even have a shot at it.”

A Former Regional Leader Told Us Axos’ Customer Base In Commercial And Multi-Family Revolved Around “Borrowers Who Couldn’t Get Loans From Other Banks”

“ if the felony was explainable, we’ll bank them,” they told us, bad “credit scores… didn’t kill a deal” and “i don’t recall there ever really being a… minimum net worth or minimum liquidity requirement”, we were told by former employees.

Former employees told us that far from applying “common sense” to its lending, there were few restrictions on the type of borrowers Axos worked with, and that Axos even banked felons and borrowers with troubled histories:

“It was the borrowers who couldn’t get loans from other banks. So it’s the dirty borrowers. Did they have a felony? If the felony was explainable, we’ll bank them, because he [the Axos CEO] was all about being well secured to the real estate.”

In a market with well-established competition for CRE deals, Axos would differentiate itself by embracing loans with problems other bank refused to touch, then try to add extra protections into the deal structuring, according to a former senior loan officer we spoke with who worked in the multifamily loan division:

“That was about the only way we could win anything, is if there was any hair. If it was a straightforward, straight up A/B kind of deal, we couldn’t compete on pricing or leverage. So there had to be a problem for us to even have a shot at it. And usually it was a borrower problem or it was an occupancy issue ..”

They went on to add that “ we couldn’t complete… unless other banks wouldn’t touch him ”, further reiterating that:

“…they [other banks] priced so much better than we did that we just couldn’t compete unless, as I said, unless the borrower was really hairy or something like that and the other banks wouldn’t touch him. Then we had a shot.”

“We weren’t doing Fannie and Freddie quality properties either. We weren’t doing those types of deals.”

A former senior loan officer also told us that at Axos, credit scores and credit problems wouldn’t typically kill the chances of a deal:

“They haven’t been a credit driven borrower, at least in the IPL [Income Property Lending] space. I mean credit scores. Well, we got them. Didn’t really, it didn’t kill a deal, let’s put it that way. If they liked the real estate… and if he [the borrower] had some credit problems, that was okay.”

A Former Credit Review Officer Uncovered Evidence Of Loan “Evergreening”, Or Providing Loans To Non-Performing Or Doubtful Borrowers, During Her Time At Axos Bank, Per Litigation Records Similar Types Of “Pretend And Extend” (aka ‘Zombie Lending’) Schemes Were Described By Former Employees During Our Investigations

Evergreening is a term used in banking circles to describe banks renewing or providing further loans to borrowers that should otherwise be recognized as non-performing or doubtful – it is sometimes referred to as ‘zombie lending’. [19] In March 2022, a former credit officer at Axos Bank brought a suit for wrongful termination claiming Axos bank retaliated against her attempts to raise concerns about compliance at the bank.

One specific allegation of evergreening she made related to “warehouse lending”. [ Pgs. 11-12 ] These are loans provided to “third-party mortgage companies”, per Axos’ 10-K . They get reported in a single segment along with single family mortgages and totaled $4.1 billion in value or 22% of Axos’ overall loan book, per Axos’ Q3 2024 financial statement. [ Pg. 18 ]

According to the court complaint , Axos Bank had financed approximately $45 million of loans by two third parties – Commerce Home Mortgage and A&D Mortgage. [ Pgs. 11-12 ] These loans turned stressed and were aging beyond Axos’ internal policy and the terms of the loan agreement, per the officer.

According to the complaint, instead of reporting these non-compliant loans, Axos decided to evergreen them, in what Axos called “hospital line” loans. [ Pg. 12 ] The term “hospital line” was a “reference to the status of the loans on life support” per the same complaint. [ Pg. 12 ]

Similar schemes – not correctly identifying borrower stress and giving extensions on loan maturities – were corroborated by our interviews with former employees, who called them “pretend and extend”.

One multifamily property manager told us that for defaulting borrowers, who couldn’t make loan payments or single payments to keep the loan regular, Axos would find another way:

“They’ll say I don’t have a million dollars. Okay, we’ll have to go find another solution. So, a lot of times, like I said, we call it pretend and extend. So if there’s a loan maturity, [or] the borrower can’t get a loan, we’ll give them more term.”

Part IV: Axos’ Rosy-Looking Credit Metrics Show Signs Of Distortion or Manipulation

Axos has seen a collapse in commercial real estate in its key markets. With its underwriting standards that embrace troubled borrowers and situations, and specific signs of stress with multiple portfolio properties, it is unfathomable that the company has dodged the issues in its markets.

Despite this, ratios that normally indicate stress (non-performing loans), likely stress (allowances for credit losses) or those that should track the value of underlying collateral (loan to value ratio), have remained essentially flat for Axos between 2021-2024.

The superior credit metrics and LTV figures that Axos touts imply that the bank is navigating the super-saturated and extremely competitive commercial real estate market with precision and finesse, and has resulted in investors awarding Axos a premium valuation.

We suspect these metrics to be highly misleading or manipulated.

#1 Suspect Key Credit Metric: Axos’ Disclosed Loan to Value (LTV) Ratios In Commercial Real Estate Stands At 39% As Of December 2023

This ratio is 17% less than the median average of 9 of its peers, nearly all of whom fall between 50% and 78% ltv.

A Loan to Value ratio (LTV ratio) compares the amount borrowed to the value of the underlying asset at the time of origination, per Axos’ disclosures . It is thus a measure of the bank’s security against the loan in case of a foreclosure situation. Axos discloses LTVs in commercial real estate at 39% as of December 2023, 17% lower on a median average basis to comparable peers. The majority of its peers report similar disclosures in commercial real estate of greater than 50%.

When we asked former employees about the outlier LTV ratios, their experience pointed to much higher LTVs at origination. A manager responsible for credit risk told us regarding the overall commercial book:

“On the commercial side, I think a lot of the properties were 65, 70 [LTV] at most, the low 70s, I think, if I recall correctly. The vast majority of them had significant equity.”

Another manager working within Commercial Real Estate specifically, and aware of construction and bridge loans, told us that LTVs would be between 50-60% on completed value for construction projects:

“Typically it would be like 50 to 60% LTV-ish, on the as-complete value.”

Even Axos’ website says that for construction and bridge loans, which make up 64% of the commercial real estate book, it will finance up to 70% LTVs depending on the property. [ 1 , 2 ] Since over $3.4 billion, or 59% of the commercial real estate loan book was originated before June-end 2023, most prior to the collapse of commercial real estate prices across the US, it is evident that on a current value basis, LTVs would likely be far greater than disclosed. In short, the disclosed LTV ratio, instead of offering the comfort of prudent underwriting, appears to be a red flag of potential data distortion, further supported by our findings in Part 3.

#1 Suspect Credit Metric Cnt’d: “The Bank Routinely Misrepresented The Average Loan To Value Ratio’s Of Its Loans To Investors”, Per A Former Credit Review Officer

A former Credit Officer, who left the bank in 2021, explained how Axos “routinely misrepresented” loan to value ratios to investors, per allegations in a legal complaint.

The complaint alleges that Axos bank was using differing metrics than true LTV, and that the denominator (i.e. collateral value) was overstated by using advance rates, which typically would take into account assets beyond just the property.

After highlighting numerous disclosure issues and loan malpractice with compliance teams and leaders at Axos, instead of resolving them, the former credit officer claims she was fired. [ Pg. 14 ] The case is ongoing, per legal records on PACER.

#2 Suspect Key Credit Metric: Axos’ Non-Performing Loans As A Percent Of Its Key Commercial Real Estate Category Have Stayed Almost Flat In The Last 2 Years, From 0.4% in 2021 to 0.43% in 2023 By Contrast, Large National Banks Like Wells Fargo And Bank of America Have Seen This Ratio Increase By 1.9%-2.0% In The Same Period If Non Performing Loans Increased To These Levels, It Would Mean $131 Million Additional Non Performing Loans For Axos, About 7% Of Tangible Book Value In 2023

At Axos, as is common across the industry, loans are classified as non-performing when a borrower has unpaid dues for more than 90 days.

While the reporting is designed to be prescriptive, deal structures (e.g. payments at the end of a term) and extensions in loans may ultimately delay the eventual recognition of loans as non-performing. As a result, tracking non-performing loan ratios versus industry peers or so called “bell-weather” stocks is vital. 

Over the last 2 years, since interest hikes have commenced regularly, large cap banks like Bank of America and Wells Fargo have seen non-performing loans in commercial real estate increase 3 to 4 times, or an increase of 2% and 1.9% respectively.

However, Axos’ non-performing commercial real estate loan ratio has remained essentially flat, increasing by just 0.03% despite being heavily concentrated in the most problematic markets like New York and California.

As of December 2023, this would have meant at least $131 million additional non-performing loans for Axos, about 7% of tangible book value. [21]

#3 Suspect Key Credit Metric: Provisions For Future Stress As A Percent Of The Commercial Real Estate Category (Called “Allowances”) Have Inexplicably Decreased 0.5% Since June 2021

Banks set aside money to create reserves for payments on loans that they do not expect to be paid back on. This is known as allowances for credit losses .

Such allowances are estimated and an area where banks have discretion. Axos states that:

“ management establishes an allowance for credit losses based upon its evaluation of the expected lifetime credit losses related to the amortized cost basis of loans on the balance sheet.” , per its 10-Q .

Since 2021 – the middle of the Covid pandemic — credit loss allowances as a % of Axos’ commercial real estate loan book have trended down, from 1.8% at the end of June 2021 to 1.3% at the end of December 2023, per its financial statements. [ 1 , 2 ] Axos’ CRE allowances were significantly lower than the industry average of 1.6% at the end of December 2023, per S&P Global Intelligence , the credit ratings agency. This is 1.5% lower than mega banks like Wells Fargo, who reported credit allowances of 2.8% at the end of December 2023. [22]

An increase in allowances by 1.5% in the CRE category would lead to an estimated $88 million negative impact, or 21% of on consensus net income for 2024, per Bloomberg. [23]

#3 Suspect Key Credit Metric Cnt’d: We Estimate That At Least $1.1 Billion Of CRE Loans Originated At Lower Interest Rates Will Face Renewal In The Coming Year, Testing The Sufficiency Of Provisions Axos’ Auditors Identify This As A Critical Audit Matter Given “The Judgmental And Subjective Nature” Of The Company’s Forecasts

The sufficiency of Axos CRE provisions will likely be tested in the next 12 months.

Axos discloses that it has $3.76 billion in short duration bridge and construction loans equivalent to ~64% of its commercial loan book as of March 31 st , 2024, per its investor presentation. [ Pg. 12 ] At Axos, bridge and construction loans have a maximum term of 3 years, according to its corporate website , and they are generally paid back at the end of the loan terms, sometimes akin to “ bullet loans ”.

­­When analyzing the vintage of the loan book, we note that Axos originated $1.65 billion in commercial real estate loans in fiscal year 2022 (i.e. July to June for Axos). [ Pg. 22 ] Assuming that 64% of Axos’ Commercial Real Estate book has a tenure of ~3 years (the maximum term advertised), this implies that approximately $1.1 billion of commercial real estate loans, or ~18% of that book are estimated due in the next ~12 months.

As stress in broader markets has spread and interest rates have increased by over 500bps since the Federal reserve started hiking rates in 2022, we expect provisions for credit losses to increase as a % of Axos’ commercial real estate book. Axos’ auditors have also noticed, raising credit loss allowances in Commercial Real Estate as a key audit matter and noting the “judgmental and subjective nature” of the company’s forecasts. [ F-1 ]

#3 Suspect Key Credit Metric Cnt’d: Axos’ Closest Bridge Loan Peer is Arbor Realty, Where Bridge Loans Make Up Virtually All Of Arbor’s Loan Book Between December 2021 and March 2024, Arbor’s Multifamily (A Commercial Bridge Loan Proxy) Non Performing Loans Rose From 0.02% to 4.4% Of That Portfolio By Contrast, Axos’ Non Performing Loans Have Increased By Only 0.18%, From 0.26% to 0.44% In Its Commercial Real Estate Portfolio If Axos’ Non Performing Bridge Loans Were In Line With Its Closest Peer, It Would Mean $234 Million Additional Non Performing Loans, About 12% Of Tangible Book Value

Bridge loans are short term financing loans that, as the name suggests, are used to “bridge” the gap until a borrower gets more permanent financing or it’s able to meet a particular financial obligation.

At Axos, these loans are generally less than 3 years, per its website , and are counted under the commercial real estate loan portfolio as part of what are known as “specialty” loans, likely owing to their more complex and structured nature.[ Pg. 12 ]

In total, Axos has $1.8 billion in outstanding bridge loans, representing 31% of its disclosed commercial real estate book . [ Pg. 12 ] Axos doesn’t itemize non-performing loans for bridge loans specifically. Axos’ closest listed peer in the bridge loan space is Arbor, where bridge loans account for 97% of its loan book, per its Q1 2024 report . Most of Arbor’s loans are multifamily loans, which are multi-unit properties and generally considered commercial real estate, making it a relevant proxy for Axos’ bridge loan book. Between December 2022 and March 2024, Arbor’s non-performing loans in its multifamily bridge loan portfolio have increased from 0.02% to 4.4% of its portfolio. At Axos, meanwhile, commercial real estate non-performing loans have only increased from 0.26% to 0.44%, or by just 0.18% during the period.

Despite being heavily exposed to bridge loans in its commercial real estate book, Axos would have its investors believe that it somehow has inexplicably managed to avoid the stress that other lenders like Arbor have experienced over the last 18 months.

Banking is an industry as old as Mesopotamia. Over the last 4,000 years, the industry’s participants have evolved in tandem. All U.S. banks are subject to the same federal funds rate and interest rate dynamics, and similar regulations and requirements. Competitive edges, along with comparable ratios and margins, are generally razor thin. This is especially true of banks operating in highly competitive markets like New York and California.

But, for as long as banks have been around, there have also been ‘too good to be true’ outliers that claim to have discovered the magical lending formula. In this case, we are meant to believe that Axos has selected only the best felons and troubled borrowers and properties to lend to, allowing it to waltz gracefully through the commercial real estate market minefield without so much as a scratch.

Note that we do not call into question Axos’ liquidity position or its depositor base. History has shown, however, it is almost always the case that these outliers are taking on undisclosed risk to fuel their optimistic numbers and justify their valuation premiums. We see significant downside ahead.

Appendix A 67% Of Axos’ Commercial Real Estate Loans Are Via Indirect Note Structures, Typically “A-B” Notes Former Employees Indicated This Structure Obscured Axos’ Visibility On The Underlying Borrower

The company claims to have unique structuring and exit strategies that provide them with greater protection than traditional commercial real estate lenders. But in interviews with former employees we were told about arrangements that are typical of senior secured bank loans.

In one common structure, the A-B note, Axos has less visibility into its underlying borrowers, posing added risks. An A-B note structure for commercial real estate deals involves two separate tranches “A” – the highest tranche, and “B” – the secondary tranche that is subordinate to the A (i.e. gets paid out later in a stressed scenario).

According to Axos’ investor presentation , $3.97 billion or 67% of its Commercial Real Estate category are structured via such types of indirect notes, where Axos has “ first payment priority ” (the “A tranche”).

Former employees recalled to us that Axos’ commercial real estate deals commonly used the A-B note structure. A former Commercial Real Estate manager at Axos explained:

“So if it’s a $100 million construction loan, we team up with like, Fortress Investment Group, for example. They were one of our big partners… So basically, Fortress would underwrite the whole loan, so they would underwrite the $100 million construction loan, and then we would give them an advance. Where basically it came out to around 70% of the whole loan. So, you know, they would write the check for $100 million, and then we would essentially write the check to Fortress for $70 million. Right? So really they were only in the deal [for] $30 million. And then they would have $70 million of like leverage essentially.”

Former employees explained that Axos often was reliant on the second partner (B or “junior lender”) for most of the information about certain projects. A credit analyst told us that the company relied on the B-lender for most of the due diligence during the loan cycle. Speaking about interfacing between A lender (Axos) and B lender (usually a credit fund), and how to assess the liquidity position of the underlying borrower after the loan had been originated, they noted:

“I would say a lot of the due diligence at that point would be coming from your B-lender.”

A consultant to a credit fund and junior “B” lender along with Axos in many deals noted the opaque dynamic:

“The borrower doesn’t usually know who it is [the A lender]…They’re [A lender] not interfacing directly with the borrower. And part of that is because if there is a problem, they don’t know how to fix it. A lot of these places don’t know how to fix it”

In short, while A-B structures might be typical in commercial real estate, as former employees explained, the lack of visibility, firsthand due diligence and connection to underlying borrower provides fertile ground for under-recognition of stressed assets.

Disclosure: We are short shares of Axos Financial, Inc. (NYSE: AX)

Legal disclaimer.

Use of Hindenburg Research’s research is at your own risk. In no event should Hindenburg Research or any affiliated party be liable for any direct or indirect trading losses caused by any information in this report. You further agree to do your own research and due diligence, consult your own financial, legal, and tax advisors before making any investment decision with respect to transacting in any securities covered herein. You should assume that as of the publication date of any short-biased report or letter, Hindenburg Research (possibly along with or through our members, partners, affiliates, employees, and/or consultants) along with our clients and/or investors has a short position in all stocks (and/or options of the stock) covered herein, and therefore stands to realize significant gains in the event that the price of any stock covered herein declines. Following publication of any report or letter, we intend to continue transacting in the securities covered herein, and we may be long, short, or neutral at any time hereafter regardless of our initial recommendation, conclusions, or opinions. This is not an offer to sell or a solicitation of an offer to buy any security, nor shall any security be offered or sold to any person, in any jurisdiction in which such offer would be unlawful under the securities laws of such jurisdiction. Hindenburg Research is not registered as an investment advisor in the United States or have similar registration in any other jurisdiction. To the best of our ability and belief, all information contained herein is accurate and reliable, and has been obtained from public sources we believe to be accurate and reliable, and who are not insiders or connected persons of the stock covered herein or who may otherwise owe any fiduciary duty or duty of confidentiality to the issuer. However, such information is presented “as is,” without warranty of any kind – whether express or implied. Hindenburg Research makes no representation, express or implied, as to the accuracy, timeliness, or completeness of any such information or with regard to the results to be obtained from its use. All expressions of opinion are subject to change without notice, and Hindenburg Research does not undertake to update or supplement this report or any of the information contained herein.

[1] Axos is formerly known as the Bank of Internet , or BofI.

[2] Excluding corporate/eliminations.  

[3] Loans reported net of allowance for credit losses.

[4] Note that for efficiency ratios , lower is considered better.

[5] Industry norms held by regulators, ratings agencies, and data providers generally classify multifamily real estate with more than 4 units as commercial real estate. Moody’s classifies multifamily with 4 units and above as commercial. S&P Intelligence also discussed it in relation to CRE loans and when determining credit allowances as a % of CRE loans. Multifamily is also included in Credit Rating Agencies framework for CMBS (Commercial Mortgage Backed Securities). [ Pg. 3 ] A Federal Reserve guidance note as far back as 2006 states that “CRE loans also include loans secured by multifamily property.” [ Pg. 2 ]) Industry and financial publications also note multifamily as part of CRE exposure, including NAREIT , an industry body representing real estate trusts, Altus , a provider of CRE data and news and data provider Bloomberg . Axos defines Multifamily & Commercial Mortgage as 4 units plus. Note that Axos re-organized its CRE definitions post July 2020, just after CRE started to generate negative headlines during the height of the Covid pandemic, creating a separate “multifamily and commercial mortgage” category, referencing Accounting Standard Update (ASU) 2016-13 (collectively ASC 326). [ Pg. 7 ]

[6] Even prior to Covid and the GFC, bank concentration in commercial real estate has been a concern for the Federal Reserve. A Federal Reserve guidance note from 2006 states historically that “concentrations in CRE lending coupled with weak loan underwriting and depressed CRE markets have contributed to significant credit losses in the past”. [ Pg. 1 ] More recently, Axos’ high concentration in CRE was highlighted in a February 2024 Reuters article , which looked at CRE exposures across U.S. regional banks.

[7] Total net loan book

[8] MSCI calls these assets “potential distress”, which it says “may precede full-blown financial trouble”.

[9] The loan is structured via an indirect note with a second lender, per the loan documents. [ Pg. 4 ] The end underlying borrower is ultimately the sponsor.

[10] Loan data referenced throughout this report is as of the most recently available public reports via the Automated City Register Information System “ACRIS” as of 5/31/2024. The loan amounts refer to sanctioned loan amounts and do not necessarily reflect the amount disbursed, given some loan types (e.g. construction loans) will be drawn down over time.

[11] Attached to the NY Department of Environmental Conservation cleanup application for 147-35 95 th Avenue, as Exhibit A, is a NY State Entity Operating Agreement which lists Solomon Feder as the sole member of Sutphin Boulevard Equities. [ Pg. 36 ] The signatory on the loan docs is Joel Zupnick, who has partnered with Solomon Feder on other projects .

[12] Our process for verifying that it was the same Solomon Feder: Firstly, Lexus Due Diligence shows a same-aged individual, with a connection to Big Apple Designers, named in the indictments. One of the other companies named in the indictment, Velocity Framers, is connected to JJ Weiss, a director at Key Developers, a Solomon Feder entity per Rocket Reach and media articles .

[13] Realdeal a real estate news aggregator confirmed that Baselice was the same individual from the 2010 case. 

[14] Lease agents, per CoStar.

[15] This was again through an indirect note exposure.

[16] Nightingale owns the property through its entity ONH 2226 Third Ave LLC, per media reports , references to the entity on the development’s website , and property records .

[17] Deal valuations per CoStar

[18] For example, the state of New York, has 133 lenders whose main specialization is “commercial lending”, per FDIC’s BankFind suite tool . This does not include credit unions or funds that are also engaged in lending activity. To recreate the search: go to the FDIC look-up tool , search by active institutions in the State of New York, download the data file and search the “SPECGRPN” column for  “Commercial Lending Specialization”. This yielded 133 results as of May 31, 2024.

[19] The term and the practice of “evergreening” are discussed in research papers by the International Monetary Fund (IMF), for example.

[20] Axos: Weighted Average of CRE Loan Book, OZK: All construction and development loans, CVB: across all CRE Seacoast Bank: Weighted Average Of CRE Non-Owner Occupied, S&T Bancorp: Average of Office CRE Loans, ConnectOne: Weighted Average Of CRE Loan Book, Trico Bancshares: Weighted Average of CRE Loan Book (including Multifamily), Renasant: Weighted Average Of Non-Owner Occupied Office, Customers Bancorp: Average Across Office and Retail CRE, Arbor: Total LTV including a small segment of single family which has lower LTVs.  [21] We used Bank of America (with lower non-performing loans) as the reference point.

[22] S&P also has a wider definition of commercial real estate, presumably including the multifamily category loans, which shows that Axos’ credit allowance as a % of total commercial real estate increased by 6bps in 2023, versus an industry average increase of 60bps.

[23] Consensus net income (adjusted) per Bloomberg is $420.3 million for 2024. CRE loan book is $5.9 billion in value, as of March 2024.

The Great Misalignment: Addressing the Mismatch between the Supply of Certificates and Associate’s Degrees and the Future Demand for Workers in 565 US Labor Markets

The great misalignment, addressing the mismatch between the supply of certificates and associate’s degrees and the future demand for workers in 565 us labor markets, full report, press release.

Los Angeles. Atlanta. Savannah. Pittsburgh. Hundreds of local labor markets fuel the American economy, and each one is driven by the needs of the local area’s mix of industries and the skills of its workers. When these needs and skills align, everyone can benefit. But many local economies are struggling to achieve strong alignment between the demand for middle-skills credentials (certificates and associate’s degrees) and the supply of these credentials produced by local institutions.

The Great Misalignment: Addressing the Mismatch between the Supply of Certificates and Associate’s Degrees and the Future Demand for Workers in 565 US Labor Markets demonstrates that in half of the nation’s labor markets, at least 50 percent of all middle-skills credentials would need to be granted in different fields of study to meet projected labor demand through 2031.

Source: Georgetown University Center on Education and the Workforce (GUCEW) analysis of data from the US Census Bureau, American Community Survey (ACS), 2012–21; US Department of Education, Integrated Postsecondary Education Data System (IPEDS), 2019–21; and GUCEW State Job Projections data set, 2023.

Note: Values may not sum to 100 percent due to rounding.

Middle-Skills Misalignment in 565 Local Labor Markets

The extent of middle-skills misalignment varies widely across the 565 local labor markets in the US that have at least one middle-skills credential provider. One-quarter of these local labor markets would need to redistribute up to 42 percent of the middle-skills credentials conferred locally to align the distribution of credentials with the distribution of projected job openings. Another one-quarter would need to redistribute more than 60 percent of middle-skills credentials. For example, the level of credentials-to-jobs misalignment is more than 70 percent higher in Los Angeles than in Atlanta.

Use the map to explore alignment metrics for each local area.

Source: Georgetown University Center on Education and the Workforce (GUCEW) analysis of data from the US Census Bureau, American Community Survey (ACS), 2012–21; US Department of Agriculture, Commuting Zones and Labor Market Areas, 2012; US Department of Education, Integrated Postsecondary Education Data System (IPEDS), 2019–21; and GUCEW State Job Projections data set, 2023.

Sources of Middle-Skills Misalignment

An important driver of misalignment in a local economy is the number of middle-skills providers in the area. Labor markets served by more middle-skills education and training providers tend to experience lower levels of credentials-to-jobs misalignment than those served by fewer providers. Our research suggests that this is because each provider prioritizes programs that fill different, but complementary, labor-market needs. Given that rural areas often have fewer providers, misalignment is typically higher in rural labor markets than in urban ones.

Another major source of middle-skills misalignment is the large percentage of credentials that have little or no direct connection to a specific occupational cluster. For example, 28 percent of all middle-skills credentials awarded nationwide have no direct occupational match. More than 90 percent of these credentials are granted in liberal arts, general studies, and humanities programs. While it is very likely that students pursuing credentials in these fields do so for reasons other than workforce preparation, such as anticipated transfer to a four-year institution, only 42 percent of certificate and associate’s degree completers in liberal arts, general studies, and humanities programs transfer to four-year institutions within six years. Given the risk that they will not successfully transfer to a four-year program, graduates in fields with no direct connection to an occupation may experience particularly difficult transitions from school to work.

Racial/Ethnic Disparities

Where individuals live has always affected their access to educational and professional opportunities. Most working-age adults in the US live within commuting distance of a local middle-skills provider. However, American Indian/Alaska Native adults are 3 to 18 times more likely than working-age adults in other racial/ethnic groups to live in an area without a local middle-skills provider, and living on tribal lands explains only some of this difference.

Additional racial/ethnic disparities arise when considering areas served by at least one middle-skills provider. In these areas, for example, Hispanic/Latino adults are 1.3 to 1.6 times more likely than adults of other racial/ethnic backgrounds to live in areas with poor alignment. Thus, the geography of opportunity is not providing middle-skills graduates of different racial/ethnic backgrounds with equal chances of working in jobs that align with their knowledge, training, and skills.

Note: The sample is restricted to adults ages 18–65 living in labor markets served by at least one middle-skills education or training provider.

Recommendations

Policymakers and stakeholders need to help middle-skills providers be more responsive to the needs of their students and economies by

• focusing on alignment across institutions to foster more coordination across local providers,
• improving data practices so efforts to align credential production with projected demand are well-informed,
• investing in counseling services that maximize students’ use of data when they select education and career paths, and
• improving career preparation and transfer pathways to bachelor’s degree programs in fields with no direct occupational match.

The Great Misalignment: Addressing the Mismatch between the Supply of Certificates and Associate’s Degrees and the Future Demand for Workers in 565 US Labor Markets examines the extent to which middle-skills providers are awarding credentials in fields that align with local workforce needs.

Comments are closed.

Home Resource Library Press Room Projects

About CEW Maps & Directions Reprint Permission Privacy Policy EIT accessibility

• Newsletters
• Infographics
• Press Releases
• Media Mentions
• Media Inquiries
• Explore Our ROI Rankings
• COVID-19 Research
• Good Jobs Data
• State Research

• Skip to main content
• Keyboard shortcuts for audio player

• Your Health
• Treatments & Tests
• Health Inc.
• Public Health

Shots - Health News

Fda advisors reject mdma therapy for ptsd, amid concerns over research, mdma recommendations.

A dose of MDMA. The drug has been studied as a treatment for PTSD and FDA is now considering whether to approve it. Travis Dove/for The Washington Post via Getty Images hide caption

A panel of experts advising the Food and Drug Administration on the use of the psychedelic MDMA for post-traumatic stress disorder found on Tuesday that the available evidence fails to show that the drug is effective or that its benefits outweigh its risks.

It represents a major setback for proponents of the drug and its sponsor Lykos Therapeutics, potentially jeopardizing FDA approval of the treatment.

Following public comment and discussion, the panel voted 9-2 that MDMA – in combination with talk therapy – is not effective for treating PTSD. And they voted 10-1 that the benefits of MDMA treatment don’t outweigh its risks.

While the FDA puts stock in the panel’s advice, it does not have to follow their recommendation.

However, that would be surprising given the host of concerns raised during the all-day meeting.

Remarks during the meeting from FDA staff and members of the advisory panel highlighted some major shortcomings in the clinical research. These include uncertainties and gaps in the data, unanswered questions about its potential for abuse and a lack of evidence supporting the psychological approach used in the therapy sessions.

“It seems like there are so many problems with the data,” said Melissa Barone, one of the panelists and a psychologist with the VA Maryland Health Care System. “Each one alone might be okay, but when you pile them up on top of each other...”

Many members of the panel brought up allegations that have surfaced about possible misconduct and bias in the trials that could have skewed the results.

“I have real concerns with the validity of the data and the allegations of misconduct,” said Elizabeth Joniak-Grant, a sociologist and a member of the panel. “I can't in good conscience support something where these many harms are being reported.”

Scientists with the FDA didn’t share details, but acknowledged the agency was investigating some of the claims, which have surfaced in a petition to the agency and outside reports on the trials.

Dr. Walter Dunn, a psychiatrist at UCLA, was one of the few who voted in favor of the treatment's efficacy. He acknowledged the misconduct allegations, but said ultimately the effect sizes of the treatment were large enough to indicate it can be effective for PTSD.

A big moment for psychedelics

The significance of the moment was not lost on those in attendance though.

There are only two FDA-approved treatments for PTSD and MDMA would be the first to come on the market in decades. It would also be a milestone for the broader effort to expand access to psychedelics.

“We are charting new territory,” said Kim Witczak, a consumer representative on the FDA’s advisory committee. “We want to set it up right.”

Representatives of Lykos emphasized the positive findings in clinical data collected during two nearly identical randomized controlled trials.

For example, one of those studies showed 67% of participants in the MDMA treatment arm no longer met the diagnostic criteria for PTSD following three dosing sessions with MDMA, compared to about 32% in the placebo group who underwent the therapy sessions but did not receive an active drug.

“In totality, these results support [that] MDMA in combination with psychological intervention provides significant and meaningful reductions in PTSD symptoms and functional impairment in patients with PTSD,” said Berra Yazar-Klosinski, chief scientific officer for Lykos.

FDA staff and the advisors did not dwell on those rosy results, though.

While the study took steps to “blind” study participants, there was considerable discussion around the fact many of those in the study could tell they had received the experimental drug, leading to what’s known as “functional unblinding,” which can ultimately affect the results.

“Although we do have two positive studies, the results are in the context of dramatic functional unblinding,” says Dr. David Millis, clinical reviewer for the FDA.

Another potential sticking point was the lack of data about how patients experienced the acute effects of the drug, including feelings like “euphoria” or “elevated mood.” That data helps inform the FDA’s assessments of the drug’s abuse potential.

“We noticed a striking lack of abuse-related adverse events,” said Millis, noting that the FDA had advised the study sponsors to collect this type of data.

While MDMA is currently listed as a Schedule I drug, the agency’s review found it has the same abuse potential as a Schedule II stimulant , a category that includes cocaine.

“We're actually managing more and more severe cases of MDMA overdose, and so I'm less concerned about the safety in the acute setting, but more chronically if they go on to abuse MDMA,” said Maryann Amirshahi, a professor of emergency medicine at Georgetown University and a member of the committee.

About 40% of those enrolled in the MDMA study had a history of using it prior to the study.

Alongside its positive findings on the short-term effects of MDMA, Lykos presented data from a follow-up observational study intended to suss out the staying power of the treatment.

While not yet published in a peer-reviewed journal, that data “suggest evidence of MDMA’s durability to at least six months,” said Yazar-Klosinski with Lykos.

However, the FDA staff highlighted various issues with that long-term data, including a dropout rate of 25% and the fact that some participants sought therapy and, in some cases, used illicit drugs, including MDMA.

The form of talk therapy used in the MDMA sessions also troubled some on the panel who noted there wasn’t strong data to support its use outside of the trial. “The psychological intervention is still for me a bit of a black box,” said Dr. Paul Holtzheimer with the National Center for PTSD. “What was described is really a relatively vague, ill-defined treatment.”

Hearing from patients

Some of the strongest arguments for approval came from patients who spoke during the public comment period about the urgent need for an effective treatment. Some had taken part in the MDMA trials and said they found the medication to be transformative.

“I was fortunate enough to enroll and get accepted, ultimately changing my life forever,” said Nick Brown, a military veteran from Colorado, who described how the treatment gave him self-compassion and allowed him to “get better sleep, have better relationships, and live what feels like a completely new life.”

Other groups representing veterans echoed these sentiments.

“I fear what will happen to them if this therapy is not approved,” said Jonathan Lubecky, a veteran who underwent MDMA-assisted therapy ten years ago, “Imagine how many lives your vote could save. Imagine how many will be lost if you did vote against this vital therapy .”

But for all the promising anecdotes of recovery, there were also many involved in psychedelic research who raised concerns about how the trials were run and the persistent problems in the data.

Harsh critics speak out

Some of the harshest criticism came from several researchers who had initially petitioned the FDA to hold the public hearing, including Neşe Devenot, a researcher at Johns Hopkins University, who said the model of therapy “incentivized boundary violations.”

Data shared from Lykos showed a range of adverse events.

The majority of those in the study had a history of suicidal ideation in their lifetime, but during the study period “the frequency of these symptoms was comparable between the two groups, said Dr. Alia Lilienstein, senior medical director for Lykos Therapeutics.

“Of note there were no suicidal behaviors or attempts reported in the MDMA group,” she said.

That point is particularly contentious because of recent allegations that certain adverse events were not reported. The petition calling for the advisory meeting outlined these concerns and others, citing an unnamed former employee of the drug company.

There is already a well-documented case of two therapists in the Phase 2 trials with a participant who said they engaged in inappropriate contact with her while she was under the influence of MDMA. The videos of the two therapists in bed with the participant were eventually made public by a podcast.

“Let's try to not gloss over this misconduct. It was sexual misconduct. That's particularly important,” said Joniak-Grant, a sociologist and a member of the panel.

Several other panelists asked questions about the potential that MDMA, once available widely in therapeutic settings, could lead to other instances of unethical behavior.

Last month, a report from the Institute for Clinical and Economic Review, a group that evaluates clinical data and drug prices, concluded there was insufficient evidence to assess the overall net benefit of MDMA-assisted therapy, after a lengthy investigation into the trial data.

That report stated that it’s possible those involved in the trials including therapists and investigators encouraged the reporting of positive events and downplayed adverse events.

The drug company has pushed back on the allegations and said it stands behind the data.

A public comment submitted to the FDA by one trial participant said her therapist encouraged her to view “worsening symptoms as evidence of healing and ‘spiritual awakening’” and that she and other participants later struggled with suicidality following the trial.

When asked if some participants may have been discouraged from participating in the long term durability study, Lillenstein said those claims had been investigated.

“All participants who were interested in participating were given the opportunity to review consent, and some chose not to participate after reviewing consent, but otherwise everyone was given the opportunity,” she said.

Correction June 4, 2024

An earlier version of this story said that MDMA is a Schedule III drug. In fact it is a Schedule I drug.

Outstanding Senior Spotlight: Jared Ura

• by College of Engineering Communications
• June 07, 2024

Jared Ura was inspired to major in materials science and engineering at the University of California, Davis, by his grandfather, who was also an engineer.   

“He sparked my interest in chemistry and physics at a young age, and I was always in awe of how he understood the world around him at a fundamental level,” Ura said. “He taught me how to dig deeper and understand topics in full, which continues to guide me in researching the structures and properties of novel materials at the smallest of scales.”  

At UC Davis, Ura was among the five founding members of the Davis Undergraduate Engineering Network , which focuses on fostering interdisciplinary collaboration among engineers at UC Davis. Through the club, Ura found lifelong connections and hopes that incoming engineering students will also find community with the legacy he is leaving behind.   

As Ura looks ahead to his Ph.D. program at the University of California, Irvine, he also looks fondly at the people who helped him achieve his academic goals. He appreciates Susan Gentry , associate professor of teaching in the Department of Materials Science and Engineering, for her dedication to her students and for teaching him marketable skills like machine learning for materials informatics.   

He also appreciates Erika La Plante , assistant professor of materials science and engineering. He never took any of her classes, but he joined her research group last fall to research the precipitation of magnesium-based cement, and her mentorship, Ura says, has been pivotal in his decision to pursue a graduate degree.   

“Professor La Plante has provided me with the tools and opportunities to realize my potential as an independent researcher. I'm beyond grateful that she took a chance on me as her first undergraduate researcher at UC Davis.”  

While Ura encourages new students to take advantage of UC Davis’ ample resources, apply for leadership positions in campus organizations and say “yes” to as much as possible, he emphasizes the value of taking the time to be present as well.     

“Your undergraduate years go by so much faster than you may think,” he said. “You're currently making the college memories that you'll look back on, so take the time to hang out with friends, watch the sunset and eat good food.”  

Primary Category