Empirical vs. Non-empirical Resilience Strategies – Outcomes and ConsequencesNo Opinions
What is resilience? Resilience is your ability to adapt to and overcome negative emotional responses in a given situation. Or is it? A general academic/peer-reviewed search of resilience results in hundreds of thousands of articles. Results are even higher among non-peer-reviewed articles. So which one has the true definition of resilience? The answer varies.
In recent years, resilience has received a lot of attention among athletes and coaches trying to gain a competitive edge. Unfortunately, due to the saturated interest of both academic and non-academic sources, a common definition of resilience has not yet been established. Some individuals have based the definition of resilience on personal experience, while others have based it on varied empirical evidence. As a result, athletes may not be getting the information they need that will enable them to excel. For the purpose of this review, resilience will be broken up into five categories: 1) resilience in empirical based approaches, 2) an example of performance outcomes based on an empirical approach 3) resilience in non-empirical based approaches, 4) an example of performance outcomes based on a non-empirical approach, and 5) things to consider when looking at general research articles. By the end of this article, one should be about to research and define resilience based on what is applicable to them and supported by empirical based/peer-reviewed research.
Resilience in Empirical Based Approaches
Empirical means the way in which one can measure an outcome that has both validity and reliability. Validity, in simplistic terms, refers to whether or not researchers are measuring what they intend to measure. Reliability is also known as consistency over time (e.g., your height/weight throughout the day). Both are equally important in research and a researcher’s ability to measure an outcome determines whether or not something may be effective or ineffective.
Generally speaking, empirical research defines resilience as one’s ability to overcome cognitive obstacles (e.g., stress, negative self-talk) and maintain composure during high stress activities. Multiple factors have been identified and linked to outcome performance related to resilience. These include, but are not limited to: determination, confidence, spirituality, and one’s ability to adapt (Gonzalez, Moore, Newton, & Galli, 2016). In order to assess these components and their impact on performance, researches have been seeking new and innovative ways of measurement. One of the more well-known measures of assessment is the Connor-Davidson Resilience Scale (CD-RISC), a multi-faceted scale which utilizes self-report measures related to performance. While self-report scales have limitations for clinical application, they provide valuable information regarding how the individual perceives performance influences. That is to say, coaches can take these assessments and compare them to other athletes in order to develop a performance plan that works for the team.
An alternative measure of resiliency is the Characteristics of Resilience in Sports Teams (CREST). CREST has shown to have increased reliability between players and between teams (Decroos et al., 2017). In other words, CREST is a valuable tool for those who wish to compare the results of one team vs. another. Similar to CD-RISC, CREST assesses multiple facets of resilience. However, unlike CD-RISC, CREST utilizes team measures (e.g., ‘the team shared a common goal’). These types of measures enable researchers to not only look at the individual and his/her success, but it enables researchers to look at the team as a whole and make predictions of success based on measurable values. These values can then be utilized to help foster further team development and future performance.
The CREST and CD-RISC are just two are a wide variety of assessments currently being used by professionals to predict performance. These assessments can help provide a valuable foundation from which to build a successful team environment for success.
Performance Outcomes Utilizing an Empirical Approach
Decroos et al. (2017) assessed 1,225 athletes across 4 separate studies and revealed that CREST helps to identify performance outcomes on numerous measurable scales. Of the scales, the most significant (p < .01) revealed that not only is a team’s ability to display resilient characteristics important, but individual acknowledgement of vulnerability may actually improve long term performance and adaptation. Based on this type of evidence, the CREST assessment may be a great way to improve team communication, synchrony and performance.
Resilience in Non-empirical Based Approaches
Non-empirical approaches, while valuable on a ‘personal belief’ or ‘common sense’ level, are immeasurable. Therefore, non-empirical sources (e.g., non-cited media reports, non-peer reviewed articles, blogs) should be viewed with caution.
Currently, there are hundreds of thousands of non-empirical ‘research’ articles related to resilience. This over-saturation of ‘research’ has the potential to not only present non-factual information, it also runs the risk of harming others. Let’s look at an example:
Title: “Improve your performance with these 3 simple tricks”
First and foremost, with a title like this, one should be hesitant. In research, there is no definitive way of saying one thing will produce another. The phrase ‘correlation does not equal causation’ is a rule that researchers know very well and work hard to avoid when writing up their study results.
Content: Researchers from highly recognized US institutions have found that if you don’t eat meat, you have a significantly longer lifespan (no source)!
Second, this statement is definitive in the sense that is states, if you do x…the result is y. Remember, correlation does not equal causation. Furthermore, there is no source from which to check this statement. What if there is a source? If a source is provided, you can use any common search engine to attempt to find it. More common than not, results from studies will have a results and limitations section from which to draw conclusions from. This is where one can see the difference between: 1) ‘this study helps show that meat may or may not be a factor in longevity, but other factors such as lifestyle, career, and family support should be considered moving forward’, and 2) ‘eating meat decreases your life span’.
Another common non-empirical way of presenting information is through general literature reviews. While literature reviews are a great way of looking at the literature related to a specific topic, more common than not, they fail to break down empirical research in their entirety. This has the potential to have individuals make definitive and non-evidence based statements centered on personal belief rather than measurable statistics.
Performance Outcomes Based on Non-empirical Approach (Consequences)
A good example of some of the fallout related to a non-empirical/non-peer reviewed research method was the creation of the US Army Comprehensive Soldier Fitness (CSF) program in 2009. In 2012, psychologists revealed that the US Army was utilizing a resiliency program which was developed on proven principles from other fields (e.g., academic, sports, business) and combined to make one large resilience strategy. While acceptable at face value, further investigation revealed the resiliency approaches had not gone under any type of combined controlled experiment. As a result, the impact of resiliency training was called under question across the research community.
In 2015, a follow-up report revealed that of the over half of the 400,000 US Army service members who took the US Army resiliency assessment were below the positive resiliency threshold. While a general search of CSF in peer-reviewed journals from 2012 – 2017 reveal numerous review articles of the structure of CSF, no articles were shown to have any empirical support. It is important to note that while no studies in this search revealed CSF to be effective, no articles were found to prove that it is ineffective either. Still, psychologists point to rising numbers of suicide and PTSD rates among personnel in recent years as considerations for future CSF effectiveness research (Griffith & Bryan, 2015). This type of resilience program not only casts doubt on future programs created by the organization, but it also runs the risk of putting others at risk for increased harm and/or decreased operational performance
Other Things to Consider Regarding Articles
The following are some general questions to consider when looking at articles of interest. While this is a basic list and professionals go through an extensive amount of training to help identify article origin and application, this list can help others who are not as familiar draw their own conclusions based on critical thinking.
- Is it empirical?
- Is it peer-reviewed?
- Are the limitations listed?
- Are there definitive statements that fall into ‘correlation = causation’?
- Has there been an extensive literature review prior to the article?
- Are there sources listed?
- Has the study been conducted in a controllable environment that measures as many possible outcomes as possible?
- What are the validity and reliability measures?
Empirical/peer-reviewed approaches are the best way to quantifiably measure and state a claim. While empirical evidence is helpful, it is not considered definitive. Not all empirical evidence is conducted the same, and therefore a certain amount of skepticism can be held based on how the study was carried out. However, when it comes to the non-empirical/non peer-reviewed approaches, high amounts of skepticism should be used regardless of how big or recognized the organization is. Large amounts of report saturation may inhibit future research due to some of the controversy surrounding background research and implementation plans. And lastly, use your judgment. If something isn’t backed up by numbers that are cited and credible, it is most likely that the article is making exaggerated claims.
As coaches and athletes, we are responsible for the well-being of each other. By knowing basic research skills we can assist in the development and implementation of proven performance strategies. This will enable us to have better confidence going into future competition and create an environment that is highly adaptable, measurable, and successful.
Decroos, S., Lines, R., Morgan, P., Fletcher, D…& Vande Broek, G. (2017). Development and validation of the characteristics of resilience in sports teams inventory. Sport, Exercise, and Performance Psychology, 6(2), 158-178. doi: 10.1037/spy0000089
Fletcher, D. & Sarkar, M. (2012). A grounded theory of psychological resilience in olympic champions. Psychology of Sport and Exercise, 13, 669-678. doi: 10.1016/j-psychsport.2012.04.007
Gonzalez, S., Moore, E., Newton, M., & Galli, N. (2016). Validity and reliability of the connor-davidson resilience scale (CD-RISC) in competitive sport. Psychology of Sport and Exercise, 23, 31-39. doi: 10.1016/j-psychsport.2015.10.005
Griffith, J., & Bryan, C. (2016). Suicides in the U.S. military: birth cohort vulnerability and the all-volunteer force. Armed Forces & Society, 42(3) 483-500. doi: 10.1177/0095327X15614552
McCarthy, M. (2014). Suicide rates double among US soldiers between 2004 and 2009, research shows. BMJ: British Medical Journal (Online), 348, doi: 10.1136/bmj.g1987
Radomski, & M., Brininger, T. (2014). Occupational therapy for service member and veteran recovery, resilience, and reintegration: opportunities for societal contribution and professional transformation. American Journal of Occupational Therapy, 68, 379-380. doi: 10.5014/ajot.2014.013060
Wagstaff, C., & Leach, J. (2015). The value of strength-based approaches in SERE and sport psychology. Military Psychology, 27(2), 65-84. doi: 10.1037/mil0000066
Buy and download up to 400 infographics!Buy infographics
About Michael Mellinger
Michael Mellinger serves as a Study Coordinator for the world's largest concussion study - the Concussion Assessment Research and Education (CARE) study - at the United States Naval Academy (USNA). He performs both military and sport psychology related consulting specific to cognitive performance enhancement strategies. Contact 'firstname.lastname@example.org' for questions surrounding his work and/or consulting.