My name is Yasmeen Krameddine and I graduated with a PhD in Psychiatry and have been developing in person and online de-escalation and mental health training for police since 2011. I work with law enforcement as well as other front-line organizations where I create, evaluate, and implement in-person and online training programs in de-escalation. I have made it my priority to keep police and the community members they interact with, safe and I do this by continuing to train groups in de-escalation, in addition to keeping up with the current research and best practices for de-escalation training.
What is Evidence Based Training?
As many of you know, evidence-based practices are ones that use current scientific evidence in making decisions about how to treat a patient (medical) or in our case, how to train an officer. Evidence-based practices for policing is defined as a research approach that evaluates policies and programs through scientific analysis to determine “what works” (Sherman, 2013). An example is to test training to see if it is making a difference in the areas you want to see changed. For de-escalation training, the main goal that we want to see is that the training is making lasting changes, such as decreases in:
- Physical force
- Weapon force
- Injuries
- Fatalities
- Complaints
The evaluation of training requires measurements taken both before and after training. Measurements of effectiveness after training should take place 3 months or more after training so long-term changes can be determined.
If use of force goes down significantly, we can state that the training is evidence based. If not, we can modify the training and re-evaluate it. If police groups already have a de-escalation program that have not been evaluated, there is an opportunity to tag-team with a local university to help evaluate the training that has been developed. Training that is not evidence based may be a waste of financial resources, because you might not know it the training is making a difference. In some cases, training can even make existing problems worse. You can tell if training is an evidence based program by searching for publications about the training that show statistically significant positive changes. When determining if a program is evidence-based, it is best to stay away from magazine or online articles that have no references and to view peer-reviewed clinical articles from reputable sources instead.
What are the differences between effective and ineffective evaluation?
As mentioned previously, it is important to evaluate training, but when evaluating we need to make sure we are looking at the right outcomes.
Ineffective evaluation can look like a few things:
- Completing training, and then proclaiming that by simply completing training you have met expectations regardless of whether training adds or removes value from officer outcomes.
- Stating that training was successful because everyone liked the training or basing it off of attitudes towards specific groups. E.g., Surveying members after training and asking them how they felt training went, such as asking them if they liked the training and if they felt they learned anything.
Training police in mental health and de-escalation should look at their stigma towards mental illness before and after training. It is often assumed if stigma goes down, that training is successful. I even thought this was the case until I conducted some research. During my PhD, I delivered training to over 600 police officers over 19 days and after we evaluated the training, we looked at their behaviours and attitudes in and towards mental health calls. We sent out surveys to gauge stigma before and after training and found that attitudes towards mental illness after training either stayed the same or got worse. That’s right, they got worse. We also looked at more direct measures, such as use of force, and found that it went down (physical force by 41% and weapon force by 26%), complaints went down by 20%, and confidence levels went up by 23%, in as long as 6 months after training. This data shows why it is so important to know the relationship between attitudes and behaviours is complex and one does not necessarily lead to changes in the other. For instance, if we send out attitude or stigma surveys, we need to make sure that we are looking at more than one aspect of training and not simply attitudes in solitude.
Time frame is also important; it is important to not just look at changes the same day, or one day after training takes place with no further follow-up. We need to make sure follow up occurs as much as 3 months or 6 months after.
Effective evaluation can look like a few things:
- Evaluating training by looking at behavioural outcomes. There are 5 outcome measure recommendations, but every organization might differ in their outcome measures depending on the data they capture, such as:
- Use of force
- Injuries
- Complaints
- Self-report of officer behaviour
- Self-report is more accurate than getting someone else to evaluate officer behaviour. In my study, I used both self-report and supervisor report, and they both showed statistically significant improvements, but please note that the research shows that self-report is a better predictor of behaviour than supervisor report.
- Communication occurrences, public trust surveys, etc.
Evidence-Based Training Evaluations
Dr. Yasmeen Krameddine
Effective | Ineffective |
Evaluating training Behaviour outcomes Outcomes:
| Completing training Attitude outcomes (alone) Outcomes:
|
To better understand the evaluative process and data measures see Krameddine & Silverstone, 2015: https://www.frontiersin.org/articles/10.3389/fpsyt.2014.00186/full
What are some other factors that have been found to be effective for de-Escalation training?
In-person training using actors:
It creates a realistic environment that is intellectually and emotionally stimulating. Although there is limited research conducted on police training, comparing lecture training versus live action training, the medical field has studied this more extensively. The medical and policing fields are similar in that they both have 2 distinct characteristics: they are lifesaving, and they both are under time pressures.
A recent meta-analysis (a study that comments on a group of studies) showed that actors and scenarios are effective in training (Williams & Song, 2016) and that having actors are better than lecture training.
Actors or police actors:
Using actors is favourable over police actors since actors have an arm’s length relationship with officers, and have been found to give more effective feedback, and portray the character with high quality and experience. However, Police actors can be used if they are highly skilled actors that are comfortable and confident in giving effective feedback to other officers. They must not be in fear of getting in trouble or stepping on someone’s toes.
Timeframe:
Training is suggested to be refreshed at least every 3 years because of the evidence available describing a decrease in memory and skill retention after this time period (Avisar, Shiyovich, Aharonson-Daniel, & Nesher, 2013; Grześkowiak et al., 2006; McKenna & Glendon, 1985; Nicol, Carr, Cleary, & Celenza, 2011).
Refreshers, in-person or online:
Refresher training can be either in-person or online. This recommendation has been created with the understanding that time is valuable and limited with increasing amounts of training needed in every organization.
If going the online route, make sure the training is engaging and realistic. We have put our training online and have made it realistic where we use in person video and a choose your adventure type training. The meter on the side goes up if the incorrect choice is chosen, and it goes down when you choose the option that de-escalates the individual. so it incorporates gamification into the training. There are many ways to make online training engaging and interactive, and this is needed especially if you are teaching “skills” online.
This is a screen shot from our online training, what is exciting is that there are groups not only in Canada and the USA that participate in our online training, but we translated it into Dutch and it was delivered to the Dutch National Police. For more information about this online program, please go to protraining.com
Length of refresher:
The length of the refresher must be at least 90-120 minutes (adults can follow with minimal attention lapses) with a quick break every 20 minutes since attention spans are re-set every 20 minutes (Cornish & Dukette, 2009).
As mentioned throughout this post, evidence-based training and evidence informed concepts are crucial for lasting changes. Additionally, effective evaluation and continued training is necessary for lasting improvements.
For more information, or to sign up for evidence-based de-escalation training, contact protraining@weknowtraining.ca
Yasmeen Krameddine, PhD
Expert: Mental health and De-escalation Training
References:
Avisar, L., Shiyovich, A., Aharonson-Daniel, L., & Nesher, L. (2013). Cardiopulmonary resuscitation skills retention and self-confidence of preclinical medical students. The Israel Medical Association Journal : IMAJ, 15(10), 622–7. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/24266089
Cornish, M. D., & Dukette, D. (2009). The essential 20: Twenty components of an excellent health care team. Retrieved from https://books.google.ca/books?hl=en&lr=&id=sf48ooRs7hgC&oi=fnd&pg=PA3&dq=Dianne+Dukette+and+David+Cornish+(2009)&ots=5niFIXvqEQ&sig=xjcyVP0QwOaW4IR6J_TTtpvfgyM
Grześkowiak, M., Bűrgi, H., Holmberg, S., Herlitz, J., Pichlmayr, I., Macfarlane, P. W., & Kellermann, A. (2006). The effects of teaching basic cardiopulmonary resuscitation—A comparison between first and sixth year medical students. Resuscitation, 68(3), 391–397. https://doi.org/10.1016/j.resuscitation.2005.07.017
Krameddine, Y. I., & Silverstone, P. H. (2015). How to improve interactions between police and
the mentally ill. Frontiers in Psychiatry, 5 (186) 1-5.
McKenna, S. P., & Glendon, A. I. (1985). Occupational first aid training: Decay in cardiopulmonary resuscitation (CPR) skills. Journal of Occupational Psychology, 58(2), 109–117. https://doi.org/10.1111/j.2044-8325.1985.tb00186.x
Nicol, P., Carr, S., Cleary, G., & Celenza, A. (2011). Retention into internship of resuscitation skills learned in a medical student resuscitation program incorporating an Immediate Life Support course. Resuscitation, 82(1), 45–50. https://doi.org/10.1016/j.resuscitation.2010.08.035
Sherman, L. W. (2013). The Rise of Evidence-Based Policing: Targeting, Testing, and Tracking. Crime and Justice, 42(1), 377–451. https://doi.org/10.1086/670819
Williams, B., & Song, J. J. Y. (2016). Are simulated patients effective in facilitating development of clinical competence for healthcare students? A scoping review. Advances in Simulation, 1(1), 6. https://doi.org/10.1186/s41077-016-0006-1
