THEY CAN TALK THE TALK, BUT ARE THEY WALKING THE WALK?
An Outcome Evaluation of the NJLCA
Matthew Bills
Many jails, probably including your own, face a growing need for motivated employees to assume leadership roles. As “Baby Boomers” and “Generation Xers” retire, the jail’s human resources are depleted (Stinchcomb & McCampbell, 2010; Stinchcomb et al., 2009). Jail leaders hold unique and challenging responsibilities requiring competence in agency operations and sound leadership skills to oversee the persistent and emerging critical issues jails face (Castle & Martin, 2006; Lambert & Paoline, 2008; Stinchcomb & Leip, 2013).
Effective leadership neither spontaneously appears in an organization nor is it only developed through observation or practice. Leadership development programs provide future leaders opportunities to build operational knowledge, skills, and abilities (KSAs). Jail-focused, non-agency/location-specific leadership development programs are limited nationwide. The National Jail Leadership Command Academy (NJLCA) is a program for emerging professionals from across the U.S. to hone the leadership KSAs needed for future success. While informal testimonials are shared with the NJLCA about the benefits of the program, the program has not been empirically evaluated.
Current Study
No jail leadership development program has been empirically evaluated to date, even with the well-documented importance of such training (Harper, 2016; Jacobs & Cooperman, 2012; Stinchcomb & McCampbell, 2007). This study addresses this gap through an impact and outcome evaluation of the NJLCA. This work uses Kirkpatrick’s four-dimension evaluation typology (1959; 1996), a widely used model developed to illustrate the ‘value-added’ of a training program. This study’s analyses centered on the model’s levels three and four, assessing the impact the training had on a participant’s job performance and on broad organizational goals being met.
Program Overview
The NJLCA is a joint initiative of the American Jail Association (AJA) and the Correctional Management Institute of Texas (CMIT) at the George Beto Criminal Justice Center at Sam Houston State University (SHSU). In 2007, AJA’s leadership recognized a need to establish a national leadership program leading to a partnership with CMIT to develop and deliver the NJLCA at SHSU. The first class was held in March 2009, and, as of November 2022, 47 sessions were concluded (including three for Indian Country jails). Class size is generally limited to 36 participants, with no more than two participants per agency in each class. This has resulted in 1,632 graduates from 382 agencies in 45 states, Washington, DC, and 44 sovereign, tribal nations. Two graduates attended from the U.S. Virgin Islands, while one attended from Tasmania, Australia.
The NJLCA curriculum was developed in consultation with subject matter experts, leading practitioners in the field, and the expertise at SHSU. The 22 jail leadership core competencies (Stinchcomb et al., 2011) serve as the substructure of the NJLCA’s focus on developing leadership KSAs rather than enhancing subject matter expertise in jail operations. These KSAs align with what scholars and practitioners identified as vital for the organizational success of jails: competent leaders who are versed in managing administrative duties, interpersonal matters, and other aspects of facility function (Jacobs & Cooperman, 2012; Russo et al., 2018).
Methodology
Structured interviews were conducted with 49 jail professionals from 41 agencies across the U.S, a geographically- and agency size-representative sample. Analysis of interview responses using Kirkpatrick’s (1959; 1996) level three and four assessments produced a detailed description of how the NJLCA led to direct benefits to jails beyond solely providing leadership development to a staff member.
Results Kirkpatrick Level Three: Behavior
Level three of the Kirkpatrick evaluation model—Behavior—assesses whether participants use the KSAs they acquired at the training to improve their job performance. Jail leaders representing 35 agencies (85.37% of the sample) provided 89 examples of how NJLCA graduates applied the KSAs to improve their overall job performance. Such examples were sorted into seven themes (see Table 1).
Respondents from 11 agencies characterized their NJLCA graduates as “accountable,” “rising to the occasion,” and embodying “professionalism.” They “produce at a higher level of excellence” and have “fine-tuned/better leadership skills,” including “making better, more sound decisions,” and “being better problem-solvers,” traits discussed by eight jail leaders. NJLCA graduates’ “leadership skills and development,” discussed by ten respondents, are the most significant impact of graduates at six jails. According to jail leaders representing 19 jails, graduates’ improved leadership capacities also involve
“recognizing the importance of networking,” and following through on communication. NJLCA graduates’ internal and external networking is particularly impactful. Eight of the 19 jail leaders mentioned it as a specific positive impact on their jail’s daily operations, and ten consider it the most significant influence of their graduates on their agency. Graduates’ ability “to see the bigger picture and having a broader perspective” is just as impactful. Jail leaders from 17 agencies provided 26 comments classified into the ‘Perspective and Understanding’ theme, with nine respondents considering it their graduates’ most significant contribution to their jail.
The NJLCA’s content is designed to expose graduates to topics they will encounter as leaders. Further, the Academy brings together individuals from across the country who represent agencies of very different sizes, organizational structures, and internal cultures. The NJLCA’s approach and instructors require participants to actively and continually work together throughout the program, allowing attendees to learn about and from one another. Collectively, respondents provided noteworthy examples of NJLCA graduates’ improved and exceptional job-related performance.
NJLCA graduates’ actions on-the-job also demonstrate that the Academy’s messages about the importance of networking were taken seriously, which broadened graduates’ understanding of jails and leadership approaches and positively impacted their job performance. As one jail leader put it, their graduates are “better able to hold positions with major responsibilities and interpersonal oversight.” NJLCA graduates’ willingness and ability to be reliable and exemplary in their agency roles positions them well for potential promotion and advancement.
Kirkpatrick Level Four: Results
Level four of the Kirkpatrick evaluation typology (1959; 1996)—Results—assesses whether the NJLCA’s targeted outcome (to prepare jail middle managers for a successful transition into senior leadership positions by imparting KSAs needed for those roles) is occurring. Level four also examines whether NJLCA graduates are positively impacting larger organizational goals. Interview responses demonstrate that NJLCA graduates impact larger organizational goals and objectives for at least over half of the jails in the sample (23; 56.10%), doing so in four areas (see Table 1).
According to respondents representing 14 jails, graduates are heavily involved in the functioning of their facility through their “contributions to system solutions.” They are positively affecting organizational sustainment by being “involved in strategic focus groups for the future,” “establishing better succession planning,” and “starting annual goals and accomplishments at the agency level.” NJLCA graduates at nine jails are “contributing to a better culture and creating more of a community,” including in four organizations establishing mentorship programs. Graduates are also uplifting and energizing their peers, boosting their morale. This may lead to broad and substantial positive impacts on the facility, including enhancing operations and reducing employee turnover. Another valuable way NJLCA graduates are impacting larger organizational goals is through amending or developing their agency’s mission, vision, and values (MVV) statements, doing so at seven agencies. These statements offer agency direction by outlining the core aims and guiding principles, helping staff align their efforts. Given the integral role a jail’s MVV statements can play, it is arguable that jails would want these statements to be well-known and accepted by staff and the community the jail serves.
According to respondents, NJLCA graduates also beneficially influence daily jail operations. The establishment of annual agency-wide goals and accomplishments by graduates, alongside succession planning, reflects a longer-term vision and fosters staff accountability for organizational sustainment. Further, respondents indicated that graduates at six agencies push their peers to pursue professional development, including attaining Certified Jail Officer (CJO) and Certified Jail Manager (CJM) certifications, two AJA-sponsored personal accomplishments of high value. These certifications demonstrate personal commitment to excellence, indicative of jail staff wanting to advance their KSAs and arguably reflecting a desire to stay in jail employment for the long term. Jail leaders indicate that NJLCA graduates are involved in improving how their agencies function, pushing their jails toward organizational improvements. Responses show that graduates are helping ensure their jails can continue to serve their communities, both through personnel professional development and broad agency improvements.
Last, respondents from six jails specifically mentioned that their graduates were promoted and currently occupy “upper branches of staff.” A good portion of the sample of jail leaders are graduates of the Academy as well. The advancement of graduates is arguably a goal many if not all jails have for the staff they send to the NJLCA. Along with the other benefits described, evidence of NJLCA graduate advancement in agencies showcases the usefulness of the Academy in readying quality candidates for senior leadership roles.
Discussion
This impact and outcome evaluation of the National Jail Leadership Command Academy is the first empirical evaluation of any jail leadership development program and is the first to use the Kirkpatrick evaluation model (1959; 1996). Analyses demonstrated that the NJLCA has a positive, multidimensional impact on its graduates and their agencies. Respondents from 38 of the 41 agencies (92.68%) in the sample provided substantial evidence of what the Kirkpatrick evaluation model’s levels three and four assess: the degree to which training impacts job performance and the achievement of broader organizational goals.
This evaluation’s findings also demonstrate that NJLCA graduates are embodying the leadership qualities and actions that prior literature identified as beneficial for the staff they supervise (Castle & Martin, 2006; Lambert & Paoline, 2008; Stinchcomb & Leip, 2013). The interviewed jail leaders described how their NJLCA graduates work better with their peers after attending the Academy, recognize the value of quality communication, and work to lift up their staff and encourage them. NJLCA graduates at many of the facilities in the sample follow through on these desirable leadership traits not only through their words but also in their actions, demonstrating their respect and commitment to their staff and their facility.
Some comments reflected respondents’ limited noticeable impact by NJLCA graduates. These observations, however, were few in number and often included a qualifier, such as explanations for why graduates’ influence may not be noticeable, or discussion of more general impacts of their graduates that are not as directly tied to NJLCA attendance. It may be the case that NJLCA graduates
are having positive effects on those jails, but these benefits may be more subtle or are unknown to the respondent. This may also be the case if leadership had changed since the graduates attended the NJLCA.
While the NJLCA does not have a beneficial impact on all agencies who sponsored attendees, the study’s findings demonstrate the Academy’s value. This is arguably bolstered by the fact that 189 out of 382 agencies (49.48%) have sponsored staff to attend at least two different NJLCA classes (meaning they have sponsored at least three staff to attend). Based on this statistic, one conclusion might be that NJLCA graduates are having a meaningful impact at those agencies, so senior leadership sees the benefit in nominating additional staff to attend. Agencies may want to nominate more of their staff to participate but attendance is cost-prohibitive, they are deterred by the competitive application process, limited class size, class schedules, or struggle to cover positions in the jail to allow staff members to attend the NJLCA. Jail leaders from 11 agencies in the sample mentioned these barriers, saying they would sponsor more staff to attend the Academy if it were not for the obstacles identified above.
This evaluation of the NJLCA offers benefits to other jail leadership development programs. Respondents demonstrated that all 22 core competencies for jail leaders (Stinchcomb et al., 2011)—the substructure of the NJLCA’s curriculum—remain relevant. Many examples were shared of graduates embodying the individual core competencies, showing how NJLCA graduates enhance their job performance, develop and enact programs/changes in their jails, and accomplish organizational goals. This evaluation documents that when such programs are designed with the core competencies as the foundation, they can produce well-equipped staff ready for senior leadership responsibilities.
Limitations
The current study has limitations. It is likely the case that the sample is not perfectly representative of all agencies whose
employees attended NJLCA. Three eligibility criteria limited the sampling frame. Only 64 of the 283 agencies which sponsored at least three individuals to attend the NJLCA (at the time the evaluation was conducted, excluding Indian Country jails) were eligible for inclusion in the sample, and jail leaders from forty-one of those 64 agencies (64.06%) were ultimately interviewed. The voluntary nature of participating in the interview may have also introduced bias in the data gathered. For example, respondents from agencies that have very positive experiences with the NJLCA may have been more motivated to participate in an interview than individuals from agencies with a negative experience with the NJLCA.
While this evaluation methodology attempted to mitigate the chance of external influences on the findings, this evaluation did not fully account for the potential of all external or internal factors that could explain the actions and benefits of NJLCA graduates that were shared by the jail leaders. It is entirely possible that there are alternative reasons why the NJLCA graduates at the jails in the sample are demonstrating high-quality job performance and helping their agencies work toward and achieve broad goals and improvements.
Related to this, one reason NJLCA graduates may be excelling is that they were already identified as potential future leaders, and their agency’s leadership had confidence in them and high expectations upon their return. Multiple respondents indicated their agencies have their own process for selecting staff to apply to attend the NJLCA, including a focus on sending emerging leaders. Given the cost of sponsoring someone to attend the NJLCA ($1,760-$1,960 plus transportation costs), and the strained budgets jails endure (Russo et al., 2018; Turney & Conner, 2019), it can be speculated that agencies do not usually have funding to sponsor their lower-performing staff to attend the NJLCA. Vetting of potential applicants by agencies (and then by the NJLCA) likely leads to the high-performing and highly motivated employees attending, staff who are more likely to excel and effect positive change after attending the NJLCA.
That said, while NJLCA graduates may have been in management or leadership positions in their agency, it does not guarantee their job performance either stays high-quality or improves, or that they will help their agency work toward its broad organizational goals. It remains up to individuals to succeed and excel.
Conclusion
This evaluation determined through structured interviews with jail leaders that the NJLCA has a positive, multidimensional impact on its graduates, their job performance, and their jails. Graduates’ investment in themselves, their peers, and their jail aids their agency in pursuing organizational sustainment by creating stronger staff culture, enhancing future planning, and developing policies and initiatives that unify staff efforts to ensure that their facility can continue to serve its community.
The NJLCA has garnered prestige among jail leaders through its instructional format and quality, informed curriculum delivered by subject matter experts and interaction among participants. NJLCA facilitators are constantly working to update the content and the learning experiences. This evaluation provides feedback that benefits this endeavor. Jail leaders’ responses are equally beneficial to those who offer other jail leadership development programs. Given the importance of quality leadership development programs for jail staff (Harper, 2016; Jacobs & Cooperman, 2012; Stinchcomb & McCampbell, 2007), programs should be regularly evaluated in broader terms. Whether conducted in-house or by an external evaluation team, constant review and assessment will help all jail leadership development programs monitor their training delivery and determine if intended outcomes are occurring. Importantly, evaluations need to go beyond surface-level assessments, such as just relying upon opinion surveys or knowledge exams to assess program effectiveness.
Interviews with jail leaders, paired with the Kirkpatrick evaluation model (1959; 1996) allowed a structured, detailed narrative of the NJLCA’s impact on its graduates and their agencies.
This is notable, as the ‘value-added’ from training—such as the application of training curriculum to improve job performance and organizational impact—can be difficult to quantify. The Kirkpatrick model’s malleability suits the aims of different evaluation types and leadership development programs. Future leadership development program evaluations and any jail-focused training program evaluation, for that matter, should consider following this approach.
Jails are currently facing and will encounter substantial leadership vacuums as the “Baby Boomers” and “Generation Xers” begin to retire en masse (Stinchcomb & McCampbell, 2010; Stinchcomb et al., 2009). This was reaffirmed by respondents from nine jails in the sample explicitly mentioning substantial leadership vacancies caused by retirements, an occurrence exacerbated by the COVID-19 pandemic. Jails need to address this wave of leadership vacancies head-on if they desire to function effectively. All steps should be taken to ensure that jails can succeed in the integral role they play for their communities. The NJLCA will continue to assist in helping jails across the country, especially if it continues to adapt to the needs of its attending jail leaders, as it has since it began.
__________________________________________
Matthew A. Bills, Ph.D., is the Research Associate for the Correctional Management Institute of Texas, where he helps CMIT pursue and complete research projects and program evaluations. Dr. Bills also works with the Specialty Court Resource Center in its endeavors to provide materials, resources, program reviews, and assistance to Texas specialty courts. He earned his degrees in Criminal Justice and Criminology from Sam Houston State University. His research has appeared in numerous academic journals. For more information, he can be reached at mab010@shsu.eduw.
References
Castle, T.L., & Martin, J.S. (2006). Occupational hazard: Predictors of stress among jail correctional officers. American Journal of Criminal Justice, 31(1), 65-80.
Harper, D.S. (2016). Correctional executives’ leadership self-efficacy and their perspectives of emotional intelligence. American Journal of Criminal Justice, 41(4), 765-779.
Jacobs, J.B., & Cooperman, K.T. (2012). A proposed national corrections college. New England Journal on Criminal and Civil Confinement, 38(1), 57-99.
Kirkpatrick, D.L. (1996). Great ideas revisited. Training and Development, 50(1), 54-59.
Kirkpatrick, D.L. (1959). Techniques for evaluating training programs. Journal of the American Society of Training Directors, 11(1), 21-26.
Lambert, E.G., & Paoline, E.A. (2008). The influence of individual, job, and organizational characteristics on correctional staff job stress, job satisfaction, and organizational commitment. Criminal Justice Review, 33(4), 541-564.
Lurigio, A.J. (2016). Jails in the United States: The “old-new” frontier in American corrections. The Prison Journal, 96(1), 3-9.
Russo, J., Woods, D., Drake, G.B., & Jackson, B.A. (2018). Building a high-quality correctional workforce: Identifying challenges and needs. Rand Corporation. https://www.rand.org/content/dam/rand/pubs/research_reports/RR2300/RR2386/RAND_RR2386.pdf
Steadman, H.J., Osher, F.C., Robbins, P.C., Case, B., & Samuels, S. (2009). Prevalence of serious mental illness among jail inmates. Psychiatric Services, 60(6), 761-765.
Stinchcomb, J. B., & Leip, L. A. (2013). Expanding the literature on job satisfaction in corrections: A national study of jail employees. Criminal Justice and Behavior, 40(11), 1209-1227.
Stinchcomb, J.B., & McCampbell, S.W. (2007). Jail leaders speak: Current and future challenges to jail administration and operation. Center for Innovative Public Policies, Incorporated. http://www.cipp.org/uploads/3/7/5/7/37578255/focusgroupreport.pdf
Stinchcomb, J.B., & McCampbell, S.W. (2010). Succession planning: A leader’s most lasting legacy. American Jails, 24(2), 9-12.
Stinchcomb, J.B., McCampbell, S.W., & Leip, L. (2009). The future is now: Recruiting,retaining, and developing the 21st century jail workforce. Center for Innovative Public Policies, Incorporated. http://www.cipp.org/the-future-is-now.html
Stinchcomb, J.B., Smith, C.J., McCampbell, S.W., & Mancini, C. (2011). National Jail Succession Planning and Leadership Development Project: Identifying core competencies and required knowledge, skills, and abilities (KSAs) for jail leaders. Center for Innovative Public Policies, Incorporated. http://www.cipp.org/uploads/3/7/5/7/37578255/final_6.6.11bja-jails_mentoring-core_competencies.pdf