CAEP Accountability Measures

The Bellarmine University Annsley Frazier Thornton School of Education (AFTSOE) prioritizes the use of data as part of its assessment and continuous improvement process. The data provided below represents a snapshot summary of survey reports, state and school district data reports, and teacher candidate performance assessment data. Data are acquired from sources including Jefferson County Public Schools (JCPS); the Kentucky Education Professional Standards Board (EPSB); Educational Testing Service (ETS); Bellarmine Career Development Center and the Bellarmine University Office of Institutional Research and Effectiveness. We have provided links to these data to assist prospective and current students, faculty, accrediting bodies, the public, and researchers.

The following programs were reviewed and approved by CAEP during our Fall 2019 accreditation visit:

  • Bachelor of Arts (B.A.) in Education
  • Master of Arts in Teaching (MAT)
  • Master of Arts in Education (MAEd)
  • Doctor of Education in K-12 District Leadership (Ed.D.)
  • Educational Specialist degree in Instructional Leadership and School Administration (Ed.S.)
  • Rank I individualized program in Curriculum and Instruction

Measures of Program Impact

Measure 1: (Initial) Completer Effectiveness and Impact on P-12 Learning and Development (CAEP Component R4.1)

Measure 1 (Initial): Completer Effectiveness (R4.1) 

The greatest number of our completers obtain work in the Jefferson County Public School (JCPS) system. To understand the teaching effectiveness of our 1st-3rd year completers, we examine data from various sources within JCPS.  To surmise the impact our 1st-3rd year teachers have on P-12 learning and development, we examine student growth in reading and math. District and state data measures have stabilized since the COVID 19 pandemic, allowing comparisons across cycles.  To surmise our 1st-3rd year completers effectiveness in applying professional knowledge, skills, and dispositions, we examine the results of a statewide survey: the Kentucky Impact Survey at district and school levels.  The Kentucky Impact Survey is administered every other year, creating a challenge when comparing data across cycles.  

Contribution to K-12 Student Learning Growth 

To infer completer impact on K-12 student learning, we compare data reflecting the performance of students taught by our 1st-3rd year completers with the growth of early career teachers in the district. Because Bellarmine sample sizes do not meet threshold for science and social studies, we limited to reviewing data in reading and math. For cycle 1 (2020-2021) and cycle 2 (2021-2022), the district provided scale scores on the Kentucky State Assessment in reading and math. Cycle 3 (2022-2023) data reporting scale scores on the Kentucky State Assessment in reading and math will be available in September of 2024. These data are delayed due to a district process that requires an MoU and utilizes the slower pace of summer to accommodate district workflow.   

According to the two cycles of data available, our completers perform at similar levels to other early career teachers with reading scores trending better than math scores. Within cycle 1, elementary reading scores approximate district average while math scores fall a bit below this mean. Middle school reading and math scores approximate the district means and high school math also approximates the district mean. The sample size for high school reading did not meet the required threshold. Within cycle 2, reading scores at elementary and high school indicate Bellarmine early completers are more successful than other early career teachers throughout the district. Middle school reading scores fall just below the district mean with fewer than four years’ experience. Overall district means exceed Bellarmine completers by one to two points in math, indicating an area for continued improvement. We continue to monitor these data with the goal of comparing data across cycles.

Completer Effectiveness in Applying Professional Knowledge, Skills, and Dispositions 

We examined the Kentucky Impact Survey for two components: Educating all Students and Managing Student Behavior. This survey is administered every two years, reflecting completer effectiveness for cycle 2 (2021-2022). There were no data available for cycle 1 (2020-2021) or cycle 3 (2022-2023). Cycle 2 data below reflect comparisons between district level data and elementary school data where our early career educators teach. For cycle 2, we were able to identify two schools where three or more of our teachers work and examined data for early career teachers within these schools. These schools present two cases through which we can consider the effectiveness of our early completers.  

Cycle 2 (2021-2022) data provide partial a snapshot of effectiveness for early completers. At schools with notable numbers of Bellarmine early completers, our completers perform consistently with early career teachers across the district when it comes to educating all students. Managing student behavior is an area for growth. While these data are generally low, indicating a challenge for most new teachers, our early completers performed notably lower than the district average with one school reporting 25% lower and the other reporting 16% lower. Given that this is only one cycle of data and the small sample size, these data are not robust enough to warrant programmatic change. We will attend to this domain as additional data are collected. 

Measure 2: (Initial and Advanced) Satisfaction of Employers Stakeholder Involvement (CAEP Components R4.2, R5.3, and RA4.1)

Employer Satisfaction

Employer surveys assess satisfaction for both initial certification completers and advance practice completers across programs. Data collection across cycles is more robust for initial certification candidates. Advanced programs data are less robust with smaller sample sizes in cycle 1 (2020-2021) and cycle 2 (2021-2022). Cycle 3 (2022-2023) advanced programs data failed to meet threshold. Overall, employers report satisfaction with Bellarmine graduates based on previous/current experience and recommend Bellarmine graduates to other school administrators. 

At the initial certification level, all three data indicate overall employer satisfaction with specific survey items aligned to InTASC standards. Cycle 2 (2021-2022) data offer comparisons both statewide and with other private institutions across Kentucky. Cycle 2 data are the result of a statewide survey piloted through the collaboration of Kentucky Association of Colleges for Teacher Education (KACTE) and the Evaluation Center at the University of Kentucky. Cycles 1 (2020-2021) and 3 (2021-2022) provide stand-alone snapshots. While the instruments across these three cycles vary, each was aligned with the Kentucky Teacher Performance Standards/InTASC Standards. This alignment allows for interpretation across cycles.  

At the advanced level, data indicate overall employer satisfaction with specific survey items tied to specific professional association standards as relevant to individual fields in cycles 1 (2020-2021) and 2 (2021-2022). Data are discussed in the aggregate before being discussed by relevant standard. We will continue to monitor overall employer satisfaction as well as employer satisfaction in relation to specific program standards (e.g. InTASC Standards, PSEL Standards, ILA Standards). Furthermore, we will continue to recruit higher response rates for advanced program employers. The majority of these employers respond to surveys for our initial certification program completers. The addition of the survey for advanced program completers may have felt duplicative to them. 

Initial Certification

Employers report positive satisfaction levels of Bellarmine completers across all three cycles of data and across all survey domains. 

Specifically, cycle 1 (2020-2021) survey results indicate that Bellarmine completers demonstrate subject matter knowledge in their area of licensure and academic knowledge in their field. Highest satisfaction items include 100% of employers agreeing completers implement developmentally appropriate instruction, build inclusive learning environments, and support collaborative learning for students. Just over 85% of employers agree completers implement rigorous instruction, use multiple learning strategies, employ diverse perspectives, implement meaningful learning experiences, promote critical thinking, allow for content mastery, use multiple assessment methods, and engage in meaningful professional development. Seventy-five percent of employers agree that completers seek appropriate professional leadership roles. 

Cycle 2 (2021-2022) data also indicate that employers believe early completers demonstrate proficiency with the InTASC standards through their instructional practices. Completers scored 100% satisfaction in demonstrating the central concepts of their discipline, planning instruction to meet all learner needs, and seeking opportunities to collaborate to support student learning. All other domains scored 75% satisfaction, with Bellarmine graduates outperforming completers from other private institutions in Kentucky in ten out of twelve domains. Cycle 3 data (2022-2023) are consistent with previous cycles with the majority of employers scoring completers as accomplished or exemplary in every domain. Qualitative comments indicated a desire for students to improve classroom management, particularly in Title 1 schools, and support for multilingual learners. Overall, employers find early completers to be effective classroom teachers. We will continue to monitor the data with a focus on these areas for growth.

Advanced Certification

Employer satisfaction surveys for advanced program graduates ask employers to report candidate preparation in relation to specific program association standards. The Teacher Leader program is aligned to the Teacher Leader Model Standards. The Principal Preparation program is aligned to the Professional Standards of Educational Leaders (PSEL). The Literacy Leader program is aligned with the standards set forth by the International Literacy Association (ILA).  Because our completers reflect a small sample size, response rates per program are small. These data will continue to be monitored across cycles. 

Teacher Leader Model Standards

Across the two available cycles of data (2020-2021 & 2021-2022), employers indicated high rates of satisfaction for teacher leader completers across all Teacher Leader Model Standards. One hundred percent of employers agree or somewhat agree completers create collaborative environments, use systemic inquiry, promote continuous improvement, model behaviors that promote instructional improvement, optimize school improvement strategies, and facilitate collaboration among stakeholders. Across cycle 1 (2020-2021) and 2 (2021-2022), 100% of employers agree that or somewhat agree completers advocate for educational policy. We will continue to monitor these data and support increased response rates to document the effectiveness of completers at advanced levels. 

Professional Standards of Educational Leaders (PSEL)

Across all three cycles of data, employers indicated high rates of satisfaction for Ed.S. completers across all PSEL standards with employers agreeing or somewhat agreeing that completers demonstrate these standards in observable ways. One hundred percent of employers agree or somewhat agree completers enact core values for academic success, demonstrate professional standards, support culturally responsive practices, support rigorous instruction, cultivate inclusivity, fosters professional community, positively engages with stakeholders, manage resources well, and practices continuous improvement.  

International Literacy Association (ILA) Standards

Across all three cycles of data, employers indicated high rates of satisfaction for Literacy Leader completers across all ILA standards with employers agreeing or somewhat agreeing that completers demonstrate these standards in observable ways. All respondents agree or somewhat agree that completers demonstrate evidence-based foundations of literacy, employ evidence-based instruction, create inclusive learning environments, utilize appropriate assessment methods, employ multiple modes of materials, and facilitate ongoing professional learning. There were no identified deficiencies across these standards. 

Stakeholder Involvement

In order to co-construct a mutually beneficial partnership between the AFTSE and our partner districts, the AFTSE meets as an entire unit twice a year with our Advisory Council on Education (ACE) committee, which includes multiple stakeholders from our partner districts. ACE committee meetings are devoted to a specific need each year identified by both our partner districts and our EPP. A smaller committee, the Bellarmine Educators Stakeholders Team (BEST), which consists of some AFTSE faculty members, administrators, teachers, and recruiters from JCPS and the Archdiocese of Louisville, meets three times a semester to provide input about individual programs, key assessments, and innovations. The AFTSE and our clinical partners share responsibility for the co-construction of key instruments used in the field, big picture curricular decisions, and programmatic changes. Over time, this collaborative process leads to purposeful changes and creates a cohesive experience for our candidates. Summaries of annual stakeholder activity are outlined in the tables below.

Measures of Program Outcomes

Measure 3 (Initial and Advanced): Candidate Competency at Completion (R.3.3)

To measure candidate competency at the initial and advanced levels, we examine licensure and end of program assessments. 

Initial Certification

Bellarmine works with ETS Praxis data to track candidate success rates on licensure assessments. Across all three cycles of data at the initial certification level, Bellarmine’s pass rates are 100% for cycles 1-3 when the data samples reach reporting threshold. Data for our alternative certification students are incomplete as cycle 1 samples are lower than ten and thus redacted. For cycles 2 and 3, 100% of our alternative certification students passed the required licensure assessments. For cycles 1, 2, and 3, our traditional certification students passed the required licensure assessments. Data across pathways and cycles exceed statewide pass rates. 

Advanced Certification

Across all three cycles of data at the advanced certification level, Bellarmine’s pass rates indicate candidate competency at completion. The Literacy Specialist assessment reflects an 80% pass rate in cycle 1 (2020-2021) and 100% pass rate in cycle 3 (2022-2023) with no completers in cycle 2. The Principal Preparation assessments reflect 100% pass rate across all three cycles. Bellarmine’s pass rates remain at or above state averages on these assessments

Measure 4 (Initial and Advanced): Ability of Completers to be Hired

Bellarmine’s Career Development Center conducts an annual First Destination Survey to collect hiring data. The last three cycles of our annual First Destination Survey indicate Bellarmine students are sought after. The Career Development Center reports a strong response rate of 89% in cycle 3 (2022-2023), 81% response rate in cycle 2 (2021-2022), and an 87% response rate in cycle 1 (2020-2021). While this survey does not clearly differentiate between initial certification and advanced program completers, the overall data indicate the vast majority of completers are employed with the remainder pursuing continuing education. This trend persists across different demographic profiles. 

Initial Certification

Cycle 3 (2022-2023) data indicate that 87% of education graduates are employed in their field with the remaining 13% pursuing continuing education. The local school district, JCPS, is identified as the third largest employer of Bellarmine graduates. Cycle 2 (2021-2022) data indicated 92% of our undergraduate completers are employed, volunteering, or in the military; 8% are pursuing graduate education. Of the 92% employed, 99% of these are working as teachers. Jefferson County Public Schools is identified as the highest employer of School of Education graduates with the Archdiocese and Oldham County Schools also employing multiple graduates. Cycle 1 (2020-2021) indicated that 67% of graduates were working in their field with 33% engaged in ongoing education. JCPS was the second largest employer of Bellarmine graduates for that year. 

Advanced Certification

Cycles 3 (2022-2023) and 1 (2020-2021) did not specify graduate student employment rates.  Data for cycle 2 (2021-2022) advanced program completers reveal 90% employment and 10% pursuing a second graduate degree. Because advanced practitioners may not immediately obtain a license upon program completion, we monitor licensure exam pass rates. These pass rates suggest the number of completers who intend to seek employment based on their new credentials.    

Employer satisfaction survey indicates that 100% of responding employers would hire another Bellarmine graduate based on previous/current experience and recommend Bellarmine graduates to other school administrators. These data remain consistent across all three cycles of data and across initial and advanced programs.