The unit has made a number of changes to its assessment system and unit processes for evaluation since the last accreditation visit in 2004, with the goal of more systematic and predictive processes to improve candidate proficiencies. These changes, along with plans for sustaining and enhancing performance, are summarized as follows:
1) Inclusion of candidate interviews (with P-12 partners) and writing prompts at Transition Point 1 for initial certification programs.
During the 2007-2008 academic year the faculty, in collaboration with the Assessment Committee and the ACE, developed collaborative interviews and writing prompts as requirements for Admission to Teacher Education. The interview was designed as an authentic task to determine candidate preparation for admission, and the writing prompt as a screening for candidate competencies. Both were piloted with undergraduate candidates applying to Transition Point I during the 2008- 2009 academic year, with both adopted by the MAT program during the 2010 academic year. The interview and writing prompts are used as an early identification system for any dispositional and skill concerns, with candidates' results reviewed by the Transition Point 1 committee and candidates' scores entered into the unit's degree audit data system (Exhibit 2.3.h.1).
2) Revision of the Dispositions Self-and Institutional Assessment in the initial certification programs.
With the revision of the Kentucky Teacher Standards in 2008, and an increase in the number of initial certification candidates experiencing dispositional concerns during the professional semester over a two-year period, the faculty initiated a review of the dispositions assessment for initial certification programs, which had been based on the previous Kentucky New Teacher Standards. The new instrument is more closely aligned with the conceptual framework and program objectives, and targets five themes based on feedback received from cooperating teachers and professional semester supervisors. Additionally, the revised dispositions assessment links an identified area for remediation with the development of a dispositions intervention plan (DIP) (Exhibit 2.3.h.2.a). Since its full implementation in 2009, faculty and cooperating teachers have initiated more DIPs at earlier points in candidates' programs in an effort to increase candidates' effectiveness in the professional semester. The unit has just completed an analysis of its DIPs on initial certification candidates to determine any patterns, which could be addressed proactively in terms of curricular or programmatic changes (Exhibit 2.3.h.2.b).
3) Revision of cooperating teacher field evaluation forms and inclusion of junior practicum and evaluation data.
The unit's initial certification field placement evaluation forms, completed by cooperating teachers and based on the previous set of Kentucky New Teacher Standards, were too abstract and not developmentally appropriate, as reported by cooperating teachers, who often left much of the form blank. The unit revised the forms to be developmental (based on the candidate's program), and added an instructional component and a comments section to each prompt for the cooperating teacher. Since the pilot and full implementation in fall 2008, the data from cooperating teachers have been more substantive, particularly with the addition of a "comments" section after each item (Exhibit 2.3.h.3.a). These data facilitate decision-making about candidate progress as well as provide feedback for possible dispositional concerns. Candidates are better able to self-assess and reflect with multiple sources of data.
The unit added a substantive field practicum experience in fall 2009 for undergraduate initial certification candidates, involving placement in a single school one day per week in the fall, and one and one-half days per week in the spring, to better equip candidates with skills critical to a successful professional semester. The practicum involves teaching lessons which are observed and evaluated by a supervisor. Beginning spring 2011, the candidate and the cooperating teacher in the practicum placement evaluate the experience to provide substantive data for analysis (Exhibit 2.3.h.3.b; see also Standard 3 for data analysis).
4) Revision of Valli reflection section on initial certification benchmarks.
Trend data from 2006-2008 revealed that a significant number of candidates in initial certification programs were scoring in the Apprentice and Proficient ranges on the Valli reflection section of each benchmark (Exhibit 2.3.h.4.a). As an essential component of every AFTSE program, Valli's types of reflection should be mastered by candidates at least to the proficient level. The former benchmarks required candidates to reflect using each of Valli's types of reflection to earn a "distinguished" rating, and a majority of types to earn "proficient." The Assessment Committee was charged with the task of determining the origins of the lack of proficiency, presenting to the unit the need to re-align specific types of reflection that best relate to the content of each benchmark rather than expect candidates to reflect on all. The unit modified each benchmark, correlating the types of reflection with the content, and affirmed the need for all programs to intentionally and clearly teach Valli's types of reflection (Exhibit 2.3.h.4.b). The unit re-affirmed its commitment to reflective teaching and leading in its self-study process, establishing the importance of candidates' knowledge about reflection in general and Valli's types of reflection in particular.
5) Changed the percentage ranges on initial certification benchmarks.
Trend data (2008-2010) on initial certification candidates' performance on the Philosophy of Education Benchmark and the Classroom Management Benchmark suggested too wide a range in performance levels, thus too many candidates scoring in the "distinguished" range (88-100 %) and the "proficient" range (63-87%). Faculty were concerned because the standard by which candidates are deemed to be "proficient" and "distinguished" was not sufficiently high to maintain an acceptable level of rigor, particularly with the challenges facing 21st century educators. Standards, and thus percentages, were raised beginning fall 2011, in order to more accurately reflect candidate performance at these levels (Exhibit 2.3.h.5).
6) Revised Principal Preparation Anchors.
In 2010, the Commonwealth of Kentucky mandated a re-design of principal preparation programs to better reflect the evolving role of the principal as instructional leader who requires extensive internship opportunities in authentic settings for effective preparation. The unit followed the state's recommended model in its design, revising its existing principal preparation program into an Education Specialist in Instructional Leadership and School Administration program. Integral to the philosophy of the EdS in Instructional Leadership program is mastery of the knowledge, skills, and dispositions required of aspiring school leaders, which is reflected in the unit's design of either an "Acceptable" or "Unacceptable" rating on required anchor assessments. The unit adopted the use of "anchor" assessments for all advanced programs, including the ratings of "Acceptable" or "Unacceptable." The current re-design of the MAED and Rank I programs in P-12 Reading and Writing to Literacy Specialist (P-12) will also include anchor assessments with the same ratings.
7) Added the Degree Audit Portlet within CRM (Constituent Relationship Module) and Full Implementation of LiveText into all advanced programs.
The unit degree audit data system has been modified to improve the specificity and functionality of candidate performance reviews both for individual candidate movement through transition points and for programmatic evaluation. Key instructional technology and unit personnel have worked closely since 2007 to enhance the unit's degree audit data system. The unit has always maintained a database to track candidates, but the new degree audit system will permit highly specific tracking across all programs as well as the ability to aggregate data by program. A candidate degree audit and student overview track both course and transcript specific data in addition to transition point progression, field experiences, and disposition plans (if applicable) (Exhibit 2.3.h.6). Transition point committees in particular use the degree audit database to make decisions about individual candidates. In addition to the upgraded degree audit database, the unit has implemented LiveText in all advanced programs. These two data management systems will enhance the unit's ability to track candidates, improve programs, and evaluate the unit's effectiveness. In particular, these data systems will improve the unit's ability to disaggregate data in the MAED programs, which will be a focus in our continuous improvement.
8) Unit Survey
The unit survey was initiated in spring 2012 for faculty and exiting candidates to assess unit proficiency (Exhibit 2.3.h.7.a, Exhibit 2.3.h.7.b). The goal of the survey was to provide a forum for full-time and adjunct faculty and exiting candidates to anonymously evaluate aspects of their programs and the unit not addressed by other venues such as course or field evaluations (candidates) or faculty meetings. The initial attempt provided comments and feedback delineating areas for growth, as well as feedback on revisions to the survey questions (Exhibit 2.3.h.7.c). The unit shared the survey results with the Advisory Committee on Education in April, 2012, with discussion on needed improvements. Revision is under way for use in the next academic year, with hopes of obtaining productive feedback for improvements.
Exhibits
2.3.h.1 Interview Protocol and Writing Prompt
2.3.h.2.a Dispositions Assessment and Intervention Plan
2.3.h.2.b Dispositions Intervention Plan Data
2.3.h.3.a Revised Cooperating Teacher Field Evaluation Forms
2.3.h.3.b Junior Field Practicum Cooperating Teacher and Candidate Evaluations
2.3.h.4.a Data Sampling for Changes in Valli Section on Benchmarks
2.3.h.4.b Changes to Valli Section of Philosophy Benchmark
2.3.h.5 Chart on Benchmark Percentage Changes
2.3.h.6 Degree Audit Portlet Sample
2.3.h.7.a Unit Final Faculty Survey
2.3.h.7.b Unit Final Candidate Exit Survey
2.3.h.7.c Unit Survey Faculty and Candidate Comments and Results