| Summary: The unit, with the involvement of its professional community, is regularly evaluating the capacity and effectiveness of its assessment system, which reflects the conceptual framework and incorporates candidate proficiencies outlined in professional and state standards. The unit regularly examines the validity and utility of the data produced through assessments and makes modifications to keep abreast of changes in assessment technology and in professional standards. Decisions about candidate performance are based on multiple assessments made at multiple points before program completion and in practice after completion of programs. Although most unit programs are new, a tracking system is in place to evaluate the relationship of program elements to future success. The unit conducts thorough studies to establish fairness, accuracy, and consistency of its assessment procedures and unit operations. It also makes changes in its practices consistent with the results of these studies. |
Unit Assessment System Overview
The unit assessment system is built upon the unit’s conceptual framework and the QNPS. The unit conceptual framework is supported by eight unit learning outcomes that come under Teaching, Scholarship, and Leadership, and the master matrix aligns the unit learning outcomes and QNPS. At a course level, faculty are expected to provide course matrix in the syllabus that clearly explains the alignment among the following items: 1) Unit Learning Outcomes; 2) QNPS; 3) Course Objectives; 4) Course Learning Outcomes; and 5) Assessment (e.g., tasks/artifacts).
Qatar University began its initial preparation to become an applicant institution for the Southern Association of Colleges and Schools (SACS) in Fall 2009. Because SACS emphasizes on the documentation of institutional effectiveness, faculty in all seven colleges and programs identified target student learning outcomes in each course for the purpose of program assessment. At the end of each semester, the program coordinators and their faculty review the candidates’ performance on pre-selected course objectives for the semester for the purpose of reflection and improvements for teaching and course content in the future. The data gathered for these pre-selected objectives are summarized and then sent to the Director of Academic Programs and Learning Outcomes Assessment for the university. This task not only contributed to university goals for program assessment, but also strengthened the unit’s assessment efforts at the course level and provided another tool to review and reflect to make program improvements.
Various instruments developed for field experiences and clinical practice also assess candidates’ disposition and proficiencies based on the unit’s conceptual framework and the QNPS. As described later in detail in this section, the unit utilizes the commercial product, TaskStream™ to assist each candidate in preparing an electronic portfolio (their E-Folio). In the E-Folio, candidates are expected to select artifacts to meet each unit learning outcome and write a rationale that includes their reasons for selecting that particular artifact and an explanation as to how that artifact demonstrates mastery of that learning outcome. They are also expected to write an overall reflection of the conceptual framework, “Together we shape the future through excellence in teaching, scholarship, and leadership.” All candidates are expected to demonstrate their knowledge, skills, and professional dispositions that support the unit’s conceptual framework by the end of their programs.
Key Assessments
The unit assessment was developed to closely monitor each candidate’s educational performance throughout the program to observe growth. Candidate performance assessments are conducted at multiple points, and the transition points are referred to as “Checkpoints.” The detailed explanation for all levels, initial and advanced candidates, is summarized in the Unit Assessment Charts.
Key assessments include course assignments and Grade Point Averages (GPA), a written Statement of Philosophy, a Diversity Questionnaire, the Classroom Performance Assessment (CPA), the Professional Practices Indicators (PPI), the Professional Dispositions Indicators (PDI), an Exit Survey, and a Post Graduation Survey. Some programs also have content tests or other program-specific requirements.
Evaluation of the Unit Assessment System
A wide range of stakeholders are involved in evaluating the unit assessment system. In addition to IRTE reviewers, the assessment system is reviewed by the university committee preparing for SACS accreditation. Candidates have opportunity to provide input via the course evaluations and the Exit Survey. Faculty members provide feedback during departmental meetings and monthly Shaping the Future meetings. Representatives from the SEC, partner schools, other colleges and educational institutes, candidates, graduates, mentors, and faculty members participate in Education Partners’ meetings twice yearly specifically the review, evaluate, and provide feedback on all unit activities.
Quality Assurance
The unit ensures fairness, accuracy, and consistency of assessment procedures by having an array of methods in the decision making process at each checkpoint throughout initial and advanced programs and by having multiple sources the assess the same criteria (triangulating data sources). As it is presented in the Unit Assessment Chart for each program level, data are collected from numerous sources, and decisions about candidate’s progress are determined by faculty members who monitor candidate growth over time.
Faculty members are trained in how to use the E-Folio rubric. During training sessions, faculty members have opportunities to practice rating samples of candidate work to see how closely they determine a particular score. The faculty members then compare and discuss where gaps may be and try to come to an agreement.
Some of the instruments are assessed by multiple individuals. For example, CPA, PPI, and PDI are assessed by three individuals. At the beginning of clinical practice, mentor teachers and college supervisors meet for orientation to go over all the instruments that will be used to evaluate the candidates. Each observational instrument and the items for rating are carefully reviewed and discussed. The CPA, PPI, and PDI are used to observe candidates’ performances by their mentor teachers and college supervisors at least twice during the clinical practice. This is another way to help to ensure fair, accurate, and consistent evaluations. In addition, candidates themselves are asked to use those three instruments for self-evaluation. All three data sets are later gathered and analyzed.
Managing and Improving Operations and Programs
The unit is committed to examining how to improve operations and programs. First, at a course level, each instructor reviews and examines the results of learning outcome measures that are collected (available onsite, Exhibit 2.3). At the program level, the coordinator and faculty members meet and review candidates’ performances at each checkpoint and the results of the identified unit learning outcomes at appropriate courses in the program. At the unit level, the dean, associate deans, department heads, and program coordinators to review and reflect on overall candidate performances by programs and discuss ways to improve the programs.
Beginning Fall 2010, a summary of data by programs will be presented at Shaping the Future faculty meetings so that the unit faculty will be able to see the overall results across programs and levels in order to participate in the discussion and provide input for improvement. The same presentation will be made at the next Education Partners Committee meeting to inform the unit’s stakeholders the candidate performances and welcome their feedback to improve the operations and programs of the unit.