2.0 Core Requirements
3.0 Comprehensive Standards
4.0 Federal Requirements
|18.104.22.168 Assessment of Educational Programs|
|Thursday, 03 September 2009 14:22|
The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas: educational programs, including student learning
Responsible Unit: Division of Academic Affairs/IPAR
Institutional Planning, Assessment and Research (IPAR), the institutional effectiveness unit at North Carolina A&T State University, is responsible for the periodic review of academic programs and administrative service units that support the academic enterprise. The cyclical review helps academic departments assess the quality and competitiveness of their programs, ascertain whether expected student learning outcomes are being achieved, and determine how to improve program quality based on assessment data. Service units examine how to improve customer service and enhance operating efficiencies. The academic departments and service units begin the assessment by reviewing how their strategic goals and objectives relate to the university's mission.
In October 2001, the university established the Institutional Planning, Assessment and Research (IPAR) unit and charged it with: (1) establishing an internal process of assessment of academic programs and service units (described in Assessment Process section below); (2) supporting academic departments' program reviews by accrediting agencies, and (3) funding external consultant visits to review academic departments in those disciplines that do not have professional accrediting agencies . The purpose is to ensure the University's adherence to sound practices of institutional effectiveness and the rigorous standards of accrediting agencies such as SACS.
In addition to the internal assessment, sixteen professional accrediting agencies review A&T academic programs and monitor the academic progress of athletes to standards set by the National Collegiate Athletic Association/Mid-Eastern Athletic Conference. IPAR reviews applications for accreditation, self-student reports, and monitoring reports. The schedule of accrediting reviews is included. Academic programs hold memberships in nearly five dozen (56) professional organizations, including the professional accrediting agencies, whose standards and criteria provide benchmarks for accessing the competitiveness and needed improvements for A&T academic programs .
Academic departments without accrediting agency reviews have the option of an invited external review. For example, in 2002 the Department of Animal Sciences invited the U.S. Department of Agriculture/ Cooperative State Research, Education and Extension Service (CSREES) for an on-site review . A five-person team comprised of CSREES representatives and other university agricultural and veterinary programs conducted the review. The purpose outlined in the department's self-study was "to examine progress to date, and determine changes that need to be addressed over the next five years so that the department remains effective in meeting its mission in instruction, research, and outreach" (p. 1). The CSREES report included numerous recommendations (30) to improve the Animal Sciences Department  to which the department responded. The department made specific changes to improve its programs. For instance, CSREES raised a concern about the heavy teaching responsibilities (p. 1) . Consequently, the Provost gave one tenure tract position to the department immediately and provided a second position in 2008. The department argued against the recommendation to limit the number of graduate students to five per major thesis advisor because of the need to grow the program and comply with the UNC-GA's requirement of a minimum of 20 students per graduate program (p. 2) .
Another example of an invited review was the assessment of the general education curriculum in October 1998. The consultant's report recommended a "restructured general education curriculum" (p. 6) , which led to a major revitalization of the core curriculum, beginning in the spring semester 2002. After four and half years of exhaustive study and University-wide engagement, A&T launched the University Studies revised general education core in fall 2006  (see SACS Principles 2.7.3 - General Education; 3.5.1 - College-level competencies, and 3.5.3 - Undergraduate program requirements).
Assessment Process: Coordinated by Institutional Planning, Assessment and Research (IPAR), the academic departments submit reports on internal assessments and evaluation every five years, with an average of 10 academic departments and programs in each of the annual cohorts undergoing review. Included are the cohort schedules for 2001-2006  and 2007-2012 . The service units-- Business and Finance, Division of Information Technology (DoIT), Division of Research and Economic Development (DORED), Enrollment Management (starting with development of a strategic plan in 2008-2009), Human Resources (starting with development of a strategic plan in 2008-2009) and Student Affairs--report annually on progress toward accomplishing the goals and objectives in their strategic plans.
Each academic department and service unit defines its goals in relationship to the University's goals  (see SACS Principle 2.5 Institutional Effectiveness). Academic programs and service units begin the assessment and evaluation process by reviewing and updating their strategic plan. If the strategic plan is outdated, the department will often begin by developing a new plan. In the spring of the program review year, completed annual assessment and evaluation reports are submitted to IPAR.
IPAR reviews these documents and uses the Annual Assessment and Program Evaluation Review Form  to provide written feedback on the strengths, weaknesses, and recommendations to the department chair or appropriate unit administrator.
These are examples of the feedback IPAR provided on five-year reports and strategic plans for academic departments: Electrical & Computer Engineering in the College of Engineering (2004-2005 cohort) , Family & Consumer Sciences in the School of Agriculture and Environmental Sciences (2005-2006 cohort) , Family & Consumer Sciences in the School of Education (2007-2008 cohort) , the strategic plan for Sociology and Social Work in the College of Arts and Sciences (2006-2007 cohort) , and the evaluation plan for University Studies  (see SACS Principle 2.7.3 General Education). These are examples of feedback IE provided for service units: the Division of Business and Finance (2006-2007 cohort), a service unit , the Division of Student Affairs (2006-2007 cohort), a service unit .
Several documents assist departments and service units in the implementation of a systematic assessment of their strategic goals and objectives as well as the student learning outcomes. Documents include the Overview of the Annual Assessment and Program Evaluation Review , which describes why it is important to create a culture of evidence and how the results of assessment should be used to improve outcomes. The Outline for Five-Year Assessment and Program Evaluation Reports  is used by the academic departments to report on the three- to five-year quantitative progress. The original form posed questions answered with narrative statements . The revised form introduced in 2006 created data tables to be completed that provided the same information . This reduced confusion about what information was being requested and made it easier for the department chairs to complete the report.
The report format for academic departments includes: (1) the department's three- to five-year strategic plan; (2) data on students retention and graduation, faculty demographics, alumni data, faculty demographic data, alumni and employer survey results, and research, scholarly and creative productivity of faculty, and faculty community service, and (3) the assessment of student learning outcomes for each degree program.
Departments complete the following forms for each degree program : (1) Form A - Student Learning Outcomes captures the content mastery expected of students when they graduate and at the gatekeeper point when students are accepted in majors such as nursing and teacher education (Praxis I knowledge). There is also a section for Overall Program Outcomes, such as job placement, graduate school enrollment, licensure exams, etc. On Program Form B - Evaluation Methods (Commercially Available Tests/Surveys) departments assess students' performance on commercial tests and licensure exams such as Praxis, Professional Engineering and NCLEX nursing exam. Most departments use locally developed assessments, which are reported on Form C - Locally Developed Methods. The analysis of data from the assessment measures on Form B and Form C lead to the program improvements reported on Form D - Major Findings and Changes Made to Program as a Result of Assessing Goals. A set of completed forms A-D  is provided as examples to help the departments understand how to state quantifiable student learning outcomes, standards of performance expected to assess the mastery of content, and a sharp analysis of the data leading to program improvements.
In 2005, IPAR refined and simplified the assessment process to prevent the confusion evident in developing strategic goals and objectives for the department and learning outcomes for students. Departments were requested to update and develop their strategic plans and submit them to IPAR for review. After approval of the strategic plan goals and objectives departments reviewed and refined the student learning outcomes for their courses. Separating the process eliminated some of the confusion in the two separate components of the assessment process.
Assessment reports are due in mid-May each year. IPAR tracks the submission of reports, noting the dates they are received, the dates revisions are due, and when the final bound copies are received. The report submission tracking logs for 2004-2005 , 2005-2006 , 2006-2007  and 2007-2008  are included.
To acquaint academic departments and service units with assessment, IPAR convenes technical assistance workshops for deans, department chairs, faculty, and service unit supervisors and staff. Technical assistance is in the form of one-on-one meetings and hands-on workshops. In plenary and small group workshop sessions, the participants have the opportunity to develop their unit's strategic goals and objectives. In department workshops, faculty members use their course syllabi to formulate measurable student learning outcomes. An agenda, a worksheet and follow up e-mail are provided for the Mechanical Engineering Department   Sociology and Social Work Department , and Visual & Performing Arts  workshop are provided as examples. In addition to these internal TA sessions, IPAR provides funding for faculty to attend assessment conferences and workshops. See the Academy for Teaching and Learning (ATL) section below for details about the Maki and Rhodes workshop, another example of technical assistance workshops.
IPAR also disseminated to department chairs The Departmental Guide and Record Book for Student Outcomes Assessment and Institutional Effectiveness, 3rd ed., by James O. Nichols and Karen W. Nichols (New York: Agathon Press, 2000), as well as an excellent, easy-to-comprehend PowerPoint presentation on the assessment of student learning outcomes, prepared by Institutional Effectiveness staff at Alabama A&M University . The step-by-step guide (slides 11-16) on how to develop measureable learning outcomes with examples of well developed and poorly developed outcomes (slides 15 and 16) are easy to follow.
Assessment: General Education: In October 1998, the university started a review of the general education core with an external review. The external reviewer's report recommended a "restructured general education curriculum" (p. 6) , which led to a major revitalization of the core curriculum, beginning in the spring semester 2002. In fall 2006, A&T launched the University Studies (UNST), the revised general education core  (see SACS Principles 2.7.3 - General Education; 3.5.1 - College-level competencies, and 3.5.3 - Undergraduate program requirements).
A comprehensive evaluation plan was implemented in academic year 2006-2007, the first year UNST was offered . Data are reported annually for the UNST foundation courses: UNST 110 Critical Writing; UNST 120 The Contemporary World; UNST 130 Analytical Reasoning; and UNST 140 The African American Experience and for UNST 100 University Experience, the first-year college orientation course   .
By 2007-2008, the second year of UNST, data analysis led to a number of program improvements. For example, the UNST faculty revised UNST 100 University Experience based on the student performance data in the four UNST foundation courses. The revised UNST 100 course emphasized the role of the UNST program and presented a broad overview of the role of UNST, its curriculum structure and rationale, and an introduction to a variety of interdisciplinary themes within the program. Introductory discussions on critical thinking, communication skills, ethics, diversity, civic engagement, and globalization were also included (p. 57) . The faculty members in UNST 110 Critical Writing developed a common rubric for assessing assignments in the course  to facilitate a fair and objective evaluation of student work across the many sections of the course that are offered. In addition, the rubric makes statistical analysis and interpretation of results consistent across sections of the course and increases confidence in the validity of the results. As a pedagogical aid to self-assessment, students are given the rubric so that they explicitly understand the criteria upon which their work is evaluated and can internalize the standards for achieving excellence. UNST faculty also do a variety of collaborative assessment activities such as peer review of writing, practice presentation critiques, and group reflection exercises. Also, in 2007-2008, faculty teaching UNST 140 The African American Experience collected data to analyze whether there was an instructor effect on the pre- and post-test scores. The results were negative (pp. 54-55) .
In 2008-2009, the UNST 110 Critical Writing faculty examined their scoring of student writing to determine the degree to which faculty members provided "objective" assessments of writing assignments. Faculty used the ETS web-based Criterion writing assessment tool as an unbiased external scoring algorithm to compare their scoring of student writing (p. 38) . The UNST 110 Critical Writing team also examined expected student learning outcomes, especially the reading challenges faced by students, and concluded that at least one more basic reading comprehension session was needed. Faculty incorporated analysis of basic comprehension using the multiple choice format of the SAT reading comprehension model (pp. 42-43) .
The Faculty Senate requested an evaluation of University Studies three years after implementation. Institutional Planning, Assessment and Research (IPAR) has scheduled an external evaluation for September 2009. The internal IPAR evaluation is scheduled for 2011-2012 (p. 9) .
Assessment: Academic Departments and Service Units: Following are examples of the five-year assessment and evaluation reports for undergraduate and graduate degree programs as well as annual reports for the administrative service units. For the academic departments, Forms A-D include the student learning outcomes, assessment tools, and how results have been used to improve the various degree programs. There are also examples of the narratives for some assessment reports. The reports show how academic departments and service units use the IPAR assessment process.
Five-Year Assessment Reports for Academic Department &
Annual Assessment Reports for Service Unit
2003 to 2008
1SAES - School of Agriculture & Environmental Sciences, CAS - College of Arts & Science, SOBE - School of Business & Economics, SOE - School of Education, COE - College of Engineering, SON - School of Nursing, SOT - School of Technology
2DoIT - Information Technology, DORED - Research & Economic Development
Certificate Programs: Since 1995 A&T has offered fifteen certificate programs in the: School of Agriculture & Environmental Science (six programs), College of Arts & Sciences (two programs), School of Education (two programs), School of Business & Economics (three programs), School of Technology (one program), and International Programs . The oldest programs are in waste management, an interdisciplinary program. The certificate programs give students the option of completing a curriculum of 11-20 credit hours in an area of concentration such as equine management, waste management, entrepreneurship, and global studies. Students take the majority of courses in their majors plus certain elective courses. Often there are other requirements, such as study abroad, internships, research papers, etc. A certificate is awarded by the department offering the certificate program, but there is no official designation of the certificate program on the student's diploma or transcript.
Professional Accreditations: Twenty-four programs are accredited by professional agencies. Most (23) are accredited by academic organizations and associations. In addition, the NCAA requires an NCAA Division I Academic Progress Rate (APR) Improvement Plan. The accreditation schedule provides a list of the agencies and on-site visit dates with notes on the actions taken . Examples of accrediting agencies are provided below.
ABET (formerly Accreditation Board for Engineering and Technology) accredits eight engineering majors in the College of Engineering. The Engineering Accreditation Commission of ABET accredits the following programs: Architectural, Biological, Chemical, Civil, Electrical, Industrial, and Mechanical Engineering. The Computing Accreditation Commission accredits the Computer Science program. Included is the self-study prepared for ABET for the 2007 on-site review , the ABET Draft Statement in response to the visit , the dean's 30-day response , and the ABET Summary of Accreditation Actions . Three of the engineering programs submitted reports in response to the summary actions. The IPAR critique of the first draft of the Chemical Engineering Department's response  and the final response report for the Chemical Engineering Department  are included.
Accrediting Council on Education in Journalism and Mass Communications (ACEJMC) awarded provisional accreditation to the Journalism and Mass Communication Program in 2003. The provisional status was removed and full accreditation awarded by a unanimous decision in 2005. The A&T revisit report and the ACEJMC notification letter of full accreditation are included . JOMC is in the College of Arts & Sciences.
American Dietetic Association (ADA) Commission on Accreditation for Dietetics Education (CADE) awarded initial accreditation to the BS Didactic Program in Dietetics in Family and Consumer Sciences in the School of Agriculture and Environmental Sciences in 2006. The ADA notification letter and on-site report are provided .
Association to Advance Collegiate Schools of Business (AACSB) has accredited the School of Business since 1979. The latest AACSB notification of the maintenance of accreditation for the undergraduate and master's degree programs was in 2007 . The Maintenance Review Visit Report (October 29-31, 2006) prepared by the School of Business  and the AACSB Consultative Report following the visit  are included. In addition, the fifth-year maintenance review applications for the accounting program to the AACSB Accounting Accreditation Committee (AAC)  and the Maintenance of Accreditation Committee (MAC)  were submitted July 2009.
Council on Rehabilitation Education (CORE) accredits the MS in Human Resources (Rehabilitation Counseling) program housed in the School of Education. CORE awarded the Rehabilitation Counseling program its three-year candidate for accreditation status designated for new programs in July 2006. CORE granted an additional one-year extension of full accreditation in 2008 . The A&T CORE Annual Monitoring Report submitted in 2008 is provided . The program will pursue reaccreditation during the 2009-2010 academic year.
Council on Social Work Education (CSWE) reaffirmed the accreditation of the Joint Masters of Social Work (JMSW) in 2004. The program, a collaboration between A&T and the University of North Carolina at Greensboro (UNCG), that has been in place since 1997 and was initially accredited by the CSWE in 2000. The administration of the JMSW program rotates between the A&T and UNCG campuses every four years. In addition, the BSW Social Work Program is also accredited by CSWE. Accreditation was restored in 2005 . The required 2005 Progress Report  was also accepted by CSWE . The social work programs are in the College of Arts & Sciences.
The National Association of Schools of Music (NASM) accredits the Bachelor of Arts degree in General Music, the Bachelor of Arts degree in Music performance and the Bachelor of Science degree in Music Education in the Music Program. NASM reaccredited the Music Program in the College of Arts & Sciences in 2007. The NASM letter of notification  and the Commission Action Report  are included. The Music Program is in the Visual and Performing Arts Department of the College of Arts & Sciences.
Distance Education: The Center for Distance Learning (DL) is the administrative unit responsible for online and extension courses and programs at North Carolina A & T State University. In academic year 2008-2009, DL offered courses for 10 degree programs at the baccalaureate, master's and doctoral levels in the School of Agriculture & Environmental Sciences (SAES) (two programs), School of Business & Economics (SOBE) (one program), and School of Technology (SOT) (seven program) .
All distance learning programs must be approved by the University of North Carolina General Administration (UNC-GA) for on-campus delivery before they can be approved for delivery via distance learning. In order for a distance learning program to be approved, the following process must be followed:
The DL Online Course Development/Enhancement Evaluation Form  is a comprehensive assessment of various aspects of the online course, including a section on learning objectives (Section B, p. 2). For example, the first criterion states: The learning objectives of the course describe outcomes that are measurable. The instructor rates the course against the defined criteria. There are three rating levels for the criteria: (1) meets criteria; (2) partially meets criteria; and (3) does not meet criteria. There is a fourth category for not applicable. Section C, Assessment and Measurement (p. 3), requires the instructor to rate the assessment strategies that are used to measure effective learning and to assess student progress by reference to stated learning objectives. The assessment of Learner Interaction in Section E (p. 6) has the instructor rate the design of the instructor-student interaction, student cooperation, and student-content interaction.
Student Course Evaluation: Starting in the fall semester of 2006, North Carolina A&T State University moved from using a paper-and-pencil form for assessing course instruction to an online version , using the content of the original paper survey instrument. A&T uses the CoursEval software from Academic Management Systems on a dedicated server. Having a server is an economic advantage as well as allowing the University to control its data. CoursEval is a web-based evaluation and reporting tool that can be used for a variety of confidential survey-driven assessment and non-assessment activities. The strength of CoursEval is its ability to adapt to an institution's assessment needs with no local programming time and very few local IT resources.
The IPAR survey manager can set up a wide variety of course, faculty, or other assessment surveys. Survey managers are able to send e-mail to each set of respondents informing them of a survey's availability, watch the progress of the survey, check for completion compliance, notify delinquent respondents, and make the results available promptly to faculty members and staff once the survey ends. The CoursEval login site is: (see login screen)
The course evaluation process consists of exporting data from the Banner integrated, collegiate administrative system. The data for each term consists of faculty, student, course and course enrollment files, which are checked and processed before being imported into the software. Once the data files are imported, an initial announcement is sent to instructors , which include the survey's starting and closing dates for the term. In this announcement instructors are asked to encourage student participation in completing the evaluations for their classes.
Evaluations are emailed to all students for each class in which they are enrolled , and reminder e-mails are also sent to instructors requesting that they encourage their students to participate . The same survey and process are used for undergraduate and graduate students. IPAR sends course evaluation surveys to any student with a valid campus e-mail who is enrolled in a course for that particular term.
Bi-weekly reminders are sent to instructors  and students  until the survey ends. Using one of the reporting features available in CourEval, comparative reports are generated for each department . In addition, the data from the term are exported into SAS, a statistical analysis system, for school/college level analyses.
Next, IPAR sends two reports to the deans: (1) the first is a summary of the overall mean scores and response rates for their respective schools and colleges , and (2) the second is the summary reports of the mean scores and response rates for each department in their respective schools and colleges . Department chairs also receive a copy of the department summary report . Finally, instructors are notified by e-mail that course evaluation results are available for each of the classes taught during the term . This e-mail allows each instructor to login and see their scores and comments from students. The Associate Vice Chancellor for Academic Affairs receives a summary report of the course evaluation results .
Using this process the University achieved a student participation rate of 50.3% for the fall semester of 2006, a participation rate of 47.2% for fall semester of 2007, and a student participation rate of 46.1% for fall semester of 2008. The average for the past three fall semesters is a 47.4% student participation rate. The student participation rate for the spring has typically been lower. The results of the student evaluations are used by department chairs to evaluate the teaching of instructors. The evaluation results are placed in the faculty member's personnel file.
Deans Annual Reports: At the Deans' Retreat in July 2009  the provost indicated that a committee of deans would be charged with revising the current annual report outline  to include a section on the achievement of student learning outcomes and assessment of general education learning outcomes. In addition, there was extensive discussion during the session on "Integrating University Studies Across the Curriculum" about the process for evaluating and assessing student learning outcomes . It was also agreed that the matrix  developed by faculty involved in the revision of the general education core curriculum will be used by academic departments as an assessment tool to determine where general education learning objectives are being reinforced and where additional attention to these objectives are needed, especially in upper division courses.
Supporting Documents [l1]
 Certificate Programs
 AACSB Maintenance Review Visit Report, October 29-31, 2006
|Last Updated ( Thursday, 18 March 2010 15:18 )|