• Decrease font size
  • Reset font size to default
  • Increase font size

Main Menu

2.0 Core Requirements

3.0 Comprehensive Standards

4.0 Federal Requirements

Submit Content

3.3.1.3 Assessment of Educational Support Services PDF Print E-mail
Thursday, 03 September 2009 14:56

3.3.1.3 Assessment of Educational Support Services

The institution identifies expected outcomes, assesses whether it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas: educational support services.

Responsible Unit: Division of Academic Affairs/IPAR

Compliance Judgment

Compliance

Narrative

North Carolina Agricultural and Technical State University (A&T) offers educational support services throughout the campus.  The services are offered in academic and student affairs.  Each educational support area is responsible for assessing its respective programs and including this information as part of its annual report.  Each annual report [1] requires the unit to indicate how the unit's mission, vision and goals align with the University's strategic plan (FUTURES) [2].

Educational support service units are required to submit annual reports that outline the following:

  • Mission
  • Vision
  • Goals
  • Progress indicators toward goals
  • Data summary
  • Outcomes
  • Significant accomplishments

Units can receive assessment training through the Office of Institutional Planning and Research (IPAR), where staff provide assistance by conducting workshops, trainings and consultations on how to measure unit effectiveness and outcomes.  The Academy for Teaching and Learning (ATL) also offers workshops throughout the academic year on assessment.  Some educational support units contract the services of outside agencies to provide assessment and evaluation services.

The educational support services offered throughout the university seek to address the academic needs of all students and assist them in progress towards their degree.  Educational support units include the Center for Academic Excellence (CAE), Summer Bridge Program, the Honors Program, and the Division of Student Affairs.

Center for Academic Excellence: The Center for Academic Excellence promotes academic success of all students and cultivates life skills critical to timely graduation and global citizenship with emphasis on freshman and sophomore undergraduate students, ultimately increasing retention. CAE offers services and programs that include academic advising, supplemental instruction, tutorials, peer mentoring, male retention initiative, student athlete academic enhancement, monitoring, study hall, learning assistance, and a curriculum offering developmental courses in reading and mathematics. CAE promotes student success by providing academic support, on-demand and intrusive advising, and facilitating choice of major and career direction to undeclared students (freshmen and sophomores) and student athletes. Student satisfaction with integration in and adjustment to the university community through a first-year orientation course (University Studies 100) is taught by many academic counselor/lecturers in CAE. [3]

Assessment Strategies: CAE utilized surveys, focus groups, and student evaluations to assess programs.  In addition, CAE has contracted with MGT of America, Inc. (MGT), a national management research and consulting firm to conduct an annual assessment of programs and services to determine student satisfaction and effectiveness of retention programs.  MGT uses both quantitative and qualitative methodologies to assess CAE.

MGT surveys and focus group results indicated the most-used services in CAE were advising, individual academic tutoring from CAE staff members, attending tutorial sessions and working with CAE staff members to develop an academic plan of action.  Results also indicated CAE has low visibility in part due to its location and should create initiatives to broaden its campus reach and appeal [4].

In order to address the deficiencies and enhance the strengths outlined in MGT's findings, CAE enacted the following improvements and department enhancements to aid in student retention as well as other initiatives to increase CAE's efficacy.

Changes/Improvements

  • Expanded tutorial hours during semester
  • Tailored tutorial subjects to meet demand
  • Implemented Study Blitz (SB), (extended tutorial hours) during finals week.
  • Implemented residence hHall advising and tutorials
  • Implemented AdviseTrack Excel document that allows advisors to monitor pertinent student information
  • Redesigned departmental website
  • Enhanced Student Athlete Academic Enhancement Program (SAAEP) Study Hall
  • Improved communication with schools/colleges regarding advising and departmental policies and procedures by holding monthly CAE staff meeting in schools/colleges.
  • Developed and taught Strategies for Academic Success (SCS 099) probationary course
  • Implemented tutorial tracking system to document number of visits, subjects, etc.

Undergraduate Academic Advising (Undeclared): CAE is responsible for the academic advisement of undecided/undeclared students.  Advisors assist students with course and major selection and provide students with tools that foster personal responsibility, accountability and commitment.  The CAE collaborated with the Office of Institutional Research to design and implement a new tracking system, AdviseTrak, which allows advisors the ability to track student data and monitor student progress.

Assessment Strategies: CAE, formerly The Center for Student Success (TCSS), was evaluated by IRC with a resulting twenty-five recommendations [4] [5]. Student surveys and focus groups were conducted by MGT of America, Inc. MGT staff conducted a survey of students who used CAE services in spring of 2008.  In addition they conducted faculty/staff surveys and focus groups.  Student survey findings yielded positive data about the quality of the services offered within CAE but brought up advisement and documentation inconsistencies.  Faculty results indicated a concern for work overload between teaching classes and a heavy advisement load would decrease their effectiveness as advisors [4].  The National Survey of Student Engagement (NSSE) is also used to assess academic advisors.  According to NSSE data, freshman and seniors overwhelmingly reported that their academic advising experience was fair, good or excellent [6].

The Center for Academic Excellence addressed the departmental concerns highlighted in the assessment report with the following actions to improve work-place efficiency and increase the quality of student service.

Changes/improvements

  • Implemented folder system to organize advisee information
  • Reassigned advisees based on alpha order instead of intended major
  • Established Engineering Intended and Nursing Intended advising programs
  • Assigned a UNST 100 section to Engineering Intended students
  • Established Male Retention Initiative to address low retention
  • Implemented AdviseTrak system
  • Developed advising training program for new and professional advisors
  • Reduced advising and teaching loads

Undergraduate Academic Advising (Declared): Academic advising for declared students is the responsibility of each individual academic department.  Each school and college has their own system of advising and is responsible for assessment of faculty advisors and the advising process.  To supplement faculty advising, CAE retention advisors are assigned to each school and college to provide assistance to declared students by monitoring their academic progress and implementing intervention strategies.

Assessment Strategies: MGT of America, Inc. conducted surveys and focus groups with declared students and faculty.  The data from these surveys and focus groups was used in conjunction with data from the National Survey of Student Engagement (NSSE) to assess the effectiveness of academic advisement efforts.

The academic advisement assessment revealed areas in which the process needs improvement as far as the information advisors are disseminating to students and to refine advisement tactics as a whole.  In response to the findings of the assessment, the following changes or improvements were made by CAE to enhance the level of academic advising for declared students.

Changes/Improvements:

  • Developed advising training program for new and existing faculty
  • Established the Faculty Academic Advising Council (FAAC)
  • Reviewed and improved the selection process for Advisor of the Year
  • Collaborated with the Academy for Teaching and Learning to conduct faculty academic advising workshops

Supplemental Instruction (SI) Program: This is a student academic assistance program that utilizes SI leaders (peer assistants) to facilitate regularly scheduled, informal weekly study sessions.  Courses selected for the SI program were among the seventeen freshmen-level courses that met the criteria of 30% of students receiving F, W, I grades (no Pass/Fail courses were included) and at least ten students enrolled in the course. SI targets first-year traditionally difficult courses to empower students with collaborative and active learning and study strategies to aid them in conquering these courses.

Assessment Strategies: To determine the effectiveness of the Supplemental Instruction (SI) Program, student surveys were administered to gain user feedback.  Grade analysis, instructor feedback and SI session observations were other strategies implemented to obtain assessment data [7].  Attendance and student satisfaction surveys are included as part of the SI assessment.  SI student leaders provided feedback on the program's effectiveness through weekly meetings and bi-annual debriefing surveys [8].

The evaluation of the SI Program is done at the end of the term after final exams and grades have been submitted in an effort to determine how well the SI Program is doing.  SI participants and non-SI participants are evaluated by measuring retention and learning.  Retention is defined for SI purposes as students remaining in the class for the full term.  Learning is measured by the mean final course grades.  The data gained from the surveys and other assessment tactics indicated that knowledge of the SI Program has limited notoriety.  Students commented that the SI leaders could benefit from formalized training in conducting the SI sessions.

SI assessments have indicated that changes need to be made to increase the overall SI attendance, percentage of students passing a particular class, and the impact of instructor/faculty satisfaction with the program. The following changes were made to the Supplemental Instruction Program to improve upon the information gained from the assessment data.  The program also implemented changes that addressed a growing concern for faculty-identified at risk students.

Changes/Improvements

  • Marketing strategies to enhance the SI program include:  Marketing the program through the University-wide Retention Committee; meeting annually with the University-wide Retention Committee to outline current progress towards goals; updating the website regularly; developing brochures, book marks and other publications; initiating close relationships with other academic departments, professors and programs
  • Improved training of SI leaders
  • Utilization of class time for SI sessions
  • Improved communication with faculty of SI classes
  • Offered SI training to other units to increase SI offerings
  • Require mandatory SI for students in University Studies general core classes

Summer Bridge Program (SBP): A&T's academic Summer Bridge Program is designed to provide regular and provisionally admitted students an opportunity to get a jump start on college.  Students are selected to participate in a rigorous five-week residential program that operates under the university's Summer Sessions and Outreach program. The program is math and English intensive with participation in skills-building workshops and seminars.

Assessment Strategies: While no formal assessment instrument has been used for the 2007 and 2008 Summer Bridge Program, each year the faculty and staff are debriefed to identify areas for improvement.  Since the program was based on sound past experiences and practices, no changes were made the first two years.

During the 2009 Summer Bridge Program, Accuplacer writing and math tests and Criterion, the Educational Testing Service writing evaluation instrument, were used as assessment tools.  We also requested weekly reflections on how students were doing in class as well as suggestions for programmatic improvements.  At the end of the program we asked everyone (i.e. faculty, students, counselors, administrators, etc.) for feedback about what went right, what went wrong, and suggestions for improvement.  Student data (grades, diagnostic scores, etc.) were tracked throughout the semester and records kept for the university and University of North Carolina General Administration.

The primary assessments are student grades in classes for the first year and their retention rate for their sophomore and junior years. The 2009 program required students to take a battery of assessments including the College Student Inventory, learning styles assessment, and pre- and post-tests in math and writing.  Students, faculty and staff of the 2009 program wrote personal reflections detailing their experience.  These reflections will be used in program assessment and improvement.  Students, as a result of the academic Summer Bridge Program:

  • Increased grammar and mechanics test scores from 62% to 78%,
  • Increased writing test scores from 73% to 82%,
  • High school students with low GPAs averaged a 24.5 point increase,
  • Low-scoring math SAT students averaged a 25.3 point increase [9].

Changes/Improvements

  • Shifted focus of Summer Bridge Program to provisionally admitted students instead of regularly admitted
  • Increased course contact hours during summer bridge courses
  • Utilized block scheduling of students for subsequent fall semester
  • Developed Academic Boot Camp to engage students prior to SBP
  • Added Supplemental Instruction (SI) component to SBP

The Honors Program: The Honors Program seeks to recruit and retain exceptional students by nurturing their academic development, broadening their cultural awareness by providing pre-professional leadership training and involving them in community service. The Honors Program offers high-achieving and motivated undergraduates the opportunity to participate in student-centered learning experiences that promote intellectual growth, cultural appreciation, international experiences, professional focus, leadership development, and civic participation, as well as helping students prepare for graduate school.

Assessment Strategies: An external review of the Honors Program was conducted by Tennessee State University's director of honors. This external assessment specified areas for improvement including the development of more effective assessment tools for students in the program and those who have graduated, increasing campus awareness of the program, and creating an honors curriculum.  The assessment also indicated that the Honors Program has matured to a point where the university should consider reorganizing into an honors college [10].

To coincide with the assessments recommendation a second external assessment was conducted by University of South Carolina Honors College as to the readiness of A&T's Honors Program to become an honors college.  The assessment conducted by dean emeritus, Peter C. Sederberg, outlined several recommendations that can position HP to transition to an honors college, including raising admission standards and developing honors sections of core courses in the most popular majors [11].

The Honors Program, in accordance with the recommendations made by both external reviews, enacted the following changes and/or improvements with a plan to initiate further developments for the following academic year.

Changes/Improvements

  • Implemented surveys of current and graduating seniors for program feedback
  • Created honors section of UNST 100
  • Development of an honors curriculum entails an honors core, major courses, experiential learning, and capstone experience
  • Development of a systematic assessment process for honors classes
  • Expand and intensify recruitment efforts
  • Expand the University Honors Lecture Series

Division of Student Affairs: A&T provides student support programs, services, and activities consistent with its mission that promote student learning and enhance the development of its students [12]. The university endorses the FUTURES strategic vision as an interdisciplinary university in a learner-centered community that develops and preserves intellectual capital through interdisciplinary learning, discovery, engagement, and operational excellence [2].

The Office of the Vice Chancellor for Student Affairs is charged with the responsibility of supporting the academic mission of the university through a broad range of opportunities for personal and professional growth and development [12].  The office partners with the Office of the Provost and Vice Chancellor for Academic Affairs to administer programs, activities, and services to extend student learning experiences beyond the classroom and enrollment management.  Thus being consistent with the institutional strategic plan, FUTURES builds upon disciplinary excellence while inspiring new possibilities beyond the field of study through cross functional teams to prepare individuals to be effective citizens in a global society. Students are the heartbeat of the campus and as such A&T embraces student support programs, services, and activities in a wide variety of units.  This section will focus on programs, services, and activities under the direction of the Division of Student Affairs.  Additional student supports services are reported in Principle 2.9 Learning Resources and Services, Principle 3.4.9 Academic Support Services and Principle 3.8.1 Library Support Services.

The Division of Student Affairs provides services to complement the academic mission by contributing to the intellectual, cognitive, social, personal, and physical competencies of students.  Furthermore, programs and services provided within the division support the community of learners and are broadly categorized into four (4) major areas:  (1) academic support, (2) personal growth, (3) professional development, (4) diversity and (5) wellness.  Specifically, the Division of Student Affairs plays a vital role in the personal growth, wellness, academic achievement, and the intellectual and professional development of each individual student by:

  • Fostering a campus culture for students to articulate their own goals for success;
  • Enhancing the learning environment and expanding learning opportunities inside and outside of the classroom;
  • Promoting a campus climate to value, accept and learn from our rich diversity by expanding the opportunities for students to acquire local, regional and international experiences; and
  • Developing leaders who believe in and lead others toward supporting civility, mutual respect, and diversity in our society through expanding workplaces that include global experiences, advanced technology and physical resources to develop students' full potential.

Assessment Strategies: To ensure that the Division of Student Affairs continues to play a vital and effective role in the lives of students at this university, several assessment tools are implemented such as surveys, site visits, evaluation forms, committee meetings and performance reports. [13] [14] [15]

As a result of the various assessment activities, the following changes and or improvements were incorporated into the operating structure of The Division of Student Affairs:

Changes/Improvements

  • Established flexible hours, locations and program advertisement for the Counseling Services Center (CSC),
  • Added an online mental health screening to the Counseling Services Center website,
  • Created walk-in hours for the Office of Career Services (OCS)
  • Added distance counseling via phone or Internet, networking events for specified majors and the implementation of an online registration system, assessments and on-campus interviews
  • Convened bi-weekly meetings for students with attention deficit hyperactivity disorder and increased contact with professors for timely interventions
  • Improved orientation registration and tracking system via org sync

The assessment program within the Division of Student Affairs has moved from a student satisfaction/program improvement program to a student learning outcome-based program-this began in October 2007. Executive directors are expected to articulate at least one significant student learning outcome that they would like students who participate in their programs or utilize their services to achieve annually.  The assessment leadership team and the executive directors have laid the foundation for evidence-based decision making and outcome-based assessment.  Student affairs programs, activities, and services are evaluated in a variety of formats, including campus-wide surveys, individual program and service satisfaction surveys, and focus groups.  A sample assessment report for the Sebastian Health Center [16] demonstrates the level of detail involved in the assessment of student affairs programs.

Academy for Teaching and Learning: Started in 2001, the Academy for Teaching and Learning (ATL) was housed in the Information Technology and Telecommunications division until it was moved to Academic Affairs in January 2004.  The director of the restructured ATL reports to the provost.  In April 2004, the director formed the ATL Advisory Board to advise on the strategic directions and initiatives of the academy, to promote communication among faculty, staff, administrators and the ATL Director, and to act as ambassadors of the teaching and learning mission of the university. [17] ATL has become a resource to help strengthen teaching.

Committed to supporting skills building in teaching and learning, the initial ATL goals were to:  (1) redefine the focus and structure of ATL and to root it in the science of learning; (2) initiate ATL activities related to the new focus; and (3) select the ATL Advisory Board (pp. 6-8) [17].  The ATL Summer Teaching Institute in May 2004 attracted 65-70 faculty members.  The Blackboard users group improved communication between faculty and information technology staff.  In addition, ATL developed an interdisciplinary theme, "Focus on Learning," for the 2004-2005 academic year with plans for workshops, faculty learning communities, and reading groups on the theme of learning and the implications for effective teaching (pp. 8-9) [17].

ATL has increased faculty knowledge of the factors that affect student learning, encouraged scholarly teaching grounded in research on student learning, and promoted the scholarship of teaching and learning, including classroom-based research, assessment of learning outcomes, public sharing of effective teaching practices, and basic research on teaching and learning issues (p. 2) [18].

For example, in 2004-2005, ATL implemented a luncheon series, "First Thursday," that continued through 2008-2009 and serves as a year-long orientation for new faculty.  Topics relevant for new faculty include:  professional expectations for promotion and tenure, writing for publication, balancing professional and personal responsibilities, teaching with technology, and mentoring (p. 7) [18].  In 2004-2005, approximately 250 faculty and staff attended the ten lunches (pp. 23-26) [18].  Since that first series, there have been ten to twenty attendees on average each year.  Many of the same faculty and staff attend the luncheons.

In academic year 2005-2006, faculty evaluations of the lunch series indicated high satisfaction with the program.  Attendees rated as most beneficial the sessions on research and grants, professional expectations, and effective teaching practices.  Responding to the recommendations from the 2004-2005 cohort, the session on professional expectations for tenure and promotion was placed on the schedule earlier in the year (p. 8) [19].

The director defined the assessment measures for determining ATL progress (p. 21) [18]:

  • Increased participation of faculty in the new faculty orientation program.
  • Increased numbers of presentations by faculty and/or participation in regional and national teaching/learning conferences, as well as disciplinary conferences on teaching/learning.
  • Increased faculty participation in sponsored research related to teaching/learning projects.
  • Increased participation of faculty in on-campus workshops, discussions, faculty learning communities, and Teaching Showcase poster sessions.
  • Continuing development of University Studies courses and development of self-contained communities of practice.
  • Increased numbers of awards for teaching/learning.
  • Development of formal criteria and rewards for documented scholarly teaching and scholarship of teaching/learning in contract-renewal, promotion, tenure, and post-tenure review decisions.
  • Increased awareness and intentional integration of teaching/learning issues in curriculum development, program review, and course assessment.

If successful, ATL programs will result in changes in the intellectual environment of the university and improved student learning and teaching effectiveness. These improvements will be visible in new courses being developed, experiments with interdisciplinary and team-based teaching, increased classroom-based research on student learning, and the development of innovative assessment practices, such as the use of student portfolios.  In addition, over time changes should occur in teaching pedagogy, curricular design, assessment of student learning outcomes, and alignment with overall learning objectives.

In 2005-2006, the director observed that, anecdotally, ATL was "making a difference" in changing campus culture and attitudes regarding teaching and learning.  In conversations with faculty members, there appeared to be greater interest in seeking out funding for teaching/learning related projects, greater emphasis on pedagogy and assessment of general education learning objectives.  ATL receives strong positive evaluations for bringing faculty together across disciplines (p. 19) [19].

ATL continues to initiate new projects and programs.  In 2006-2007, in response to Collaborative on Academic Careers in Higher Education (COACHE), the national survey of junior faculty job satisfaction, a twelve-member task force was convened to make recommendations for improving the work environment for junior faculty (p. 3) [20].  In 2007-2008, twenty-six faculty members and students worked on the Carnegie Academy for the Scholarship of Teaching and Learning (CASTL) project to develop measurable learning outcomes for university-wide student learning expectations and summarizing currently available institutional data on student learning outcomes [21].  Participation in the Wabash National Study of Liberal Arts Education provided rich insights into student learning and the factors that lead to student success, along with recommendations for improving student learning outcomes, retention, and graduation rates (p 2) [22].  The Academy for Teaching and Learning plays a central role in building and sustaining academic excellence at North Carolina A&T State University [23].

Supporting Documents

[1]        Annual Report Outline, Academic Affairs 2008

[2]        FUTURES, Uncomprimising Excellence

[3]        Annual Report, Center for Academic Excellence 2008

[4]        Assessment Report, MGT of America CAE

[5]        Assessment/Evaluation, CAE Responses to Institutional Research Consultants (IRC) Team's Findings and Recommendations for The Center for Student Success (TCSS)

[6]        Survey, National of Student Engagement (NSSE),

[7]        Annual Report, Supplemental Instruction

[8]        Survey, Supplemental Instruction Debrief

[9]        Preliminary Report, Academic Summer Bridge Program

[10]      Annual Report, Honors Program 2008

[11]      Assessment Report, Honors Program External

[12]      Annual Report, Student Affairs, 2008-09

[13]      Annual Report, Counseling Services Center 2009

[14]      Annual Report, Career Services, 2008-09

[15]      Website, First Year Experience

[16]      Evaluation, Career Services Workshop

[17]      Student Affairs, Sebastian Health Center report

[18]      Annual Report, ATL 2003-2004

[19]      Annual Report, ATL 2004-2005

[20]      Annual Report, ATL 2005-2006

[21]      Annual Report, ATL 2006-2007

[22]      Annual Report, ATL 2007-2008

[23]      Annual Report, ATL 2008-2009

Go to Top

 

 

Last Updated ( Monday, 08 February 2010 14:52 )
 

Get Adobe Reader

To view supporting documents you will need Adobe Reader.

The SACS website and tools have been optimized for the Firefox web browser.