Alfred State College’s Assessment Plan:

Assessment as a Systematic, Scholarly, and Sustainable Process for Improvement

Contents:

Overview

Philosophy

Foundation for Assessment at Alfred State College

Assessment Implementation: Process

Assessment Implementation Objective List

Appendix A: Selection of Measurement Instruments

Appendix B: Assessment Plan Essential Elements

Appendix C: Glossary of Assessment Terms

Appendix D: Academic Assessment Sample Plan Template

Appendix E: Non-academic Assessment Sample Plan Template

Overview

Alfred State College has a longstanding commitment to excellence in higher education and student achievement. In order to evaluate and improve student learning and the college experience on a continuous basis, this plan is devised to assess student and institutional outcomes and requires the support and participation of the entire college community. Given the college mission as a student-centered institution, this plan focuses on students’ feedback and record of achievement. The results of this examination of student achievement will be one index used in the evaluation of the college’s instructional and administrative programs. The plan focuses on instruction, growth opportunities, and services, although administrative areas and enrollment management will also assess their efforts in achieving the mission of the college. The faculty will take an active leadership role because they represent the best resource of discipline-specific standards and their direct, ongoing contact with students in their pursuit of a degree. Student affairs areas are essential because of the proportion of the students' time spent outside the classroom. The Assessment Coordinator (AC) will administer the assessment program.

Philosophy

Assessment at Alfred State College (ASC) is created to enhance student learning and institutional effectiveness through a rigorous, systematic, and continuous process. The process is defined by a department-centered, outcomes-based plan that examines institutional activities for the purpose of improving learning. The process is inclusive and assesses all aspects of the college. Assessment information is used to produce necessary change or affirm best practices. The assessment plan and process is consistent with the SUNY Assessment Initiative, ASC Faculty Senate endorsed General Education Assessment Plan, and ASC Mission and Core Values, as evidenced by the focus on student learning and the examination of teaching efforts to produce the best outcomes in education. External assessment requirements (i.e., accreditation aspects, stakeholders' issues, and consumer desires) will be addressed through assessment evidence that demonstrate the attainment of desired student learning outcomes. The multidimensional and cohesive nature of assessment will allow the college to meet its mission, support its core values, and excel in the SUNY system with evidence-based teaching and learning activities.

Foundation for Assessment at Alfred State College

With continuous improvement of student achievement and institutional effectiveness as the objectives, assessment at ASC is supported by four convictions:

1) Departments / Programs are the basic unit of analysis and will be evaluated on the basis of data collected in the assessment of student outcomes.

Student academic assessment will be derived from these fundamental questions:

· What should students know (concept, skill, knowledge)?

· How will we know that they know (or can)?

· How can assessment data be used to improve the program?

Student affairs assessment will be derived from these fundamental questions:

· How is student learning enhanced by the services and activities developed, provided, and facilitated by the division?

· What impact does the division's services and activities have on a safe campus environment?

· How does the division promote the development of healthy lifestyles by students?

· In what ways does the division promote an appreciation for, and an understanding of diversity?

· How can assessment data be sued to improve the activities and services of the division?

Administrative affairs assessment will be derived from these fundamental questions:

· Is the division meeting department objectives in the most efficient and cost-effective manner possible?

· Is the institution fiscally sound?

· Is the campus physical plant in good condition and does a plan exist for future improvements?

· Is the institution in compliance with federal, state, and SUNY regulations?

· How effective is the division at meeting the needs of all institutional constituencies?

· How can assessment data be used to improve the division?

Marketing/Enrollment management assessment will be derived from these fundamental questions:

· Did the institution meet its student enrollment targets and enhance its student profile?

· Was the awareness and image of ASC enhanced?

· Were appropriate academic program ideas generated, researched, and appropriate program proposals prepared?

· Do students obtain the necessary information to make informed decisions regarding scholarships and types of financial aid in order to finance their education?

· How can assessment data be used to improve the division?

Institutional Advancement assessment will be derived from these fundamental questions:

· What is IA's rate of return on investments?

· Is the percentage of participation for all annual fund appeals increasing each year?

· Is the average annual fund gift amount increasing each year?

· Is there 100% participation in giving by the Alfred State College Development Foundation board, Alumni Association board, College Council and the Campus leadership team?

· Are we continually increasing our pool of major donor prospects?

· How can assessment data be used to improve the division?

2) Teaching faculty has direct control of the objectives that underlie course and curricular assessment.

Given that teaching faculty has traditionally been responsible for curriculum development and has substantial contact with students, this approach seems the most logical. Furthermore, program assessment is really the culmination of the sum total of course assessments for the entirety of a student’s academic career. Given that faculty has borne the responsibility for each of those parts or elements, it is consistent for them to share responsibility for the collective result.

The faculty and staff are the authors of the departmental program goals and communicate those goals to the AC for review and comment. After comment from the AC, it will be those same faculty who will define what student outcomes best illustrate the accepted program goals and ascertain how to assess learning outcomes. The program administrators, whether faculty or staff, must then determine what instruments will be used to measure those outcomes and to use these instruments appropriately.

3) Instituting compliant assessment processes will be undertaken by every member of the college as quickly as possible. Full and comprehensive implementation will be expected no later than the fall semester of 2002.

Every department will facilitate and enable assessment development and implementation by coordinating all needed activities and ensuring that objectives for outcomes are established quickly. Providing for implementation may also require a reallocation of resources as there are limited resources dedicated to this effort. Departments must also conduct internal audits to ensure that outcomes expected from the program are provided for somewhere in the courses required.

4) The assessment plan itself will be subject to continuous assessment.

No assessment plan is ever final and is always subject to revision as a department or program implements it. The college expects and anticipates that every member of the organization will proceed with assessment with a very critical eye on the process at all levels. This requires an engaged and committed faculty and staff. The procedures and criteria for plans will vary considerably, reflecting each department’s sense of its own nature, needs, intentions and capabilities. This flexibility is viewed as a strength of this plan, to ensure assessment customized and tailored specifically to each program or area of the organization.

General Education

A Faculty Senate standing committee will oversee the implementation and assessment of general education since it has no traditional home academic department. The responsibility for coordinating the collection and storage of data pertaining to student performance in the areas of general education shall be the responsibility of the dean of arts and sciences, which will provide data as requested by the Faculty Senate. The composition of the Faculty Senate committee shall be at the discretion of the Faculty Senate.

Assessment Implementation: Process

As noted, every person in the organization has some level of responsibility for implementing the assessment program at Alfred State College. In order for individuals to identify their role, a general structure must be created to ensure a comprehensive program in which everyone understands their role. As such, the following table summarizes the general area being assessed and defines which individual or constituency has primary responsibility for assessment of its respective area.

Assessment Responsibility Matrix

	President	VP for Admin. Affairs	VP for Academic Affairs	VP for Student Affairs	VP for Marketing and Enrollment	Academic Deans	Department Heads	Faculty Senate	Instructors	Assessment Committee	Assessment Coordinator
Administrative Affairs		X					X
Academic Affairs			X			X	X
Curricular Majors							X
Student Affairs				X			X
Institutional Advancement							X
Marketing and Enrollment Management					X		X
General Education								X
Classes									X
College Assessment Plan										X	X
Institutional Effectiveness	X

Responsibility for developing appropriate assessment initiatives lies with each of those parties defined in the right-hand column. Overall, the faculty and staff are responsible for developing the plan and then implementing it.

Initially, each group will write goals and intended student outcomes relevant to that area. Once outcomes are established, the next phase is to decide assessment criteria and procedures, including the appropriate measurement instruments and standards. Those responsible units must consider the standards of external agencies and be certain to incorporate those into their plans. Whether stimulated by accreditation requirements or local recommendations, an effective plan will reflect appropriate considerations. At this point, the preliminary plan design should be submitted to the AC for comment.

The plan will provide two essential results. First, the plan provides a formal opportunity for a program to communicate the results of self-assessment to interested constituencies. Second, and of no less importance, is the catalyst the plan provides to participants for regular, directed reflection and effective decision-making, for the purpose of improving education.

Once the comprehensive plan is in place, faculty and staff use the plan, collect data and determine the value of the results. Each department or responsible unit must give careful consideration to precisely how the plan and its results will be integrated into planning functions. On a yearly basis, the AC uses the results of the assessment process to provide example implementation for review and comment. The AC will provide feedback at each step as resources and time allow.

Once data has been amassed from the implementation, the faculty and staff will respond to the results in an annual report to the AC. The annual report is to include:

• A copy of the department / program assessment plan
• An overall assessment summary (data and measurement criteria)
• The results report
• Examples of areas of need (conclusions) and intended actions
• General comments on the assessment process.

The AC has the responsibility for ensuring that the responsible units move from assessment of one objective to another, using multiple measures to assess each outcome. Furthermore, the AC oversees compliance and adherence to the plan as laid out initially to facilitate a fair and productive review of the plan’s effectiveness.

Assessment Plan Objective List

What follows is the proposed implementation schedule through until the end of the 2001-2002 academic year. It is based on the premise that serious work on assessment began in spring of 2001, in preparation for a program launch in fall of 2002. The first year is devoted primarily to education of the faculty and staff and development of the program. It is anticipated that this first stage will produce a comprehensive assessment program that can be fully implemented in fall of 2002.

The implementation consists of three main areas:

· EDUCATION of faculty, staff, and administration

· DEVELOPMENT of appropriate assessment methods

· PROMOTION of efforts to the stakeholders of Alfred State College

When this first year and a half have been completed, the college will have achieved the following objectives:

· Creation of a half-time assessment coordinator position

· Creation of a comprehensive assessment program that includes assessment of both the major and general education, as measured by a detailed plan submitted by the AC in April 2002 and approved by both Faculty Senate and the VPAA

· Education of at least thirty percent of the faculty regarding the principles underlying successful assessment, as measured by attendance at workshops and surveys, coordinated by the AC.

· Support of at least thirty percent of the faculty for the assessment initiative as measured by surveys

· Promotion of the college’s assessment efforts to the following stakeholders: students, faculty, College Council, and SUNY system administration, as measured by the presentation of information to these groups.

Appendix A: Selection of Measurement Instruments

Much latitude is extended to assessing units in the decisions regarding how student achievement will be measured. It is imperative, however, that instruments are valid and subject to reasonable interpretation addressing the objectives identified. It is anticipated that instruments prepared and designed internally, nationally (or regionally) normed tests, as well as vocation-dictated examinations will factor into the process at some point. In keeping with principles of sound statistical theory, assessment plans should include multiple measures.

In the event that units plan to use examination data as indication of student mastery of content, these must be explained in the Assessment Plan.

A full range of instruments may be applied, including:

Instructor examinations Practical examinations

Professional testing Portfolio review

Student surveys Self-assessment tools

Employer surveys Alumni surveys

Rate of acceptance into other programs

Appendix B: Assessment Plan Essential Elements

Every assessing unit must draft a plan including the following:

1. A statement of the program’s purpose
2. A series of intended student outcomes or objectives for the program
3. A series of assessment procedures, criteria and standards that can be used to demonstrate level of achievement of those outcomes or objectives
4. An explanation of how the department or area intends to use the assessment results for improvement.

The idea behind these plans is to focus on program performance as evidenced by student attainment of knowledge, skills, and reasoning abilities. These statements must be directly connected so that each part is read in relation to the part preceding it. Thus, an assessing unit’s statement of purpose (as it relates to student learning) is made specific by its objectives which are directly addressed by the procedures and criteria identified.

Statement of Purpose-

A statement of overall purpose defines the plan’s larger contexts (e.g., department > campus > college > SUNY). Developing this portion of the plan allows departments programs and administrative groups to reaffirm their specific roles within the college’s more generally stated purpose.

Intended Outcomes and Objectives-

The plan specifies what graduates will know or be able to do as a result of this group’s influence. What concept, skill or knowledge is to be assessed? Optimally, these will be stated in three to five clear statements indicating specifically what students are expected to achieve. It is important to be concise, clear, and specific.

Assessment Procedures and Criteria-

This part of the plan should show how assessment would demonstrate some level of achievement of the outcomes and objectives identified in the previous step. Is reasoning, memory or process the subject of this assessment? In many cases, assessment instruments are already being used, so these will be identified. It is important to ask whether the stated procedures and their associated criteria actually, adequately and specifically measure or demonstrate the identified outcome.

It is desirable that all assessing units have consensus on the statements contained in their overall plan, as each member of that unit will be required to contribute in some way to student attainment of those outcomes and objectives. Furthermore, all members will want to be intimately involved in devising the strategy for assessment as they relate to their own specific instructional areas. This ensures that faculty, staff, and administration are working in concert for best effect and that students will be consistently prepared when the assessment is administered.

Appendix C: Glossary of Assessment Terms

Assessment: an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance. When it is embedded effectively within larger institutional systems, assessment can help focus the collective attention, examine assumptions, and create a shared educational culture dedicated to assuring and improving the quality of higher education (Thomas A. Angelo, AAHE Bulletin, November 1995, p.7). At Alfred, assessment will not take place at the level of the individual student, but rather at the level of groups of students, courses, programs, and the college as a whole.

Criterion-Referenced Test: a measurement of achievement of specific criteria or skills in terms of absolute levels of mastery. The focus is on performance of an individual as measured against a standard or criteria rather than against performance of others who take the same test, as with norm-referenced tests.

Diagnostic Test: an intensive, in-depth evaluation process with a relatively detailed and narrow coverage of a specific area. The purpose of this test is to determine the specific learning needs of individual students and to be able to meet those needs through regular or remedial classroom instruction.

Domain-Referenced Test: a test in which performance is measured against a well defined set of tasks or body of knowledge (domain). Domain - referenced tests are a specific set of criterion-referenced tests and have a similar purpose.

Evaluation: determination of the overall success of individual students.

Informal Test: a non-standardized test that is designed to give an approximate index of an individual's level of ability or learning style; often teacher-constructed.

Measurement: determination of the success of individual students at meeting objectives.

Norm: performance standards that are established by a reference group and that describe average or typical performance. Usually, testing a representative group and then calculating the group’s test performance determine norms.

Norm-Referenced Test: an objective test that is standardized on a group of individuals whose performance is evaluated in relation to the performance of others; contrasted with criterion-referenced test.

Objective: a desired result. An objective should contain an explanation of the intended result and reference to the method used to measure success in attaining the result.

Outcomes: the result of a particular program.

Performance Test: designed to evaluate general intelligence or aptitudes. Consists primarily of motor items or perceptual items because verbal abilities play a minimal role.

Rating Scales: subjective assessments made on predetermined criteria in the form of a scale. Rating scales include numerical scales or descriptive scales. Forced choice rating scales require that the rater determine whether an individual demonstrates more of one trait than another.

Raw Score: the number of items that are answered correctly.

Reliability: the extent of which a test is dependable, stable, and consistent when administered to the same individuals on different occasions. Technically, this is a statistical term that defines the extent of which errors of measurement are absent from a measurement instrument.

Screening: a fast, efficient measurement for a large population to identify individuals who may deviate in a specified area, such as the incidence of maladjustment or readiness for academic work.

Specimen Set: a sample set of testing materials that are available from a commercial test publisher. May include a complete individual test without multiple copies or a copy of the basic test and administration procedures.

Standardized Test: a form of measurement that has been normed against a specific population. Standardization is obtained by administering the test to a given population, and then calculating means, standard deviations, standardized scores, and percentiles. Equivalent scores are then produced for comparisons of an individual score to the norm group's performance.

Standard Scores: a score that is expressed as a deviation from a population mean.

Validity: the extent to which a test measures what it was intended to measure. Validity indicates the degree of accuracy of either predictions or inferences based upon a test score.

Appendix D: Academic Assessment Sample Plan Template

What follows below is an assessment plan model that meets the criteria outlined in this report. The plan need not be lengthy or overly detailed. A department’s plan may vary significantly from this model with respect to number or type of outcomes, number or type of assessment instruments or methods, or timetable. This template is intended only as a guide for departments who believe they would benefit from seeing an example of an assessment plan.

Program: Curriculum 880 (AS degree in Philosophy)

Program Purpose: The purpose of this program is twofold: First, the program provides graduates with a comprehensive foundation in the liberal arts for the purpose of successful college transfer; second, the program provides graduates with the skills to critically analyze and improve their own lives and the world around them through the application of established philosophical principles.

Program Objectives:

• Differentiate between facts, opinions and inferences as measured by successful completion of the course-specific learning objectives of PHIL 1401 (Introduction to Philosophy).
• Compare and contrast the relative merits of opposing arguments and interpretations, moving between the main points of each position as measured by successful completion of the course-specific learning objectives of COMP 1503 (Composition), PHIL 1401 (Introduction to Philosophy), and PHIL 1503 (Critical Thinking).
• Identify common errors in reasoning as measured by successful completion of the course-specific learning objectives of PHIL 1503 (Critical Thinking).
• Explain at least two examples where cultural factors have influenced the method of reasoning and/or moral decision-making process as measured by successful completion PHIL 2073 (Philosophy Across Cultures).
• Recount the general history of ethics as measured by successful completion PHIL 1401 (Introduction to Philosophy) and PHIL 2053 (Ethics).
• Manage information, which involves such tasks as sorting information, ranking data for significance, and synthesizing facts, concepts and principles, as measured by successful completion FYCE 1053 (Freshman Year Experience) and PHIL 1503 (Critical Thinking).
• Communicate thoughts clearly and concisely in written and verbal form as measured by successful completion COMP 1503 (Freshman Composition), LIT 2603 (Introduction to Literature), and PHIL 1503 (Critical Thinking).

Assessment Procedure

Our program assessment consists of five main strategies:

1. Evaluation of existing student work through sampling
2. Evaluation of graduates through surveys
3. Evaluation of course content through course evaluation forms and instructor feedback
4. Evaluation of program through periodic external review conducted by American Philosophical Association
5. Evaluation of program assessment plan

Strategy detail:

1. Existing student work: A standing department committee will choose one learning objective per year for evaluation. This committee shall design a procedure for evaluating a sample of student work that demonstrates the degree to which the outcomes are met. The committee shall recommend action that (1) makes suggestions for improving student attainment of that objective, and (2) suggests improvements, if any, to the objective itself. The department will operate on the assumption that some significant change needs to be made if less that 60% of students are meeting the selected objective.
2. Alumni surveys: One department member shall be responsible for distribution and analysis of an alumni survey once every three years. The survey shall be created by a departmental committee, and shall be designed to measure at least one purpose of the program as well as one objective. Depending on the participation level of the survey (minimum participation 25%), the department will operate on the assumption that significant change needs to be made if less that 75% of respondents are meeting the selected purpose, or if less that 60% of the respondents are meeting the selected objective.
3. Evaluation of course content through course evaluation forms and instructor feedback: At the end of each academic year, the department will meet to discuss the content of one course. Instructors of the course being discussed shall provide from their course outlines at least one recurring or important criticism by a student with regard to the course content. The instructors of the course shall make improvements to the course as they see fit and make a note of these changes for possible inclusion in the department’s annual report. The following year, the instructors involved will report briefly at a department meeting on the impact of the improvement on student learning.
4. Evaluation of program through periodic external review conducted by American Philosophical Association: Our department participates in a voluntary external review process that occurs every five years. This result of this review will be discussed and acted upon, and we will consider including the review as an appendix in our department’s annual report.
5. Evaluation of program assessment plan: In the year following our external review, a departmental committee will meet to discuss and revise the department’s assessment plan. Prior to the meeting, the department will solicit comments on the plan from the assessment coordinator and school dean.

Appendix E: Non-academic Assessment Sample Plan Template

Department: Conference and Events Office (CEO)

Departmental Responsibilities:

1) Planning and executing “annual” campus-wide events – New Student Convocation, Honor’s Convocation, Commencement

2) Providing logistical support for other campus-wide events as required

3) Assisting departments in hosting campus-related conferences

4) Marketing the campus as a host site for “outside” conferences

5) Coordinating campus support services for “outside” users

Departmental Objectives:

• “Regularize” annual events so that they become traditions for the campus
• Maximize appropriate constituent participation in campus-wide events
• Compare services and pricing to regional competitors
• Increase Alfred State’s name recognition by hosting more SUNY-wide meetings and conferences
• Increase external revenue by hosting more non-affiliated conferences

Assessment Procedures:

Faculty, professional staff participation in campus-wide events	Participation of 75% or more of the faculty shall be considered acceptable. If less than 75% then we need to consult the representative organizations (UUP, Faculty Senate) for feedback. Regardless of percentage, all faculty and staff not in attendance should be contacted to find out what we could have done to ensure their participation.
Student participation in campus-wide events	Participation of 90% or more of appropriate students shall be considered acceptable. If less than 90% then we need to consult the Student Senate. We will need to determine if students did not know of the events or if they did not care to attend. Depending upon the feedback, we will review publicity for the events and/or re-design programs to include more interesting content for the students.
Family attendance at campus-wide events	10% increase (over last year) of families of participating students is desired. If successfully attained, we should note timelines and publicity procedures for the future. If not, then we need to sample families to find out what is desired in programs to make them valuable for them.
Faculty, student and alumni opinion surveys	Appropriate constituents will be surveyed to determine the trends in response to events.
Surveys of existing facilities and services (including personnel)	These will be compared with other regional hosts of conferences and events to determine if we are charging appropriate and competitive rates for facilities and services.

Faculty survey	This will be used to determine what SUNY-wide and professional organizations faculty and professional staff belong to and if they consider Alfred a suitable host for meetings and gatherings. If so, we can assist in promotion of ASC as a host. If not, what are the concerns? Are these addressable?
Focus groups of local businesses of marketing materials	If the materials are determined effective, we will begin a campaign of marketing the college to non-affiliated organizations. If not, then we will solicit feedback on improving the materials.
Evaluation of department assessment plan	The department will annually review its assessment plan. Evaluation will include the solicitation of comments from departmental users and campus service providers. Areas of concern will be focused on for improvement.