CSAM Summary Report on Program Assessment

April 1, 2003

College of Science and Mathematics Assessment Committee

• Jim Fulmer, Committee Chair, Department of Mathematics

• Cindy Gilbert, Department of Nursing

• Larry Coleman, Department of Physics and Astronomy

• Marian Douglas, Department of Chemistry

• Paul Akers, Department of Biology

• Thea Spatz, Department of Health Sciences

• Jeff Connelly, Department of Earth Science

The College of Science and Mathematics used to a 0 to 4 point scale to rate this year’s program assessment reports. This scale corresponds to the Exceptional (4), Reasonable (3), Limited (2), and Inadequate (1) criteria distributed by the Provost’s Office. A score of 0 indicates that the area of the report was not present. The following table is a summary of the scores for each program.

All but one of the undergraduate programs in the College of Science and Mathematics are rated at reasonable level, or above, in program assessment based on the PAAG criteria rubric. In general the undergraduate programs are doing a good job with assessment, although there is variable across the college and room for improvement. It is apparent that there is a culture of assessment that is now imbedded in the undergraduate programs in the college. The step between reasonable and exceptional in every category is a difficult step to make. Two programs have made that step, the B.S. in Geology and A.S. in Nursing. Two other programs have made the step to exceptional in at least one category, the B.S./B.A. in Mathematics and the M.S. in Applied Mathematics.

The graduate programs have the most room for improvement in their assessment activities. There are two newer graduate programs (Biology and Integrated Science) and two established programs (Applied Mathematics and Chemistry) in the college. The Applied Mathematics program is the only graduate program in the college that has a reasonable/exceptional assessment plan. Both the Chemistry and the MSISM program have limited assessment activities in place. The Biology program did not submit a report this year, but has submitted an assessment plan to the college committee. Therefore, the graduate program in Biology will submit an assessment progress report next year.

Strengths:

• There is considerably more assessment data being collected. It is obvious that more implementation has occurred and this has resulted in more useful assessment data.
  
• Most programs have good learning objectives that are linked to the goals of their programs. In addition most programs have methods to assess the learning objectives.
  
• There is evidence that the “assessment feedback loop” is closed in more programs this year than in previous years.
  
• There has been improvement in program assessment in the programs that comprise CSAM over the past few years. As a result of continued emphasis on assessment, the college is developing faculty expertise in the area of program assessment in the programs that are scoring well in the review process.
  
• A benefit of participation in the assessment process is that each program has had to examine and evaluate student learning in respect to learning objectives. Programs have discovered areas in which they are doing a good job as well as areas in which they need improvement. Faculty have become involved in assessment and have a more comprehensive idea of the program and its goals as a whole rather than the narrower view of individual courses.
  
• Data are being collected from a wider range of sources, covering more diverse areas of assessment.
  
• Validity and reliability are being measured by some programs.

Areas of concern:

• It concerns to be apparent that MS programs are struggling with the assessment process.
  
• Departments are collecting information from stakeholders, but are we sharing the results of assessment with our stakeholders (students, alumni, and employers)?
  
• How can the Assessment Committee determine the reliability and validity of the program evaluations?
  
• There is some concern regarding the assessment and programs with statistics involving small number

Recommendations and Comments:

• There is a risk that programs and people external to the assessment process may look at a score as a measure of worth or worthless. A high score does not indicate a good academic program and a poor score does not mean that the academic program is bad.
  
• We recommend that a reward system be developed for programs that are doing exceptional assessment. Possible reward is release time or stipend.
  
  • Evaluations that receive a rating of 3 or better be moved to a 2-year written cycle. This does not mean that they do not need to assess the next year. The program’s next report would simply contain the assessment data of a 2-year cycle. This reward is less report writing, not doing less assessment.
• We suggest that college think about a college-wide assessment that is applicable to all programs. Specifically, we recommend exploring the use of a critical thinking exam college-wide in capstone courses. All programs have critical thinking embedded in their program goals in some way. This may provide a useful way of doing assessment at both the program and at the college level.
  
• An external assessment expert would be beneficial, especially if this is a yearly event. The Assessment Committee itself would benefit from consulting a professional in this area.
  
• A formula be developed for the distribution of “assessment monies” that includes a constant amount across programs and also includes numbers of majors served within the programs. Programs should decide on their use of these monies, thus eliminating the time spent by the assessment committee concerning these plans
  
• Use Excellent instead of Exceptional as a descriptor for “highest level” assessment score.
  
• The assessment funds are not being made available in the appropriate time frame. They are made available in October and then must all be spent by the end of the fiscal year.
  
• Mentoring could assist programs in preparation of assessment reports.
  
• Assessment is an evolving process and programs should view their plans as works in progress no matter what their program rating. There is no room for improvement in all aspects of their assessment plans.
  
• Recognize the limits and reliability of the assessment evaluation on our 11 programs.
  
• Remind and enforce the 10-page limit rule with 10-12 font size.
  
• Change the order of items on the assessment progress report form to
  1) approach, 2) use, 3) faculty/stakeholder, 4) overall.