Chemistry

Bachelor of Arts/Bachelor of Science in Chemistry (49/50)

Contact Person:
Dr. Robert Steinmeier
Chemistry Department
(501)569-8840
rcsteinmeier@ualr.edu

UNIVERSITY OF ARKANSAS AT LITTLE ROCK Plan No. 49/50
Assessment Progress Report Form - Calendar Year 2004

I. USE OF ASSESSMENT FOR PROGRAM BUILDING AND IMPROVEMENT:

1. Findings of the past year's assessment activities:

1. The Major Field Achievement Test (MFAT) gives a group score taken from data from several questions that assess the group’s ability to use critical thinking and reasoning ability.  ETS reported this information for only the Spring, 2004 seminar class because the Fall, 2004 seminar classes had fewer than five students. The Spring group scored at the 50 percentile on the Critical Thinking and Reasoning Ability Assessment Indicator which is lower than 2003’s percentile of 75. 

2. A review of the student portfolios revealed that seven of the ten students included a safety artifact. The artifacts were safety guidelines in the submitted lab reports, chemical safety section in their seminar paper or the Materials Safety Data Sheets (MSDS) seminar assignment.
3. Of the ten students who completed the seminar course in 2004, five based their presentation on laboratory work they conducted. These student presentation scores averaged five percentage points higher than the five students that presented seminars based on a literature search. The average score for all oral presentations was 86%. The students also submit a written paper describing their work. The average score on these seminar papers was 88%.  These scores are consistent with previous graduates’ performance in seminar.  These indicators tell us that the graduates who do conduct undergraduate research are acquiring the skills required to conduct scientific inquiries.
4. The Departmental Exit Exam was given for the eleventh year, (see data 2004 on Page 10). Ten students took the exam this year. This is the fifth year for the revised exam. This year’s mean score of 47% that is comparable to previous years (Graph Page 10) and the highest individual score this year of 77% was near the all time high of 83%.  The average department exam score (47% mean score) correlates well with the average percentile on the MFAT (Graph Page 9) that placed them in the 49rd percentile nationally.  The individual student’s Department Exit Exam scores did not correlate well with their percentiles on the MFAT. On the MFAT, three of the ten students scored at or above the 90^th percentile, but four of the ten scored in the 30rd percentile. 
5. In 2002, the faculty asked how the general chemistry preparation of transfer students compared to that of students who took general chemistry at UALR. To seek the answer, a diagnostic exam created by the analytical faculty was given at the beginning of Analytical Chemistry I and ACS General Chemistry exam was given at the beginning of Organic Chemistry l. The following data was reported for calendar year 2003 and 2004.

    

   Analytical Chemistry I Students	2003 (n=34)	2004 (n=38)
   UALR Gen. Chem	45%	48%
   Transfer Gen Chem	30%	35%

    

   Organic Chemistry I Students	2003 (n=85)	2004(n=90)
   UALR Gen. Chem	22%	39%
   Transfer Gen Chem	25%	38%

    

USE OF ASSESSMENT FINDINGS FOR PROGRAM IMPROVEMENT

a.  The MFAT and Departmental Exit Exam scores continue to correlate very well for our graduates.  The MFAT percentile ranking allow us to compare our program to others nationally. The Department Exit Exam percentages allow us to examine specific topics on chemistry addressed in our program.  We plan to continue administering both exams each semester.  This dual testing allows us to both compare our students to national norms and also allows us to test items unique to our curriculum.

b.  Our goal that 100% of our graduates will include at least one safety artifact and include chemical safety information in their seminar paper and presentation is closer to being met than in past years. In response to concerns held by all our stakeholders regarding the knowledge our graduates demonstrate regarding chemical safety issues, a new course in Fall, 2004 entitled CHEM 4399/5399 Selected Topics in Chemistry: Chemical Safety was offered.  All graduate teaching assistants, several faculty, and two undergraduates took the course.  As a result, we hope that all stakeholders will be better prepared to deal with chemical incompatibilities, disposal and chemical health concerns.

c.   As a result of the Chemical Safety course the following changes were made:

1.    All teaching assistants were required to complete the three credit hour Chemical Safety course, CHEM 5399.

2.   All faculty and teaching assistants completed a course in First Aid and CPR training.

3.   Departmental Safety committee meets weekly.

4.    The university committee on Environmental Health and Safety was reactivated by the Chancellor.

5.    The 5399 students prepared 29 Standard Operating Procedures to guide the department in safety revisions.

6.   Safety inspection forms were developed and used to conduct inspections of teaching and research labs.

7.    The student laboratory safety agreement forms were revised.

8.   The laboratory incident report form was revised.

9.   The contents of laboratory first aid kits were updated.

10.  The department is working to establish a better culture of safety.

d.   In order to instruct our students about safety issues such as chemical incompatibilities and toxicities, we ask students to obtain the Materials and Safety Data Sheet for at least one compound used in their research and interpret the information given.  This information is to be included in their oral and written presentations.  The newly-revised (Fall, 2003) Seminar Critique Sheet allows us to collect data on Safety in the oral presentation. The ten students the year scored an average of 73% in safety. 

e.  We are not ready to draw any conclusions regarding the general chemistry knowledge of students who take general chemistry at UALR versus students who transfer general chemistry.  The preliminary data (see 1e above) indicate some difference but the sample populations seems to indicate a trend that students taking general chemistry at UALR scored better on the two diagnostic exams than those that transferred after completing general chemistry.

f.   In response to past roundtable forum comments and score on seminar presentations, two hours of undergraduate research are required for a BS degree.

II. FACULTY AND STAKEHOLDER INVOLVEMENT:

* Faculty Involvement: The Chemistry Department Assessment Committee of four of the twelve faculty leads the assessment effort.

1. The entire faculty agrees on the assessment approach and objectives to be assessed at the beginning of the Fall Semester.
2. The faculty is apprised of assessment exam results, and they receive copies of each year's Assessment Progress Report.
3. Faculty mentor student research and listen to seminar presentations and critique them according to the seminar rubric.
4. Each seminar student has asked a faculty member to assist in the preparation of the seminar presentation in a Student/Mentor contract.
5. The faculty participates in the exit survey/round table discussions held with the graduating seniors. Students are more relaxed and communication is honest. Student comments are taken seriously, and are considered in the institution of changes in curriculum, policy, and attitudes.

Students: Our students actively participate in the assessment process. The students understand that their performance on the MFAT and the Departmental Exit Exams are crucial in our attempt to assess and improve our chemistry program. The student portfolios are a central part of the assessment plan, and students are mailed the portfolio guideline sheets as soon as they declare chemistry as a major. The students participate in the evaluation of the oral presentations in seminar.  The average of their numerical scores comprises 10% of the student’s seminar grade. At the end of their last semester, the students complete an exit survey (anonymously).  The responses are compiled and the faculty and students discuss the responses openly and honestly. Some consistent facts have been revealed from written surveys and discussions.

In summary our senior students in the seminar roundtable and exit surveys stated they

1. appreciate the “quality and enthusiasm of faculty” and they think the faculty is knowledgeable.
2. appreciate the difficulty of the chemistry courses – “They were challenging and made you want to work harder.”
3. value the laboratory and research experience in chemistry
4. believe that the “classrooms and buildings for chemistry students were very good.”
5. have the following concerns:
   1. Some faculty are not interested in students
   2. Facilities for organic and analytical chemistry need to be improved
   3. Lack of "chemical writing

Alumni and Employers/Supervisors of our Bachelors Alumni:

Our alumni and their employers/supervisors continue to be a source of feedback and support. A web-based alumni survey was planned but the CSAM Assessment Committee decided to conduct a college-wide telephone survey during the summer of 2005. The Institute of Government will conduct the survey, and results should be available for Fall, 2005.

III  APPROACH:

1. Program goals and relationships of goals to program and UALR mission:
   
   B.A. Program:
   The goals for this program reflect the program mission which is "to provide a strong, broad background in chemistry which could be combined with related undergraduate disciplines such as biology, physics, mathematics, or psychology in the pursuit of postgraduate degrees in medicine, dentistry, veterinary science, and other health related professions, and to provide an ample chemistry background for post graduate training in secondary science education."
   
   B.S. Program:
   The goals of this program reflect the program mission that is "to provide intensive, focused education in chemistry, providing the graduate with the background required for post graduate study in chemistry and/or for professional positions as a chemist in private industry or government service."
   
2. Student Learning Objectives Geared To Both Bachelors Programs

   NOTE: The student learning objectives for the two degree programs are different and were listed in our assessment plan. Due to page constraints, the student learning objectives for each program were omitted, and only those listed below are common to both degrees and are assessed.

   1. Chemical knowledge: Students should have competence in the following basic chemical knowledge areas: Atomic and molecular structure, stoichiometry, periodicity, stereochemistry, thermodynamics, reaction kinetics, synthetic transformations, nomenclature, equilibria, spectroscopy, electrochemistry, oxidation-reduction, chemical and physical properties, and solutions. This objective is geared to B.A. and B.S. goal a.
      
   2. Scientific inquiry competence: Graduating majors will have obtained the ability to carry out an independent library or laboratory research project, and to present their results in both a written and oral presentation using modern technology (e.g. Microsoft Word, Power Point, etc.) This objective is geared to B.A. goals d. and f. and B. S. goals b. and c.
      
   3. Competency in writing formal laboratory and technical reports: Students will have mastered the ability to write clear, concise reports of completed laboratory experiments, organized into a prescribed format using modern word processing, spreadsheet, and structure drawing technology. This objective is tied to B.A. goals e. and f. and B.S. goal c.
      
   4. Competence in the use of chemical instrumentation and the analysis of the results obtained therefrom: Specifically, students will learn to operate and analyze data obtained from UV-visible and infrared spectrophotometers, gas chromatographs, electrophoresis equipment, bomb calorimeters, and polarimeters. This objective is tied to B.A. goal c. and B.S. goal e.
      
   5. Knowledge and practice of chemical safety: Students will learn to read and comprehend information provided on Material Safety Data Sheets and thus be aware of safety hazards associated with specific chemical used in experiments and be able to address emergencies with the proper procedures. This objective is geared to B.A. goal c. and B.S. goal d.
      
3. Assessment Methods Employed in this Year for the Three Objectives in the 2004 Calendar Year:

   We continue to view our graduating seniors as the focal point for program assessment.

• Objective 6a, Chemical Knowledge:
  Students take two exit exams during their last semester: the Departmental Exit Exam and the Major Field Achievement Exam. Our goal is an average score of 65% on each these exams.
  
  To compare the general chemistry knowledge of students who complete general chemistry at UALR as compared to other campuses, all students are given the ACS General Chemistry Exam or a faculty-created exam during the beginning of Analytical Chemistry I and Organic Chemistry I.  

• Objective 6c, Competency in writing formal laboratory and technical reports:
  Students a written report in senior seminar describing their laboratory experience in undergraduate research or the work of others if they conducted a literature search. Our goal is an average score of 65% on the written report.
  
  Students prepare written laboratory reports for all lab courses except General Chemistry I.  Our goal is an average score of 50% for laboratory reports.   
  
• Objective 6e, Safety:
  Review of the Materials Safety Data Sheet assignment for seminar students. The goal is for 100% of our students to complete the assignment and use that information in their oral and written presentation.
  
  Review of the portfolios of the students to determine the number of safety artifacts included with a goal that 100% of the students have a least one safety artifact.
  
  Review of seminar papers to determine if safety issues were included with the goal that 100% of the students address chemical safety issues in their written report.
  
  Review of the seminar critique sheets to determine the students’ score on chemical safety in their seminar presentation.

Last Year's Feedback:

Based on feedback from the roundtable sessions, the faculty discussed requiring all majors to conduct undergraduate research.  All BS graduates must do so, but it was decided to continue to strongly encourage the BA students to complete an undergraduate research project instead of making it a requirement.

As a result of faculty discussion about students who transfer to UALR after completing general chemistry, a diagnostic exam covering general chemistry topics is conducted at the beginning of Analytical Chemistry I and Organic Chemistry I.

In response to all stakeholders’ concern about chemistry safety, the Chemical Safety course was offered.

Possible assessment approach changes for next year:

1. We are planning to use the telephone survey of  alumni and employers/supervisors as a means to improve our program.
2. Request that all lab courses discuss and test chemical safety emphasizing chemical incompatibilities and toxicities. Alumni of our program have indicated a weakness in this area.

Timeline For Assessment Activities:
A timeline for assessment activities was established in 1998 for the years 1998-2003. The included timeline was extended through the year 2007.

Click to see Assessment Timeline