Department of Mathematics and Statistics

Bachelor of Arts/Bachelor of Sciences in Mathematics (60/61)

Contact Person:
Dr. Thomas C McMillan
Mathematics and Statistics Department
(501) 569-8102
tcmcmillan@ualr.edu

UNIVERSITY OF ARKANSAS AT LITTLE ROCK 
Plan No. 60, 61
Assessment Progress Report Form - Calendar Year 2004

Introduction:

The Department of Mathematics and Statistics revised its assessment plan during the 2002-2003 academic year.  This occurred as a result of the participation by two faculty members, Jim Fulmer and Tom McMillan, in a 15-month workshop, Supporting Assessment of Undergraduate Mathematics (SAUM).  The team attended a series of three workshop meetings, which were sponsored by the Mathematical Association of America with funding from the National Science Foundation.  This revised plan has been approved by the Department Gold Committee, consisting of the tenured and tenure-track faculty members.  It was implemented effective with the 2002 calendar year and this is the third year for the new plan.  In revising its plan, the department faculty kept as an overarching principle, “To make it work, keep it simple.” This revised assessment plan consists of the following model:   Mission Statement,    Program Goals,

Student Learning Objectives, Assessment Criterion, Assessment Method, and Assessment Instruments

In this model, for each student learning objective, there is at least one assessment criterion and an assessment method for measuring that assessment criterion.  Currently, our assessment plan consists of six student learning objectives, which are stated in the Approach section of this report.  Our plan had been to assess two of the student learning objectives each year, so that over a three-year cycle all student learning objectives will have been assessed.  However, since we administer the ETS-MFT test each year  and we consider the comparison with a national standard to be a strength of our assessment plan, we have decided to measure Objective B each year.  Since we are in the process of revising our student exit survey and student exit interview, we are not assessing objective A and C  for the 2004 assessment year.  For the assessment year 2004, we are assessing only student learning objective B.  The assessment model that we are using, consisting of a student learning objective, assessment criterion, and assessment method, is illustrated in the Approach section of this report.

We feel that a primary strength of our assessment plan is that we are assessing our students relative to a national benchmark.  All students enrolled in our Senior Seminar/Capstone course are required to take the Educational Testing Service – Major Field Test in Mathematics, a nationally recognized examination.  We are strongly encouraging our mathematics majors to take the MFT test during their junior year.  In this way, we are obtaining two scores on each student which can be used for assessment purposes.  In these ways, we are using student performance and input as a part of the feedback mechanism in program assessment to improve our program. 

As a result of recommendations of two recent assessment consultants which visited our campus, Dr. Trudy Banta and Dr. Ed White, we have decided to simplify and shorten our Assessment Progress Report and make it more concise.

I. USE OF ASSESSMENT FOR PROGRAM BUILDING AND IMPROVEMENT:

For the first time, we are using the Planning for Learning and Assessment format,  which was recommended by Dr. Trudy Banta in her recent visit to our campus as an assessment consultant.   It consists of the six questions listed below. 

1)    What general outcome are you seeking?   

• Objective B:  Mathematics majors acquire the mathematical knowledge and skills  necessary for success in their program or career.

2)    How would you know it (the outcome) if you saw it? (What will the student know    or be able to do?)  

• The ETS Major Field Test will be the assessment instrument, which measures five assessment indicators:  calculus, algebra, applied, routine, and non-routine.

3)     How will you help students learn it?  

• Students will acquire this knowledge through their mathematics courses, including the Senior Seminar/Capstone course.

4)     How could you measure each of the desired behaviors listed in Objective B?  

• Scores for each student on the ETS Major Field Test will be the assessment instrument, which measures five assessment indicator areas:  calculus, algebra, applied, routine, and non-routine.

5)     What are the assessment findings?  

• Graph A represents the ETS Major Field Test Percentile of Average Score (Seniors) History over the years 1996 to 2004.

Click to View Graph A

• Nine students took the Major Field Test in April 2004.  According to Graph A, this number has varied from a low of two to a high of 16 over the past nine years. 

• According to Graph A, the average percentile score for 2004 is the 56^th percentile.  This represents an increase over the previous year of about six percentile points.

• According to Graph A, the highest percentile was in the year 2000.   We feel that this number is skewed as only two students took the test that year and one made an unusually high score.

• According to Graph A, the scores since the year 2000 are somewhat better than the scores for 1996 to 1999.

• Graph B represents the 2004 ETS Major Field Test Assessment Indicators for the nine students.

Click to View Graph B

• According to Graph B, the best mean percent correct(+/- standard error)  for 2004 is in Calculus with a score of 45.8; the lowest mean percent correct is in Nonroutine (Problem Solving) area with a score of 27.9.  

• According to Graph B, scores in all the assessment indicators were below the 50^th percentile.

Click to View Graph C

• Graph C represents the Assessment Indicator History in percent correct for the four years 2001 to 2004.

• According to Graph C,  the percent correct increased from 2003 in the assessment indicator areas of Calculus, Routine, and Nonroutine .

• According to Graph C, the percent correct decreased from 2003 in the assessment indicator areas of Algebra and Applied.

• According to Graph C, there was an increase in three areas and a decrease in two areas.

6)  What improvements might be based on assessment findings?

• We are concerned that the assessment indicator nonroutine problems continues to be our lowest score.  This is an indicator that measures student problem-solving skills.  We are not sure if this is an indication of actual student difficulties or an artifact of the relatively unfamiliar testing environment.  Our plan is to use some time in the senior seminar to acquaint students with the MFT testing format and to increase the emphasis in the Senior Seminar and in all mathematics courses on solving problems that require knowledge from various mathematical disciplines.

• We are concerned that our students mean score continues to be below the 50^th percentile in all five assessment indicator areas.

• We are concerned that the number of students taking the MFT in 2004 dropped to single digits.  We plan an active recruitment effort to increase this number back to double digits. 

• We need to address the reasons that our scores in the assessment indicator  areas of Algebra and Applied for 2004 are at the lowest level for the four years 2001 to 2004.

•  A change for next year is the implementation of a written student exit survey and exit interview, which will be given to students in the Senior Seminar course near the end of the course.  This will enable us to use student responses near the end of their program as a part of the feedback loop in improving assessment.  

•  Another change is that the student score on the ETS-MFT is now a part of the course grade.  It is felt that with this change, students will exhibit a more serious and conscientious attitude toward the test and will strive to give it their best effort.  In the past, it was felt that students finished the test too quickly since their performance on the test was not a part of the grade for the course; thus their score did not represent their best effort.

• We continue to be concerned about problem solving skills, which on the MFT assessment indicators are referred to as nonroutine problems. We are concerned that this continues to be the assessment indicator for our students most in need of improvement.

II. FACULTY AND STAKE HOLDER INVOLVEMENT:

• The Gold Committee, tenure and tenure-track faculty, functions as a department assessment committee (nine fulltime faculty).  Among the tasks accomplished in these meetings are the development of a rubric for evaluating student portfolios, considera­tion of rubrics for the evaluation of student project presentations, how to assess the portfolio, ways to encourage students to take the Major Field Test during their junior year, developing a rubric for the exit survey, and decision to implement an exit interview.  The assessment plan contains a schedule for all assessment activities.
• We are considering changing our method of assessing portfolios.  We feel there is a better way than having faculty/professors responsible for collecting assignments to put in the portfolio.  Two recent assessment consultants to the campus, Dr. Trudy Banta and Dr. Ed White, both have recommended that the student should be responsible for creating and maintaining their own portfolio.   The department will make a decision on this very soon.  As a result of this pending decision, we decided to not assess portfolios during the current 2004 assessment year.
•  In the past, the department  had two two-member teams of  faculty attending a series of three workshops (Strengthening Assessment of Undergraduate Assessment – SAUM) sponsored by the Mathematical Association of America at the Joint Meetings of the Mathematical Association of America and the Ameri­can Mathe­matical Society.  One team (Jim Fulmer and Tom McMillan) concentrated on program assessment and the other team (Melissa Hardeman and Tracy Watson) concentrated on core assessment.  This workshop series was funded by a National Science Foundation grant.  The department teams participated in this assessment project for a period of fifteen months and the department has adapted its assessment plan to conform to standards for the assessment of undergraduate mathematics developed at these meetings.
•  The  CSAM Assessment Committee has decided to have a college-wide alumni and employer telephone assessment administered by Cindy Boland of the Institute of Government.  This assessment process will be done during late spring 2005.  As a result, there is no assessment data from alumni or employers for the 2004 assessment year.
•  The department has implemented several changes in the Senior Seminar/Capstone course. This is due to assessment feedback and our revised assessment plan, which has indicated a need to have more activities in the course to provide additional data for measuring our student learning objectives.    The changes are a)  expand the course from one to three credit hours, b) move the course from the spring semester to the fall semester, and c) make the student score on the ETS-MFT a part of the course grade.  It is felt that these changes will enable the course to expand the required student project in the course, cause students to make a better effort to score their best on the MFT, provide time for review and preparing students to take the ETS-MFT, and to help students prepare papers to be submitted for presentation at the student section of the annual Oklahoma-Arkansas Section of the Mathematical Association of America meeting, which occurs during the spring semester.  By having UALR students presenting at this meeting would reflect favorably on our department. 

III. APPROACH

• Goal :  To prepare our students to enter graduate school, to teach at the  secondary level, and to be employed and act in a consulting capacity on matters concerning mathematics.
• Objective A: That mathematics majors develop an appreciation of the variety of mathematical areas and their interrelation.

  Assessment criteria A1: Students should be able to name several different fields of mathematics they have studied.
  Assessment method A1: In senior seminar exit interview, ask students about the variety of fields they have studied.
  Assessment criteria A2: Students should demonstrate at least one relationship between different mathematical fields.
  Assessment method A2a: In portfolio, look for items which demonstrate such a relationship.
  Assessment method A2b: In senior seminar exit interview, ask students to demonstrate such a relationship.
  

• Objective B: That mathematics majors acquire the mathematical knowledge and skills necessary for success in their program or career.

  Assessment criteria B1: Students should achieve an acceptable score on a nationally recognized test with comparison to national percentiles.
  Assessment method B1a: In senior seminar, students are required to take the ETS Major Field Test.  Score reports show their national percentile rank and various assessment indicators.
  Assessment method B1b: In alumni/student survey, ask for opinions and comments on this.
  

• Objective C: That mathematics majors develop the ability to read, discuss, write, and speak about mathematics.

  Assessment criteria C1: Students should make a presentation to their peers, including department faculty.
  Assessment method C1:  In senior seminar, students are required to develop and prepare a mathematics project, prepare a written handout of the project including solution, and then make a presentation to other members of the seminar and the department faculty.
  

• Objective D: That mathematics majors develop the ability to work both independently and collaboratively on mathematical problems.
  Assessment criteria D1: Students should demonstrate the ability to solve a variety of mathematics problems working on their own.

  Assessment method D1a: In portfolio, review the individual assignments and examinations.  Look for whether there have been improvements over the students careers at UALR.
  Assessment method D1b: In employer survey, ask for opinions and comments.
  Assessment criteria D2: Students should demonstrate the ability to solve a variety of mathematics problems working collaboratively in a team setting.
  Assessment method D2a: In senior seminar, assign problems to be solved in a small  group setting.
  Assessment method D2b: In employer survey, ask for opinions and comments.
  

• Objective E: That mathematics majors develop an appreciation for the roles of intuition, formalization, and proof in mathematics.

  Assessment criteria E1: Students will show that they can reason both intuitively and rigorously.
  Assessment method E1a: In portfolio, look for examples of both kinds of reasoning.
  Assessment method E1b: In senior seminar, ask students to demonstrate both kinds of reasoning.
  Assessment criteria E2: Students will show that they can reason both inductively and deductively.
  Assessment method E2: In portfolio, look for examples of both kinds of reasoning.
  

• Objective F:  That mathematics majors develop problem-solving skills.

  Assessment criteria F1: Students will show they have problem-solving skills.
  Assessment method F1a: In portfolio, look for examples of problem-solving skills.
  Assessment method F1b:  In senior seminar, students are required to take the ETS Major Field Test.  One of the assessment indicators on the score reports is nonroutine problems.
  Assessment method F1c: In employer survey, ask for opinions and comments.
  Assessment method F1d: In alumni/student survey, ask for opinions and comments on this.
  

• Assessment instruments:  Six assessment instruments are used in our assessment plan: student scores on the ETS Major Field Test, the student portfolio, the senior seminar presentation, the student exit survey, the alumni survey, and the employer survey.  We plan to add a seventh assessment instrument, student exit interview, during the next assessment year.

•  Timeframe for future assessment activities: The following table gives the timeframe for fu­ture as­sessment activities.  Because they involve relatively small population sizes, the alumni and employer surveys will be conducted every three years and will involve more than one graduating class.  The cur­rent assessment round is labeled “2004.”

Click to see Timeframe for Future Assessment Activities

Addressing evaluator’s comments from last year:

• Last year, evaluators asked how we could use the alumni and employer survey as an instrument for evaluating learning objectives every year, considering that it is to be administered only every three years.  Our plan is that it would consist of questions connecting to the various objectives, which would then be used in the appropriate year that objective was being measured.  In this way, data from the alumni and employer survey would provide information for several objectives, not necessarily those that were being measured during the year that the survey was administered.
• Last year, evaluators said plans for improving non-routine problem solving look to be too concentrated in the senior seminar.  We plan to address this with more non-routine problem solving in all mathematics courses and perhaps in activities of the Math Club.