University of Arkansas at Little Rock

Information About:

• Academics
• Academic Calendar
• Assessment Central
• Faculty Development
• Faculty Senate
• Policies and Forms
• Staff

Provost's Office
Admin South 311 UALR
2801 S. University Ave.
Little Rock, AR 72204
(501) 569-3204

HIST 1311 - 1312
History of Civilization

Fall 2005

I. Objectives

The History of Civilization core courses are designed to provide students with an introduction to the major themes in world history from the earliest times to the post-World War II “global world”. In approach, content, and methodology the History of Civilization courses meet the Goals of the History Program and Student Learning Objectives, both of which link directly to the University’s Core Curriculum Competencies. Specifically, the History of Civilization courses convey knowledge of key historical people and events, explore patterns of change and transition over time, make connections between the past and present, and facilitate critical and analytical thinking and writing.

Goals of the History Program
1. To present basic information on the main stages of human cultural development
2. To illustrate the interrelationships among historical events through an understanding of the process of change, continuity, and causation over time
3. To offer opportunities to analyze the implications of a broad range of social economic, political, and cultural problems and developments.
4. To improve student reading, writing, and critical thinking skills

Student Learning Objective for Core Courses
1. Students will demonstrate a knowledge of historical information such as names, dates and chronologies, events, terms, and concepts.
2. Students will demonstrate an understanding of the diversity and complexity of the historical context that shapes human experience.
3. Students will demonstrate an understanding of the inter-relatedness of historical events as expressed in such concepts as continuity and change, causation, interdependence of cultures, and the interaction between differing groups and societies.
4. Students will organize and articulate their ideas through an essay that presents a thesis relevant to the question.
5. Students will support their ideas with historical evidence and will reach conclusions based on that evidence.

Core Curriculum Competencies

The Goals of the History Program and Student Learning Objectives meet six of the ten Core Curriculum Competencies:
• Critical Thinking
• Ethical and Moral Consciousness
• Historical Consciousness
• International Awareness
• Social and Cultural Awareness
• Verbal Literacy

Relationship between Student Learning Objectives and the Goals of the History Program
• Objective 1 fulfills Goal A
• Objective 2 fulfills Goal B
• Objective 3 fulfills Goal C
• Objective 4 fulfills Goal D
• Objective 5 fulfills Goals A-D

Relationship between Student Learning Objectives and Core Curriculum Competencies
• Objective 1 relates to Ethical & Moral Consciousness, Historical Consciousness, International Awareness, and Social and Cultural Awareness.
• Objective 2 relates to Critical Thinking, Ethical & Moral Consciousness, Historical Consciousness, International Awareness, and Social and Cultural Awareness.
• Objective 3 relates to Critical Thinking, Ethical & Moral Consciousness, Historical Consciousness, International Awareness, and Social and Cultural Awareness.
• Objective 4 relates to Critical Thinking and Verbal Literacy
• Objective 5 relates to Critical Thinking and Verbal Literacy

II. Methods

In order to assess the success of the History of Civilization courses in meeting the Goals and Objectives outlined above, pre- and post-tests are administered in all History of Civilization Courses (1311 and 1312) during the Spring semester. The pre-test, given during the first or second week of the semester, is comprised of three questions from which students must choose one and write a comprehensive essay. The scope of the questions is broad enough to account for the diverse approaches that faculty members take to teaching History of Civilization courses. The same questions are asked on the post-test, which is given during the second to last week of the semester. Although the questions are the same, students are not required to answer the same question on the pre-and post-test. However each individual student’s pre- and post-tests are matched up, so only students who complete both a pre- and post-test are included in the final results.

During the summer and/or fall semester all of the pre- and post-tests are scored by two evaluators, one of whom is an adjunct instructor and the other of whom is a full-time faculty member who teaches both halves of the History of Civilization course. The essays are evaluated on five separate criteria that link to the Goals and Objectives outlined above. These criteria are:

1. Essay includes accurate and relevant information
2. Essay shows an awareness of historical context.
3. Essay shows an awareness of the interrelatedness of events.
4. Essay organizes ideas in an argument and articulates it effectively.
5. Essay presents an appropriate thesis.

Each criteria is scored on a range from zero to three. Zero implies that the student has not met any of the objectives being measured; one suggests a very limited achievement of objectives; two implies a fair to good achievement of the objectives; and three suggests that the essay fully meets the objectives outlined above. After all of the pre- and post-tests have been evaluated all of the scores are entered into a spreadsheet and a mean is derived for each category in each class. This allows us to both identify discrepancies in scoring between the two evaluators, and to hone in on specific objectives in total and by section/instructor.

III. Results

The scores in the following tables are based on the evaluations described above. An average score for each category was calculated, and the overall average on the pre-test was subtracted from the overall average on the post-test. The difference represents the degree of change presumably resulting from the course. One might think of it as the “value added” to each student’s Core Curriculum Competencies by the course.

History of Civilization I (1311)

Section	Average Change (Evaluator A)	Average Change (Evaluator B)
01	.56	.32
02	.78	1.04
06	.46	.32
97	.72	1.10

History of Civilization II (1312)

Section	Average Change (Evaluator A)	Average Change (Evaluator B)
02	.62	.70
06	.86	1.10
09	.16	.16
991	1.04	1.04
999	1.08	1.20

The overall average scores are useful in providing a general snapshot of student improvement over the course of a semester, particularly in those cases (as, for example, in the case of 1312.991) when the two evaluators’ scores are identical or nearly identical. What the above scores suggest is that while there was an across the board improvement from the pre-test to the post-test, the improvement was more marked in some courses than in others. This information can be used to evaluate the effectiveness of individual instructors and of different teaching strategies and methodologies.

The nature of the information on a criteria-by-criteria basis is extensive enough as to prohibit its reproduction here. However, an analysis of change according to each criteria, rather than on an overall basis, provides us with far more information with which to work in critically evaluating how well the History of Civilization courses meet the objectives outlined above. In most cases the average overall change was not much larger than the change within each category. But there were cases, for example in 1311.02, where there was a significant difference between the overall average change and the change within specific criteria. In the case of 1311.02 the overall change, according to Evaluator A, was .78; however, the overall change in criteria three was 1.2, suggesting that much of the overall change derived from the fact that students did a particularly good job at showing the interrelatedness of events, and a particularly bad job (resulting in only a .4 rate of change) in providing correct and relevant information. Providing faculty with assessment numbers that give overall rates of change, as well as rates of change within individual criteria will enable them to give thought to how they approach the task of meeting each of the objectives outlined above.

For the most part the overall evaluations provided by the two evaluators are close enough not to warrant undue concern that they are evaluating the exams according to different assumptions about what each of the criteria and scores mean. In cases where there are significant discrepancies, for example in 1311.02 and 1311.97, it might be useful for the evaluators to discuss their results and ensure that they are on the “same page” with regard to the criteria on which they based their scores.

In general the scores are probably lower than the actual value added. The success of the program depends on students doing as well as possible on each of the essays. It is likely that students do not put as much effort as they might into their answers, particularly at the end of the course. Anecdotal evidence indicates that some instructors do not stress, indeed may discount, the importance of the assessment system, and that some of the post-tests were given on the day of the final exam, after students had taken the final exam and when they were disinclined to take the post-test seriously. Indeed a major weakness in the system is that its effectiveness depends on the level of effort and seriousness with which instructors and students approach the task.

In spite of some of the shortcomings and challenges outlined above, the pre-test post-test system is a useful assessment tool and should be retained. It can provide a valid assessment of “value added” because it is based on criteria that are closely related to the history department’s Student Learning Objectives and to the Core Curriculum Competencies. It can also provide a significant degree of reliability so long as the evaluators are consistent in the way they score the essays. The pre-test/post-test system can provide not only a measurement of “value added” but also a benchmark that could be used to measure improvement over time and also a means of evaluating individual instructors.

IV. Decision Making

Overall these results provide ample fuel for discussions within the department about not only how we assess our core courses, but how we structure our courses to meet the Goals of the History Program, Student Learning Objectives, and Core Curriculum Competencies. The History Department will hold a special meeting to discuss the results of both the History of Civilization and U.S. History assessments. Particular attention will be placed on how we do assessment, and how assessment can help us all do our jobs better, particularly when it comes to meeting both the Student Learning Objectives and the Core Curriculum Competencies. Individual faculty members whose courses were evaluated will receive copies of the pre-tests and post-tests and the evaluations; these tools should spark some discussion amongst individual faculty members and within the department on teaching styles and methodologies.

V. Response to Recommendations of the Assessment Committee to the Department

Last year the Assessment Committee recommended a number of changes in order to provide better results and make assessment a more meaningful part of the department’s overall program for core courses. First, they suggested that pre-tests and post-tests be administered by Assessment Team members who will impress on students the importance of the program and of their role in providing reliable material to assess. The Assessment Team members would also ensure that the tests are given on an appropriate date at the beginning and end of the semester. Although we find merit in this recommendation, we did not implement it this year because responsibility for assessment shifted during the Spring 2005 semester due to changes in job duties of the faculty member responsible for assessment. However this is a change that we will consider implementing in the future.

Second, the Assessment Committee recommended that the evaluators meet prior to scoring the tests to discuss evaluation of sample tests in order to reach agreement on the general nature of scoring. This would, presumably, increase the reliability of the scoring system. Due to the circumstances described above, this change was not implemented for the current assessment cycle, but it will be implemented prior to the next assessment cycle. Third, the Committee suggested that pre-tests and post-tests be administered in the Spring semester and in all sections. We did administer the tests during the Spring semester this year, but only a handful of courses were surveyed. In part this is a function of faculty cooperation ­ some faculty were more cooperative than others in actually administering the pre-and post-tests to their sections. Utilizing Assessment Team members to administer the tests should eliminate the problem of faculty cooperation.

Fourth, the Committee recommended that the essay questions used on the pre- and post-tests be drawn from a larger sample of questions to ensure that instructions do not “teach the test.” Although we agree that a larger sample of questions might be useful, our experience generally is that instructors do not, in fact, “teach the test.” In fact, quote the opposite could be true. The amount of material that could potentially be covered in two semesters of History of Civilization is vast, and instructors must make some difficult choices about what to include and what to exclude. Because of this, it is entirely possible that some instructors do not teach any of the material covered by the pre- and post-test questions.

Finally, the Committee suggested that the concept of “value added” as measured by the assessment should become a factor in the teaching evaluation of the instructor. This is something the Department will consider further, although there are no plans at the moment to integrate this into annual evaluations.

---

University of Arkansas at Little Rock | 2801 S. University Avenue | Little Rock, Arkansas 72204
UALR is accredited by The Higher Learning Commission.
All Rights Reserved.

About UALR | Jobs at UALR | Give to UALR | A-Z Index | Map & Directions | Contact UALR