UALR | College of Science and Mathematics

Earth Science

Bachelor of Science in Geology (52)

Contact Person:
Dr. Steve Leslie
Department of Earth Science
(501) 569-8061
saleslie@ualr.edu

UNIVERSITY OF ARKANSAS AT LITTLE ROCK Plan No. 52
Assessment Progress Report Form - Calendar Year 2004

I. USE OF ASSESSMENT FOR PROGRAM BUILDING AND IMPROVEMENTS:

Four of the five Geology Program’s Student Learning Objectives were assessed for the Calendar Year (CY) 2004. Objective 3 was not assessed because the Paleobiology course in which this objective is assessed was not taught during CY2004. See the Approach (Section 3) of this report for a more detailed description of each Student Learning Objective.

Objective 1- Students will be able to recognize and describe common minerals and rocks as well as some minerals and rocks that are relatively uncommon but that are important (e.g., in interpreting earth history, economically, etc.).

This objective is assessed primarily with the laboratory final exams in Mineralogy (ERSC 2320), Igneous and Metamorphic Petrology (ERSC 3311), and Sedimentology (ERSC 3350). These laboratory exams emphasize mineral and rock identification. Exams are used to assess this objective because they directly correspond to the learning objective and can be easily quantified. A minimum score of 70% on each of the exams is considered acceptable for meeting this objective. The rubrics for this exam are designed such that a 70% score would be the lowest “C” (lowest satisfactory score). The objective nature of these exams minimizes reliability issues. Results from the rock and mineral lab exams from CY2002, 2003 and 2004 are presented in Table 1.

Table 1. Results from the Mineralogy, Igneous and Metamorphic Petrology, and Sedimentology Lab exams on Rock and mineral identification for CY2002, 2003, and 2004.

Click here to see Data Table 1

This is the third year that Objective 1 has been addressed. The total sample size is small, however the results are mostly positive.

Faculty assessment discussions have resulted in changes implemented in the exams and rubrics related to the concept of Universal Design, (the incorporation of best practices for the inclusion of multiple learning styles) – see discussion of Universal Design under Summary of Use of Assessment for Program Building below.

Objective 2 - Students will be able to make systematic scientific observations of rock samples, outcrops, and stratigraphic sections, and interpret these observations using sound scientific principles.

One of the capstone experiences for students in the Geology Program is a six-week, six credit-hour intensive Geology Field Camp generally offered in the summer. We do not offer this course at UALR because of the high cost of running it and the relatively small number of students who take it each year - a situation faced by many geology programs nationally. We require that our students take this course with another university. A benefit of this for assessment purposes is that it allows for an external assessment of our geology program. The department must approve the course before the student participates in it. Field course exercises include observations of geologic phenomena in the field, identification of rocks, structures, fossils, stratigraphic section measurement, synthesis of this information usually in the form of geologic mapping, and interpretation of the synthesized data in the form of written reports.

Click here to see Data Table 2

Data compiled in Table 2 from all majors who have completed field camp indicate that, in general, our student perform at the A or B level. This shows that our students are exhibiting the ability to think scientifically in the field of geology.

Instructor evaluation forms (attached) have been developed to be provided to field camp instructors immediately following completion of the field course. We expect these results will provide us with more detailed information on specific program strengths and weaknesses in the future. Since the implementation of the field camp instructor evaluation form only one student has attended field camp (CY 2004). Unfortunately the instructor evaluation form for this student was not returned.

We experimented in 2002-2003 with offering the equivalent of a traditional six-week, six credit-hour field course through our own department. This was initiated partly by disability accommodation issues. We offered three two-credit hour courses (Field Camp I, II, and III) over three consecutive semesters focusing on three geologically diverse regions. The two students who chose this option completed all three during Summer 02 – Spring 03. This is another example of the implementation of Universal Design in our program. Other institutions are realizing that a 6 week summer commitment can be a hardship and barrier to choosing Geology as a degree path. We are continuing to discuss the issues related to the field camp requirement in our program, most recently at a Department of Earth Science faculty retreat (December 2004) on curricular issues.

Objective 3 - Students will be able to recognize common invertebrate fossils, interpret the ancient environment in which the fossil lived, and assess their utility as indicators of geologic time.(Not assessed CY 2004 – Paleobiology not taught CY 2004)

The laboratory final exam in Paleobiology is used to assess Objective 3. The average score over the past 3 times that this course has been offered is just over 71%. This is a challenging exam, and a score of 70 or above is considered a good score in the opinion of the professor, and a score below 55 is a poor score (Table 3).

The professor discusses poor scores with the students who receive them, and in general, it is lack of preparation [studying lab specimens before the exam for a sufficient amount of time, and lack of time learning characters on the fossils that are used in systematics (i.e., the Kingdom, Phylum, Class, Order)] that is the reason for the poor scores.

Click here to see Data Table 3

In the past, we have used the ACAT test as an external assessment of Objective 3. However, we have suspended this method until we are confident scoring issues are resolved. We are presently considering alternative standardized tests including the ASBOG (Association of State Boards of Geology), or the Geology GRE.

Objective 4 - Students will be able to construct and interpret geologic maps and geologic cross-sections, evaluate models of geometry of rock units at depth, and graphically present surface and subsurface geologic and topographic data.

Objective four is assessed at three stages of the program: during Geologic Methods (ERSC 2320), Structural Geology (ERSC 3330), and Field Geology (Field Camp) (ERSC 4626).

Students continue to score above the minimum satisfactory 70% on the Geologic Methods final project/poster where students present their semester-long field-mapping project as a poster session (posters are made using presentation software and plotted on a large-format plotter) (Table 4). The departmentally developed rubric for this project is designed such that a 70% score would be the lowest “C” (lowest satisfactory score).

These projects are rated by all in the Earth Science Program faculty, and by students in the class using a rubric designed with input from all faculty. Analysis of the scores on the rubric yield no particular pattern, therefore no changes to the curriculum were made based on this data. We believe that the results provide one line of evidence that Objective 4 is being addressed and met in this course.

Click here to see Data Table 4

Click here to see Data Table 5

Objective 4 is also assessed in the Structural Geology course in a semester-long structural synthesis project. Results from this project are shown in Table 5. Students in this course have performed above the acceptable 70%.

Lastly, Objective 4 is assessed in the 6-week Field Course at the culmination of the Geology degree. For further details, see the results and discussion under the assessment of Objective 2.

Objective 5 - Students will demonstrate the ability to utilize the theoretical basis of geology, to critically evaluate and analyze data and conclusions of others, and to develop appropriate competence in communicating geologic information in written and oral form.

Based on peer student and faculty evaluation of oral and poster presentations for senior seminar this past year, all students received averaged scores better than the minimum 70% for both poster and oral presentations. The departmentally developed rubric for this assessment instrument is designed such that a 70% score would be the lowest “C” (lowest satisfactory score). To ensure interrater reliability, all faculty were involved in determining student grades.

Summary of Use of Assessment for Program Building

Most students are meeting learning objective criteria and major program changes do not appear to be needed. Careful evaluation of assessment results, however, will continue as specific areas in need of improvement can be identified. Because of the small number of geology majors we typically have in our upper-level classes where most of the assessment is occurring, statistical analysis of our data in many instances is not yet meaningful. We continue to develop or modified several assessment instruments that have or will highlight program areas in need of change.

As stated in objective 2, we are in the process of re-evaluating our field camp requirement. This was a focus of a departmental retreat in December of 2004. The potential effects of changing the field camp requirement has caused us to closely evaluate the stages in the curriculum where certain skills are introduced and reinforced (e.g. map and compass skills, field identification of rocks and structures, field applications of technology, etc.).

In addition, we are putting together an external advisory board composed of stakeholders from government agencies, industry, and secondary education. The first meeting of this board will be in either April or May 2005. One of the charges for this board will be to examine our ideas generated at the retreat regarding program changes.

Our evaluation of areas where some students are not meeting the learning objective criteria established by the department has led us to the concept of Universal Design for learning. This concept, as applied to higher education, is an approach to teaching, learning and the development of curriculum, and assessment that incorporates and encourages different individual learning styles. The concept comes from the field of Architecture where it was recognized that designing for the divergent needs of special populations increases usability for everyone. For example, curb cuts, originally designed to help those in wheel chairs also help those pushing carriages, riding skateboards, pulling suitcases, or simply walking. Examples of our application of Universal Design include color coded mineral chemistry notes and the use of 3-D structure models in Mineralogy, the posting of lecture notes in multiple formats (i.e. accessible to text readers for those who learn audibly) in Paleobiology, and the modular 3-part Field Course. As we implement “best practices” of Universal Design, we anticipate that our Assessment methods, instruments and potentially our Learning Objectives will change.

II. FACULTY AND STAKE HOLDER INVOLVEMENT:

All Earth Science faculty and interested geology student majors/laboratory assistants meet weekly. Initially, this was done to discuss our Blue Ribbon core courses and the laboratory manual design and revision. This session has been expanded to discuss, among other things, the curriculum and assessment issues. At each meeting, following discussion of the status of our weekly introductory geology laboratories, assessment (core and program) issues are discussed. Thus, all Earth Science faculty members and some geology students have been active participants in the development of the Geology program's assessment strategy and implementation. The Geology Club is proving to be an active vehicle for sharing assessment information with students.

Geology majors have the opportunity to be active participants in the development and implementation of the departmental assessment plan. Considerable student input on curriculum design and connectedness is being derived from required student portfolios in the Senior Seminar capstone class. Geology majors begin to assemble this portfolio as soon as the major in geology is declared. Items such as artifacts (exams, research papers, etc.) and reflective writing (discussion strengths and weaknesses of the Geology Program) are included. Portfolio guidelines and a grading rubric with criteria for evaluation of portfolios are attached.

Faculty keep in close contact with most alumni of the geology program (approximately 30 since program inception) through work, professional meetings, and the annual departmental picnic. Although formal survey instruments are very useful, we feel we can gain more in-depth information concerning program satisfaction and suggestions for program improvement with “interviews” of alumni. We found through recent discussions with several alumni, particularly those who are not practicing professional geologists or in graduate school, that they felt a need for a more well-rounded liberal arts type education. These alumni have not used many of the non-geology science courses required for a BS in geology (e.g., two semesters of physics and chemistry plus labs, three semesters of calculus/math) in their workplace. Based in this alumni input, Earth Science faculty have decided to propose a BA degree in geology to better meet the needs of this population of students. We are currently in the late planning stage for this proposed new program. We feel that this is a good example of how stakeholder involvement in the assessment process will make a significant positive change in the Earth Science department’s curriculum.

In addition to alumni interviews, an alumni survey form has been developed and will be administered through the college-wide phone survey effort. This assessment instrument will help us reach those alumni not in the central Arkansas area. The goal of the survey is to assess the strong and weak points of the required courses in the geology major and the students' overall satisfaction with the degree program. A plan for employer evaluation of Geology graduates is also being developed. It is expected that this will also be a part of the college-wide phone survey effort.

III. APPROACH:

The educational goals of the Geology Program are: 1) to develop basic critical thinking, communication, technical, and teamwork skills, which are critical to any chosen career path, 2) to provide academic preparation and to develop the skills and competencies necessary for an entry-level position in the geologic profession or for graduate studies in the Geosciences, and 3) to provide a solid content background in the Earth Sciences and Integrated Sciences for K-12 science teachers.

Secondary goals of the Geology Program include the discovery, integration, and dissemination of knowledge of geologic systems, use of this knowledge in ways that will contribute to society through the application of faculty and student resources, and the application of their research skills to the community, state, and nation

These goals are consistent with the Earth Science program's mission statement, which is to encourage students to relate the geoscience disciplines to intelligent living on Earth, to thoughtfully understand the interplay between humanity and the earth, and to obtain a scientific understanding of the earth - its anatomy, processes, resources, history, and potential future. The geology curriculum provides a foundation of knowledge that allows the student to pass through progressively higher, connected levels of understanding to achieve a professional competence and to acquire the ability for professional growth through life-long learning. After these experiences, the graduate will have grown sufficiently in self-motivation to deal effectively with a constantly changing and expanding professional environment.

This mission statement is consistent with the stated mission of UALR, which includes developing the intellect of students, discovering and disseminating knowledge, and serving and strengthening society by enhancing awareness in the scientific arena. The program "emphasizes the liberal education of undergraduate students" by providing students an opportunity to understand the physical earth on which all of humanity resides, and to explore the earth and life through time. Like UALR, the Geology Program should instill a life-long desire to learn, use knowledge in ways that will contribute to society, and apply its resources and research skills to the service of the city, state, and nation.

The Geology Program has the following five central student learning objectives:

Students will be able to recognize and describe common minerals and rocks as well as some minerals and rocks that are relatively uncommon but that are important (e.g., in interpreting earth history, economically, etc.).
Students will be able to make systematic scientific observations of rock samples, outcrops, and stratigraphic sections, and interpret these observations using sound scientific principles.
Students will be able to recognize common invertebrate fossils, interpret the ancient environment in which the fossil lived, and assess their utility as indicators of geologic time.
Students will be able to construct and interpret geologic maps and geologic cross-sections, evaluate models of geometry of rock units at depth, and graphically present surface and subsurface geologic and topographic data.
Students will demonstrate the ability to utilize the theoretical basis of geology, to critically evaluate and analyze data and conclusions of others, and to develop appropriate competence in communicating geologic information in written and oral form.

We believe that the strengths of our assessment plan are the well-focused, measurable learning objectives, and the use of multiple measurements of each objective. Our primary weakness stems from the small number of geology majors we typically have in our upper-level classes where most of the assessment is occurring. We have found that statistical analysis of our data from just a small number of students is not meaningful; therefore it is difficult to make decisions concerning program changes. This has made it difficult to “close the feedback loop” in some areas.

Objectives 1 through 5 were addressed this year. A schedule showing a timeframe of assessment activities is shown below.

Click to see Schedule

Methods used to measure assessed objectives:

This skill is first introduced in Physical Geology laboratory (ERSC 1102), but assessment of this objective occurs primarily in upper level laboratory courses. The primary means of assessment are the laboratory final exams in Mineralogy (ERSC 2320), Igneous and Metamorphic Petrology (ERSC 3311), and Sedimentology (ERSC 3350). Rock and mineral identification are also assessed in a component of Field Camp described in Objective 2 below. The laboratory exams emphasize mineral and rock identification. Exams are used to assess this objective because they directly correspond to the learning objective and can be easily quantified. A minimum score of 70% on each of the exams is considered acceptable for meeting this objective. The rubrics for this exam are designed such that a 70% score would be the lowest “C” (lowest satisfactory score). The objective nature of this exam minimizes reliability issues.

Objective 2 is also assessed during ERSC 4626, Field Geology, a required six-week summer geology field course/camp that is required before graduation. Students generally take this course after their Junior year (structural Geology is a prerequisite) or just prior to graduation in their senior year. Field Geology, like Senior Seminar, is considered a capstone course. It is in this course that the students must utilize information acquired in all major courses and synthesize it to complete several geologic mapping projects and interpret geologic settings and histories of real areas. Students typically take this course through another university (some enroll at UA-Fayetteville's course in Dillon, Montana). This capstone course therefore functions to provide us with an external assessment of our students by non-UALR faculty and a comparison to students in other universities that also take this course (several universities are generally represented at each field camp). An additional assessment form was developed to be given to the appropriate field geology faculty following the field geology courses. This form provides information concerning student performance in specific areas (e.g., mapping, interpreting structures, petrology, stratigraphy, etc.) than an overall course grade provides. A "C" in this course and an average score of 3 or below on the external faculty assessment form is considered the minimum acceptable grade for meeting this objective. Validity of this assessment instrument is addressed by the fact that students must receive departmental approval for the field camp that they sign up for. Reliability is addressed by a comparison with the students overall GPA with the grade received in field camp. Overall GPA of all program graduates is 3.1. (we have not yet calculated geology GPA of students). Field camp GPA is just slightly higher but comparable at 3.4.

Objective 3 - Students will be able to recognize common invertebrate fossils, interpret the ancient environment in which the fossil lived, and assess their utility as indicators of geologic time.

This objective is assessed primarily during the laboratory final exam in ERSC 3360, Paleobiology, a required course taken during the student's sophomore or junior year. An exam is used to assess this objective because it directly corresponds to the learning objective and can be easily quantified. Students are required to identify the fossil (Phylum, Class, and Order typically), provide an age range, and provide additional information on morphology, preservation, etc. A minimum score of 70% on this exam is considered acceptable for meeting this objective. The departmentally developed rubric for this exam is designed such that a 70% score would be the lowest “C” (lowest satisfactory score). The objective nature of this exam minimizes reliability issues.

In the past, Objective 3 was also assessed externally using student performance on the paleontology content area of the Area Concentration Achievement Test (ACAT) in Geology. We have discontinued use of the ACAT because this exam was designed to be a comparison against a national sample of similar departments. Decreasing participation and outdated content caused us to discontinue using this assessment tool. We are presently exploring using other external assessment in place of the ACAT. The most promising is the fundamental geology portion of the Association of State Boards of Geologists (ASBOG) exam. We are currently in discussions with ASBOG and the Geology program at Mississippi State University (MSU). MSU has adopted use of the ASBOG as a key component of their program assessment. We hope to be able to use the ASBOG effectively in our external assessment in the near future.

This objective is assessed at three stages in the geology major's undergraduate career, all of which are required for graduation. Students are first exposed to geologic mapping and cross-section construction during ERSC 2320, Geologic Methods. This course, an outgrowth of UALR's "Reforming the Major" project, is usually taken during the student's sophomore year. Students spend several weeks learning basic map and field techniques, including geologic mapping. Cross sections depicting subsurface structure are then constructed based on the geologic map. As a final project, maps, cross sections, stratigraphy, and the geologic history of an area are then presented in poster format and evaluated by peers and faculty. The evaluation form used to assess this objective is attached. This evaluation form was developed with the consensus of all Earth Science faculty thus addressing the validity of this instrument. Reliability is addressed through examining the “spread” of the ratings. Because there has not been a large spread in faculty ratings, reliability between raters appears to be high. A score of 70% on the evaluation form is considered the minimum acceptable for meeting this objective.

Objective 4 is also assessed in the laboratory portion of ERSC 3330, Structural Geology, where more complex maps, cross sections, and other structures are introduced. This required course is usually taken during a student’s junior year. Throughout the semester, students solve various laboratory exercises (including more sophisticated methods of cross-section construction) related to a geologic map. Near the end of the semester, students synthesize this information into a written summary of the geologic and structural history of an area. A score of 70% on this structural synthesis assignment is considered the minimum acceptable for meeting this objective.

Objective 4 is also assessed during ERSC 4626, Field Geology, which is discussed in detail above under Objective 2.

This objective is taught at all stages of a geology major’s undergraduate career, but is assessed primarily during ERSC 4190, Senior Seminar, a required capstone course in the Earth Science curriculum. Senior seminar is usually taken after completion of all required geology courses. One component of Senior Seminar is an original research project defined and agreed upon in advance with the instructor coupled with a written report and oral report or poster session presentation. Research is considered a very important component of assessment as it requires students to use skills that they are expected to acquire in the geology program such as library research, field techniques, laboratory techniques, data analysis and evaluation, use of the scientific method, and effective written and oral communication. The product of this research is rated by departmental faculty and peer students. A score of 70% on the oral or poster evaluation form is considered the minimum acceptable for meeting this objective.

Student research reporting and presentation occurs in the department's Geology Colloquium held twice annually. All students enrolled in the required Senior Seminar participate in the Colloquium. Our students have seen professional seminars the previous four years where they observe appropriate presentation techniques by professional scientists and other students. The installation of the poster session format as an assessment tool (final) in the new Geologic Methods Course (ERSC 2320) expands the techniques available to evaluate student projects and undergraduate research methodology used on the road to Senior Seminar. A copy of a revised evaluation form for Geologic Methods posters is attached. Validity is addressed through complete faculty involvement in the development of the evaluation forms. Reliability is addressed by having all faculty, including guest faculty from related disciplines and community geologists not on the UALR faculty, evaluate all of the student poster and oral presentations.

In addition to the colloquium, students commonly present Senior Seminar projects or other independent research results at professional meetings, including the Geological Society of America, the Arkansas Undergraduate Research Conference, and the Arkansas Academy of Science.

Assessment Tools