Nursing
Associate of Science in Nursing (10)
Contact Person:
Dr. Ann Schlumberger
Nursing Department
(501) 569-8081
abschlumberger@ualr.edu
UNIVERSITY OF ARKANSAS AT LITTLE ROCK Plan No. 10
Assessment Progress Report Form - Calendar Year 2004
I. USE OF ASSESSMENT FOR PROGRAM BUILDING
AND IMPROVEMENT:
As a component of the nursing
department’s formal written assessment plan, nursing students
are tested at the end of the first year of the program (Custom)
and prior to graduation (Exit) using a national nursing
examination developed by Health Education Science, Incorporated
(HESI). Data provide students with individual feedback on
knowledge in key areas. This is used to assist students in
working on areas of weakness and provides group data specific to
course content and program outcomes that is used by faculty to
strengthen the curriculum. In 2004, score calculation converted
from percentage to the HESI Predictability Model (HPM), a
proprietary mathematical model. Research studies have found the
HESI HPM to be highly accurate in predicting success on the
nursing licensure exam, NCLEX-RN. Based on HESI research data, a
score of 900 is identified as the recommended level of
performance, with 850 identified as acceptable.
A score of 850 has been established as the benchmark that
has been incorporated into program outcomes. The class mean is
established as the benchmark for consistency in reporting data
correlated with NCLEX-RN data.
Students are required to score 850 on the
Custom HESI and two attempts are allowed. The exam is
administered at the completion of the first year of the program.
Students scoring below 850 on the second take are encouraged to
use the tutorial resources available in the Learning Resource
Center, but progression is not interrupted. Individual score
reports indicate areas of strengths/weaknesses which is used to
guide students in remediation. Custom HESI data have proven to
be an indicator of success in the second year of the program. In
2004, a team faculty review was conducted to verify correlation
of the HESI Custom exam content to curricular concepts. Eleven
revisions were made and will be reflected on the Spring 2005
exam.
As a result of data analysis, a score of
850 on the HESI Exit Exam is required for successful completion
of Nursing 2910. The exam is administered at the end of the
course. Students are allowed three attempts to achieve the
required score, with remediation between attempts. To date, all
students have achieved the required score and NCLEX-RN pass
rates continue to exceed state and national first-write pass
rates. The 1999-2004 retake pass rate is 100%.
Analysis of May 2004 HESI Exit Examination
data indicated established levels of achievement were met or
exceeded in all categories of Program Outcomes/Goals. The
performance on the Exit Examination shows improvement in all
areas.
The current method of assessment provides
the program with trended, aggregated data for use in program
improvement. Annually the faculty review data as a component of
the systematic Plan for Evaluation. The use of assessment
findings for program improvement contributed to the 2001 UALR
Department of Excellence Award and the maximum 8-year
accreditation by the National League for Nursing Accreditation
Commission (NLNAC Fall 2002 Accreditation Visit). Data indicate
continued improvement in graduate performance on the HESI Exit
Exam and on NCLEX-RN. Curricular revisions related to admission
and progression were implemented in 2004. No curricular changes
will be made for 2005.
Assessment data are also obtained from
annual evaluations. First and second year students complete
Program Evaluations. Student response rate is 78%--100%.
Graduates and Employers complete a Graduate/Employer
Evaluation Survey 6 months after graduation. This survey has a
Content Validity Index of 1.0. Response rate ranges from
24%-33%. In an effort to increase response rates, the department
contracted with the Institute of Government Survey Research
Center (SRC) to conduct a telephone survey of 2004 Graduates and
Employers. Response rate increased to 54% and 93% respectively.
During 2004, the department assessed each
Program Outcomes/Goals and two Program Objectives.
Program Outcome (Goal 1)
Benchmark: First and second year students will have a
mean score of 850 or higher on the overall HESI examination and
on subscores in the areas of critical thinking, communication
abilities, and therapeutic nursing interventions.
Click
to see HESI Test results *Prior to
the score reporting change, the benchmark for HESI testing was
85%. Benchmark: 90% or more of
first and second year students who respond to the Program Evaluation will rate
program preparation in the areas of critical thinking, communication
abilities, and therapeutic nursing interventions as
satisfactory.
Click
to see Program Evaluation
Program Outcome (Goal) 2
Benchmark: 95% or more of RN graduates who
actively seek employment will be employed as RNs.
Benchmark not met. 83%
of RN graduates who actively sought employment are employed as
RNs. A component of the curriculum includes a capstone course
(Nursing 2910) with a preceptorship in which students work with
selected registered nurses in a practice setting. Data indicate
that most graduates are employed by an institution that supports
the preceptorship component of the curriculum.
In Spring 2004, preceptor sites were expanded to include
facilities in adjacent counties where increasing numbers of
nursing students reside. This action was to meet student and
area healthcare workforce demands. This number reflects
employment secured prior to graduation. The remaining students
were pursuing employment leads or indicated they would seek
employment following graduation.
Program Outcome (Goal) 3
Benchmark: 95% or more of RN graduates will
initially practice in the State of Arkansas. Benchmark not met.
89% of RN graduates of 2004 are initially practicing in
Arkansas. The remaining reported plans for initial licensure in
other states, mainly due to spousal employment, military duty,
or family location. States listed included Texas, Maryland, New
York, Alaska, and Pennsylvania.
Program Outcome (Goal) 4
Benchmark: Graduates will meet or exceed
the average Arkansas NCLEX-RN pass rates on first writes.
Benchmark exceeded since 1999. The May 2004 class pass rate is
98%. The Arkansas pass rate is 86%. Of note, UALR nursing
graduates exceeded both the Arkansas and national pass rate
(85%) and have consistently done so for the past 6 years.
Program Outcome (Goal) 5
Benchmark: 90% or more of graduates who
respond to the Graduate Survey will report overall satisfaction
with preparation provided by the program.
Benchmark exceeded. The Graduate Survey is distributed
(per NLNAC guidelines) 6 months following graduation. 98% of
respondents of the reported overall satisfaction with
preparation provided by the program.
Benchmark: 90% of more of first and second
year students who respond to the Program Evaluation will report
overall satisfaction with preparation provided by the program.
Benchmark exceeded; Program evaluations completed in May 2004 by
first year students indicated 96% satisfaction and second year
students, 92% satisfaction.
Program Outcome (Goal) 6
Benchmark: 90% or more of employers who
respond to the Employer Survey will report overall satisfaction
with the preparation provided by the program.
Benchmark exceeded; 100% satisfaction
reported by employers.
Program Outcome (Goal) 7
Benchmark: Annual graduation rate will be
50% or higher. Benchmark not met. The graduation rate for the
class of 2004 was 49%. This increased from 2003 following
development and implementation of a retention plan. Retention
efforts included the restructuring of the first semester course
to 3 teaching teams to provide continuity between classroom and
clinical instructors, free tutoring in the department, and the
addition tutorial resources for the Learning Resource Center.
Admission criteria were revised and Anatomy & Physiology I
is required for program admission and support courses (Anatomy
& Physiology II, Chemistry 1400 and Biology 2401) must be
completed during the program, beginning Fall 2005. Data are
being collected to evaluate the effectiveness of an RN
Pre-Admission Examination as an admission requirement. This
study will be completed in 2006 to determine the reliability of
this exam in predicting program success for students entering
the program. In 2004, an 8-hour “Survivor Course” was
implemented to introduce new students to the program,
facilities, available resources and support, as well as
providing information relating to study, time-management, and
test-taking skills required for program success. Enrollment was
limited to fifty students and 47 completed the course. The
course will be offered again this fall and the enrollment limit
will be doubled. Faculty volunteer to serve as “guides” for
the course. In addition to retention efforts, data are collected
via chairperson interview as students exit the program to
determine factors that contributed to course failure/withdrawal.
For the Class of 2005, 18% cited poor study habits and testing
skills as the reason for course failure; 11% cited family or
health problems and work interference. 7% withdrew prior to
completing the course and the majority of these students cited
social issues as interfering with success. While attrition and
retention assessment is required by the NLNAC, it should be
noted that our retention rate is consistent with the national
average (NLNAC, 2004).
This year the faculty also assessed two
Program Learning Objectives:
1. Use critical thinking while applying the nursing process
to provide effective nursing care to human beings with unmet
biophysical and psychosocial needs who require promotion,
maintenance, and/or restoration of health.
2. Practice accountability and responsibility by adhering to
legal and ethical standards of practice.
Program/Learning Objective – Nursing
Process
The nursing process includes assessment,
analysis, planning, implementation, and evaluation. Benchmark:
First and second year students will have a mean HESI score of
850 or higher on nursing process subscores (assessment,
analysis, planning, implementation, evaluation).
Click
to see HESI Results
Benchmark: 90% or more of first and second
year students who respond to the Program Evaluation will rate
program preparation in nursing process as satisfactory.
Click
to see Program Evaluation results
Program/Learning Objective –
Legal/Ethical
Practice accountability and responsibility
by adhering to legal and ethical standards of nursing.
Benchmark: First and second year students
will have a mean score of 850 and higher on legal/ethical
subscores.
Click
to see HESI Testing Data
Benchmark: 90% or more of first and second
year students who respond to the Program Evaluation will rate
program preparation in legal/ethical standards as satisfactory.
Click
to see Program Evaluation Data
Benchmark: 90% or more of graduates and
employers who respond to the Graduate/Employer Survey will
report preparation in legal/ethical standards as satisfactory.
Click
to see Follow Up Graduate/Employer Survey
II. FACULTY AND STAKEHOLDER INVOLVEMENT
The department has a systematic method for
program assessment and evaluation that was implemented in 1998.
Under this governance model, faculty serve on department
committees. A revision in 2003 resulted in committee assignments
based on individual faculty strengths and preferences as well as
department needs. This structure allows for continued systematic
program assessment. Bi-annual
Systematic Evaluation Meetings (December and May) serve as a
forum for presenting assessment data, reviewing program
progress, and soliciting faculty input. Each committee
chairperson completes a written report using a designated report
format. Faculty are provided with copies of all reports in
advance of the meeting. Decisions regarding program maintenance,
revision, and/or development are based on trended data analysis
and are approved by faculty at these meetings.
The department underwent an accreditation
visit by the NLNAC in September 2002. We were cited as having a
“fully implemented Systematic Plan of Evaluation that serves
as a model for assessment activities in nursing education!”
Patterns of strength that were cited in the official
reporting document included, “the outstanding contributions of
the faculty members within the department in all areas;”
“the UALR DON is truly a Department of excellence, not only at
UALR but also within the broader nursing education community;”
“the full achievement of each of the 22 NLNAC criterion for
accreditation.” Additionally, the NLNAC National Review Panel
cited the program as a national model and encouraged the program
to publish the Self-Study Report as an example of an exemplary
report.
The department consistently earns high
ratings (3.7—4.0) in college assessment evaluations. For
several years we have submitted a poster at the University
Assessment Expo outlining our assessment process.
In 2001, a Student Advisory Committee was
developed consisting of student representatives from the first
and second year classes. The committee replaced a system that
required students to attend numerous departmental committee
meetings in which student participation was limited. Student
participation in the Student Advisory Committee has resulted in
monthly meetings in which student/faculty dialogue regarding
concerns and suggestions for departmental improvement.
Additional stakeholders involved in the
ongoing assessment process include the institutions and
community agencies in which nursing students participate in
clinical laboratories as students and subsequently as registered
nurses. Employers and alumni alike provide formal assessment
feedback to the program each January via the Graduate/Employer
Follow up Survey. In 2004, we hosted the first annual “Day of
Dialogue,” in which representatives from area healthcare
agencies were invited to meet with faculty to discuss issues
relating to program assessment and improvement. Feedback from
the meeting was positive and resulted in several nursing student
policy revisions.
The community of health care consumers in
the metropolitan area of central Arkansas participates in the
program’s assessment process. These are accomplished by
evaluation of direct care received by students/graduates as part
of extensive service learning projects and nursing care provided
by the nursing students/graduates.
Data from all stakeholders is used as a
part of the formal systematic plan for evaluation. It is used to
evaluate and modify teaching/learning methods, content, and
objectives, as trended data indicate.
Beginning Fall 2003, the first year of the program was
offered at the UALR Benton site and the second year is in
progress. This was in response to the community’s request for
nursing course availability in Benton and the awarding of a
$826,000 Workforce Grant. Public and agency support for the
course offerings has been generous and continuation is planned.
III.
APPROACH
Program Outcomes (Goals)
Program Outcomes (Goals) are specified by
the National League for Nursing Accreditation Commission (NLNAC).
Benchmarks are established by the department.
1. Critical Thinking, Communication
Abilities, Therapeutic
Nursing Interventions
· First and second year students will have a mean score of
850 or higher on the overall HESI examination score and on
subscores in the areas of critical thinking, communication
abilities, and therapeutic nursing interventions.
· 90% or more of first and second year students who respond
to the Program Evaluation will rate program preparation in the
areas of critical thinking, communication abilities, and
therapeutic nursing interventions as satisfactory.
2.
Graduation Rates
· Annual graduation rate will be 50% or higher.
3.
Job Placement Rates
· 95% or more of registered nurse graduates who actively
seek employment will employed as registered nurses.
4.
Patterns of Employment
· 95% or more of registered nurse graduates will initially
practice in Arkansas.
5.
NCLEX-RN Pass Rates
· UALR Department of Nursing graduates will meet or exceed
the average Arkansas NCLEX-RN pass rate on the first write.
6.
Program Satisfaction (students/graduates)
· 90% or more of graduates who respond to the Graduate
Survey will report overall satisfaction with the preparation
provided by the program.
· 90% or more of first and second year students who respond
to the Program Evaluations will report overall satisfaction with
the preparation provided by the program.
7.
Employer Satisfaction
· 90% or more of employers who respond to the Employer
Survey will report overall satisfaction with the preparation
provided by the program.
Central Student Learning Objectives
(Program Objectives)
Program Objectives for the nursing program
are required to be linked to concepts in the philosophy (i.e.
nursing process, communication, etc.)
At the conclusion of the Associate Degree
Nursing Program, the student is prepared to:
1. Use the nursing process to deliver effective nursing care
for human beings with unmet biophysical and psychosocial needs
who require promotion, maintenance, and/or restoration of
health.
2. Use effective communication to assist individuals,
families, significant others, and health care providers.
3. Practice accountability and responsibility by adhering to
legal and ethical standards of practice.
4. Manage nursing care for a small group of individuals by
establishing priorities for care and collaborating with other
health care providers.
5. Pursue self-established goals for continued
self-development and maintenance of clinical competencies.
6. Prepare to contribute to society through participation in
community and/or professional activity.
The department uses the following data
collection tools that are rated consistent and valid by
established content validity indices:
· Program Evaluations: These are completed each April by
both first and second year students. In 2003 the evaluations
were done electronically via WebCT. Student response was
favorable and this method will continue. In addition to Likert scale items, students are asked
to provide suggestions for improvement for any item not rated as
satisfactory. Students are also provided with the opportunity to
document general comments regarding the program. The Program
Evaluation tool was developed using NLNAC guidelines and is
assessed annually. Data obtained from Program Evaluations are
analyzed by the Assessment Committee and distributed to faculty
at the August Faculty Meeting. Data are reported as counts and
percentages.
· Graduate and Employer Surveys: Surveys are conducted six
months following graduation to each graduate and direct
supervisor of the graduate. In an effort to increase the
response rate to this survey, they are being conducted by the
ISR via telephone. Surveys were developed using NLNAC guidelines
and are evaluated annually by the Assessment Committee. Data
from the surveys are presented at the May SPE faculty meeting.
Data are reported as counts and percentages. Faculty implement
developments and/or revisions resulting from data analysis the
following fall semester.
· Heath Education Science, Incorporated (HESI) testing:
This is a proven external prediction model for NCLEX-RN pass
rates and a valid measurement tool for use in evaluating nursing
curricula. Reliability is calculated using the Kuder-Richardson
Formula 20 (KR-20). The
KR-20 has indicated high reliability for our students since we
began using the HESI exam.
· NCLEX-RN results: Results are provided by the National
Board of Nursing and the Arkansas State Board of Nursing. Data
are reported as counts and percentages.
Feedback from the
CSAM 2004 Assessment Report:
“Excellent stakeholder involvement”.
“Using HESI and licensure exams as
measures laudable”
“Strengthening admission requirements to
increase retention laudable”
“Multiple faculty committee involved in
assessment activities”
Suggestions from CSAM 2004 Assessment
Report:
“Encouraged to address population that does not
succeed”
“85%-100% pass rate is great but do you need to look at
…
retention..”
All
Program Outcomes/Goals will be evaluated annually. In addition,
the Program Objectives will be evaluated per this schedule:
Click to see
Scheduled Timeframe
|
