ACRL Planning, Assessment, and Outcomes Measurement Questions
Planning
The library should have a mission statement and goals to serve as a framework for its activities. The mission and goals should be compatible and consistent with those developed by the institution. Assessment of the quality and effectiveness of the library should be linked closely with the specific mission and goals of the institution. In order to build its programs and services in the context of the institution the library should be involved in the overall planning process. Formal planning procedures and methods, such as strategic planning, are used frequently. These planning methods require input from a broad spectrum of the institution’s community. They help the institution prepare for the future by clearly defining a vision and mission, by setting goals and objectives, and by implementing specific strategies or courses of action designed to help meet those ends. Strategic planning is an iterative process that includes evaluation, updating, and refinement. This process helps the community focus on its essential values and provides an overall direction that helps to guide day-to-day activities and decisions.
Assessment
Comprehensive assessment requires the involvement of all categories of library users and also a sampling of non-users. The choice of clientele to be surveyed and questions to be asked should be made by the administration and the staff of the library with the assistance of an appropriate advisory committee. Questions should relate to how well the library supports its mission and how well it achieves its goals and objectives. Library users should be encouraged to offer signed or anonymous comments and suggestions. Opportunities for making suggestions should be available both in the library and through remote electronic access. All categories of users should be given an opportunity to participate in the evaluation. The weight given to responses should be consistent with the focus and mission of the library. A program of assessment and evaluation should take into consideration the changing rhythm of the academic year. Evaluation, whether it involves some or all of the techniques listed below, should be an ongoing process. Formal evaluation tools may include the following:
General library knowledge surveys (or "pre-tests") offered to incoming first year students, re-offered at a mid-point in the students' careers and again near graduation, to assess whether the library's program of curricular instruction is producing more information-literate students.
- Evaluation checklists for librarian and tutorial instruction to gather feedback from students, other librarians and teaching faculty.
- Student journal entries, or information literacy diaries, used to track their library use.
- Focus groups of students, faculty, staff, and alumni who are asked to comment on their experiences using information resources over a period of time.
- Assessment and evaluation by librarians from other institutions and/or other appropriate consultants.
- Reviews of specific library and information service areas and/or operations.
Assessment Outcomes
Outcomes assessment will increasingly measure and affect how library goals and objectives are achieved. It will address the accountability of institutions of higher education for student achievement and cost effectiveness. It should take into consideration libraries' greater dependence on technology, their increasing use of online services, their growing responsibility to provide information literacy skills, their increasing reliance on consortial services, the possibilities of dwindling financial resources for collection development, and new developments in the ways in which scholarly information is published and distributed.
Outcomes assessment can be an active mechanism for improving current library practices. It focuses on the achievement of outcomes that have been identified as desirable in the library's goals and objectives. It identifies performance measures, such as proficiencies, that indicate how well the library is doing what it has stated it wishes to do. Assessment instruments may include surveys, tests, interviews, and other valid measuring devices. These instruments may be specially designed for the function being measured, or previously developed instruments may be used. It is critical, however, to choose carefully the instrument, the size of the sample, and the method used for sampling. The instrument should be valid, and the way it is used should be appropriate for the task. Colleagues at peer institutions may render invaluable assistance by suggesting assessment questions and sample sizes, by sharing lessons learned, and suggesting alternative methods for measuring outcomes.
1. Is the library’s mission statement clearly understood by the library staff and the institution’s administration? Is it reviewed periodically?
The University Library’s staff and administration reviews its mission statement, as well as its vision statement, periodically in order to maintain its viability. Members of the staff are cognizant of both documents in order to effectively serve patrons. A subcommittee is organized to introduce and familiarize both the staff and administration of amendments to these documents. Since the last accreditation review in FY2000, a subcommittee was formed to review these statements in FY2002, FY2003, FY2006, and FY2008.
2. How does the library incorporate the institution’s mission statement into its goals and objectives?
The Library has a vision statement that highlights its goals and objectives. These goals and objectives incorporate the institution’s mission statement by echoing its purposes and supporting its pledged developments. The foundation of the Library’s Mission Statement, as clarified in the Vision Statement, supports the commitments of the institution’s Mission.
In the process of comparing the University Library to peer institutions it was revealed that the M. Louis Salmon Library at the University of Alabama at Huntsville maintains a mission statement and goals outlined in the Directions for Strategic Development. According to the institution’s SACS Compliance Certification Report, section 3.8.1, the “Library’s mission and goals are reviewed and updated regularly in relationship to the University’s mission and goals and in relationship to the Library’s services.”
Supporting Documentation:
1. Mission Statement
2. Vision Statement
3. How does the library maintain a systematic and continuous program for evaluating its performance, for informing the institution’s community of its accomplishments, and for identifying and implementing needed improvements?
The University Library maintains a systematic and continuous program for evaluating its performance by administering user satisfaction surveys, assessing user responses, and taking steps to correct or to improve deficiencies as indicated by user survey responses. The General Library Survey administered by the Circulation Department has been used periodically for this purpose and more recently, LibQUAL+, a web-based online user satisfaction survey developed by the Association of Research Libraries, was administered for the first time during the fall of 2007 as part of the Library’s ongoing commitment to improving its library services. A Collection Development survey was designed in early 2003 to evaluate subject areas needing enhancement. However, it has also served as an overall survey as respondents make comments about all facets of library services. The Library’s Webpage provides a “Feedback” section where users can contact librarians with research needs, questions and suggestions. Library departments such as the National Center for the Study of Civil Rights and African-American Culture may also evaluate its performance by soliciting user survey responses in conjunction with specific annual programs, tours or other educational services. A suggestion box is placed at the The National Center to encourage visitor participation in identifying areas where services need improvement.
The Library informs its community of faculty, staff, students and other users of its accomplishments through e-mail updates, through the posting of informational flyers within the facility, and through announcements in The Libretto, the official print and online newsletter of the library. Faculty, staff and students also have opportunities to receive relevant information through staff meetings, training sessions and through other print and online publications of Alabama State University, i.e., The Hornet Tribune and ASU Today.
The Library identifies needs for improvement by regularly reviewing survey responses and other user response mechanisms that are in place to help identify deficiencies. Through reviews of existing policies, the Library is able to determine if guidelines are already in place and are being followed to address perceived deficiencies. If policies are not already in place to address specific deficiencies or grievances, steps are taken to determine how such deficiencies or grievances should be handled. Staff meetings, discussions and other brainstorming sessions are regularly held to address these issues. The implementation of needed improvements is guided by decisions on feasibility, administrative approval and Library resources, including staff and budget.
4. Is the library's assessment plan an integral component of the institution's assessment and accreditation strategies? For example, does the library revise and update its assessment procedures in conjunction with campus-wide planning and the actions of academic departments?
The University Library's Planning Committee works to develop the library's mission, vision, and value statements. The University Library's goals and objectives are formulated in accordance with the mission, vision, and value statements. Each library department establishes goals and objectives that are evaluated on a monthly basis through the How-Goes-It-Reports. The goals and objectives are an integral part of the institutional and unit goals for the Office of Academic Affairs. As part of the planning process, the goals and objectives for each library department are reviewed and updated on an annual basis to ensure that the focus is in keeping with the overall mission for Alabama State University (ASU).
Through the How-Goes-It-Reports, the University Library's assessment plan is conducted in a very systematic manner to ensure customer satisfaction. The assessment plan also incorporates several types of measures to ensure learning opportunities, resources, facilities, and services to support teaching, research, scholarship, and cultural activities for Alabama State University and its local and global communities. The monthly How-Goes-It-Report includes internal and external assessment activities such as holdings reports and general library surveys. In FY08, the University Library conducted the LibQUAL+ web-based survey to assess the quality of library services campus-wide. The LibQUAL+ survey was administered to undergraduate, graduate, faculty, and staff at Alabama State University during October 15, 2007 - November 9, 2007. Furthermore, in FY07, the library participated in the Academic Library Survey (ALS) previously referred to as the Integrated Postsecondary Education Data System (IPEDS) survey. This biennial survey is a regular part of the assessment conducted by the ASU Office of Institutional Research as well as other academic departments. The data collected from the Academic Library Survey is used as a basis for peer comparison for the Southern Association of Colleges and Schools (SACS) accreditation report for the University Library.
5. How does the library assess itself? (e.g., What quantitative and qualitative data does the library collect about its performance? How does it take into account special needs, such as those of physically challenged users?)
The library is assessed, quantitatively and qualitatively, by collecting and maintaining statistics that measure the output, which leads to the outcomes. These statistics are collected and documented in monthly departmental How Goes It reports, on the Levi Watkins Library Intranet, in staff email alerts, and in the ASU University Library & Learning Resources Annual Report. Reporting of output and outcomes of the LibQUAL+ survey, conducted in November 2007, is posted on the Intranet as well.
Special needs of users are met with library compliance of "Library Services for People with Disabilities Policy." Special needs of physically challenged users that may not be immediately evident are favorably addressed when requested.
6. What outcomes does the library measure, and how does it measure these outcomes?
Outcomes the Library measures fall into three categories. The first category measures successful access to resources. Resources include print materials (able to match, locate, and utilize books and/or journals), electronic resources (databases with successful searches and hits, number of turnaways), and electronic collections (able to search and view articles within the collection). The second category measures improved understanding of resources. Most of the Library’s instruction comes from sessions coordinated by the Information Literacy Department. The remaining sessions are handled by individual departments (Special Collections, Health Sciences, and Education). Training of the library’s student assistants to be more familiar with library resources is also measured. For each of these cases, pre- and post-tests are given and evaluated. The third categories of outcomes concern fulfillment of materials’ requests. Inter-library loan requests, whether borrowing or lending, fall under this category. Book and journals requests are also here. Many of these outcomes are evaluated via the annual ACRL and biannual IPEDS reports.
7. How does the library compare itself with its peers?
The Library compares relatively well with its peers for a school of its size. The 2004 and 2006 statistics in the table below came from the NCES Library Peer Analysis system. The Alabama State University library appears to compare favorably, but some of the peer libraries’ averages reflect special cases of individual institutions.
Institution |
Year Rev- iewed |
FTE Enroll-ment |
Book Volumes |
Volumes per Student |
Current Serials |
Audio-visual Materials | Micro- forms |
Materials Expenditures |
Total Expenditures |
Library Expend- itures per Person Enrolled |
Libra-rians |
Alabama A & M (AL) |
2004 |
5627 |
621,485 |
110.45 |
1718 |
5919 |
140934 |
$1,417,449.00 |
$2,400,341.00 |
$426.58 |
9 |
| 2006 | 5888 | 448,787 | 76.22 | 1718 | 6338 | 142162 | $1,122,300.00 | $2,176,465.00 | $369.64 | 10 | |
Auburn University - Montgomery (AL) |
2004 |
3894 |
319,669 |
82.09 |
2202 |
25223 |
2421686 |
$759,163.00 |
$1,870,348.00 |
$480.32 |
10 |
| 2006 | 4120 | 331,513 | 80.46 | 2280 | 23449 | 2424915 | $985,042.00 | $2,249,981.00 | $546.11 | 10 | |
Clark Atlanta University (GA) |
2004 |
4216 |
378,848 |
89.86 |
1419 |
11702 |
867237 |
$1,632,300.00 |
$4,465,000.00 |
$1,059.06 |
21 |
| 2006 | 4336 | 380,022 | 87.64 | 30,083 | 10152 | 994228 | $2,074,573.00 | $8,637,979.00 | $1,992.15 | 25 | |
Columbus State University (GA) |
2004 |
5541 |
376,242 |
67.9 |
1763 |
17517 |
456910 |
$248,233.00 |
$1,279,153.00 |
$230.85 |
10 |
| 2006 | 6305 | 386,733 | 61.34 | 1707 | 516,236 | 1127021 | $669,108.00 | $1,952,543.00 | $309.68 | 11 | |
Florida A&M University (FL) |
2004 |
12122 |
879,458 |
72.55 |
19483 |
76,070 |
208153 |
$3,091,608.00 |
$5,779,646.00 |
$476.79 |
29 |
| 2006 | 11651 | 927,412 | 79.6 | 64,678 | 76426 | 425941 | $4,106,911.00 | $6,285,008.00 | $539.44 | 30 | |
Jacksonville State University (AL) |
2004 |
7249 |
674,818 |
93.09 |
1688 |
35636 |
1406037 |
$924,712.00 |
$2,275,910.00 |
$313.96 |
15 |
| 2006 | 7771 | 694,365 | 89.35 | 57,100 | 36814 | 1404381 | $981,484.00 | $2,450,861.00 | $315.39 | 16 | |
Troy University (AL) - 2 branches |
2004 |
15541 |
411,224 |
26.61 |
1397 |
16473 |
1578145 |
$850,162.00 |
$1,596,129.00 |
$103.30 |
8 |
| 2006 | 20213 | 540,801 | 26.76 | 3263 | 32972 | 1977283 | $1,217,477.00 | $2,563,165.00 | $126.81 | 16 | |
University of Alabama - Huntsville (AL) |
2004 |
5530 |
327,665 |
59.25 |
1051 |
2677 |
579663 |
$980,164.00 |
$1,979,384.00 |
$357.94 |
9 |
| 2006 | 5661 | 334,684 | 59.12 | 870 | 2677 | 572297 | $1,082,134.00 | $2,051,210.00 | $362.34 | 9 | |
University of North Alabama (AL) |
2004 |
5028 |
371,123 |
73.81 |
3540 |
11368 |
1047372 |
$868,728.00 |
$1,972,102.00 |
$392.22 |
11 |
| 2006 | 5340 | 380,361 | 71.23 | 3760 | 11869 | 1059711 | $1,028,456.00 | $2,182,059.00 | $408.63 | 10 | |
University of South Alabama (AL) |
2004 |
11102 |
1,300,653 |
117.15 |
5804 |
33558 |
1502658 |
$2,499,839.00 |
$5,133,290.00 |
$462.38 |
26 |
| 2006 | 11673 | 1,075,857 | 92.17 | 5947 | 34562 | 1532110 | $3,468,709.00 | $6,396,005.00 | $547.93 | 24 | |
AVERAGE - 10 univ's |
2004 |
7585 |
566,119 |
79.28 |
4007 |
23,614 |
1157052 |
$1,327,236.00 |
$3,263,510.00 |
$430.34 |
15 |
| 2006 | 8296 | 550,054 | 72.39 | 17,141 | 75,150 | 1151789 | $1,673,620.00 | $3,476,882.00 | $551.81 | 16 | |
Alabama State University (AL) |
2004 |
4889 |
408896 |
83.64 |
1935 |
42711 |
2636683 |
$1,286,622.00 |
$2,581,928.00 |
$528.11 |
16 |
| 2006 | 5134 | 417404 | 81.3 | 2082 | 43318 | 2657119 | $1,233,391.00 | $2,925,615.00 | $569.85 | 15 | |
Difference |
2004 |
-2696 |
-157,223 |
4.36 |
-2,072 |
19,097 |
1479631 |
($40,614.00) |
($681,582.00) |
$97.77 |
1 |
Difference |
2006 |
-3162 |
-132,650 |
8.91 |
-15,059 |
-31,832 |
1505330 |
($440,229.00) |
($551,267.00) |
$18.04 |
same |
Alabama State University’s library is compared with 8 other institutions in Alabama, 2 in Georgia, and 1 in Florida. While full-time enrollment and volume holdings for Alabama State University are below the 10-university average, the ratio of books per student is still higher. The current serials category difference is due to three larger than average numbers which belonged to Clark Atlanta University, Florida A&M University, and Jacksonville State University. Without these three institutions, the average would become 2,443 current serials, resulting in a difference of 361 current serials.