Source
For more complete information,
see the published studies listed below

What didn’t work:
Terms reported as
being misunderstood, not understood,
or not preferred

What did work:
Terms reported as being
understood or preferred;
Successful strategies for presenting
or explaining terms

Appalachian State University Library and Georgia Southern University Library
This test compared the sites of these two libraries and that of the University of Arizona Library.

Test method: user observation.

Test participants: 32 freshmen

See:
W. Bede Mitchell et al., "Testing the Design of a Library Information Gateway", ACRL Tenth National Conference (Denver, 2001); also published in Southeastern Librarian 49 (2001), 4-10.

Periodical
Databases ('without referring to magazines, periodicals, or articles")

Databases and Periodical Article Indexes [for finding a newspaper article]; "If the precise term, such as Newspaper ... did not appear in the description of an option, many students thought it was probably not to be found there."

------
Special Collections "did not convey to the freshmen anything other than ... catalogs of materials besides books. ... [It] was also chosen in desperation for other searches as well, indicating that this is not a good term to use on an opening library Web site screen if it is not further defined."

"The Arizona site's icon clearly represented magazines and newspapers with the word articles prominently displayed, making it easy for students to find the best search option."

------
Library Catalog "led to more correct responses [on finding a book] than ... a prominent icon which featured a book [or] ... Books and more"

Bucknell University Information Services & Resources

Pathfinders Usability Study (2000)

Test method: Questionnaire.

Test participants: 21

Information provided by Isabella O'Neill, Bucknell University.

Pathfinders
[as a generic term for subject guides]

Research by Subject
[now used on production site]

Other terms suggested by participants:
ResearchFinder
Research Home
Research Source
Getting Started
Research Pathfinders
Beginning Research Page
Where to Start
Researching? Start Here

California State University, Long Beach Library

Test method: user observation.

Test participants: 9 students at the University of Arizona.

See:
Tiffini Anne Travis and Elaina Norlin, "Testing the Competition: Usability of Commercial Information Sites Compared with Academic Library Web Sites," College & Research Libraries 63 (2002), 433-448.

COAST [library catalog]
"... the annotation beneath the link to COAST did not mention the word 'books'. In fact, nothing on the home page or the secondary electronic resources page ever said the word 'book.' Instead, ambiguous terms such as 'items' and 'resources' were used."

"By contrast, students did read the descriptions under the Research Databases link. They noticed the words 'articles' and 'research.' As a result, students did not encounter the same difficulties finding articles as they did finding books ..."

[Since this test, this section of the home page now reads as follows:
Electronic Resources
Find books (COAST), articles (Research Databases), Course Reserves, Research Guides to help you with your research, links to other libraries, & more!]

College of Charleston Libraries

Test method: user observation.

Test participants: 15 students.

See:
Debbie Vaughn and Burton Callicott, "Broccoli Librarianship and Google-Bred Patrons, or What's Wrong with Usability Testing?", College & Undergraduate Libraries 10 (2003), 1-18.

DISCLAIMER: The fact that I am quoting data from this article does not imply agreement with the authors' expressed views on library terminology, web site usability, or test methods. In this case, the characterization of Databases as an unsuccessful link label is my interpretation only.

Databases

"Several participants experienced difficulty when asked to find a magazine or journal article about The Great Gatsby. Eight of the 15 participants were unable to complete the task and many who eventually completed the task indicated that they were not confident that they had been successful. ... By watching and listening to the participants, it was clear that when the terms magazine, journal, or article did not appear on the libraries' home page, their confidence and success in their ability to complete this task was lowered."

Hunter College Libraries

Test method: user observation.

Test participants: 24 undergraduates, 2 graduate students, 2 others.

See:
Laura Cobus, Valeda Frances Dent, and Anita Ondrusek, "How Twenty-Eight Users Helped Redesign an Academic Library Web Site", Reference & User Services Quarterly 44 (Spring 2005), 232-46.

Databases
(described by one student as "the base that holds the data")

Electronic Journals
("I would go to [that link] just because it says 'journals.")

Reference Shelf
"... very general. You don't know what to expect as it could be anything.")

Web Guides

Archives

FAQs

Finding an Article

Finding a Book

Library Instruction

Tutorials

Subject Directories (Websites reviewed by librarians)

MIT Libraries
"Big Test" usability test, November 2002

Test method: user observation.

Test participants: 21 students (9 undergrads + 12 graduate students)

For detailed reports on these and other tests, plus useful guidelines and policies, see the MIT Web Advisory Group site.

MIT Libraries
Usability test results, March-April 2001

Test method: user observation.

Test participants: 3 undergrad students, 4 grad students, 1 faculty member.

Accompanying the name Barton
with MIT Libraries' online catalog

Accompanying the name VERA
with A listing of our databases and e-journals

In the VERA electronic resources system, use Display List instead of Search to show a list of resources whose titles contain certain words.

For MIT users only (to replace MIT only)

MIT Libraries
Barton catalog test, February 2001

Test method: user observation.

Test participants:7 undergrad students, 3 library staff

MIT Libraries
Card-sorting exercise, Summer 1999

Test method: manual card sorting

Test participants: 9 volunteers

Test instructions

Databases
"people used this word to mean many different things, a much broader definition of it than we usually think of (no clear trend other than this)"

Resources
"No one used the term, resources, to describe anything."

Major categories created by more than one participant:

Thesis information
Information by Course
New materials & library news
Ordering materials
Access policies for our library and other libraries
About the MIT Libraries
Services
Reference
Materials/Content/Collections

See full descriptions

MIT Libraries
Category identification Test, Fall 1999

Test method: survey

Test participants: 21 library users

Description

Particpants associated:

New Books with Library News & Updates
Electronic Journals with Searchable Resources
Hours with About the Libraries
Barton: Online Catalog of the MIT Libraries with Searchable Resources
Locations of MIT Libraries with About the Libraries
Specifications for Thesis Preparation with Subject/Course-related Information
Databases on the Web with Searchable Resources
Interlibrary Borrowing with Services
Library Course Page for 6.763 with Subject/Course-related Information
Research Assistance with Services

MIT Libraries
Web Site Usability Test, March 1999

Top 5 observed problems included:

1. Unclear link names: Barton, RSC, ILB, virtual reference

2. Vague and unclear category names: resources, services, subjects

Memorial University of Newfoundland Libraries

Test methods:  user task performance with some automated data collection; questionnaire.

Test participants: 17 undergraduates, 2 graduate students, 14 faculty members.

See:
Louise McGillis and Elaine G. Toms, "Usability of the Academic Library Web Site: Implications for Design," College & Research Libraries (July 2001), 355-367.

Journal article
 
Internet Resources vs. Databases

Internet Resources by Subject vs.Internet Search Engines vs. Electronic Books vs. Internet Reference Sites

Research Help vs. Request Forms

Local terms:
Webcat (library catalog)
Unicorn (library catalog)
Do-it-Yourself in Unicorn

Brief descriptions under main menu items on the home page. "[T]hese annotations ... were the most effective aid in assisting participants to havigate the menus because they provided hints about what might be found on the next menu level. These were much more specific than the menu choice and able to add discriminating power".

“Some participants suggested it would be more helpful if each category’s annotation contained all subsequent choices”.

Minnesota Library Information Network (MnLINK)

Test method: Task-based questionnaire using paper prototypes.

Test participants: 101 students enrolled in writing courses at Minnesota State University, Mankato.

See:
Joan Roca and Roland Nord, "Usability Study of the MnLINK Gateway," OCLC Systems & Services 17 (2001), 26-33.

Library Catalogs
Databases

Basic search
Advanced search

Help
"More than 90 per cent of the subjects identified the help button and its purpose; however ... only 1 per cent of the subjects stated that they would use the help button."

North Carolina State University Libraries

Test method: user observation.

Test participants: 6 undergraduates in each of 3 trials.

See:
Find Articles Usability Tests, 2005.
This report includes videos, viewable by special arrangement. Thanks to Amanda French for posting this information.

"Many of the users are choosing E-journal Finder when they should be choosing Database Finder ... This indicates that neither page is clearly titled. The term database is often unfamiliar, so that users who are looking for a journal article tend to click on E-journal Finder rather than on Database Finder."

"The term journal, used to refer to periodicals in general, is not sufficiently explained nor exemplified in the text on the site. The term's use in the brand E-journal Finder obscures the fact that this collection of tools encompasses other kinds of periodicals, most especially newspapers and magazines. The term articles, which usefully applies to items in most kinds of periodicals, appears only in underemphasized, unhyperlinked descriptive text on the home page. Users don't see it, especially if they're in the catalog, where there is nothing to point them to articles."

"... the Find Articles term and structure is indeed successful."

User success rate went from 53% on the original site to 61% and 89% on two different redesigns. The report cited at left includes screenshots of all three. Terminology, graphic design, and information architecture may all contribute to the varying rates.

Norwich University Library

Test method: user observation.

Test participants: 7 undergraduates (freshmen and sophomores).

Data provided by Meredith Farkas; selected results are posted on her blog as "Website redesign at MPOW: What Im Learning," 2005.

Looking for journal articles on a specific subject, most participants did not first choose the correct link, Databases and Indexes. Unsuccessful choices included Periodicals, Library Catalog, Internet Resources, and Special Collections.

"No student knew what a Reference Desk was."

"Only one of them knew what [Interlibrary Loan] was called."

"Only one person knew that Circulation had anything to do with borrowing."

Student comment: "I couldn't find a few sites due to the fact that they were under different names, or I was never taught how to find them."

Internet Resources to find websites on a specific subject.

Student suggestion: "To make the site better, you can perhaps give examples of what each category deals with."

Pennsylvania State University Libraries

Test method: user observation in "focus groups"; participants were asked to complete tasks using the Penn State website and those of 20 other academic libraries.

Test participants: "Freshmen with limited experience in library use" (number unknown).

See:
Lesley Moyo and Ashley Robinson, "Library Jargon as a Factor in Information Design for Web Usability: Survey Report (Summary)," 16th Annual Computers in Libraries 2001 (Medford, NJ: Information Today, Inc., 2001), pp.157-165.

"professional terms"

"natural terms"

"Pages that worked best were those with explanatory notes below each library term used and those sites which popped up explanatory information on mouseover."

Pennsylvania State University Libraries
Usability study for website redesign (2004)
Test method: user observation.

Test participants: Not specified.

See:
University Libraries Web Redesign / Usable

The CAT
ILLIAD
database
E-resources
research guide
location
General subject terms, e.g., Life sciences

Specific subject terms, e.g., Biology

Recommendations on terminology:

"Short term - use patron-friendly terminology in place of library or vendor specific terminology when possible. When this isn't possible, provide a brief explanation of the terms in-place.

"Long term - determine a way to address the needs of the novice user and experienced user while providing learning opportunities for the novice user. This could include creating multiple sites or allowing patrons to set preferences."

Roger Williams University Library

Test method: user observation.

Test participants: 19 students in two rounds of testing.

See:
Susan McMullen, "Usability Testing in a Library Web Site Redesign Project", Reference Services Review 29 (2001), 7-22.
And:
Usability Testing and Library Web Site Redesign at Roger Williams University

“… users do not perceive the link, Online Databases and Indexes, as the resource choice to make [for] periodical articles.”

“Students do not understand basic library terms, such as database or index.”

”… one remarked that she thought of databases as spreadsheets.”

”Users can quickly identify the online library catalog.”

“Placement of links, color, and size does make a difference. Links should be easy to identify. It is easier to spot links when they are not embedded in text.”

”Built-in redundancy works.”

Texas A&M University Libraries

Test method: focus groups.

Test participants: 26 people, including undergraduates, graduate students, faculty, librarians, and university staff.

See:
Gwyneth H. Crowley et al., "User Perceptions of the Library's Web Pages: A Focus Group Study at Texas A&M University,"  Journal of Academic Librarianship 28 (July 2002), 205-210.

Find titles by keyword - participants thought this meant article titles rather than e-journal or database titles.

Information
Reserves
Reference
Database
Remote Access
Forms
Online Help
TN3270
VT100

Database subject categories:
Humanities
Social Sciences
Science & Engineering

Local or vendor terms:
PAM (Public Access Menu system for electronic resources)
Citrix

Participants suggested:
Using ALT text for mouseover explanations of graphical buttons.
FAQ page.
Glossary of terms.

University at Buffalo Libraries

Test method: user observation.

Test participants: 11 undergraduates

See:
Brenda Battleson, Austin Booth, and Jane Weintrop, ”Usability Testing of an Academic Library Web Site: A Case Study,” Journal of Academic Librarianship 27 (May 2001), 188-198.

Web Search ("students erroneously assumed [this] led only to Web search engines and the Internet, when, in fact, it included links to site-specific search features...").

Need Help (participants were not satisfied with help pages provided).

Online Resources  (most chose “Libraries Catalog” for non-catalog research tasks such as finding articles).

Databases by Title
Reference Resources
Quick Start

Libraries Catalog (most chose this correctly for normal catalog functions)

"Although not always appropriate, Databases by Subject was selected most often, while virtually every other link on the 'Online Resources' screen was ignored."

University at Buffalo Libraries

Test method: manual card sorting

Test participants: 9, including 2 graduate students and 6 undergraduates

See:
Website Nomenclature Test [RTF]
Posted by permission of the authors, Brenda Battleson and Jane Weintrop.

Catalog
Course Reserve
Reference
Electronic Resources

"... there was little or no consensus when it came to terminology related to library tasks. Subjects could find few terms that could easily be linked to tasks. They really had no idea what to 'call things.' ... Surprisingly, what librarians considered to be the most basic of terms ... were not identified by the subjects as such. They seemed to grasp the concepts, but were 'hung up' on the terminology."

Find
Search[ing]
Internet
Professor
Class[es]

University of Arizona Library, SABIO Information Gateway

Test methods: heuristic evaluation, design walk-through, card sorting, user observation.

Test participants: 8-12 students per round of tests.

See:
Ruth Dickstein and Vicki Mills, “Usability Testing at the University of Arizona Library: How to Let the Users in on the Design”, Information Technology and Libraries 19 (September 2000).

Catalog
Index
Resources
Databases
Reference

”We learned that if students have no idea why or when they should use an index, they will not choose a link labeled Index, no matter how well designed the Web page is.”







”… an Indexes page with twelve broad categories, such as social science, humanities, life sciences, etc. -- words supplied by librarians rather than students”.

(1) Graphical buttons
incorporating additional wording:
CATALOGS of Books & More/What We Own
Indexes to ARTICLES & More/Electronic Journals
Web SEARCH
Online REFERENCE/Ency/Dictionary
Multi-SEARCH
Research by SUBJECT

(2) “How to Find” pop-up menu.
Items include:
How to find MAGAZINES owned by the library
How to find MAGAZINE articles

(3) “Tips pages located at the point of need”

(4) “Research by Subject” page

(5) Subject menu using  “a scroll box that allowed as many specific subjects to be listed as needed. … The scroll box also enabled synonyms to be included for some subjects, such as both Health and Medicine.”

University of California, Berkeley Library

Survey of users of Pathfinder online catalog, November 2004

Test method: online survey.

Test participants: 254 responses, including 47 undergraduates, 119 graduate students, 22 faculty and 25 staff members.

Margin of error: +/- 6%

Survey form
Survey report

Title words   (28.3%)
Title  (6.3%)

------
Title phrase  (15.7%)
Title (exact)   (9.1%)

------
Text-only
Text-based
(though most did not think these meant the system contained full text).
Terminal-style

 Title keyword(s)  (62.2%)


------
Title begins with ...   (70.9%)


------
Command-line
Telnet


------
"Both Journal Title and Title of Journal searches were understood correctly (as covering journal titles) by almost all respondents. About one quarter of respondents and slightly over one third of undergraduates incorrectly assumed these searches would cover article titles, and a smaller number thought they would include full text. Differences between the two alternative names were within the margin of error for this question."

University of California, Berkeley Library

Usability test of library home page, April 2004. [MS Word]

Test method: user observation.

Test participants: 4 undergraduates.

"No participant successfully found journal articles. ... Participants were confused by the distinction between journals and articles, and they used the terms interchangeably. As in previous usability testing, some participants were drawn to the Journals (by title) link when searching for journal articles."

"Participants were unclear on the differences between GLADIS and Pathfinder [and] between Pathfinder and Melvyl."

"Participants were confused by the distinction between GLADIS and ERes [the library's electronic reserves system]."

"Participants were confused by the term telnet."

"Half of the participants understood the Search function, while the other half thought it would retrieve library resources (books, articles and more)."

"Participants seemed uncertain about the More… link, describing it as vague, unimportant and easy to miss."

"All four participants ... seemed to understand that they would find [online dictionaries, encyclopedias, atlases, statistics, etc.] through the Electronic resources link under the Find Information tab."

Participants gravitated toward Services when looking for course reserves, library workshops, and privileges for undergraduates.

Student comment: "I would try to have something that said Employment Opportunities." Each participant, though, mentioned a different term; the others said they were looking for Join Our Team, Apply Here, and Work For Us.

University of California, Berkeley Library

Link Choice Test on Prototype Home Page, November 2003

Test method: paper questionnaire.

Test participants: 18 responses, including
9 undergraduates, 3 graduate students, 4 staff members, 2 visiting scholars.

Indexes and Abstracts  (2 responses)



------
Research Tools (2 responses)
Reference Sources (2 responses)

Article Databases  (12 responses)
4 participants suggested terms including "Journal" (note that the question specifically referred to "journal articles").

------
Electronic Resources  (14 responses)
A follow-up test, with 15 participants, showed Electronic Resources getting 6 responses and Reference Tools getting 5. The results in the first round may have been skewed by the test design.

University of California, Berkeley Library

Unpublished usability study of prototype Electronic Resources Database interface, March 2002.

Test method: user observation.

Test participants: 3 undergraduates; 2 graduate students; 1 reference librarian.

Article Indexes – some student participants didn’t understand this term.  One commented: “Article index doesn’t mean ‘journals’ to me. Professors say ‘journal articles’”.

Library Catalogs
Gateway site

Doe/Moffitt Libraries, University of California, Berkeley

Usability studies of draft Doe-Moffitt Libraries web page:
Round 1: August 2002
Round 2: October 2002
Round 3: November 2002

Test method: user observation.

Test participants: 5 undergraduates; 3 graduate students; 1 visiting faculty member.

Journals (by Title) - most thought this “would lead [directly] to scholarly journal articles”.

Articles – most thought this “would lead … to a more global search of articles … in magazines, journals, newspapers and popular level materials”.

Library Catalogs
Instruction
Liaison
Telnet
Proxy
Reference
Text only
Contacts by subject
Units

Local or vendor terms:
Pathfinder (library catalog system)
CDL (California Digital Library)
WorldCat

Find Books, Articles, Etc. - participants chose this over Library catalogs.

Library Services – most gravitated to this term when given tasks requiring them to find:
Loan periods, Renewing books online, Copiers, Tours, Reserves, Laptop, Word processing, Connecting from off campus, Ask a reference question online, Contacts by subject.

University of California, San Diego Libraries

Test methods: link choice test using paper form.

Test participants: 10 staff, 10 undergraduates and 8 graduate students

See:
University of California, San Diego Libraries. About This Site: Usability Testing.

This test generated an unusually rich body of data on what terms were most successful as link names. See the report linked above for "terminology test" results. The examples at right show the "aggregate preference" as a percentage of respondents choosing each link option.

[To find a book]
Library Catalog   (preferred by 18.4%)


[To find an article]
CDL/MELVYL   (preferred by 7.8%)
Browse by Subject/Type of Material   (7.8%)

[To find a book]
Find Books and more...   (preferred by 34.7%)
ROGER (Library catalog)   (30.6%)

[To find an article]
Find Articles and more...   (preferred by 33.3%)
Choosing Article Databases   (27.5%)

University of Illinois at Chicago Library

Test methods: user observation.

Test participants: 12 students.

See:
Susan Augustine and Courtney Greene, “Discovering How Students Search a Library Web Site: a Usability Case Study,” College & Research Libraries 63 (2002), 354-65.

Article indexes
Pathfinders
Workshops

“... all participants but one used the internal Web site search engine to complete tasks rather than navigating through the pages by following links. ... These search habits ... indicate that more attention should be paid to metadata and a strong internal search engine...”

University of Rochester, River Campus Libraries

Test method: not specified.

Test participants: not specified.

See:
Jennifer Bowen et al., "Serial Failure," The Charleston Advisor 5 (2004)

Databases

Find Articles

"On the new home page, the word Databases was replaced with Find Articles, which resulted in a dramatically reduced failure rate in getting to the list of databases. ... From a serial failure rate of nearly 100 percent in 1997, we now can claim some success in actual task completion."

University of Rochester, River Campus Libraries

Test method: not specified.

Test participants: not specified.

See:
Brenda Reeb and Susan Gibbons, "Students, Librarians, and Subject Guides: Improving a Poor Rate of Return," portal: Libraries and the Academy 4 (2004), 123-130.

Subject discipline names

"At the University of Rochester, librarians repeatedly observed in usability testing that undergraduates lack an understanding of an academic discipline. ... Some never grasp the concept of a 'discipline.' Others may gain an understanding in their majors, but do not transfer this comprehension to other academic domains. The concept of disciplines is not usually part of a student's mental model; therefore, the collocation of resources by discipline is not recognized."

Course-specific guides are suggested as a solution.

University of Saskatchewan, Health Sciences Library

Test methods: "preference test" using paper questionnaire (20 participants); user observation (5 participants).

See:
Vicky Duncan and Darlene Fichter, "What words and where? Applying usability testing techniques to name a new live reference service," JMLA: Journal of the Medical Library Assocation 92 (2004), 218–225.

Invitations to chat:

   Don't give up!
   Ask us and we'll help!

Invitations to click:

   LiveHelp

   Ask me!

   Questions Online

   Answers Online

Combined phrases:

   Questions Online
   Click Here

   Answers Online
   Click Here

Other terms found to be problematic in usability testing:

   Database
   Interlibrary Loan
   E-journals

Invitations to chat:

   Need help finding information?
   Chat online with a librarian

   Got a question?
   Chat with a librarian NOW!

Invitations to click:

   Ask a Librarian

   Click Here

Combined phrases:

   LiveHelp
   Ask a Librarian

Term adopted for production site:

   Ask a Librarian

University of South Florida Virtual Library

Test methods: user task performance and survey with automated data collection.

Test participants: 32 undergraduates.

See:
Maryellen Allen, “A Case Study of the Usability Testing of the University of South Florida’s Virtual Library Design,” Online Information Review 26 (2002), 40-53.

E-Journals
Databases
100% of participants chose E-Journals instead of Databases in order to “research journal or magazine articles”.

"... few users were familiar with the term 'interlibrary loan' request, even though they were familiar with the service itself."

Local terms:
WebLUIS (library catalog)

Online renewal (92.3% chose this correctly).

How do I find an article? (76.9% chose this correctly in the second round of testing. This link combined E-Journals and Databases, and led to an “intermediate page where the option for choosing direct subscriptions to e-journals and those articles found in databases were offered and explained”).

“… the simple, straightforward approach as seen in the ‘how do I’ questions invariably produced more successful participant behaviours than those using jargon.”

University of Southern California, Norris Medical Library

Test method: user observation.

Test #1 participants: 5 medical school faculty, 1 student, 1 staff member (medical library)
Teat #2 participants: 2 faculty, 4 staff members.

See:
Candice Benjes and Janis F. Brown, "Test, Revise, Retest: Usability Testing and Library Web Sites," Internet Reference Services Quarterly 5 (2001), 37-54.

"For the redesign, the text was rewritten in a more colloquial style, replacing "library speak" such as document delivery with the more commonly known phrase interlibrary loan."

After a "successful publicity campaign ... students, staff, and faculty began asking for Ovid - not MEDLINE, CINAHL, [etc.]. In our setting, Ovid is shorthand for 'I want to find articles' ... "

University of Tennessee, Knoxville Libraries

Test method: user observation.

Test participants: 13 graduate students and faculty, most having experience with both the library web site and research in their subject areas.

See:
Thura Mack et al., "Designing for Experts: How Scholars Approach an Academic Library Web Site," Information Technology and Libraries (2004), 16-22.

Databases
"Five participants went to Databases on their first click, and three more eventually discovered the correct path to an article, for a success rate of 62 percent."

Subject Guides [as a way to find a word definition]
"Only one of the participants went first to Subject Guides, and seven went to Internet Search Engines."

Kudzu [local brand name for OPAC]

AskUs.Now [local brand name for electronic reference service]

"The authors discovered that most participants were not confused by terms used on the library Web pages, with a few notable exceptions."

University of Virginia Libraries

Article Finder/OpenURL Resolver Usability Test
September-October, 2004

Test method: user observation

Test participants: 3 undergraduates; 2 Library staff; 2 faculty

Data provided by Leslie Johnston, 2/3/05

86% did not initially use the Find at UVa button in Web of Science to invoke the Resolver. In a following question, 71% successfully used it.

"The label Find@UVa on our Resolver button that appears in vendor databases fails to communicate anything meaningful to the uninitiated, or may even be misleading. Alternatives were suggested, however each had problems of its own. Given the constraints on the size of the button, and the difficulty of clearly explaining its function in 10 characters or fewer, this is a problem that cannot be effectively addressed by button design or labeling; if users don't know what a button does, they still won't click it."

The eventual success of most testers suggests that once a user perceives and understands the function of the "Find@UVa" button, s/he will use it, regardless of the labeling.

University of Virginia Libraries

VIRGO OPAC Usability Test
May-October, 2002

Test method: user observation

Test participants: 3 faculty, 2 graduate students, 4 undergraduates, 4 library staff

Data provided by Leslie Johnston, 2/3/05

Serials
Periodicals

------
Ask VIRGO (inquire about services or ask for help)

------
31% were unsure what about the difference between Ejournals and Journal and Newspaper Articles as navigation options.

------
"There was much confusion and a good deal of incomprehension about the difference between a journal and an index to a journal."

Navigation now consistently uses Journals (although the others terms do additionally appear in some page content).

------
Contact Us instead of Ask VIRGO

------
Navigation now reads:
Ejournal lists
Journal articles
Newspaper articles

University of Washington Libraries

Usability Study of the Subject Pages (2004)

Test method: user observation

Test participants: 8 undergraduates, 2 graduate students, 1 faculty

Core Resources to designate recommended databases.

------
By Subject

"Users are more likely to use Best Bets than Core Resources".

------
"Users slightly preferred Browse Subjects to By Subject"

University of Washington Libraries

Card Sorting Usability Study (2001)

Test method: card sorting

Test participants: 1 undergraduate, 1 graduate student, 1 faculty member, 2 staff members

Starting Points (users didn't know what to expect)

Connecting (unclear that this referred to a proxy server)

Cascade (brand name of a service).

Users suggested FAQ's instead of Starting Points

University of Washington Libraries
Willow (text-based interface, no longer in use)

Test method: link choice questionnaire

Test participants: 145 undergraduates

See:
Karen Eliasen et al., "Navigating Online Menus: A Quantitative Experiment", College & Research Libraries (1997), 509-517.

UW Libraries Catalog

------
Databases ... Arts & Humanities
[and other similar subject categories]

Books, Periodicals, & Other Resources at the UW Libraries

------
Arts & Humanities: Indexes of Articles & Other Sources
[etc.]

Washington State University Libraries

Test methods: user observation.

OPAC test participants: 3 undergraduates, 3 graduate students, 1 "other".
Web site test participants: 6 students.

See:
Janet Chisman, Karen Diller, and Sharon Walbridge, "Usability Testing: A Case Study," College & Research Libraries 60 (November 1999), 552-69.

WebPac

"MARC format terminology in drop-down lists", e.g., Proj medium

Other Library Catalogs [many thought this could be any library on campus other than the one they normally use]

Dates of coverage [of article databases]

4 of 6 participants correctly chose Article Indexes, Full Text, and More to "find information ... in periodicals (also known as serials or journals)"

Western Michigan University Libraries

Test method: user observation

Test participants: 29 undergraduates, 10 graduate students, 10 faculty.

See:
Barbara J. Cockrell and Elaine Anderson Jayne, "How Do I Find an Article?  Insights from a Web Usability Study,"  Journal of Academic Librarianship 28 (May 2002), 122-132.
This article contains an excellent discussion of terminology issues.

Electronic Journals vs.Databases and Indexes

"At least one participant, who chose Electronic Journals rather than a database when looking for a journal article ... indicated he made this choice because the associated annotation included both of the words 'journals' and 'articles'."

"Several participants said they did not understand the term database [which] was coupled with indexes ... because some of these databases were not only indexes. Unfortunately, this may be a case where an attempt to give precise and complete information actually confuses users trying to distinguish among options."

Serials vs.Periodicals vs.Magazines vs. Journals

”Almost half the participants thought the OPAC was the source to use for magazine and journal articles.”

”Users were not very discriminating in their choice of indexes. … [they often] selected the first item on any given list ..”

Changes made as a result of the study:

Placed links on the home page leading to “mini-tutorials” for:
Find an Article
Find a Book

Placed a “Find Articles” link in the online catalog, leading to a page providing more guidance and links to databases.

”… links were made more descriptive and readily distinguishable, specialized terminology was simplified as much as possible, and a glossary of library terms … is being developed.”

Western Wyoming Community College Library

Test method: user observation

Unpublished study, 2003
(per Usability4Lib listserv posting, "how do we measure success?" by Robert Kalabus, 1/27/2005, quoted by permission)

Interlibrary Loan
Periodical
New Acquisitions
Database
Webliography
Search Engine

 
Other Studies

Joseph Barker, "'Now Which Buttons Do I Press to Make These Articles Appear on the Screen?", Serials Review 25 (November 1999), 49-54.

“Most undergraduates do not come to Cal knowing what a journal article is, what a journal is, what an index is, or what is scholarly and reputable.”

”Only about 25 percent understand journal index citations and the difference between a journal title and an article title.”

“If we seriously expect undergraduates to find e-journals, we must unify bibliographic control and make links to them stand out like giant billboards along the research paths undergraduates routinely travel.”

Abdus Sattar Chaudhry and Meng Choo, "Understanding of Library Jargon in the Information Seeking Process," Journal of Information Science 27 (2001), 343-349.

Test method: Questionnaire given to 40 clients of the National Reference Library of Singapore and "acquaintances of the staff" of the Library Support Services, National Library Board of Singapore, to test their understanding of "technical terms commonly used by librarians during reference interviews."

Overall correct answer rate: 76.9%

Note: Division into "what didn't work" and "what did work" is my own, based on an arbitary cutoff point of 66%.

Resource file   (45% correct)
Holdings   (47.5%)
Citation   (55%)
ILL (interlibrary loan)   (55%)

Book drop   (97.5% correct)
OPAC (Online Public Access Catalogue)   (95%)
Self-check terminal   (95%)
User education   (90%)
Call number   (85%)
Keyword   (85%)
Title/subject search   (85%)
Free text   (85%)
Bibliographical list/bibliography   (85%)
Author catalogue   (85%)
Full text   (75%)
Bibliographic/catalogue record   (75%)
Search term   (72.5%)
DDC (Dewey Decimal Classification   (72.5%)
Document delivery service   (70%)
Microfilm   (67.5%)

Norman B. Hutcherson, "Library Jargon: Student Recognition of Terms and Concepts Commonly Used by Librarians in the Classroom," College & Research Libraries 65 (2004), 349-354.

Test method: Questionnaire given to 297 first- and second-year undergraduates at California State University, Bakersfield who had completed a "seven-week library skills lab," to test their understanding of "terms derived from library literature, reference desk experience, and classroom observation."

Overall correct answer rate: 62.31%

Note: Division into "what didn't work" and "what did work" is my own, based on an arbitary cutoff point of 66%.

Boolean logic   (8.1% correct)
Bibliography   (14.9%)
Controlled vocabulary   (18.1%)
Truncation   (27.7%)
Precision   (31.8%)
Information need   (34.9%)
Descriptors   (35.8%)
Abstract   (36.2%)
Article   (47%)
Citation   (51.7%)
Bibliographic information   (54.73%)
Authority   (57.7%)
Collection   (59.7%)
Catalog   (61.62%)
Search statement   (63.8%)

Plagiarism   (100% correct)
Reference services   (94.6%)
Research   (94%)
Copyright   (91.58%)
Table of contents   (90.5%)
Synonym   (89.9%)
Audiovisual materials   (89.2%)
Editor   (86.5%)
Call number   (81.48%)
Reference books   (75%)
Journal   (74.5%)
Edition   (72.3%)
Fair use   (67.6%)

Rachael Naismith and Joan Stein, "Library Jargon: Student Comprehension of Technical Language Used by Librarians," College & Research Libraries 50 (Sept. 1989), 545.

Test method: Questionnaire given to 100 freshman English students at Carnegie Mellon University, to test their understanding of terms "derived from actual reference interviews and library handouts."

Overall correct answer rate: 51.3%

Note: Division into "what didn't work" and "what did work" is my own, based on an arbitary cutoff point of 66%.

Multi-volume set   (11% correct)
Proceedings   (20%)
Command search   (25%)
Citation   (35%)
Clearinghouse   (38%)
Nonprint materials   (40%)
University archives   (41%)
Viewing carrel area   (45%)
Primary source   (45%)
Library rep   (47%)
Pre-search   (52%)
Online database searches   (53%)
Search statement   (53%)

Call number   (83% correct)
Bound journals   (82%)
Interlibrary loan   (75%)
Microform   (74%)
Search terms   (71%)
Catalog screen   (68%)
Online catalog   (68%)

Mark A. Spivey, "The Vocabulary of Library Home Pages: An Influence on Diverse and Remote End-Users," Information Technology and Libraries 19 (September 2000), 152-156.

Test methods:
Evaluations of terms appear to result from the author’s experience in public service work.
Article also reports a year-long survey of terminology used on 60 library home pages.

Reference
Reserves
Periodicals
Periodical indexes
Indexes
Special collections
Citation

Library acronyms:
BI, ILL, OPAC etc.

Vendor acronyms:
CINAHL, CARL, JSTOR etc.

Author advises using:

(1) “Embedded explanations [such as] appositives, prepositional phrases, parenthetical examples or descriptions, and categorical headers.”
Examples:

BI (library instruction)
ILL (interlibrary loan of books not held [here]
Reference for information assistance
Reserves for checking out class material
Special Collections for local material
Archives (university records)
WildCat, Lagoon University’s catalog

(2) “A glossary of library idioms and examples of local use, available as links from the home page.”