Metalib Usability Test at the Medical Library
June 2005
Result Summary
6 Subjects
5 nursing graduate students, 1 graduate student in Cellular Biology
Library use:
never 0
once per month 0
one a week 1
2-3 per week 4
4 + per week 1
4 Endnote users, 2 RefWorks
1. a and b [Starting at generic page] If you were looking
for a book about the law and medical ethics, please show me from this page
where you would first go.
6 out of 6 went to QuickStart
Search type: 3 keyword, 2 subject, 1 title search
Choice of QuickStart set: 4 local catalog, 2 books anywhere
all felt search was successful
1. c. General discussion of the display.
Not one subject noted that there were more items not yet displayed. Several wanted to limit by English. 3 wanted to limit more (look for limit to English language).
1.d Pick any one
of the books and show me how you find would find out if the book is available
to be checked out.
4 found the Orbis link under holdings, 2 went straight to SFX first. In all cases, successful at determining that the book was available. Finding the Orbis link took a little searching.
6 out of 6 were successful
2. Please show me how you would go about finding some
general background information in articles about living wills.
4 chose Quick Search, 2 went to MultiDatabase Search.
Of the 4 Quick, 3 chose Articles, and 1 chose Local Catalogs (failed to notice that they needed to change the default selection).
Of the 4 Quick, 3 chose subject search and 1 chose keyword.
2 chose Multi DB search
Of the 2 who chose MultiDB search, 1 failed to complete the task, 1 found the quick sets and ran the search in articles, but used subject searching.
Problem: too many people choose subject searching.
Once in Multidatabase searching, people are very confused about how to proceed.
2b. Please pick any article that looks interesting and
show me how you would find
more about that article
full text or
print copy of the journal
6 out of 6 were able to do this, but they followed different routes, and were influenced by what they already knew.
1 chose to go to the native interface (recognized Academic Search Premier)
1 went directly to SFX
4 went to the Detailed Display by clicking on title, then
2 found SFX
2 found the native display and then either SFX or link to full text there
3. We're looking at this tool to replace the list of databases
I showed you earlier. Can you show me
how you would use this tool to find a database you know, say Pubmed, and
connect directly to it to do a search?
6 out of 6 went to Find a database, clicked on P and scrolled to Pubmed.
There was no problem seen in completing this task.
4. Now I'm going to ask you to use this tool to find a set of databases you might use to search about a subject where you'd like to get some background information from the sociological literature.
4a. Show me how you'd identify databases from sociology.
This was not simple for any of the 6 subjects. All struggled to one degree or another.
All went to multidatabase first.
3 tried to search for sociology.
Once there, 4 of the 6 succeeded in finding Categories/Social Sciences/Sociology.
4b. Select a set to search across, and let's look for information again about living wills. After being shown how to find a set of databases, 4 of 6 selected some of the databases and did a search. 2 failed to realize that they needed to click on a box to select a database.
3 did a subject search and 3 did a keyword search.
Several people voiced puzzlement over what some of these databases were, but none thought to click on the I (information button).
3 people were then shown a result set of 3 items: none noticed without prompting that there were more results available in other databases.
5. Now let's use the service to search across databases
you might already be familiar with.
Let's say you want to search Pubmed and Web of Science for living
wills.
5a. Please locate Pubmed and Web of Science, and find a
way to search across both databases at one time.
Only one person was able to do this (chose category for Health/Clinical Medicine)
People went back and forth between Find and Multidatabase search.
2 searched for sociology and then realized that this wasn't going to work.
4 successfully found one database but then couldn't figure out how to save it with the other database.
This was very frustrating for people and usually we would stop the question after a few minutes.
General Debriefing Quiestions
All said they would use the search again for very specific situations: usually when they needed to research a topic outside of their field of expertise.
Pointed out that it was easy to use and get search results. Liked that it could save time having to search in multiple databases.
Liked the ability to find a list of databases in a subject area they didn't know.
Worst features:
Multidatabase is confusing and hard to use.
Not enough information on the entry page.
No limits available.
Improve
Help buttons with concise information about what to do at point of need (although two specifically also said they don't often read explanatory text on screen).
More information on entry page.
2 people mentioned that "Add to My Research" should be check boxes.