Talk to those users


Tefko Saracevic pointed out that “users have many difficulties with digital libraries.” As a librarian or a librarian to be, how would you address Saracevic’s concerns?

– they usually do not fully understand them

– they hold different conception of a digital library from operators or designers

– they lack familiarity with the range of capabilities, content and interactions provided by a digital library.

– they often engage in blind alley interactions.


Saracevic brought up these concerns specifically as examples of the versus hypothesis. This hypothesis states that digital libraries and digital library users are on opposite sides of a fence with their assumptions about the other side going unchecked. So any needs from one side are not being addressed by the other simply because the “other” is unknown.

The chapters in Evaluating Digital Libraries: A User-Friendly Guide mentions user focus groups as a survey method used for answering questions about user perspectives and recommended enhancements, it might also help to understand the public preconceptions about the library. It also mentions usability inspections (using real world terms rather than jargon, making actions visible, ensuring that there is help text and basic documentation about the site) which would help to bridge the gap between the two sides. One of the things that I’ve learned from using databases and other online portals to information is that having an introductory paragraph about the purpose and contents of the site and help text in plain English that is specific to page the user is on really helps to set expectations and to assist the user in using the site.

The importance of making sure to evaluate your work as you go is outlined well by Morse, and making sure to involve users and evaluate as you build can help to mitigate the alienation of your audience. Once the library is live; the librarians could monitor usage, and to ensure that you’re getting the truth rather than the reported use, you could track a user through their interactions with the site. The users chosen would have to be anonymous and picked at random, but it would help the library to determine what users are coming to the site to find and whether or not they’re finding that item. It would also help to determine true user workflows for testing after enhancements or upgrades.


Three popular methods for usability evaluation are focus group, ethnography, and survey. Please describe your experience in any of these, if any. Otherwise, please describe their pros and cons of each method within the context of digital library evaluation.

I have a lot of experience with focus groups  when it comes to IT implementations.  Focus groups are excellent, but they’re difficult to control. People with strong emotions about a topic will overpower the group and potentially shut out people with dissenting opinions. I’ve found that it’s best to get individual opinions through surveys so that you can address specific concerns or opinions with the group. It is also really easy to get sidetracked with a focus group, so having an agenda and specific questions for the group is very important, you can’t just allow it to be a free-form conversation.


7 responses to “Talk to those users

  1. just curious… has your experience with surveys been good then? do enough people respond to give you useful information when you put them out there? i’ve always wondered about this, because I definitely don’t respond to every survey I’m prompted for (libraries as well as other sites).

    • My experience with surveys is mixed. For this post, I really only touched on surveys as a precursor to focus groups. If a group knows that it will be part of a discussion, and you let them know it’s a required task prior to the focus group, you’ll get decent information. For surveys that you’re just pushing to a set of users and hoping for a response, there are a couple of ways to make sure you get useful information. I’ve found that if you ask a question in a few different ways (open end and a scale of 1-10) your answer set is more useful. In terms of making people do the survey? That’s an age-old question, usually answered with bribes, threats or guilt. You can give people something for answering, let them know that it’s required for their job or directly email them until they give in. None of these are perfect, and you’ll mostly get answers from the people who just want to complain, but you’ll get some answers. The key is to cast a wide enough net that having a small percent of the survey returned will be enough.

  2. It also might be easier for a library to implement surveys than focus groups because the response rate for in person focus groups and interviews is so low. In my LIS Research Methods class I ran across a number of studies in the LIS that had trouble attracting more than five to fifteen subjects for in person research. But I have similar concerns to spd because I think we are almost reaching a point of survey over-saturation.

    • I think it’s different outside of an academic setting. I work for hospitals and we bring people in to talk through almost every step of an implementation. In my case, I turn people away from some meetings because everyone has a stake in the outcome and they all really want to be involved in the process.

  3. For seeking info from people with strong emotions, I heard in-depth interview is useful. But the interviewer may need to have higher skills.

  4. Bribes! Bribes work almost every time. For example, you wouldn’t believe what undergrads will do for a slice of pizza or a doughnut! From the private sector side, most often a coupon for some service or product works well. Don’t overlook the ability of your staff to tease feedback out of even the most stubborn patrons in an informal way.

  5. I am still a noob here, but I can’t help thinking that surveys and user groups would be really useful if nobody had ever made a web page before… but, like, aren’t the principles sort of well established at this point? And to a certain extent, isn’t that all a self-fullsilling prophesy? I mean, do you ever get done with a focus group and find, wow, a really innovative idea that i’ve literally never seen on another website, or do people tend to tell you that websites should ideally be just like they already are? I guess I see a lot more value in trying to interpret the server logs, for all the problems there, because for one, you are already generating that information, and two, you have it on hand when you need it, you don’t have to convene a bunch of people and get someone to write effective questions.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s