Evaluating Digital Collections

After this week’s course readings by the DELOS Working Group, Ying Zhang, Tefko Saracevic, and the NISO Framework, I am excited to learn more about Nielsen’s heuristic evaluation methods and start evaluating a few collections.

After examining resources and background information for this week’s Information Package, the idea of defining digital library criteria in a way that many researchers can agree on and, more importantly, specifying how those criteria will be measured or assessed, seems to be the biggest challenge.

The Information Retrieval aspect of digital libraries is relatively well established, but the User-Centered approach is filled with criteria that, until now, have been seen as largely subjective.

Some of the criteria discussed in Zhang’s article, “Developing a holistic model for digital library evaluation,” such as “ease of use,” “courtesy,” and “attractiveness” could vary enormously among individuals.

How can researchers ensure, when discussing these concepts with users, librarians, or designers, that everyone is talking about the same thing?

Perhaps detailed surveys with numerical rating systems (such as a 1-5 scale) and a definition of each criteria before each question would be one way to begin to define these criteria more narrowly.