Next-Generation Library Catalogs and the Problem of Slow Response Time

Response time as defined for this study is the time that it takes for all files that constitute a single webpage to travel across the Internet from a Web server to the end user’s browser. In this study, the authors tested response times on queries for identical items in five different library catalogs, one of them a next-generation (NextGen) catalog. The authors also discuss acceptable response time and how it may affect the discovery process. They suggest that librarians and vendors should develop standards for acceptable response time and use it in the product selection and development processes.

Response time as defined for this study is the time that it takes for all files that constitute a single webpage to travel across the Internet from a Web server to the end user's browser.In this study, the authors tested response times on queries for identical items in five different library catalogs, one of them a next-generation (NextGen) catalog.The authors also discuss acceptable response time and how it may affect the discovery process.They suggest that librarians and vendors should develop standards for acceptable response time and use it in the product selection and development processes.

N
ext-generation, or NextGen, library catalogs offer advanced features and functionality that facilitate library research and enable Web 2.0 features such as tagging and the ability for end users to create lists and add book reviews.In addition, individual catalog records now typically contain much more data than they did in earlier generations of online catalogs.This additional data can include the previously mentioned tags, lists, and reviews, but a bibliographic record may also contain cover images, multiple icons and graphics, tables of contents, holdings data, links to similar items, and much more.This additional data is designed to assist catalog users in the selection, evaluation, and access of library materials.However, all of the additional data and features have the disadvantage of increasing the time it takes for the information to flow across the Internet and reach the end user.Moreover, the code that handles all this data is much more complex than the coding used in earlier, traditional library catalogs.Slow response time has the potential to discourage both library patrons from using the catalog and library staff from using or recommending it.During a reference interview or library instruction session, a slow response time creates an awkward lull in the process, a delay that decreases confidence in the mind of library users, especially novices who are accustomed to the speed of an open Internet search.
The two-fold purpose of this study is to define the concept of response time as it relates to both traditional and NextGen library catalogs and to measure some typical response times in a selection of library catalogs.Libraries Mathews posted an article called "5 Next Gen Library Catalogs and 5 Students: Their Initial Impressions." 7Here he shares student impressions of several NextGen catalogs.Regarding slow response time Mathews notes, "Lots of comments on slowness.One student said it took more than ten seconds to provide results.Some other comments were: 'that's unacceptable' and 'slow-motion search, typical library.'"Nagy and Garrison, on Lauren's Library Blog, emphasized that any "cross-silo federated search" is "as slow as the slower silos." 8Any search interface is as slow as the slowest database from which it pulls information; however, that does not make users more likely to wait for search results.In fact, many users will not even know-or care-what is happening behind the scenes in a NextGen catalog.
The assertion that slow response time makes wellintentioned improvements to an interface irrelevant is supported by an article that analyzes the development of Semantic Web browsers.Frachtenberg notes that users, however, have grown to expect Web search engines to provide near-instantaneous results, and a slow search engine could be deemed unusable even if it provides highly relevant results.It is therefore imperative for any search engine to meet its users' interactivity expectations, or risk losing them. 9This is not just a library issue.Users expect a fast response to all Web queries, and we can learn from studies on general Web response time and how it affects the user experience.Huang and Fong-Ling help explain different user standards when using websites.Their research suggests that "hygiene factors" such as "navigation, information display, ease of learning and response time" are more important to people using "utilitarian" sites to accomplish tasks rather than "hedonistic" sites. 10In other words, response time importance increases when the user is trying to perform a tasksuch as research-and possibly even more for a task that may be time sensitive-such as trying to complete an assignment for class.

■ ■ Method
For testing response time in an assortment of library catalogs, we used the WebSitePulse service (http://www .websitepulse.com).WebSitePulse provides in-depth website and server diagnostic services that are intended to save e-business customers time and money by reporting errors and Web server and website performance issues to clients.A thirty-day free trial is available for potential customers to review the full array of their services; however, the free Web Page Test, available at http://www.website server and arrive sequentially at the computer where the request was initiated.
While the World Wide Web Consortium (W3C) does not set forth any particular guidelines regarding response time, go-to usability expert Jakob Nielsen states that "0.1 second is about the limit for having the user feel that the system is reacting instantaneously." 1 He further posits that 1.0 second is "about the limit for the user's flow of thought to stay uninterrupted, even though the user will notice the delay." 2 Finally, he asserts that: 10 seconds is about the limit for keeping the user's attention focused on the dialogue.For longer delays, users will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done.Feedback during the delay is especially important if the response time is likely to be highly variable, since users will then not know what to expect. 3en though this advice dates to 1994, Nielsen noted even then that it had "been about the same for many years." 4

■ ■ Previous Studies
The chief benefit of studying response time is to establish it as a criterion for evaluating online products that libraries license and purchase, including NextGen online catalogs.Establishing response-time benchmarks will aid in the evaluation of these products and will help libraries convey the message to product vendors that fast response time is a valuable product feature.Long response times will indicate that a product is deficient and suffers from poor usability.It is important to note, however, that sometimes library technology environments can be at fault in lengthening response time as well; in "Playing Tag In the Dark: Diagnosing Slowness In Library Response Time," Brown-Sica diagnosed delays in response time by testing such variables as vendor and proxy issues, hardware, bandwidth, and network traffic. 5In that case, inadequate server specifications and settings were at fault.
While there are many articles on NextGen catalogs, few of them discuss the issue of response time in relation to their success.Search slowness has been reported in library literature about NextGen catalogs' metasearch cousins, federated search products.In a 2006 review of federated search tools MetaLib and WebFeat, Chen noted that "a federated search could be dozens of times slower than Google." 6More comments about the negative effects of slow response time in NextGen catalogs can be found in popular library technology blogs.On his blog, ■ ■ Findings: Skyline Versus

WorldCat@Auraria
In figure 2, the bar graph shows a sample load time for the permalink to the bibliographic record for the title Hard Lessons: The Iraq Reconstruction Experience in Skyline, Auraria's traditional catalog load time for the page is pulse.com/corporate/alltools.php, met our needs.To use the webpage test, simply select "Web Page Test" from the dropdown menu, input a URL-in the case of the testing done for this study, the permalink for one of three books (see, for example, figure 1)-enter the validation code, and click "Test It." WebSitePulse returns a bar graph (figure 2) and a table (figure 3) of the file activity from the server sending the composite files to the end user's Web browser.Each line represents one of the files that make up the rendered webpage.They load sequentially, and the bar graph shows both the time it took for each file to load and the order in which the files were received.Longer segments of the bar graph provide visual indication of where a slow-loading webpage might encounter sticking points-for example, waiting for a large image file or third-party content to load.Accompanying the bar graph is a table describing the file transmissions in more detail, including DNS, connection, file redirects (if applicable), first and last bytes, file transmission times, and file sizes.results for Skyline (traditional) catalog record requested at items 8, 14, 15, 17, 26, and 27.The third parties include Yahoo!API services, the Google API service, ReCAPTCHA, and AddThis.ReCAPTCHA is used to provide security within WorldCat Local with optical character recognition images ("captchas"), and the AddThis API is used to provide bookmarking functionality.At number 22, a connection is made to the Auraria Library Web server to retrieve a logo image hosted on the Web server.At number 28, the cover photo for Hard Lessons is retrieved from an OCLC server.The files listed in figure 6 details the complex process of Web browsers' assembly of them.Each connection to third-party content, while all relatively short, allows for additional features and functionality, but lengthens overall response.As figure 6 shows, the response time is slightly more than 10 seconds, which, according to Nielsen, "is about the limit for keeping the user's attention focused on the dialogue." 12While widgets, third-party content, and other Web 2.0 tools add desirable content and functionality to the Library's catalog, they also do slow response time considerably.The total file size for the bibliographic record in WorldCat@Auraria-compared to Skyline's 84.64 KB-is 633.09KB.As will be shown in the test results below for the catalog and NextGen catalog products, bells and whistles added to traditional 1.1429 seconds total.The record is composed of a total of fourteen items, including image files (GIFs), cascading style sheet (CSS) files, and JavaScript (JS) files.As the graph is read downward, the longer segments of the bars reveal the sticking points.In the case of Skyline, the nine image files, two CSS files, and one JS file loaded quickly; the only cause for concern is the red line at item four.This revealed that we were not taking advantage of the option to add a favicon to our III catalog.The Web librarian provided the ILS server technician with the same favicon image used for the Library's website, correcting this issue.The Skyline catalog, judging by this data, falls into Nielsen's second range of user expectations regarding response time, which is more than one second, or "about the limit for the user's flow of thought to stay uninterrupted, even though the user will notice the delay." 11Further detail is provided in figure 3; this table lists each of the webpage's component files, and various times associated with the delivery of each file.The column on the right lists the size in kilobytes of each file.The total size of the combined files is 84.64 KB.
In contrast to Skyline's meager 14 files, WorldCat Local requires 31 items to assemble the webpage (figure 4) for the same bibliographic record.Figures 5 and 6 show that this includes 10 CSS files, 10 JavaScript files, and 8 images files (GIFs and PNGs).No item in particular slows down the overall process very much; the longestloading item is number 13, which is a wait for third-party content, a connection to Yahoo!'s User Interface (YUI) API service.Additional third-party content is being total time for each permalinked bibliographic record to load as reported by the WebSitePulse tests; this number appears near the lower right-hand corner of the tables in figures 3, 6, 9, 12, and 15.We selected three books that were each held by all five of our test sites, verifying that we were searching the same three bibliographic records in each of the online catalogs by looking at the OCLC number in the records.Each of the catalogs we tested has a permalink feature; this is a stable URL that always points to the same record in each Using a permalink approximates conducting a known-item search for that item from a catalog search screen.We saved these links and used them in our searches.The bibliographic records we tested were for these books; the permalinks used for testing follow the books:  ■ ■ Gathering More Data: Selecting the

Books and Catalogs to Study
To broaden our comparison and to increase our data collection, we also tested three other non-Auraria catalogs.We designed our study to incorporate a number of variables.We decided to link to bibliographic records for three different books in the five different online catalogs tested.These included Skyline and WorldCat@Auraria as well three additional online public access catalog products, for a total of two instances of Innovative Interfaces products, one of a Voyager catalog, and one of a SirsiDynix catalog.We also selected online catalogs in different parts of the country: We gathered the data for thirteen days in early November 2009, an active period in the middle of the semester.For each test, we recorded the response time total in seconds.The data is displayed in tables 1-3.We searched bibliographic records for three books in five library catalogs over thirteen days (3 x 5 x 13) for a total of 195 response time measurements.The WebSitePulse data is calculated to the ten thousandth of a second, and we recorded the data exactly as it was presented.

university of colorado Denver: skyline (innovative interfaces)
As previously mentioned, the traditional catalog at Auraria Library runs on an Innovative Interfaces integrated library system (ILS).Testing revealed a missing favicon image file that the Web server tries to send each time (item 4 in figure 3).However, this did not negatively affect the response time.The catalog's response time was good, with an average of 1.2930 seconds, giving it the fastest average time among all the test sites in the testing period.As figure 1 shows, however, Skyline is a typical legacy catalog that is designed for a traditional library environment.

library of congress: Online catalog (voyager)
The average response time for the LCOC was 2.0076

■ ■ Results
The data shows the response times for each of the three books in each of the five online catalogs over the thirteenday testing period.The raw data was used to calculate averages for each book in each of the five online catalogs, and then we calculated averages for each of the five online catalogs (table 4).The averages show that during the testing period, the response time varied between 1.2930 seconds for the Skyline library catalog in Denver to 11.5734 seconds for WorldCat@Auraria, which has its servers in Ohio.

university of colorado Denver: Worldcat@Auraria
WorldCat@Auraria was routinely over Nielsen's tensecond limit, sometimes taking as long as twenty seconds to load all the files to generate a single webpage.As previously discussed, this is due to the high number and variety of files that make up a single bibliographic record.The files sent also include cover images, but they are small and do not add much to the total time.After our tests on WorldCat@Auraria were conducted, the site removed one of the features on pages for individual resources, namely the "similar items" feature.This feature was one of the most file-intensive on a typical page, and its removal should speed up page loads.However, WorldCat@Auraria had the highest average response time by far of the five catalogs tested.results for Library of Congress online catalog record item 14 is a script, that while hosted on the ILS server, queries Amazon.com to return cover image art (figures 11-12).The average response time for UT Austin's catalog was 3.4997 seconds.This example demonstrates that response times for traditional (i.e., not NextGen) catalogs can be slowed down by additional content as well.

university of southern california: Homer (sirsiDynix)
The average response time for USC's Homer catalog was 4.1085 seconds, making it the second slowest after seconds.This was the second fastest average among the five catalogs tested.While, like Skyline, the bibliographic record page is sparsely decorated (figure 7), this pays dividends in response time, as there are only two CSS files and three GIF files to load after the HTML content loads (figure 9). Figure 8 shows that initial connection time is the longest factor in load time; however, it is still short enough to not have a negative effect.Total file size is 19.27 KB.As with Skyline, the page itself (figure 7) is not particularly end-user friendly to nonlibrarians.

university of texas at Austin: library catalog (innovative interfaces)
UT Austin, like Auraria Library, runs an Innovative Interfaces ILS.The library catalog also includes book cover images, one of the most attractive NextGen features (figure 10), and as shown in figure 12, third-party content is used to add features and functionality (items 16 and 17).UT Austin's catalog uses a Google JavaScript API (item 16 in figure 12) and LibraryThing's Catalog Enhancement product, which can add book recommendations, tag browsing, and alternate editions and translations.Total content size for the bibliographic record is considerably larger than Skyline and the LCOC at 138.84 KB.It appears as though inclusion of cover art nearly doubles the response time;  results for University of Texas at Austin's library catalog record completed.Added functionality and features in library search tools are valuable, but there is a tipping point when these features slow down a product's response time to where users find the search tool too slow or unreliable.Based on the findings of this study, we recommend that libraries adopt Web response time standards, such as those set forth by Nielsen, for evaluating vendor search products and creating in-house search products.Commercial tools like WebSitePulse make this type of data collection simple and easy.Testing should be conducted for an extended period of time, preferably during a peak period-i.e., during a busy time of the semester for academic libraries.We further recommend that reviews of electronic resources add response time as an WorldCat@Auraria, and the slowest among the traditional catalogs.This SirsiDynix catalog appears to take a longer time than the other brands of catalogs to make the initial connection to the ILS; this accounts for much of the slowness (see figures 14 and 15).Once the initial connection is made, however, the remaining content loads very quickly, with one exception: item 13 (see figure 15), which is a connection to the third-party provider Syndetic Solutions, which provides cover art, a summary, an author biography, and a table of contents.While the display of this content is attractive and well-integrated to the catalog (figure 13), it adds 1.2 seconds to the total response time.Also, as shown in item 14 and 15, USC's Homer uses the AddThis service to add bookmarking enhancements to the catalog.Total combined file size is 148.47 KB, with the bulk of the file size (80 KB) coming from the initial connection (item 1 in figure 15).

■ ■ Conclusion
An eye-catching interface and valuable content are lost on the end user if he or she moves on before a search is

Figure 1 .
Figure 1.Permalink screen shot for the record for the title Hard Lessons in Auraria Library's Skyline catalog

Figure 2 .
Figure 2. WebSitePulse webpage test bar graph results for Skyline (traditional) catalog record

Figure 3 .
Figure 3. WebSitePulse webpage test table results for Skyline (traditional) catalog record

Figure4.
Figure4.Permalink screen shot for the record for the title Hard Lessons in WorldCat@Auraria

Figure 7 .
Figure 7. Permalink screen shot for the record for the title Hard Lessons in the Library of Congress online catalog

Figure 10 .
Figure 10.Permalink screen shot for the record for the title Hard Lessons in University of Texas at Austin's library catalog

Figure 13 .
Figure 13.Permalink screen shot for the record for the title Hard Lessons in Homer, the University of Southern California's catalog

Table 1 .
Response Times for Book 1

Table 2 .
Response Times for Book 2

Table 3 .
Response Times for Book 3