“Good Night, Good Day, Good Luck”

Applying Topic Modeling to Chat Reference Transcripts

  • Megan Ozeran University of Illinois at Urbana-Champaign
  • Piper Martin University of Illinois at Urbana-Champaign


This article presents the results of a pilot project that tested the application of algorithmic topic modeling to chat reference conversations. The outcomes for this project included determining if this method could be used to identify the most common chat topics in a semester and whether these topics could inform library services beyond chat reference training. After reviewing the literature, four topic modeling algorithms were successfully implemented using Python code: (1) LDA, (2) phrase-LDA, (3) DMM, and (4) NMF. Analysis of the top ten topics from each algorithm indicated that LDA, phrase-LDA, and NMF show the most promise for future analysis on larger sets of data (from three or more semesters) and for examining different facets of the data (fall versus spring semester, different time of day, just the patron side of the conversation).


Jo Kibbee, David Ward, and Wei Ma, “Virtual Service, Real Data: Results of a Pilot Study,” Reference Services Review 30, no. 1 (Mar. 1, 2002): 25–36, https://doi.org/10.1108/00907320210416519.

David Ward and M. Kathleen Kern, “Combining IM and Vendor-Based Chat: A Report from the Frontlines of an Integrated Service,” Portal: Libraries and the Academy 6, no. 4 (Oct. 2006): 417–29, https://doi.org/10.1353/pla.2006.0058.

JoAnn Jacoby et al., “The Value of Chat Reference Services: A Pilot Study,” Portal: Libraries and the Academy 16, no. 1 (Jan. 2016): 109–29, https://doi.org/10.1353/pla.2016.0013.

David Ward, “Using Virtual Reference Transcripts for Staff Training,” Reference Services Review 31, no. 1 (2003): 46–56, https://doi.org/10.1108/00907320310460915.

Robin Brown, “Lifting the Veil: Analyzing Collaborative Virtual Reference Transcripts to Demonstrate Value and Make Recommendations for Practice,” Reference & User Services Quarterly 57, no. 1 (Fall 2017): 42–47, https://doi.org/10.5860/rusq.57.1.6441.

Maryvon Côté, Svetlana Kochkina, and Tara Mawhinney, “Do You Want to Chat? Reevaluating Organization of Virtual Reference Service at an Academic Library,” Reference & User Services Quarterly 56, no. 1 (Fall 2016): 36–46, https://doi.org/10.5860/rusq.56n1.36.

Donna Goda and Corinne Bisshop, “Frequency and Content of Chat Questions by Time of Semester at the University of Central Florida: Implications for Training, Staffing and Marketing,” Public Services Quarterly 4, no. 4 (Dec. 2008): 291–316, https://doi.org/10.1080/15228950802285593.

Kelsey Keyes and Ellie Dworak, “Staffing Chat Reference with Undergraduate Student Assistants at an Academic Library: A Standards-Based Assessment,” The Journal of Academic Librarianship 43, no. 6 (2017): 469–78, https://doi.org/10.1016/j.acalib.2017.09.001.

Michael Mungin, “Stats Don’t Tell the Whole Story: Using Qualitative Data Analysis of Chat Reference Transcripts to Assess and Improve Services,” Journal of Library & Information Services in Distance Learning 11, no. 1–2 (Jan. 2017): 25–36, https://doi.org/10.1080/1533290X.2016.1223965.

Shu Z. Schiller, “CHAT for Chat: Mediated Learning in Online Chat Virtual Reference Service,” Computers in Human Behavior 65 (Dec. 2016): 651–65, https://doi.org/10.1016/j.chb.2016.06.053.

Ellie Kohler, “What Do Your Library Chats Say?: How to Analyze Webchat Transcripts for Sentiment and Topic Extraction,” in Brick & Click Libraries Conference Proceedings (Brick & Click, Maryville, MO: Northwest Missouri State University, 2017), 138–48, https://www.nwmissouri.edu/library/brickandclick/presentations/eproceedings.pdf.

Guan-Bin Chen and Hung-Yu Kao, “Re-Organized Topic Modeling for Micro-Blogging Data,” in Proceedings of the ASE BigData & SocialInformatics 2015, ASE BD&SI ’15 (New York, NY: ACM, 2015), 35:1–35:8, https://doi.org/10.1145/2818869.2818875.

X. Cheng et al., “BTM: Topic Modeling over Short Texts,” IEEE Transactions on Knowledge and Data Engineering 26, no. 12 (Dec.2014): 2,928–41, https://doi.org/10.1109/TKDE.2014.2313872.

Chenliang Li et al., “Topic Modeling for Short Texts with Auxiliary Word Embeddings,” in Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval (ACM Press, 2016), 165–74, https://doi.org/10.1145/2911451.2911499.

HyunSeung Koh and Mark Fienup, “Library Chat Analysis: A Navigation Tool,” (Poster, Dec. 5, 2018), https://libraryassessment.org/wp-content/uploads/2018/11/58-KohFienup-LibraryChatAnalysis.pdf.

How to Cite
Ozeran, M., & Martin, P. (2019). “Good Night, Good Day, Good Luck”. Information Technology and Libraries, 38(2), 49-57. https://doi.org/10.6017/ital.v38i2.10921