Screenshot reading Welcome back to Research AI

Semantic Search & Research Insights : How Large Language Models are Changing Library Search Experiences

Stop me if you’ve heard this one before: “Your research question or thesis statement is useful but insufficient for starting your library search. You’ll need to extract some words or phrases from that sentence and probably look up a few synonyms or related terms for those keywords as backups.”

In the first quarter of the 21st century, keyword searching has been the most heavily used technique for online information retrieval by students, faculty, staff, and other academic library patrons. This was a much more user-friendly option for many than previous systems, but translating an information need or topic into relevant keywords was still an abstract process. 

Introducing Semantic Search

A different search paradigm is becoming more common: multiple Champlain College Library databases now offer optional modes supporting semantic or prompt-based search. When using Statista’s Research AI or O’Reilly Answers, patrons can ask questions or enter statements based on their research topic or information need. These tools then use artificial intelligence (AI) technologies such as natural language processing, sometimes augmented by large language models (LLMs), to interpret the question and run a database search based on that interpretation.

What those two tools do with those results is a crucial point of difference. Statista’s Research AI uses the LLM Claude 3 to generate a synthesis of the top results available in Statista that best address the user-provided question or prompt. Statista’s tool uses a technique called Retrieval Augmented Generation that restricts the model from basing those syntheses and insights on anything but sources findable and verifiable in the Statista database. However, the technique is not perfect, and errors may still occur.

O’Reilly Answers, meanwhile, points users directly to the most relevant passages from that database’s ebooks and videos after processing the user-provided question or prompt, meaning that the tool does not create generative AI-derived text.

More library databases will integrate new features such as semantic search, generated research insights, and more in the coming months, and many of these tools will be powered by large language models and generative AI. 

Access Statista Research AI from Within that Database

Select the Research AI link in the top-level navigation. That link may be consolidated on smaller screens or mobile devices under a hamburger button.

Screenshot of Research AI (New) link in Statista's main menu.

Access O’Reilly Answers from Within that Database

Select the Start Learning link in the top-level navigation. Then, select Answers. That link may be consolidated on smaller screens or mobile devices under a hamburger button. Note that you cannot directly link to the O’Reilly Answers interface; you will need to use the general O’Reilly database link.

Screenshot of O'Reilly. The Start Learning option in the menu is selected, which opened a drop-down in which one of the options in Answers.