Humans use vast stores of information to complete their thoughts and tasks. If I’m baking a cake, I will use a recipe book (or the box instructions). If I’m trying to install a 3-way electrical switch, I might go to YouTube for the information I need. We call this process information retrieval.
As humans, we need and rely on various sources or information stores to retrieve relevant information. Our minds have a natural ability to infer or retrieve based on context. The word “cucumber” has a different meaning if I’m making a salad or doing behavior-driven design in software development.
This contextual awareness is what creates complexity when we try to retrieve information relevant to language models to get a desired output. The retrieval process plays a critical role in ensuring accurate and meaningful responses. But today’s AI hype cycle unduly displays vector databases as the ultimate solution for information categorization and retrieval.
In this blog post, we will explore the hype surrounding vector databases, analyze their limitations, highlight where they are very useful, and hopefully illuminate broader thoughts on AI information retrieval. Colin Harman’s enlightening article on Tunnel Vision with AI inspired this discussion.
When large language models interact with text, they convert each word (or portion of a word called a token) into a vector. A vector is a series of numbers used to represent that word or token. In the case of ChatGPT, 1,536 vectors represent each token. These vectors describe everything from the word root to relationships of preceding and succeeding words.
Vector databases refer to databases designed to store and manage these high-dimensional vectors efficiently. With ChatGPT, vector databases have gained monumentally in popularity. Vector databases are great at similarity searches – finding items that are mathematically near your query. If you search for the word “food,” similarity searches will find words like “hungry,” “thirsty,” “food,” or “drink.” What Vector databases can do, though, is perform math on the vectors of those words so that:
drink - food + hungry = thirsty
Searches, filters and mathematical operations make vector databases invaluable for large language models. It is essential to understand the role vector databases play in the development workflow and consider alternative approaches to AI information retrieval.
Information retrieval is the art of selecting the right information to pass on to a language model. Like humans naturally filter information to answer questions, we must provide language models with relevant context.
If I’m an employee asking about vacation days, I would naturally switch context to my current employer. When we use tools like ChatGPT to ask about vacation days, they do not have awareness of my employer. Therefore, we typically pass this contextual information on, either by copying our HR policy directly or via retrieval from a vector database.
This information retrieval process is crucial for language models to have proper context and domain knowledge to generate accurate responses. However, a practical constraint exists with the amount of information we can pass to a language model in a single prompt, which necessitates careful and strategic information selection.
If we take a closer look at the current AI marketing landscape, we can easily see how prominent vector databases are. There are many blog articles and tutorials that indicate this is the ultimate solution for AI information storage and retrieval. Unfortunately, this misconception has resulted in many developers attempting to force vector databases into use cases where they are suboptimal. The consequence is poorly architected products that fail to deliver the desired results.
So why is there still so much hype about vector databases?
Startup companies are always eager to stand out in a competitive market, contributing to the vector database hype. These startups want to differentiate themselves, which is key to getting noticed. This is a great opportunity for startups to latch onto something new and cutting-edge.
By adopting vector database technology, companies can position themselves as being at the forefront of AI innovation. Additionally, this buzzword enables them to raise capital from venture capitalists keen to invest in AI-oriented enterprises. Consequently, vector databases have become synonymous with AI in public perception.
Don’t get us wrong – vector databases are crucial to AI and large language models in particular. They help provide a means for semantic and similarity search. They also offer a means to categorize and store information. But, it is crucial to look at any tech solution by considering the specific requirements and constraints at hand rather than blindly adopting the latest trend.
When it comes to information retrieval, the choice between using a vector database or some alternate method depends on the nature of the information you wish to retrieve. For example, if you have a database of structured information, such as purchase history, you should continue to use a traditional database and retrieval method. This is what developers call a “solved problem.” Moving purchase history into a vector database does not help solve purchase information retrieval, nor are they more efficient than traditional databases.
Another example is deterministic rules-based systems, such as shipping calculations or coupon applications. When we know context and rules and can repeatedly calculate the outcome, then a rules-based information retrieval system is best.
There are many cases in which you will need a language model to transform the user’s intent into a structured query, but you should not force vector databases onto every use case. Vector search excels when the output form is unknown, making it ideal for retrieving context, paraphrasing or conducting semantic searches. Just don’t forget that sometimes, regular keyword searches can also go a long way at no additional cost.
Next time you come across an opportunity to employ vector search, take a step back and widen your perspective. The world of information retrieval is far more extensive than what vector databases alone can offer. While vector databases do play a role, it is essential to recognize they are only one piece of the puzzle. There are various other approaches and techniques available for effective retrieval. By exploring the broader landscape, we can make informed decisions and build robust products that deliver meaningful results.
Remember, context is king in AI, and the appropriate approach to information retrieval plays a pivotal role in ensuring accurate, meaningful interactions with AI language models. So, whenever you’re considering implementing technologies such as vector databases, don’t follow the hype – instead, follow the facts, your unique needs, and look for an expert to guide you to success.
