By now, everyone’s used ChatGPT (or my personal favorite Claude). It’s fantastic to get quick answers for most things instead of delving through endless search results. But there are some things that AI chatbots don’t do well. Ask it about the weather this week, the latest news or new music releases, or about your company’s knowledgebase and it will respond along the lines of, “I’m sorry, Dave, I’m afraid I can’t do that.”
To overcome this, the previous traditional approach was to include your material and then re-train the AI on the new corpus of material. But re-training takes a lot of time and resources, and can lead to unpredictable or unwanted outcomes, such as not including your material with enough priority (weights), or too much weight. (Example: You proudly proclaim your company is “the best in the world” in your documentation, and when you ask about the biggest company in the world based on revenue or something, it insists it your company.)
In 2020, a new method was proposed, and just this week it’s started to really gain attention. This is called Retrieval Augmented Generation or RAG for short. Instead of re-training the entire chatbot, you instead use tools like LangChain to vectorize your datasets separately (that is, convert from human readable text to the numbers that the mathematical models use for LLMs). Then, when asking questions to a chatbot, it is geared to both use it’s main training set, plus your datasets separately, in order to more accurately retrieve that information.
In my dayjob, we are currently working with some partners to implement this methodology for our new customer support chatbot. In addition to just vectorizing data for LLM use, we are implementing an AWS Lambda that can interact with our API in order to fetch realtime data. The intersection of AI + API is really the future of automated systems and intelligent agents.
In other AI news, Claude announced some new functionality about being able to “use a computer” like a human, to search info, to browse, interact, or scrape webpages, etc. Soon, it seems robots will be surfing the web for us, leaving us more time for goat farming or whatever it is humans do for fun.