Decoding Better Encodings

Every business has documents with unstructured yet valuable information. Accessing these insights improves decision making but has conventionally come at a high cost in time and effort. Retrieval augmented generation (RAG) leverages generative AI to accelerate research and successful decision making. RAG encodes documents in large vectors, then compares those encodings to find the information most relevant to the question, making a RAG model only as good as its encodings. In this talk, we will dive deep into what information is probably being encoded, the geometry of the encodings, and benefits of quantization. At the end, we will tie it all together to identify some practical improvements for your RAG system.