article thumbnail

Super charge your LLMs with RAG at scale using AWS Glue for Apache Spark

AWS Machine Learning Blog

Large language models (LLMs) are very large deep-learning models that are pre-trained on vast amounts of data. This enables you to preprocess your external data in the phases including cleaning, sanitization, chunking documents, generating vector embeddings for each chunk, and loading into a vector store.

LLM 107