chunking-optimization

BetterRAG: Powerful RAG evaluation toolkit for LLMs. Measure, analyze, and optimize how your AI processes text chunks with precision metrics. Perfect for RAG systems, document processing, and embedding quality assessment.

This package, developed as part of our research detailed in the Chroma Technical Report, provides tools for text chunking and evaluation. It allows users to compare different chunking methods and includes implementations of several novel chunking strategies.

An LLM-RAG powered application where you can talk with your own pdf.

An Overview of the Latest Document Chunking Research

AI-powered semantic matching system that leverages local embedding technology (via @xenova/transformers) and Pinecone's vector database to perform intelligent matching based on skills, experience, and job requirements.

Chunking In Enterprise Document Processing Pipeline for building Retrieval-Augmented Generation (RAG) and Enterprise AI Knowledge Assistants.

A minimal, dependency-light Retrieval-Augmented Generation (RAG) pipeline that answers questions from a PDF document.

A RAG-powered knowledge assistant that answers style guide questions for translators. Built with Weaviate, Amazon Bedrock (Claude 3 Sonnet), and Streamlit.

LumenFlow is an AI-assisted customer support operations platform featuring local LLM workflows, retrieval-augmented generation, grounded drafting, human review, and evaluation-first system design.

cython based chunking package

A deep dive into text chunking for Retrieval-Augmented Generation systems