ai-chunk-overlap Edge Function — Ai

ai-chunk-overlap — Split text into overlapping chunks for RAG pipelines

Splits a document into overlapping text chunks with configurable size, overlap, and boundary snapping — no external API calls required.

---

API

POST /api/ai-chunk-overlap

Request body

Field	Type	Required	Default	Description
text	string	✅	—	Text to split into chunks
chunkSize	number	❌	1000	Target chunk size in characters
overlap	number	❌	200	Characters to repeat between adjacent chunks
boundary	"char" | "word" | "sentence"	❌	"word"	Boundary snapping mode

Success response (200)

{
  "success": true,
  "data": {
    "chunks": [
      { "text": "...", "start": 0, "end": 1024, "index": 0 },
      { "text": "...", "start": 824, "end": 1848, "index": 1 }
    ],
    "count": 2,
    "totalChars": 5000
  }
}

Error responses

Code	HTTP	When
INVALID_INPUT	400	Missing text or invalid parameters
INTERNAL_ERROR	500	overlap >= chunkSize

---

Usage

cURL

curl -X POST "$FUNCTION_URL" \
  -H "Content-Type: application/json" \
  -d '{"text": "...", "chunkSize": 500, "overlap": 100, "boundary": "sentence"}'

TypeScript / JavaScript (HTTP)

const response = await fetch(FUNCTION_URL, {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ text: documentText, chunkSize: 1000, overlap: 200 }),
});
const { data } = await response.json();
for (const chunk of data.chunks) {
  await embedAndStore(chunk.text, chunk.start, chunk.end);
}

Direct import (Node / Bun / Deno)

import { aiChunkOverlap } from '@aerostack/functions/ai-chunk-overlap';

const { chunks } = aiChunkOverlap({ text: longDocument, chunkSize: 800, overlap: 150 });

---

Use Cases

• Preparing documents for embedding by splitting them into overlapping chunks before storing in a vector database.
• Chunking transcripts or long-form articles for RAG retrieval, ensuring context spanning chunk boundaries is preserved.
• Splitting code or documentation files into manageable pieces for LLM-based analysis.

---

Notes

• start and end are character offsets into the original text — text.slice(start, end) reproduces each chunk.
• Overlap ensures that phrases or sentences crossing a chunk boundary appear in both adjacent chunks.
• boundary: 'sentence' snaps to the nearest ., !, or ? character.
• overlap must be less than chunkSize — equal or greater values throw an error.