chunkipy.text_chunker.base_overlap_text_chunker

Classes

BaseOverlapTextChunker([chunk_size, ...])

Base class for chunkers that assemble chunks with overlap from text parts.
class chunkipy.text_chunker.base_overlap_text_chunker.BaseOverlapTextChunker(chunk_size=None, size_estimator=None, overlap_ratio=0.0)[source]

Bases: BaseTextChunker, ABC

Base class for chunkers that assemble chunks with overlap from text parts.

Parameters:
chunk(text)[source]

Chunk text by splitting first and then assembling chunk objects.

Return type:

Chunks

Parameters:

text (str)

abstract split_text(text)[source]

Split text into parts consumed by overlap-aware chunk assembly.

Return type:

Generator[TextPart, None, None]

Parameters:

text (str)