chunkipy.text_splitters.basic_text_splitters

Classes

ColonTextSplitter()

Split text on ``: `` boundaries.

CommaTextSplitter()

Split text on ``, `` boundaries.

FullStopTextSplitter()

Split text on ``. `` sentence-like boundaries.

NewlineTextSplitter()

Split text on newline boundaries.

SemicolonTextSplitter()

Split text on ``; `` boundaries.

SeparatorTextSplitter(separator)

Split text using a fixed separator while preserving the separator.

WordTextSplitter()

Split text on spaces while preserving trailing whitespace.
class chunkipy.text_splitters.basic_text_splitters.ColonTextSplitter[source]

Bases: SeparatorTextSplitter

Split text on ``: `` boundaries.

class chunkipy.text_splitters.basic_text_splitters.CommaTextSplitter[source]

Bases: SeparatorTextSplitter

Split text on ``, `` boundaries.

class chunkipy.text_splitters.basic_text_splitters.FullStopTextSplitter[source]

Bases: SeparatorTextSplitter

Split text on ``. `` sentence-like boundaries.

class chunkipy.text_splitters.basic_text_splitters.NewlineTextSplitter[source]

Bases: SeparatorTextSplitter

Split text on newline boundaries.

class chunkipy.text_splitters.basic_text_splitters.SemicolonTextSplitter[source]

Bases: SeparatorTextSplitter

Split text on ``; `` boundaries.

class chunkipy.text_splitters.basic_text_splitters.SeparatorTextSplitter(separator)[source]

Bases: BaseTextSplitter

Split text using a fixed separator while preserving the separator.

Parameters:

separator (str)

property separator: str

Return the delimiter used by the splitter.

class chunkipy.text_splitters.basic_text_splitters.WordTextSplitter[source]

Bases: SeparatorTextSplitter

Split text on spaces while preserving trailing whitespace.