Installation

You can install chunkipy in several ways, depending on your preferred environment or package manager. The library is lightweight at its core, with optional dependencies for specific functionalities such as language detection or advanced text splitting.

The goal is to let you install only what you really need — while still providing an easy way to install everything at once.

Note

You can install optional dependencies as feature groups or individually. Feature groups (e.g. language-detection, nlp, ai) are recommended for convenience.

Optional dependencies overview

Feature groups (recommended for convenience):

  • language-detection – enables both LangdetectLanguageDetector and FastTextLanguageDetector (langdetect + fasttext).

  • nlp – enables semantic sentence splitters with Stanza and spaCy backends (stanza + spacy).

  • ai – enables LLM integration with OpenAI and token-based size estimation (openai + tiktoken).

  • all – installs all optional dependencies.

Individual packages (for granular control):

  • stanza – enables the StanzaSentenceTextSplitter for sentence-level splitting using the Stanza NLP library.

  • spacy – enables the SpacySentenceTextSplitter for sentence-level splitting using spaCy models.

  • langdetect – enables the built-in LangdetectLanguageDetector (used by default in semantic splitters).

  • fasttext – enables the FastTextLanguageDetector for FastText model-based language identification.

  • openai – enables the OpenAISizeEstimator for OpenAI-compatible token counting.

  • tiktoken – enables token-based size estimation using OpenAI’s tokenizer.

Install using pip

You can install chunkipy directly from PyPI using pip.

Installs only the core library (no optional dependencies).

pip install chunkipy

Install feature groups for common use cases.

pip install "chunkipy[language-detection]"  # Language detection
pip install "chunkipy[nlp]"                  # NLP backends
pip install "chunkipy[ai]"                   # LLM integration
pip install "chunkipy[all]"                  # Everything

Installs chunkipy with specific optional dependencies.

pip install "chunkipy[spacy,langdetect]"
pip install "chunkipy[stanza,fasttext]"

Mix and match feature groups and individual packages.

pip install "chunkipy[nlp,language-detection,openai]"

Install using Poetry

If you use poetry for dependency management, you can add chunkipy with extras directly to your project.

poetry add chunkipy

poetry add chunkipy[language-detection]
poetry add chunkipy[nlp]
poetry add chunkipy[ai]
poetry add chunkipy[all]

poetry add chunkipy[spacy,langdetect]

poetry add "chunkipy[nlp,language-detection]"

Install using uv

uv is a fast Python package manager designed for modern workflows. It fully supports PEP 621-style extras and can dramatically speed up installations.

uv add chunkipy

uv add chunkipy[language-detection]
uv add chunkipy[nlp]
uv add chunkipy[ai]

uv add chunkipy[all]

Install using pipx (for CLI or isolated usage)

If you want to experiment with chunkipy in isolation, or use it in a CLI-style environment without polluting your main environment:

pipx install "chunkipy[spacy,stanza,langdetect,fasttext,openai,tiktoken]"

This installs chunkipy in a virtual environment managed by pipx — great for trying it out quickly or keeping your global environment clean.

Verification

Once installed, you can verify imports:

import chunkipy
from chunkipy import FixedSizeTextChunker, RecursiveTextChunker

print("Chunkipy import OK")

Next steps

  • 📘 Continue to the Quickstart section to see how to start chunking text.

  • ⚙️ Check the API Reference reference for all available classes and configuration options.

  • 🤝 Explore the Contributing guide to learn how to contribute to the project.

  • 🔍 Explore Overview to learn about different size estimation methods.

  • 🧩 Explore Overview to learn about the different chunking strategies.

  • ✂️ Explore Overview to learn about the different text splitting options.

  • 🌍 Explore Overview to learn about the available language detector APIs.