Upstream software project

sentencepiece

Unsupervised text tokenizer and detokenizer

About sentencepiece

Unsupervised text tokenizer and detokenizer

This project links 18 native package records across 5 recorded operating-system releases. Compare the retained versions and architectures below, then open the package for your own release.

These are catalog observations, not a guarantee of installation, compatibility, or upstream support.

Project pictures and package coverage

sentencepiece repository preview from GitHub
Project repository preview, supplied by GitHub. Open source repository
Debian 12 (Bookworm): 4 package records; Debian 13 (Trixie): 4 package records; Fedora 43: 4 package records; Fedora 44: 4 package records; openSUSE Tumbleweed: 2 package records. Catalog coverage diagram, not an application screenshot.sentencepiece: recorded package coverageDebian 12 (Bookworm)4 recordsDebian 13 (Trixie)4 recordsFedora 434 recordsFedora 444 recordsopenSUSE Tumbleweed2 records
OpenFactory diagram of linked package records. It is not an application screenshot.

Project identity

Project
sentencepiece
Publisher
Not authoritatively mapped
Native package records
18
Operating systems
debian-12, debian-13, fedora-43, fedora-44, opensuse-tumbleweed
License expression
Apache-2.0
Source repository
Open source repository

Source-reported description

The fullest retained description is shown with its source. Distribution packaging descriptions may include downstream details.

SentencePiece is an unsupervised text tokenizer and detokenizer mainly for Neural Network-based text generation systems where the vocabulary size is predetermined prior to the neural model training. SentencePiece implements subword units (e.g. byte-pair-encoding and unigram language model) with the extension of direct training from raw sentences. This package provides the Python bindings for SentencePiece.

Description source

Packages by operating system

Compare recorded versions, then open a package for dependency, file, checksum, and repository evidence. Version strings are distribution-specific, not a ranking of newer software.

Debian 12 (Bookworm)

  1. Package libsentencepiece0

    bookworm (Debian 12 (Bookworm)) (libs): Library files of SentencePiece

    0.1.97-3+b1: amd64 arm64

    Source: sentencepiece

  2. Package libsentencepiece-dev

    bookworm (Debian 12 (Bookworm)) (libdevel): Header files of SentencePiece

    0.1.97-3+b1: amd64 arm64

    Source: sentencepiece

  3. Package python3-sentencepiece

    bookworm (Debian 12 (Bookworm)) (python): SentencePiece binding for Python3

    0.1.97-3+b1: amd64 arm64

    Source: sentencepiece

  4. Package sentencepiece

    bookworm (Debian 12 (Bookworm)) (science): Unsupervised text tokenizer and detokenizer

    0.1.97-3+b1: amd64 arm64

    Source: sentencepiece

Debian 13 (Trixie)

  1. Package libsentencepiece0

    trixie (Debian 13 (Trixie)) (libs): Library files of SentencePiece

    0.2.0-1+b4: amd64 arm64

    Source: sentencepiece

  2. Package libsentencepiece-dev

    trixie (Debian 13 (Trixie)) (libdevel): Header files of SentencePiece

    0.2.0-1+b4: amd64 arm64

    Source: sentencepiece

  3. Package python3-sentencepiece

    trixie (Debian 13 (Trixie)) (python): SentencePiece binding for Python3

    0.2.0-1+b4: amd64 arm64

    Source: sentencepiece

  4. Package sentencepiece

    trixie (Debian 13 (Trixie)) (science): Unsupervised text tokenizer and detokenizer

    0.2.0-1+b4: amd64 arm64

    Source: sentencepiece

Fedora 43

  1. Package python3-sentencepiece

    43 (Fedora 43) (Unspecified): Python module for SentencePiece

    0.2.0-8.fc43: aarch64 x86_64

    Source: sentencepiece

  2. Package sentencepiece-devel

    43 (Fedora 43) (Unspecified): Libraries and header files for SentencePiece

    0.2.0-8.fc43: aarch64 i686 x86_64

    Source: sentencepiece

  3. Package sentencepiece-libs

    43 (Fedora 43) (Unspecified): Runtime libraries for SentencePiece

    0.2.0-8.fc43: aarch64 i686 x86_64

    Source: sentencepiece

  4. Package sentencepiece-tools

    43 (Fedora 43) (Unspecified): Tools for SentencePiece

    0.2.0-8.fc43: aarch64 x86_64

    Source: sentencepiece

Fedora 44

  1. Package python3-sentencepiece

    44 (Fedora 44) (Unspecified): Python module for SentencePiece

    0.2.1-1.fc44: aarch64 x86_64

    Source: sentencepiece

  2. Package sentencepiece-devel

    44 (Fedora 44) (Unspecified): Libraries and header files for SentencePiece

    0.2.1-1.fc44: aarch64 i686 x86_64

    Source: sentencepiece

  3. Package sentencepiece-libs

    44 (Fedora 44) (Unspecified): Runtime libraries for SentencePiece

    0.2.1-1.fc44: aarch64 i686 x86_64

    Source: sentencepiece

  4. Package sentencepiece-tools

    44 (Fedora 44) (Unspecified): Tools for SentencePiece

    0.2.1-1.fc44: aarch64 x86_64

    Source: sentencepiece

openSUSE Tumbleweed

  1. Package python313-sentencepiece

    tumbleweed (openSUSE Tumbleweed) (Unspecified): Unsupervised text tokenizer and detokenizer

    0.2.2-1.2: x86_64

    Source: python-sentencepiece

  2. Package python314-sentencepiece

    tumbleweed (openSUSE Tumbleweed) (Unspecified): Unsupervised text tokenizer and detokenizer

    0.2.2-1.2: x86_64

    Source: python-sentencepiece

Project resources and further reading

Mapping provenance

Only source-backed identity signals create public cross-OS links. A reviewer can later approve or dispute an inferred relationship without rewriting native package history.

No field-level source record is published yet.