Source Package: python-sentencepiece (0.2.2-1.2)
Unsupervised text tokenizer and detokenizer
SentencePiece is an unsupervised text tokenizer and detokenizer mainly for Neural Network-based text generation systems where the vocabulary size is predetermined prior to the neural model training. SentencePiece implements subword units (e.g. byte-pair-encoding and unigram language model) with the extension of direct training from raw sentences. This package provides the Python bindings for SentencePiece.
Package information
- Homepage
- [https://github.com/google/sentencepiece]
- License
- Apache-2.0
- Mantenedor del paquete downstream
- Desconocido: no consta en la fuente indexada
The following binary packages are built from this source package:
- python313-sentencepiece
Unsupervised text tokenizer and detokenizer
- python314-sentencepiece
Unsupervised text tokenizer and detokenizer
