sterzik/tokenizer

Library for creating an universal tokenizer

Maintainers

Package info

github.com/marek-sterzik/tokenizer

pkg:composer/sterzik/tokenizer

Statistics

Installs: 1

Dependents: 1

Suggesters: 0

Stars: 0

Open Issues: 0

Security

Advisories: 0

Aikido package health analysis

dev-master 2025-10-14 22:57 UTC

Requires

  • php: >=8.1

MIT fc988e4e6afff8f6576f2dbc3c3171ff65916e41

  • Marek Sterzik <marek.sterzik.woop@gmail.com>

This package is auto-updated.

Last update: 2026-02-14 23:40:29 UTC

README

This library is currently in the very first stage of implementation and therefore not really useful.

But maybe in the future, there will be an univerzal tokenizer library.