theseer/tokenizer

A small library for converting tokenized PHP source code into XML and potentially other formats

Maintainers

Details

github.com/theseer/tokenizer

Source

Issues

Fund package maintenance!
theseer

Installs: 562 336 462

Dependents: 12

Suggesters: 0

Security: 0

Stars: 5 168

Watchers: 9

Forks: 22

Open Issues: 0

1.2.3 2024-03-03 12:36 UTC

Requires

  • php: ^7.2 || ^8.0
  • ext-dom: *
  • ext-tokenizer: *
  • ext-xmlwriter: *

BSD-3-Clause 737eda637ed5e28c3413cb1ebe8bb52cbf1ca7a2

  • Arne Blankerts <arne.woop@blankerts.de>

This package is auto-updated.

Last update: 2024-12-10 18:09:16 UTC

README

A small library for converting tokenized PHP source code into XML.

Test

Installation

You can add this library as a local, per-project dependency to your project using Composer:

composer require theseer/tokenizer

If you only need this library during development, for instance to run your project's test suite, then you should add it as a development-time dependency:

composer require --dev theseer/tokenizer

Usage examples

$tokenizer = new TheSeer\Tokenizer\Tokenizer();
$tokens = $tokenizer->parse(file_get_contents(__DIR__ . '/src/XMLSerializer.php'));

$serializer = new TheSeer\Tokenizer\XMLSerializer();
$xml = $serializer->toXML($tokens);

echo $xml;

The generated XML structure looks something like this:

<?xml version="1.0"?>
<source xmlns="https://github.com/theseer/tokenizer">
 <line no="1">
  <token name="T_OPEN_TAG">&lt;?php </token>
  <token name="T_DECLARE">declare</token>
  <token name="T_OPEN_BRACKET">(</token>
  <token name="T_STRING">strict_types</token>
  <token name="T_WHITESPACE"> </token>
  <token name="T_EQUAL">=</token>
  <token name="T_WHITESPACE"> </token>
  <token name="T_LNUMBER">1</token>
  <token name="T_CLOSE_BRACKET">)</token>
  <token name="T_SEMICOLON">;</token>
 </line>
</source>