README

Status
About the project
Purpose
Reporting
Why PHP?
License
Contributing
Install
Getting started
Inline XBRL
Links
Case Study

Find much more information in the wiki.

Status

Conformance suite tests

These tests are performed nightly to provide an overview of the status of the source code. All tests have been run on Linux and Windows.
This project does not support HHVM.

Statistics

This project comprises 102535 lines in 251 files

About the project

The XBRLQuery project started as an idea to take the regulatory information companies must produce and extend that use within organizations. The benefit of using XBRL to represent corporate data is that it's verifiable and published and in an agreed, transparent format, a format accepted by all major regulatory bodies around the world.

We realised the ability to work as a community to provide useful information from internal or published financial data will allow organizations to plan, analyse and react to business driven factors and by doing so via an OpenSource project, it becomes beneficial to all.

An overview of the project is provided below. There is more information available in the Wiki including and explanation and documentation of the source code and examples of using the source to create reports.

Validating

The processor can be validating. Set a variable to true and both the schemas and instance documents are validated. Set the variable to false and the processor just processes and assumes you know the schemas and instance documents are valid and consistent.

Specifications supported

XBRL 2.1 except there is no support for reference linkbases.
XBRL Dimensions are supported.
XBRL Generics including generic references, generic links, generic labels and assertions.
XBRL Formulas.
XBRL Taxonomy Packages (including support for the various legacy SEC packages).
XBRL Extensible Enumerations 1.0 and 2.0 PWD.
In-line XBRL 1.1 and tranformations to TRR 4

XBRL Formulas includes support for:

All the recommended specifications
The Formula Tuples and Variable Scope draft specifications.
The full set of functions registry (XFI) are also supported including both recommended and draft functions.

Notable exception:

Only instance documents using the XML format (XHTML) can be read as the processor does not support any of the OIM mappings.

These exceptions exist because there has been no reason to add support. At the moment where iXBRL is being used, submitters are also providing data in XML formatted instance documents.

Taxonomy package support

There are many instances of zip files containing taxonomies. If a zip file content follows the XBRL Taxonomy Package specification then the processor will support it. However, there are many instances of zip files that contain taxonomies which were packaged this way long before the packaging specification existed. The US SEC packages are an example. In fact there are two versions of legacy packages available from the SEC: one that uses a JSON file to catalog the package content; and one that uses an XML file. The processor supports both.

In addition, the Danish IFRS package is supported as is the taxonomy from the Danish Business Agency (Erhvervsstyrelsen). Neither of these packages support the XBRL Taxonomy Packaging specification though the taxonomies are provided in a zip file.

Sometimes it is helpful to extend the core XBRL processinging class to add specific functionality for a taxonomy. This is especially true when a taxonomy may be extended. Examples of extended XBRL processing classes are IFRS and ESMA ESEF taxonomies. In these cases, the base packaging class, such as XBRL_TaxonomyPackage can be extended to return the name of the XBRL processor class to use to processes the taxonomies. Examples provided include:

XBRL-US-TaxonomyPackage.php
XBRL-IFRS-Package.php
XBRL-ESMA-ESEF-Package.php

Conformance

The project passes almost all conformance tests, the omissions arising because there is no support for reference linkbases.

A separate project XBRL-tests provides a means to verify the project passes the conformance test. This project is a copy of the test harness we use to run the conformance tests.

Signing

Instance documents and taxonomies can also be signed. The signing techniques used are similar to those use to sign Word or PDF documents or emails. Documents are signed using the private key of a certificate you have so that others are able to verify the contents of a document have not changed since you created it.

Signing does not impact the original document but is important for recipients of a taxonomy to know they are working with an unaltered copy of the relevant taxonomy and for recipients of instance documents to know the contents have not be altered since the document was prepared.

Signing uses only open standards for encryption and verification such as public/private key certificates. Although the project provides code to verify a signed instance document, the same process is possible using a standard distribution of most programming languages such as, but not limited to, Java, C#, Python and C++.

Purpose

This project has been created by Lyquidity Solutions Limited using the website name of www.xbrlquery.com to provide PHP applications with access to validated corporate data contained in XBRL instance documents. We also provide consulting services around these XBRL technologies. Please contact us at info-at-xbrlquery.com.

XBRL stands for 'eXtensible Business Reporting Language'. Naively I thought that word 'Reporting' in this context could be prefixed with words like 'Management' or 'Budgeting'. I thought the overwhelming focus on using XBRL to create submissions to regulatory authorities like the US SEC or the UK HMRC was a consequence of having so many accounting and governmental organizations involved in XBRL community. But the reality is the purpose of XBRL 'reporting' is reporting to regulatory bodies. It is rarely, if ever, used for other types of reporting.

Most of the tools I have seen that are available commercially or are in the public domain such as Arelle, Gepsio or ABRLAPI focus on preparing instance documents and/or validating those documents against a taxonomy and reporting errors. Clearly ensuring data contained in instance documents are consistent with the base taxonomy they claim to represent this is an important first step. In the context of 'reporting' to statutory authorities it probably is the only goal. Surely there are other potential uses for data that can be recorded in a document and that can be validated against a taxonomy.

Budgeting and management reporting are examples of processes all organizations have to perform regularly. They are processes which may have specific requirements in different departments, geographies and responsibility levels. These processes need to generate, transform, transfer and report information in a consistent way. A goal of these processes may be to record data in a repository such as a database but that data has to be captured in a variety of locations and needs to be captured in a format that allows verification of the data and that the correct data is being captured - tasks for which XBRL is ideally suited.

So the focus of XBRL Query is to provide a platform in which XBRL can be used as the means of representing data that is used in corporate processes such as budgeting and management reporting; to create a platform that uses XBRL as a means to transfer data between the actors in those business reporting processes in a reliable, repeatable way; to create a platform that allows anyone to validate the content of instance documents on which business decisions may be made; and that any document being used is genuine and unchanged.

XBRL Support

This project includes support for the full XBRL 2.1 specification as well as the XBRL Dimensions 1.0, Generics and XBRL Formulas specification (including the XFI functions). The project allows an application to validate taxonomy and instance documents and is able to report any errors discovered. Error reports will document each element in a taxonomy or instance document that fails identifying the offending element and it will include a reference to the specification.
In the case of the dimensions and formula validation, the message will include the approrpriate error constant as defined in the XDT (dimensions) specification.

Reporting

The processor supports instance document rendering based on the information provided by the taxonomy author that is available in the presentation linkbase. This means you can create renderings of an instance document easily without complex code as the presentation linkbase defines the layout of each network.

There are many examples in the Digital Financial Reporting examples page including an example of the code to create a rendering of an instance document. You can also visit www.xbrlquery.org to see live examples or even create a rendering of your own instance documents. The case study example also includes an example of rendering Danish commerce authority taxonomy based instance documents.

This example shows that the XBRL formulas and calculation linkbase rules compute correctly based on the data available (rules that do not compute will be highlighted in red).

Rendering can also include details of the structure of the report and a breakdown of calculation and formula results.

Why PHP?

PHP is one of the most widely used languages by web servers. Independent and up-to-date research by w3techs shows PHP is used on approximately 82% of web sites where the server-side language is known. PHP is simple to integrate with all major web servers such as Apache, IIS and nginx. As a language it supports object-oriented development, polymorphism, namespaces, closures, and all the other features you'd expect of a modern programming language. This background means basing the project on PHP allows the platform to have strong HTTP-based server capabilities wide support and can be integrated into many widely used web platforms.

No compile step/No library chaos

Like all scripting languages PHP does not require a compile step. There are advantages to having a compile step the main one being that the compiler can perform additional code checks. But there are downsides as well. The ability to write code and run can boost productivity especially when prototyping ideas.

PHP versions and development tools

To work with the code you will need to use PHP 7.0 or later. PHP 8.0 is the latest and greatest and while it is supported, we currently develop using PHP 7.4.13. PHP 8.0. PHP 7.0 and later are much faster than earlier version and it is likely we have used features of the language that are only available in PHP 7.+.

We have not made use of functions that require PHP extensions not in the standard distribution.

PHP version relative performance

Every night our test suites are run which includes around 50,000 unit tests covering the XPath 2.0 conformance suite, the XBRL 2.1, Formulas and Dimensions conformance suites plus example compilation and report rendering. There a 12 discrete tests (each is a separate invocation of PHP) that in total take roughly 3-4 minutes to complete. Because they are intensive and exhaustive they provide some indication of which versions of PHP are best for this type of application.

The table below shows run times of the same tests on the same hardware with the same load. The times are minutes:seconds. x(f) means that opcache is enabled and that file caching is enabled so the benefit of the opcache persists across invocations. When opcache is enabled with file cache, the option to check timestamps is enabled. Note that opcode file file caching is not the same as web site file caching.

We have a winner! It's clear that the best performance is using version 8.0 with JIT (JIT requires the opcache is enabled) and opcode file caching.

The best version 7.2.6 time is 3:31. The best 7.4.13 time is 2.55, a 21% improvement. The best version 8.0 time is 2:25 a 35% improvement over version 7.2.6 and 18% better than 7.4.13.

Memory

The default memory limit defined in php.ini is suitable for sites generating regular web pages. However taxonomies can be large and the source makes liberal use of memory to boost performance. We recommend that when you execute examples that you ensure the memory limit is raised by including the following line at the top of the entry point file you use to run your program implementations:

ini_set( 'memory_limit', '512M' );

XBRL Formulas relies on XPath 2.0 and XPath 2.0 query execution relies on recurive calls. XPath 2.0 queries generated by XBRL Formulas can result in very heavily nested calls so it is recommended that the PHP 'xdebug.max_nesting_level' value is increased. This only affects while debugging using XDebug

ini_set('xdebug.max_nesting_level', 512);

We use phpDocumentor to create the source code documentation from the embedded comments.

We use Eclipse Neon and the PDT package as the IDE. XDebug is used to provide debugging support. At the time of writing the Zend debugger does not support PHP 7.0 while XDebug does. Nothing about the project or the source has any dependencies on these tools.

Dependencies

This project has dependencies on the following projects:

Example loading and validating an instance document

Using the project code is straight-forward. This simple example shows how to read an instance document file, check the validity instance document and report any discovered issues.

$instance = XBRL_Instance::FromInstanceDocument( 'my_instance_document.xml' );
$instance->validate();
if ( XBRL_Log::getInstance()->hasConformanceIssueWarning() )
{
    echo "Validation error\n";
}

License

This project is released under GPL version 3.0

What does this mean?

It means you can use the source code in any way you see fit. However, the source code for any changes you make must be made available to others and must be made available on the same terms as you receive the source code in this project: under a GPL v3.0 license. You must include the license of this project with any distribution of the source code whether the distribution includes all the source code or just part of it. For example, if you create a class that derives from one of the classes provided by this project - a new taxonomy class, perhaps - that is derivative.

What does this not mean?

It does not mean that any products you create that only use this source code must be released under GPL v3.0. If you create a budgeting application that uses the source code from this project to access data in instance documents, used by the budgeting application to transfer data, that is not derivative.

Contributing

We welcome contributions. See our contributions page for more information. If you do choose to contribute we will ask you to agree to our Contributor License Agreement (CLA). We will ask you to agree to the terms in the CLA to assure other users that the code they use is not going to be encumbered by a labyrinth of different license and patent liabilities. You are also urged to review our code of conduct.

Install

The project can be installed by composer. Assuming Composer is installed and a shortcut to the program is called 'composer' then the command to install this project is:

composer require lyquidity/xbrl:dev-master lyquidity/xpath2:dev-master lyquidity/utilities:dev-master lyquidity/xml:dev-master --prefer-dist

Or fork or download the repository. It will also be necessary to download and install the XML, utilities and pear/Log projects.

Getting started

The example.php file in the examples folder includes illustrations of using the classes.

Assuming you have installed the library using composer then this PHP application will run the examples:

<?php
// This MUST be set before the autoload because the XBRL class autoloaded uses it
global $use_xbrl_functions; $use_xbrl_functions = true;
require_once __DIR__ . '/vendor/autoload.php';
include __DIR__ . "/vendor/lyquidity/xbrl/examples/examples.php";

Read the getting started section in the Wiki where you will find more examples showing how the source can be used to query taxonomies instance documents and present their contents.

Inline XBRL transforms

The in-line XBRL specification describes how to embed XBRL instance data in an xHTML document. This project provides a validating iXBRL processor that can tvalidate and transform an iXBRL docment into an XBRL instance document. With an XBRL instance document it can be processes like any other.

Assuming you've been able to successfully run the code in Getting started then the following line will transform an input iXBRL document:

$documents = XBRL_Inline::createInstanceDocument( $name, $documentSet, $cacheLocation, $validate );

If the documents are validated successfully, the createInstanceDocument() function will return an array of of DOMDocument instances, one for each document in the input document set. The array will be indexed by the relevant targets. For more about targets, review the iXBRL specification.

The returned documents then be saved. The example below assumes there is a suitable output folder in $outputFolder and that $name used above is a useful filename. $target is the name defined in the iXBRL document to distinguish the output instance documents if more than one is to be produced.

foreach( $documents as $target => $document )
{
    $document->formatOutput = true;
    $xml = $document->saveXML();
    file_put_contents( "$outputFolder/$name-$target.xbrl", $xml );
}

lyquidity / xbrl

Maintainers

Details