Thursday, 3 July 2025

Generating llms.txt files with PHP

To make your website or project site provide LLM-friendly content, there's a new standard on the horizon named /llms.txt. It suggests adding a llms.txt containing human and LLM readable content in a fixed Markdown format.

Using llms-txt-php

In the PHP ecosystem, there's a relatively fresh PHP 📦 for writing, reading, and validating llms.txt Markdown files. The following snippet shows how to programmatically model such a file.
use Stolt\LlmsTxt\LlmsTxt;
use Stolt\LlmsTxt\Section;
use Stolt\LlmsTxt\Section\Link;

$section1 = (new Section())->name('Section name')
    ->addLink((new Link())->urlTitle('Link title')
        ->url('https://link_url')->urlDetails('Optional link details')
    );
$section2 = (new Section())->name('Optional')
    ->addLink((new Link())->urlTitle('Link title')
        ->url('https://link_url')
    );

$llmsTxt = (new LlmsTxt())->title('Test title')
  ->description('Test description')
  ->details('Test details')
  ->addSections([$section1, $section2])
  ->addSection($section2)
  ->toString();
Reading a llms.txt file and its parts are done the following way.
use Stolt\LlmsTxt\LlmsTxt;

$llmsText = (new LlmsTxt())->parse('/path/to/llmsTxt.md');

if ($llmsText->validate()) {
    $title = $llmsText->getTitle();
    $description = $llmsText->getDescription();
    $details = $llmsText->getDetails();
    $sections = $llmsText->getSections();
}
The llms-txt-php package also provides a minimalistic CLI to interact with an existing llms.txt file or boostrap it's creation. The following console output lists the currently available commands.
php bin/llms-txt list

llms-txt-php 1.6.1

Usage:
  command [options] [arguments]

Options:
  -h, --help            Display help for the given command. When no command is given display help for the list command
      --silent          Do not output any message
  -q, --quiet           Only errors are displayed. All other output is suppressed
  -V, --version         Display this application version
      --ansi|--no-ansi  Force (or disable --no-ansi) ANSI output
  -n, --no-interaction  Do not ask any interactive question
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Available commands:
  check-links  Check links of a llms txt file
  completion   Dump the shell completion script
  help         Display help for a command
  info         Get metadata info of a llms txt file
  init         Create an initial llms txt file
  list         List commands
  validate     Validate the given llms txt file

Friday, 17 January 2025

Developing a PHP package, 2025 edition

Since my blog post about the anatomy of a dope PHP package repository a lot has changed in the PHP ecosystem and it's surrounding tools. Time to reflect these changes and provide, yet another, guide to develop a PHP package in 2025.

Idea to implementation

The hardest part to get started is finding an idea worth implementing. Sources of inspiration can be visiting conferences, visiting local user groups, or the good old blog post reading. Also, daily business might generate an idea worth investing time. The idea might be very niche at first but that shouldn't stop you from going for it; the learnings might advance your self-confidence and career.

Getting started

Template repositories are a GitHub feature released in June 2019, which allows you and others to generate new Git repositories with the same directory structure and files based on the given template repository. Some templates to look into are spatie/package-skeleton-php and ergebnis/php-package-template. If you're developing a dedicated Laravel package spatie/package-skeleton-laravel is a good starting point. For checking if a package is PDS conform, there are for one the command-line tools of the pds/skeleton project or the package analyser which is a very opinionated validator/analyser.

Must have development tools

Some of the above-mentioned package templates come with preconfigured development tools like PHPUnit or the PHP Coding Standards Fixer. Other tools you might have to add or switch yourself. For example when you're developing a Laravel package you might want to replace PHPUnit with the traction gaining Pest testing framework and the PHP Coding Standards Fixer with Pint. For aiding you in development, tools like PHPStan and Rector have become quite mandatory. Also, worth checking out is their thriving extension ecosystem e.g. Larastan and rector-laravel. For the CI part, GitHub Actions have replaced former CI environments like Travis CI. So it might be a good invest to look into the usage and definition of GitHub Actions.

The automation of dependency updates for dependant Composer packages can currently be handled via Dependabot but might soon be replaced by Conductor.

Identifying wording or grammar mistakes in repositories might become easier once we can add Peck, a wrapper around GNU Aspell, into the mix.

To practice Datensparsamkeit you can add the lean-package-validator, which ensures that no unnecessary package artifacts end up in the distributed dist archives. For some additional development tools, it's also worth checking out Tomas Votruba's tools selection.

Filling the gaps with polyfills

In case you're stuck with an older PHP or PHPUnit version, there are polyfills (polyfill-php83, polyfill-php84, and PHPUnit-Polyfills) available for filling the gaps; which eases later migrations to current PHP or PHPUnit versions.

Registering your package

After having finished the implementation of the package you need to Git tag it, ideally following semantic versioning, and submit it at Packagist, the main PHP package repository.

Package promotion

To promote your package start with a care- and heartful crafted README.md file. If you want to rise the visibility of your package invest some time to create a catching logo or delegate its creation to your bubble. Other channels to promote your work are X, the awesome PHP list, and giving a presentation about it at your local user group.

Thursday, 23 November 2023

Automating the backslash prefixing for native PHP function calls

After reading the blog post Why does a backslash prefix improve PHP function call performance by Jeroen Deviaene I was looking for a way to automate it for the codebase of the Lean Package Validator, to shave off some miliseconds for it's CLI. The PHP Coding Standards Fixer has a rule named native_function_invocation which does the very exact task.

Configuring the PHP Coding Standards Fixer

.php-cs-fixer.php
<?php

use PhpCsFixer\Config;
use PhpCsFixer\Finder;

$finder = Finder::create()
    ->in([__DIR__, __DIR__ . DIRECTORY_SEPARATOR . 'tests']);

$rules = [
    'psr_autoloading' => false,
    '@PSR2' => true,
    'phpdoc_order' => true,
    'ordered_imports' => true,
    'native_function_invocation' => [
        'include' => ['@internal'],
        'exclude' => ['file_put_contents']
    ]
];

$cacheDir = \getenv('HOME') ? \getenv('HOME') : __DIR__;

$config = new Config();

return $config->setRules($rules)
    ->setFinder($finder)
    ->setCacheFile($cacheDir . '/.php-cs-fixer.cache');
To make this rule executeable I needed to add the --allow-risky=yes option to the PHP Coding Standards Fixer calls in the two dedicated Composer scripts shown next.

composer.json
"scripts": {
    "lpv:test": "phpunit",
    "lpv:test-with-coverage": "export XDEBUG_MODE=coverage && phpunit --coverage-html coverage-reports",
    "lpv:cs-fix": "php-cs-fixer --allow-risky=yes fix . -vv || true",
    "lpv:cs-lint": "php-cs-fixer fix --diff --stop-on-violation --verbose --dry-run --allow-risky=yes",
    "lpv:configure-commit-template": "git config --add commit.template .gitmessage",
    "lpv:application-version-guard": "php bin/application-version --verify-tag-match=bin",
    "lpv:application-phar-version-guard": "php bin/application-version --verify-tag-match=phar",
    "lpv:static-analyse": "phpstan analyse --configuration phpstan.neon.dist", 
    "lpv:validate-gitattributes": "bin/lean-package-validator validate"
},
After running the lpv:cs-fix Composer script the first time the tests of the system under test started failing due to file_put_contents being prefixed with a backslash when using phpmock\MockBuilder's setName method, so I had to exclude it as shown in the PHP Coding Standards Fixer configuration above.

Monday, 15 January 2018

Documenting Composer scripts

For open source projects I'm involved with, I developed the habit to define, and document the steady growing amount of repository and build utilities via Composer scripts. Having Composer scripts available makes it trivial to define aliases or shortcuts for complex and hard to remember CLI calls. It also lowers the barrier for contributors to start using these tools while helping out with fixing bugs or providing new features. Finally they're also simplifying build scripts by stashing away complexity.

Defining Composer scripts

If you've already defined or worked with Composer scripts or even their npm equivalents you can skip this section, otherwise the next code snippet allows you to study how to define these. The here defined Composer scripts range from simple CLI commands with set options (e.g. the test-with-coverage script) to more complex build utility tools (i.e. the application-version-guard script) which are extracted into specific CLI commands to avoid cluttering up the composer.json or even the .travis.yml.

composer.json
{
  "scripts": {
    "test": "phpunit",
    "test-with-coverage": "phpunit --coverage-html coverage-reports",
    "cs-fix": "php-cs-fixer fix . -vv || true",
    "cs-lint": "php-cs-fixer fix --diff --stop-on-violation --verbose --dry-run",
    "configure-commit-template": "git config --add commit.template .gitmessage",
    "application-version-guard": "php bin/application-version --verify-tag-match"
  }
}

Describing Composer scripts

Since Composer 1.6.0 it's possible to set custom script descriptions via the scripts-descriptions element like shown next. It's to point out here that the name of a description has to match the name of a defined custom Composer script to be recognised at runtime. On another note it's to mention that the description should be worded in simple present to align with the other Composer command descriptions.

composer.json
{
  "scripts-descriptions": {
    "test": "Runs all tests.",
    "test-with-coverage": "Runs all tests and measures code coverage.",
    "cs-fix": "Fixes coding standard violations.",
    "cs-lint": "Checks for coding standard violations.",
    "configure-commit-template": "Configures a local commit message template.",
    "application-version-guard": "Checks that the application version matches the given Git tag."
  },
  "scripts": {
    "test": "phpunit",
    "test-with-coverage": "phpunit --coverage-html coverage-reports",
    "cs-fix": "php-cs-fixer fix . -vv || true",
    "cs-lint": "php-cs-fixer fix --diff --stop-on-violation --verbose --dry-run",
    "configure-commit-template": "git config --add commit.template .gitmessage",
    "application-version-guard": "php bin/application-version --verify-tag-match"
  }
}
Now when running $ composer via the terminal the descriptions of defined custom scripts will show up sorted in into the list of available commands, which makes it very hard to spot the Composer scripts of the package at hand. Luckily Composer scripts can also be namespaced.

Namespacing Composer scripts

To namespace (i.e. some-namespace) the custom Composer scripts for any given package define the script names with a namespace prefix as shown next. As the chances are very high that you will be using the one or other Composer script several times, while working on the package, it's recommended to use a short namespace like in the range from two to four characters.

composer.json
{
  "scripts": {
    "some-namespace:test": "phpunit",
    "some-namespace:test-with-coverage": "phpunit --coverage-html coverage-reports",
    "some-namespace:cs-fix": "php-cs-fixer fix . -vv || true",
    "some-namespace:cs-lint": "php-cs-fixer fix --diff --stop-on-violation --verbose --dry-run",
    "some-namespace:configure-commit-template": "git config --add commit.template .gitmessage",
    "some-namespace:application-version-guard": "php bin/application-version --verify-tag-match"
  }
}
Now this time when running $ composer via the terminal the defined custom scripts will show up in the list of available commands in a namespaced manner giving an immediate overview of the available Composer script of the package at hand.

$ composer
 ... ommitted content
Available commands:
  ... ommitted content
 some-namespace
  some-namespace:application-version-guard  Checks that the application version matches the given Git tag.
  some-namespace:configure-commit-template  Configures a local commit message template.
  some-namespace:cs-fix                     Fixes coding standard violations.
  some-namespace:cs-lint                    Checks for coding standard violations.
  some-namespace:test                       Runs all tests.
  some-namespace:test-with-coverage         Runs all tests and measures code coverage.
To use any namespaced Composer script, e.g. to fix coding standard violations after a substantial refactoring, it has to be called with its namespace e.g.$ composer some-namespace:cs-fix, which is the one disadavantage of Composer script namespacing.

Saturday, 25 March 2017

Keeping your CLI integration tests green on Windows

Lately on a Windows system, some failing integration tests for CLI commands utilising the Symfony Console component caused me some blip headaches by PHPUnit insisting that two strings are not identical due to different line endings. The following post documents the small steps I took to overcome these headaches.

First the assertion message produced by the failing test, see the console output below, got me thinking it might be caused by different encodings and line endings; though the project was utilising an .editorconfig from the early start and the related files were all encoded correctly and had the configured line endings. The Git configuration e.g. core.autocrlf=input also was as it should be.

1) Stolt\LeanPackage\Tests\Commands\InitCommandTest::createsExpectedDefaultLpvFile
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
 #Warning: Strings contain different line endings!
-Created default 'C:\Users\stolt\AppData\Local\Temp\lpv\.lpv' file.
+Created default 'C:\Users\stolt\AppData\Local\Temp\lpv\.lpv' file.
Another deeper look at the CommandTester class yielded that it’s possible to disable the command output decoration and also to normalise the command output. So a change of the SUT preparation and a normalisation of the console output, visualised via a git diff -U10, brought the solution for this particular test.

diff --git a/tests/Commands/InitCommandTest.php b/tests/Commands/InitCommandTest.php
index 58e7114..fb406f3 100644
--- a/tests/Commands/InitCommandTest.php
+++ b/tests/Commands/InitCommandTest.php
@@ -48,21 +48,21 @@ class InitCommandTest extends TestCase
     /**
      * @test
      */
     public function createsExpectedDefaultLpvFile()
     {
         $command = $this->application->find('init');
         $commandTester = new CommandTester($command);
         $commandTester->execute([
             'command' => $command->getName(),
             'directory' => WORKING_DIRECTORY,
-        ]);
+        ], ['decorated' => false]);

// ommitted  code

-        $this->assertSame($expectedDisplay, $commandTester->getDisplay());
+        $this->assertSame($expectedDisplay, $commandTester->getDisplay(true));
         $this->assertTrue($commandTester->getStatusCode() == 0);
         $this->assertFileExists($expectedDefaultLpvFile);
Since the SUT had a lot of integration test for its CLI commands, the lazy me took the shortcut to extend the CommandTester and using it, with desired defaults set, instead of changing all of the related command instantiations.

<?php

namespace SUT\Tests;

use Symfony\Component\Console\Tester\CommandTester as ConsoleCommandTester;

class CommandTester extends ConsoleCommandTester
{
    /**
     * Gets the display returned by the last execution of the command.
     *
     * @param bool $normalize Whether to normalize end of lines to \n or not
     *
     * @return string The display
     */
    public function getDisplay($normalize = true)
    {
        return parent::getDisplay($normalize);
    }

    /**
     * Executes the command.
     *
     * Available execution options:
     *
     *  * interactive: Sets the input interactive flag
     *  * decorated:   Sets the output decorated flag
     *  * verbosity:   Sets the output verbosity flag
     *
     * @param array $input   An array of command arguments and options
     * @param array $options An array of execution options
     *
     * @return int The command exit code
     */
    public function execute(
        array $input,
        array $options = ['decorated' => false]
    ) {
        return parent::execute($input, $options);
    }
}
So it's a yay for green CLI command integration tests on Windows from here on. Another measure for the SUT would be to enable Continuous Integration on a Windows system via AppVeyor, but that’s a task for another commit 85bdf22.