Showing posts with label CLI. Show all posts
Showing posts with label CLI. Show all posts

Thursday, 3 July 2025

Generating llms.txt files with PHP

To make your website or project site provide LLM-friendly content, there's a new standard on the horizon named /llms.txt. It suggests adding a llms.txt containing human and LLM readable content in a fixed Markdown format.

Using llms-txt-php

In the PHP ecosystem, there's a relatively fresh PHP 📦 for writing, reading, and validating llms.txt Markdown files. The following snippet shows how to programmatically model such a file.
use Stolt\LlmsTxt\LlmsTxt;
use Stolt\LlmsTxt\Section;
use Stolt\LlmsTxt\Section\Link;

$section1 = (new Section())->name('Section name')
    ->addLink((new Link())->urlTitle('Link title')
        ->url('https://link_url')->urlDetails('Optional link details')
    );
$section2 = (new Section())->name('Optional')
    ->addLink((new Link())->urlTitle('Link title')
        ->url('https://link_url')
    );

$llmsTxt = (new LlmsTxt())->title('Test title')
  ->description('Test description')
  ->details('Test details')
  ->addSections([$section1, $section2])
  ->addSection($section2)
  ->toString();
Reading a llms.txt file and its parts are done the following way.
use Stolt\LlmsTxt\LlmsTxt;

$llmsText = (new LlmsTxt())->parse('/path/to/llmsTxt.md');

if ($llmsText->validate()) {
    $title = $llmsText->getTitle();
    $description = $llmsText->getDescription();
    $details = $llmsText->getDetails();
    $sections = $llmsText->getSections();
}
The llms-txt-php package also provides a minimalistic CLI to interact with an existing llms.txt file or boostrap it's creation. The following console output lists the currently available commands.
php bin/llms-txt list

llms-txt-php 1.6.1

Usage:
  command [options] [arguments]

Options:
  -h, --help            Display help for the given command. When no command is given display help for the list command
      --silent          Do not output any message
  -q, --quiet           Only errors are displayed. All other output is suppressed
  -V, --version         Display this application version
      --ansi|--no-ansi  Force (or disable --no-ansi) ANSI output
  -n, --no-interaction  Do not ask any interactive question
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Available commands:
  check-links  Check links of a llms txt file
  completion   Dump the shell completion script
  help         Display help for a command
  info         Get metadata info of a llms txt file
  init         Create an initial llms txt file
  list         List commands
  validate     Validate the given llms txt file

Monday, 15 January 2018

Documenting Composer scripts

For open source projects I'm involved with, I developed the habit to define, and document the steady growing amount of repository and build utilities via Composer scripts. Having Composer scripts available makes it trivial to define aliases or shortcuts for complex and hard to remember CLI calls. It also lowers the barrier for contributors to start using these tools while helping out with fixing bugs or providing new features. Finally they're also simplifying build scripts by stashing away complexity.

Defining Composer scripts

If you've already defined or worked with Composer scripts or even their npm equivalents you can skip this section, otherwise the next code snippet allows you to study how to define these. The here defined Composer scripts range from simple CLI commands with set options (e.g. the test-with-coverage script) to more complex build utility tools (i.e. the application-version-guard script) which are extracted into specific CLI commands to avoid cluttering up the composer.json or even the .travis.yml.

composer.json
{
  "scripts": {
    "test": "phpunit",
    "test-with-coverage": "phpunit --coverage-html coverage-reports",
    "cs-fix": "php-cs-fixer fix . -vv || true",
    "cs-lint": "php-cs-fixer fix --diff --stop-on-violation --verbose --dry-run",
    "configure-commit-template": "git config --add commit.template .gitmessage",
    "application-version-guard": "php bin/application-version --verify-tag-match"
  }
}

Describing Composer scripts

Since Composer 1.6.0 it's possible to set custom script descriptions via the scripts-descriptions element like shown next. It's to point out here that the name of a description has to match the name of a defined custom Composer script to be recognised at runtime. On another note it's to mention that the description should be worded in simple present to align with the other Composer command descriptions.

composer.json
{
  "scripts-descriptions": {
    "test": "Runs all tests.",
    "test-with-coverage": "Runs all tests and measures code coverage.",
    "cs-fix": "Fixes coding standard violations.",
    "cs-lint": "Checks for coding standard violations.",
    "configure-commit-template": "Configures a local commit message template.",
    "application-version-guard": "Checks that the application version matches the given Git tag."
  },
  "scripts": {
    "test": "phpunit",
    "test-with-coverage": "phpunit --coverage-html coverage-reports",
    "cs-fix": "php-cs-fixer fix . -vv || true",
    "cs-lint": "php-cs-fixer fix --diff --stop-on-violation --verbose --dry-run",
    "configure-commit-template": "git config --add commit.template .gitmessage",
    "application-version-guard": "php bin/application-version --verify-tag-match"
  }
}
Now when running $ composer via the terminal the descriptions of defined custom scripts will show up sorted in into the list of available commands, which makes it very hard to spot the Composer scripts of the package at hand. Luckily Composer scripts can also be namespaced.

Namespacing Composer scripts

To namespace (i.e. some-namespace) the custom Composer scripts for any given package define the script names with a namespace prefix as shown next. As the chances are very high that you will be using the one or other Composer script several times, while working on the package, it's recommended to use a short namespace like in the range from two to four characters.

composer.json
{
  "scripts": {
    "some-namespace:test": "phpunit",
    "some-namespace:test-with-coverage": "phpunit --coverage-html coverage-reports",
    "some-namespace:cs-fix": "php-cs-fixer fix . -vv || true",
    "some-namespace:cs-lint": "php-cs-fixer fix --diff --stop-on-violation --verbose --dry-run",
    "some-namespace:configure-commit-template": "git config --add commit.template .gitmessage",
    "some-namespace:application-version-guard": "php bin/application-version --verify-tag-match"
  }
}
Now this time when running $ composer via the terminal the defined custom scripts will show up in the list of available commands in a namespaced manner giving an immediate overview of the available Composer script of the package at hand.

$ composer
 ... ommitted content
Available commands:
  ... ommitted content
 some-namespace
  some-namespace:application-version-guard  Checks that the application version matches the given Git tag.
  some-namespace:configure-commit-template  Configures a local commit message template.
  some-namespace:cs-fix                     Fixes coding standard violations.
  some-namespace:cs-lint                    Checks for coding standard violations.
  some-namespace:test                       Runs all tests.
  some-namespace:test-with-coverage         Runs all tests and measures code coverage.
To use any namespaced Composer script, e.g. to fix coding standard violations after a substantial refactoring, it has to be called with its namespace e.g.$ composer some-namespace:cs-fix, which is the one disadavantage of Composer script namespacing.

Saturday, 25 March 2017

Keeping your CLI integration tests green on Windows

Lately on a Windows system, some failing integration tests for CLI commands utilising the Symfony Console component caused me some blip headaches by PHPUnit insisting that two strings are not identical due to different line endings. The following post documents the small steps I took to overcome these headaches.

First the assertion message produced by the failing test, see the console output below, got me thinking it might be caused by different encodings and line endings; though the project was utilising an .editorconfig from the early start and the related files were all encoded correctly and had the configured line endings. The Git configuration e.g. core.autocrlf=input also was as it should be.

1) Stolt\LeanPackage\Tests\Commands\InitCommandTest::createsExpectedDefaultLpvFile
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
 #Warning: Strings contain different line endings!
-Created default 'C:\Users\stolt\AppData\Local\Temp\lpv\.lpv' file.
+Created default 'C:\Users\stolt\AppData\Local\Temp\lpv\.lpv' file.
Another deeper look at the CommandTester class yielded that it’s possible to disable the command output decoration and also to normalise the command output. So a change of the SUT preparation and a normalisation of the console output, visualised via a git diff -U10, brought the solution for this particular test.

diff --git a/tests/Commands/InitCommandTest.php b/tests/Commands/InitCommandTest.php
index 58e7114..fb406f3 100644
--- a/tests/Commands/InitCommandTest.php
+++ b/tests/Commands/InitCommandTest.php
@@ -48,21 +48,21 @@ class InitCommandTest extends TestCase
     /**
      * @test
      */
     public function createsExpectedDefaultLpvFile()
     {
         $command = $this->application->find('init');
         $commandTester = new CommandTester($command);
         $commandTester->execute([
             'command' => $command->getName(),
             'directory' => WORKING_DIRECTORY,
-        ]);
+        ], ['decorated' => false]);

// ommitted  code

-        $this->assertSame($expectedDisplay, $commandTester->getDisplay());
+        $this->assertSame($expectedDisplay, $commandTester->getDisplay(true));
         $this->assertTrue($commandTester->getStatusCode() == 0);
         $this->assertFileExists($expectedDefaultLpvFile);
Since the SUT had a lot of integration test for its CLI commands, the lazy me took the shortcut to extend the CommandTester and using it, with desired defaults set, instead of changing all of the related command instantiations.

<?php

namespace SUT\Tests;

use Symfony\Component\Console\Tester\CommandTester as ConsoleCommandTester;

class CommandTester extends ConsoleCommandTester
{
    /**
     * Gets the display returned by the last execution of the command.
     *
     * @param bool $normalize Whether to normalize end of lines to \n or not
     *
     * @return string The display
     */
    public function getDisplay($normalize = true)
    {
        return parent::getDisplay($normalize);
    }

    /**
     * Executes the command.
     *
     * Available execution options:
     *
     *  * interactive: Sets the input interactive flag
     *  * decorated:   Sets the output decorated flag
     *  * verbosity:   Sets the output verbosity flag
     *
     * @param array $input   An array of command arguments and options
     * @param array $options An array of execution options
     *
     * @return int The command exit code
     */
    public function execute(
        array $input,
        array $options = ['decorated' => false]
    ) {
        return parent::execute($input, $options);
    }
}
So it's a yay for green CLI command integration tests on Windows from here on. Another measure for the SUT would be to enable Continuous Integration on a Windows system via AppVeyor, but that’s a task for another commit 85bdf22.