Saturday, 4 July 2009

Scaffolding, implementing and using project specific Zend_Tool_Project_Providers

Working on a project involving several legacy data migration tasks, I got curious what the Zend_Tool_Project component of the Zend Framework offers to create project specific providers for the above mentioned tasks or ones of similar nature. Therefore the following post will try to show how these providers can be developed in an iterative manner by scaffolding them via the capabilities of the Zend_Tool_Project ProjectProvider provider, enlived with action/task logic, and be used in the project scope.

Scaffolding project specific providers

All following steps assume there is a project available i.e. recordshelf initially created with the Zend_Tool_Project Project provider and that the forthcoming commands are issued from the project root directory against the zf command line client. The scaffolding of a project specific provider can be triggered via the create action of the ProjectProvider provider by passing in the name of the provider i.e. csv and it's intended actions. As the next console snippet shows it's
possible to specify several actions as a comma separated list.
sudo zf create project-provider csv importSpecials,importSummersale
After running the command the project's profile .zfproject.xml has been modified and a new providers directory exists in the project root directory containing the scaffolded Csv provider. The next code snippet shows the initial Csv provider class skeleton and its two empty action methods named importSpecials and importSummersale. At the point of this writing, using the Zend Framework 1.8.4 and PHP 5.2.10 on a Mac OS X system the generated Csv provider code or the mapping in the .zfproject.xml is incorrect, but can be fixed by renaming the class from CsvProvider to Csv.
<?php

require_once 'Zend/Tool/Project/Provider/Abstract.php';
require_once 'Zend/Tool/Project/Provider/Exception.php';

class CsvProvider extends Zend_Tool_Project_Provider_Abstract
{

public function importSpecials()
{
/** @todo Implementation */
}

public function importSummersale()
{
/** @todo Implementation */
}


}

Implementing the action logic

Having the project provider class skeleton ready to get going, it's time to enliven the actions with their intended features by using either other components of the Zend Framework, any suitable third party library or plain-vanilla PHP. For the sake of brevity I decided to implement only the importSpecials action which transforms the data of a known CSV file structure into a relevant database table. The CSV parsing steps shown next might not be that sophisticated, as their sole purpose is to illustrate an exemplary implementation of a project specific provider action.
<?php

require_once 'Zend/Tool/Project/Provider/Abstract.php';
require_once 'Zend/Tool/Project/Provider/Exception.php';

class Csv extends Zend_Tool_Project_Provider_Abstract
{
private function _isProjectProviderSupportedInProject(Zend_Tool_Project_Profile $profile,
$projectProviderName)
{
$projectProviderResource = $this->_getProjectProfileResource($profile,
$projectProviderName);
return $projectProviderResource instanceof Zend_Tool_Project_Profile_Resource;
}

private function _isActionSupportedByProjectProvider(Zend_Tool_Project_Profile $profile,
$projectProviderName, $actionName)
{
$projectProviderResource = $this->_getProjectProfileResource($profile,
$projectProviderName);
$projectProviderAttributes = $projectProviderResource->getContext()
->getPersistentAttributes();
return in_array($actionName, explode(',', $projectProviderAttributes['actionNames']));
}

private function _getProjectProfileResource(Zend_Tool_Project_Profile $profile,
$projectProviderName)
{
$profileSearchParams[] = 'ProjectProvidersDirectory';
$profileSearchParams['ProjectProviderFile'] =
array('projectProviderName' => strtolower($projectProviderName));
return $profile->search($profileSearchParams);
}

public function importSpecials($csvFile, $env = 'development')
{
$relatedTablename = 'specials';

if (!$this->_isProjectProviderSupportedInProject($profile, __CLASS__)) {
throw new Exception("ProjectProvider Csv is not supported in this project.");
}
if (!$this->_isActionSupportedByProjectProvider($profile, __CLASS__, __FUNCTION__)) {
$exceptionMessage = "Action 'importSpecials' is not supported by "
. "the Csv ProjectProvider in this project.";
throw new Exception($exceptionMessage);
}

if (!file_exists($csvFile)) {
throw new Exception("Given csv-file '{$csvFile}' doesn't exist.");
}

$importEnvironment = trim($env);
if ($importEnvironment !== 'development' && $importEnvironment !== 'production') {
throw new Exception("Unsupported environment '{$importEnvironment}' provided.");
}

$csvHandle = fopen($csvFile, "r");

if (!$csvHandle) {
throw new Exception("Unable to open given csv-file '{$csvFile}'.");
}

$config = new Zend_Config_Ini('./application/configs/application.ini',
$importEnvironment);
$db = Zend_Db::factory($config->database);

$db->query("TRUNCATE TABLE {$relatedTablename}");
echo "Truncated the project '{$relatedTablename}' database table." . PHP_EOL;

$rowCount = $insertCount = 0;

while (($csvLine = fgetcsv($csvHandle)) !== false) {
if ($rowCount > 0) {
$insertRow = array(
'product_name' => $csvLine[0],
'product_image_path' => $csvLine[1],
'price' => $csvLine[2],
'special_until' => $csvLine[3]
);
$db->insert($relatedTablename, $insertRow);
++$insertCount;
}
++$rowCount;
}
fclose($csvHandle);
$importMessage = "Imported {$insertCount} rows into the project "
. "'{$relatedTablename}' database table.";
echo $importMessage;
}

...
}

Making providers and actions pretendable

To make project specific providers its actions pretendable and thereby providing some kind of user documentation the provider classes have to implement a marker interface called Zend_Tool_Framework_Provider_Pretendable. For making a action of a provider pretendable and giving some feedback to the user, the request is checked if the action has been issued in the pretend mode; which is possible by adding -p option to the issued zf command line client command. The next code snippet shows how the above stated Csv provider and its importSpecials action is made pretendable.
<?php

require_once 'Zend/Tool/Project/Provider/Abstract.php';
require_once 'Zend/Tool/Project/Provider/Exception.php';

class Csv extends Zend_Tool_Project_Provider_Abstract implements
Zend_Tool_Framework_Provider_Pretendable

{

public function importSpecials($csvFile, $env = 'development')
{
...

if ($this->_registry->getRequest()->isPretend()) {
$pretendMessage = "I would import the specials data provided in {$csvFile} "
. "into the project '{$relatedTablename}' database table.";
echo $pretendMessage;
} else {
...
}

}
...
}

Using project specific providers

To use the bundled up capabilities of project specific providers, these have to made accessable to the zf command line client by putting them in the include_path. Currently I discovered no best practice for doing so only for single project scopes and simply added the path to the project to my php.ini and thereby global include_path; another approach might be to add the project name as a prefix to the Provider. After doing so it's possible to get an overview of all with the Zend_Tool_Project shipped providers plus the project specific providers and their offered actions by issuing the zf --help command as shown in the next screenshot. To ensure that project specific providers and its actions are only runnable in projects which support them, it is necessary to check if these and the offered action exists as resources in the project its profile .zfproject.xml file as shown in the implementation of the importSpecials action in one of above code snippets.

Provider overview

As shown in the previous screenshot the first character of the project specific providers are omitted, this is another minor bug which might be fixed in one of the forthcoming Zend Framework releases. The current workaround for this issue is simply to type the command exactly as shown in the help. The outro screenshot shows how the import-specials action of the project specific Csv provider is issued against the zf command line client and its provided user feedback after an successfull import against the projects development database.

Calling the import-specials action

Sunday, 10 May 2009

Testing Phing buildfiles with PHPUnit

While transforming some of the Ant buildfile refactorings described in Julian Simpson's seminal essay into a Phing context, it felt plainly wrong that I didn't have any tests for the buildfile to back me up on obtaining the pristine behaviour throughout the process. While Ant users can rely on an Apache project called AntUnit there are currently no tailor-made tools available for testing or verifying Phing buildfiles. Therefor I took a weekend off, locked myself in the stuffy lab, and explored the abilities to test Phing buildfiles respectively their included properties, targets and tasks with the PHPUnit testing framework. In case you'd like to take a peek at the emerged lab jottings, keep on scanning.

Introducing the buildfile under test

The buildfile that will be used as an example is kept simple, and contains several targets ranging from common ones like initializing the build environment by creating the necessary directories to more specific ones like pulling an external artifact from GitHub. To get an overview of the buildfile under test have a look at the following listing.
<?xml version="1.0" encoding="UTF-8"?>
<project name="test-example" default="build" basedir=".">

<property name="project.basedir"
value="." override="true" />
<property name="github.repos.dir"
value="${project.basedir}/build/github-repos" override="true" />

<target name="clean" depends="clean-github-repos"
description="Removes runtime build artifacts">
<delete dir="${project.basedir}/build" includeemptydirs="true"
verbose="false" failonerror="true" />
<delete dir="${project.basedir}/build/reports"
includeemptydirs="true" verbose="false" failonerror="true" />
</target>

<target name="clean-github-repos" description="Removes runtime build artifacts">
<delete dir="${github.repos.dir}" includeemptydirs="true" failonerror="true" />
</target>

<target name="-log-build" description="A private target which should only be invoked internally">
<!-- omitted -->
</target>

<target name="build" depends="clean" description="Builds the distributable product">
<!-- omitted -->
</target>

<target name="database-setup" description="Sets up the database structure">
<!-- omitted -->
</target>

<target name="init" description="Initalizes the build by creating directories etc">
<mkdir dir="${project.basedir}/build/logs/performance/" />
<mkdir dir="${project.basedir}/build/doc" />
<mkdir dir="${project.basedir}/build/reports/phploc" />
</target>

<target name="init-ad-hoc-tasks"
description="Initalizes the ad hoc tasks for reusability in multiple targets">
<adhoc-task name="github-clone"><![CDATA[
class Github_Clone extends Task {
private $repository = null;
private $destDirectory = null;

function setRepos($repository) {
$this->repository = $repository;
}
function setDest($destDirectory) {
$this->destDirectory = $destDirectory;
}
function main() {
// Get project name from repos Uri
$projectName = str_replace('.git', '',
substr(strrchr($this->repository, '/'), 1));

$gitCommand = 'git clone ' . $this->repository . ' ' .
$this->destDirectory . '/' . $projectName;

exec(escapeshellcmd($gitCommand), $output, $return);

if ($return !== 0) {
throw new BuildException('Git clone failed');
}
$logMessage = 'Cloned Git repository ' . $this->repository .
' into ' . $this->destDirectory . '/' . $projectName;
$this->log($logMessage);
}
}
]]></adhoc-task>

<echo message="Intialized github-clone ad hoc task." />
</target>

<target name="github" depends="init-ad-hoc-tasks, clean-github-repos"
description="Clones given repositories from GitHub">
<github-clone repos="git://github.com/raphaelstolt/phploc-phing.git"
dest="${github.repos.dir}" />
</target>
</project>

Testing the buildfile

All tests for the buildfile under test will be bundled, like 'normal' tests, in a class i.e. BuildfileTest extending the PHPUnit_Framework_TestCase class. When testing buildfiles it's possible to build some tests around the actual buildfile XML structure, by utilizing the xpath method of PHP's SimpleXMLElement class and asserting against the XPath query results, or around the dispatching of specific targets and asserting against the expected build artifacts. Furthermore these two identified groups, structure and artifact, can be used to organize the accumulating tests via PHPUnit's @group annotation.

To be able to dispatch specific build targets and feed them with properties if necessary I additionally developed a very basic build runner shown in the next code listing.
<?php
class Phing_Buildfile_Runner {

private $_buildfilePath = null;

public function __construct($buildfilePath) {
if (!file_exists($buildfilePath)) {
throw new Exception("Buildfile '{$buildfilePath}' doesn't exist");
}
$this->buildfilePath = realpath($buildfilePath);
}
public function runTarget($targets = array(), $properties = array()) {
$runTargetCommand = "phing "
. "-f {$this->buildfilePath} ";
if (count($targets) > 0) {
foreach ($targets as $target) {
$runTargetCommand.= $target . " ";
}
}
if (count($properties) > 0) {
foreach ($properties as $property => $value) {
$runTargetCommand.= "-D{$property}={$value} ";
}
}
exec(escapeshellcmd($runTargetCommand), $output, $return);
return array('output' => $output, 'return' => $return);
}
}
Out of the box PHPUnit's assertion pool provides all the utilities to test buildfiles; although it would be cleaner to create domain specfic assertions for this testing domain this technique will be ignored for the sake of brevity.

After an initial 1000ft view on how to test buildfiles let's jump into the actual testing of a structural aspect of the buildfile under test. The test to come shows how to verify that a clean target is defined for playing along in the build orchestra by querying a XPath expression against the buildfile XML and asserting that a result is available.
/**
* @test
* @group structure
*/
public function buildfileShouldContainACleanTarget() {
$xml = new SimpleXMLElement($this->_buildfileXml);
$cleanElement = $xml->xpath("//target[@name='clean']");
$this->assertTrue(count($cleanElement) > 0,
"Buildfile doesn't contain a clean target");
}
The next artifactual test raises the bar an inch, by verifying that the defined init target of the build does initialize the build environment correctly, or to pick up the orchestra metaphor again that the specific instrument plays along and holds the directed tone. Therefor the build runner executes the target and afterwards asserts a list of expected artifacts against the current state of the build process.
/**
* @test
* @group artifact
*/
public function initTargetShouldCreateInitialBuildArtifacts() {
$this->_isTearDownNecessary = true;
$this->_buildfileRunner->runTarget(array('init'));
$expectedInitArtifacts = array(
"{$this->_buildfileBasedir}/build",
"{$this->_buildfileBasedir}/build/logs/performance/",
"{$this->_buildfileBasedir}/build/doc",
"{$this->_buildfileBasedir}/build/reports"
);

foreach ($expectedInitArtifacts as $artifact) {
$this->assertFileExists($artifact,
"Expected file '{$artifact}' doesn't exist");
}
}
The next code listing shows the whole picture of the BuildfileTest class containing additional test methods verifying different aspects of the buildfile under test and also the innards of the setup and teardown method.

The main assignment of the setup method is to load the XML of the buildfile under test and to intialize the build runner so an instance is available for an use in artifactual tests. The teardown method its sole responsibility is to reset the build state by running the clean target of the buildfile.
<?php
require_once 'PHPUnit/Framework.php';
require_once 'Phing/Buildfile/Runner.php';

class ExampleBuildfileTest extends PHPUnit_Framework_TestCase {

protected $_buildfileXml = null;
protected $_buildfileName = null;
protected $_buildfileBasedir = null;
protected $_buildfileRunner = null;
protected $_isTearDownNecessary = false;

protected function setUp() {
$this->_buildfileName = realpath('../../build.xml');
$this->_buildfileBasedir = dirname($this->_buildfileName);
$this->_buildfileXml = file_get_contents($this->_buildfileName);
$this->_buildfileRunner = new Phing_Buildfile_Runner(
$this->_buildfileName);
}

protected function tearDown() {
if ($this->_isTearDownNecessary) {
$this->_buildfileRunner->runTarget(array('clean'));
}
}

/**
* @test
* @group structure
*/
public function targetBuildShouldBeTheDefaultTarget() {
$xml = new SimpleXMLElement($this->_buildfileXml);
$xpath = "//@default";
$defaultElement = $xml->xpath($xpath);
$this->assertSame('build', trim($defaultElement[0]->default),
"Buildfile doesn't have a default target named 'build'");
}
/**
* @test
* @group structure
*/
public function propertyGithubReposDirShouldBeSet() {
$xml = new SimpleXMLElement($this->_buildfileXml);
$xpath = "//property[@name='github.repos.dir']/@value";
$valueElement = $xml->xpath($xpath);
$this->assertTrue($valueElement[0] instanceof SimpleXMLElement,
"Buildfile doesn't contain a 'github.repos.dir' property");
$this->assertGreaterThan(1, strlen($valueElement[0]->value));
}
/**
* @test
* @group structure
*/
public function buildfileShouldContainACleanTarget() {
$xml = new SimpleXMLElement($this->_buildfileXml);
$cleanElement = $xml->xpath("//target[@name='clean']");
$this->assertTrue(count($cleanElement) > 0,
"Buildfile doesn't contain a clean target");
}
/**
* @test
* @group structure
*/
public function targetLogBuildShouldBeAPrivateOne() {
$xml = new SimpleXMLElement($this->_buildfileXml);
$nameElement = $xml->xpath("//target[@name='-log-build']");
$this->assertTrue(count($nameElement) > 0,
'Log build target is not a private target');
}
/**
* @test
* @group structure
*/
public function targetBuildShouldDependOnCleanTarget() {
$xml = new SimpleXMLElement($this->_buildfileXml);
$xpath = "//target[@name='build']/@depends";
$dependElement = $xml->xpath($xpath);
$this->assertTrue(count($dependElement) > 0,
'Target build contains no depends attribute');
$dependantTasks = array_filter(explode(' ',
trim($dependElement[0]->depends)
));
$this->assertContains('clean', $dependantTasks,
"Target build doesn't depend on the clean target");
}
/**
* @test
* @group structure
*/
public function allDefinedTargetsShouldHaveADescriptionAttribute() {
$xml = new SimpleXMLElement($this->_buildfileXml);
$xpath = "//target";
$targetElements = $xml->xpath($xpath);
$describedTargetElements = array();
foreach ($targetElements as $index => $targetElement) {
$targetDescription = trim($targetElement->attributes()->description);
if ($targetDescription !== '') {
$describedTargetElements[] = $targetDescription;
}
}
$this->assertEquals(count($targetElements),
count($describedTargetElements),
'Description not for all targets set');
}
/**
* @test
* @group structure
*/
public function githubCloneAdhocTaskShouldBeDefined() {
$xml = new SimpleXMLElement($this->_buildfileXml);
$xpath = "//target[@name='init-ad-hoc-tasks']/adhoc-task";
$adhocElement = $xml->xpath($xpath);
$this->assertSame('github-clone',
trim($adhocElement[0]->attributes()->name),
"Ad hoc task 'github-clone' isn't defined");
}
/**
* @test
* @group artifact
*/
public function initTargetShouldCreateInitialBuildArtifacts() {
$this->_isTearDownNecessary = true;
$this->_buildfileRunner->runTarget(array('init'));

$expectedInitArtifacts = array(
"{$this->_buildfileBasedir}/build",
"{$this->_buildfileBasedir}/build/logs/performance/",
"{$this->_buildfileBasedir}/build/doc",
"{$this->_buildfileBasedir}/build/reports"
);

foreach ($expectedInitArtifacts as $artifact) {
$this->assertFileExists($artifact,
"Expected file '{$artifact}' doesn't exist");
}
}
/**
* @test
* @group artifact
*/
public function sqlFilesForDatabaseSetupTargetShouldBeAvailable() {
$expectedSqlFiles = array(
"{$this->_buildfileBasedir}/sqlfiles",
"{$this->_buildfileBasedir}/sqlfiles/session-storage.sql",
"{$this->_buildfileBasedir}/sqlfiles/acl.sql",
"{$this->_buildfileBasedir}/sqlfiles/log.sql"
);

foreach ($expectedSqlFiles as $sqlFile) {
$this->assertFileExists($sqlFile, "SQL file '{$sqlFile}' doesn't exist");
}
}
/**
* @test
* @group artifact
*/
public function githubTargetShouldFetchExpectedRepository() {
$this->_isTearDownNecessary = true;
$this->_buildfileRunner->runTarget(array('github'));
$expectedGitRepository = "{$this->_buildfileBasedir}/build/"
. "github-repos/phploc-phing/.git";
$this->assertFileExists($expectedGitRepository,
"Github target doesn't fetch the expected 'phploc-phing' repository");
}
}
The outro screenshot shows the above stated test class run against the example buildfile on a Mac OS X system utilizing the --colors option; which by the way comes in really handy in combination with Stakeout.rb during the process of refactoring or extending/creating buildfiles the test-driven way.

PHPUnit console output

Saturday, 18 April 2009

Creating and using Phing ad hoc tasks

Sometimes there are build scenarios where you'll badly need a functionality, like adding a MD5 checksum file to a given project, that isn't provided neither by the available Phing core nor the optional tasks. Phing supports developers with two ways for extending the useable task pool: by writing 'outline' tasks that will end up in a directory of the Phing installation or by utilizing the AdhocTaskdefTask, which allows to define custom tasks in the buildfile itself. The following post will try to outline how to define and use these inline tasks, by sketching an ad hoc task that enables the build orchestra to clone Git repositories from GitHub during a hypothetical workbench setup.

Creating the inline/ad hoc task

The AdhocTaskdefTask expects a name attribute i.e. github-clone for the XML element which will later referr to the ad hoc task and a CDATA section hosting the task implementation. Similar to 'outline' tasks the ad hoc task extends Phing's Task class, configures the task via attributes and holds the logic to perform. Unfortunately inline task implementations don't allow to require or include external classes available in the include_path, like Zend_Http_Client which I initially tried to use for an example task fetching short Urls from is.gd. This limits the available functions and classes to craft the task from to the ones built into PHP. The following buildfile snippet shows the implementation of the github-clone ad hoc task which is wrapped by a private target to encourage reusability and limit it's callability.
<target name="-init-ad-hoc-tasks" 
description="Initializes the ad hoc task(s)">
<adhoc-task name="github-clone"><![CDATA[
class Github_Clone extends Task {

private $repository = null;
private $destDirectory = null;

function setRepos($repository) {
$this->repository = $repository;
}
function setDest($destDirectory) {
$this->destDirectory = $destDirectory;
}
function main() {
// Get project name from repos Uri
$projectName = str_replace('.git', '',
substr(strrchr($this->repository, '/'), 1));

$gitCommand = 'git clone ' . $this->repository . ' ' .
$this->destDirectory . '/' . $projectName;

exec(escapeshellcmd($gitCommand), $output, $return);

if ($return !== 0) {
throw new BuildException('Git clone failed');
}
$logMessage = 'Cloned Git repository ' . $this->repository .
' into ' . $this->destDirectory . '/' . $projectName;
$this->log($logMessage);
}
}
]]></adhoc-task>
<echo message="Initialized github-clone ad hoc task." />
</target>

Using the ad hoc task

With the ad hoc task in the place to be, it's provided functionality can now be used from any target using the tasks XML element according to the given name i.e. github-clone in the AdhocTaskdefTask element earlier and by feeding it with the required attributes i.e. repos and dest. The next snippet allows you to take a peek at the complete buildfile with the ad hoc task in action.
<?xml version="1.0" encoding="UTF-8"?>
<project name="recordshelf" default="init-work-bench" basedir=".">

<property name="github.repos.dir" value="./github-repos" override="true" />

<target name="init-work-bench"
depends="-init-ad-hoc-tasks, -clone-git-repos"
description="Initializes the hypothetical workbench">
<echo message="Initialized workbench." />
</target>

<target name="-clean-git-repos"
description="Removes old repositories before initializing a new workbench">
<delete dir="${github.repos.dir}" includeemptydirs="true" failonerror="true" />
</target>

<target name="-init-ad-hoc-tasks"
description="Initializes the ad hoc task(s)">
<adhoc-task name="github-clone"><![CDATA[
class Github_Clone extends Task {

private $repository = null;
private $destDirectory = null;

function setRepos($repository) {
$this->repository = $repository;
}
function setDest($destDirectory) {
$this->destDirectory = $destDirectory;
}
function main() {
// Get project name from repos Uri
$projectName = str_replace('.git', '',
substr(strrchr($this->repository, '/'), 1));

$gitCommand = 'git clone ' . $this->repository . ' ' .
$this->destDirectory . '/' . $projectName;

exec(escapeshellcmd($gitCommand), $output, $return);

if ($return !== 0) {
throw new BuildException('Git clone failed');
}
$logMessage = 'Cloned Git repository ' . $this->repository .
' into ' . $this->destDirectory . '/' . $projectName;
$this->log($logMessage);
}
}
]]></adhoc-task>
<echo message="Initialized github-clone ad hoc task." />
</target>

<target name="-clone-git-repos" depends="-clean-git-repos"
description="Clones the needed Git repositories from GitHub">
<github-clone repos="git://github.com/abc/abc.git"
dest="${github.repos.dir}" />
<github-clone repos="git://github.com/xyz/xyz.git"
dest="${github.repos.dir}" />
</target>

</project>

Favouring inline over 'outline' tasks?

The one big advantage of using inline tasks over 'outline' tasks is that they are distributed with the buildfile and are instantly available without the need to modify the Phing installation. Some severe disadvantages of inline tasks are the limitation to use only the core PHP functions and classes for the implementation, the introduction of an additional hurdle to verify the task behaviour via PHPUnit as it's located in a CDATA section of the buildfile and the fact that the use of several inline tasks will blow up the buildfile, and thereby obfuscate the build flow.

Regrettably Phing doesn't provide an import task like Ant which might enable a refactoring to pull the ad hoc task definitions into a seperate XML file and include them at buildtime; in case you might have some expertise or ideas for a suitable workaround hit me with a comment. So far I tried to get it working, with no success, by utilizing Phing's PhingTask and XML's external entities declaration.

Tuesday, 31 March 2009

Using Haml & Sass from a Rake task

Haml logoSome time ago I had the 'lightning' idea to implement another Rake automation to support my current blogging workflow, which at the moment consists of finding a sparkling idea to blog about, write it out in WriteRoom and refine the post in TextMate before publishing. As this process was a recurring and copy & paste driven event, I strove for an automation supporting this workflow. So unsurprisingly the post will show my current solution to achieve this goal by utilizing Rake, Haml and Sass.

So what's that Haml and Sass thingy?

Haml (HTML Abstraction Markup Language) is a templating language/engine with the primary goal to make Markup DRY, beautiful and readable again. It has a very shallow learning curve and therefor is perfectly suited for programmers and designers alike. Haml is primarily targeted at making the views of Ruby on Rails, Merb or Sinatra web applications leaner, but as you will see later the Ruby implementation also can be used framework independently.

Sass (Syntactically Awesome StyleSheets) is a module which comes bundled with Haml providing a meta-language/abstraction on top of CSS sharing the same goals and advantages as Haml.

Gluing Haml and Sass into a Rake task

To get going you first have to install Haml and Sass by running the gem command shown next.
sudo gem install haml
With Haml and Sass available it's about time to identify and outline the parts you want to automate, in my case it's the creation of a WriteRoom and/or a XHTML draft document for initial editings. So the parameters to pass into the task to come are the targeted editor(s), the title of the blog post to draft and a list of associated and whitespace separated category tags.

The XHTML document skeleton content and it's inline CSS are defined each in a separate Haml and Sass template file and will be rendered into the outcoming document along with the content passed into the Rake task. While the document skeleton for the WriteRoom draft document, due to it's brevity, is defined inside of the task itself. The following snippets are showing the mentioned Haml and Sass templates for the XHTML draft output file, which are located in the same directory as the Rake file.

 Haml
!!! 1.1
%html
%head
%title= "#{title} - Draft"
%style{ :type => 'text/css' }= inline_css
%body
%h3= title
%h4.custom sub headline
%pre.consoleOutput console command
%pre.codeSnippet code snippet
%br/
= "Tags: #{tags.join ', '}"
 Sass
body
:margin 5
:line-height 1.5em
:font small Trebuchet MS, Verdana, Arial, Sans-serif
:color #000000
h4
:margin-bottom 0.3em
.consoleOutput
:padding 6px
:background-color #000
:color rgb(20, 218, 62)
:font-size 12px
:font-weight bolder
.codeSnippet
:padding 3px
:background-color rgb(243, 243, 243)
:color rgb(93, 91, 91)
:font-size small
:border 1px solid #6A6565
To inject the dynamic content into the Haml template and have it rendered into the outcoming document, the values i.e. draft_title, draft_tags and draft_inline_css have to be made available to the template engine by passing them in a bundling Hash into the to_html alias method of the Haml Engine object like shown in the next Rake task.
task :default do
Rake::Task['blog_utils:create_draft_doc'].invoke
end

namespace :blog_utils do

desc 'Create a new draft document for a given title, category tags and editor'
task :create_draft_doc, [:title, :tags, :editor] do |t, args|
draft_title = args.title
draft_tags = args.tags.split(' ')
draft_target_editor = args.editor

raise_message = 'No title for draft provided'
raise raise_message if draft_title.nil?

raise_message = 'No tags for draft provided'
raise raise_message if draft_tags.nil?

draft_target_editor = '*' if draft_target_editor.nil?

raise_message = 'Unsupported target editor provided'
raise raise_message unless draft_target_editor == 'Textmate' ||
draft_target_editor == 'Writeroom' || draft_target_editor == '*'

if draft_target_editor == 'Writeroom' || draft_target_editor == '*'
draft_output_file = draft_title.gsub(' ', '_') + '.txt'

File.open(draft_output_file, 'w') do |draft_file_txt|
draft_file_txt.puts draft_title
draft_file_txt.puts
draft_file_txt.puts "Tags: #{draft_tags.join ', '}"
end
end

if draft_target_editor == 'Textmate' || draft_target_editor == '*'

template_sass_content, template_haml_content = ''

['haml', 'sass'].each do |template_type|
template = File.dirname(__FILE__) + "/draft_template.#{template_type}"
raise_message = "#{template_type.capitalize} template '#{template}' not found"
raise raise_message if !File.exists?(template)

template_sass_content = File.read(template) if template_type === 'sass'
template_haml_content = File.read(template) if template_type === 'haml'
end

require 'sass'
require 'haml'

draft_inline_css = Sass::Engine.new(template_sass_content).to_css
draft_document_content = Haml::Engine.new(template_haml_content).to_html(
Object.new, { :title => draft_title , :tags => draft_tags ,
:inline_css => draft_inline_css } )


draft_output_file = draft_title.gsub(' ', '_') + '.html'
File.open(draft_output_file, 'w') do |draft_file_html|
draft_file_html.puts(draft_document_content)
end
end

end
end

Easing invocation pain with alias

Now as the Rake task is implemented and waiting for demands it can be invoked by calling the task as shown in the next console snippet.
sudo rake -f $HOME/Automations/Rakefile.rb blog_utils:create_draft_doc['Title','Tag1 TagN','Editor']
As I'm not even close to being a console ninja and probably will have forgotten the task call structure before initiating the next blog post, I decided to add an easing and more memorizable alias to $HOME/.profile as shown next.
alias createdraft='sudo rake -f $HOME/Automations/Rakefile.rb blog_utils:create_draft_doc[$title,$tags,$editor]'
The created alias now allows to invoke the Rake task in a nice and easy way as shown in the next console command.
createdraft title='Using Haml & Sass from a Rake task' tags='Rake Ruby' editor='Textmate'

Taking a peek at the generated draft document

After running the described Rake task I end up with the XHTML document shown in the outro code snippet, which then can be used for the further editing process. Of course I could have setup a TextMate Snippet to get me going, but that way I would have missed the opportunity to mess around with another amazing Ruby tool.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html>
<head>
<title>Using Haml & Sass from a Rake task - Draft</title>
<style type='text/css'>
body {
margin: 5;
line-height: 1.5em;
font: small Trebuchet MS, Verdana, Arial, Sans-serif;
color: #000000; }

h4 {
margin-bottom: 0.3em; }

.consoleOutput {
padding: 6px;
background-color: #000;
color: rgb(20, 218, 62);
font-size: 12px;
font-weight: bolder; }

.codeSnippet {
padding: 3px;
background-color: rgb(243, 243, 243);
color: rgb(93, 91, 91);
font-size: small;
border: 1px solid #6A6565; }

</style>
</head>
<body>
<h3>Using Haml & Sass from a Rake task</h3>
<h4>sub headline</h4>
<pre class='consoleOutput'>console command</pre>
<pre class='codeSnippet'>code snippet</pre>
<br />
Tags: Rake, Ruby
</body>
</html>

Sunday, 22 February 2009

Phplocing your projects with Phing

When I started to play around with Ruby on Rails, my attention got somehow soon drawn to it's Rake stats task, which provides developers or more likely project managers with an overview of the actual project size. Exactly one month ago Sebastian Bergmann, of PHPUnit fame, started to implement a similar tool dubbed phploc which can give you an overview of the size for any given PHP project. As I wanted to automate the invocation of this handy tool and collect it's report output out of a Phing buildfile, I invested some time to develop a custom Phing task doing so. Thereby the following post will show you a possible implementation of this task and it's use in a buildfile.

Installing phploc

To setup phploc on your system simply install the phploc PEAR package available from the pear.phpunit.de channel as shown in the next commands. In case you already have installed PHPUnit via PEAR you can omit the channel-discover command.
sudo pear channel-discover pear.phpunit.de
sudo pear install phpunit/phploc

Implementing the phploc task

As I already blogged about developing custom Phing task I'm only going to show the actual implementation and not dive into any details; alternatively you can also grab it from this public GitHub repository.
<?php
require_once 'phing/Task.php';
require_once 'phing/BuildException.php';
require_once 'PHPLOC/Analyser.php';
require_once 'PHPLOC/Util/FilterIterator.php';
require_once 'PHPLOC/TextUI/ResultPrinter.php';

class PHPLocTask extends Task
{
protected $suffixesToCheck = null;
protected $acceptedReportTypes = null;
protected $reportDirectory = null;
protected $reportType = null;
protected $fileToCheck = null;
protected $filesToCheck = null;
protected $reportFileName = null;
protected $fileSets = null;

public function init() {
$this->suffixesToCheck = array('php');
$this->acceptedReportTypes = array('cli', 'txt', 'xml');
$this->reportType = 'cli';
$this->reportFileName = 'phploc-report';
$this->fileSets = array();
$this->filesToCheck = array();
}
public function setSuffixes($suffixListOrSingleSuffix) {
if (stripos($suffixListOrSingleSuffix, ',')) {
$suffixes = explode(',', $suffixListOrSingleSuffix);
$this->suffixesToCheck = array_map('trim', $suffixes);
} else {
array_push($this->suffixesToCheck, trim($suffixListOrSingleSuffix));
}
}
public function setFile(PhingFile $file) {
$this->fileToCheck = trim($file);
}
public function createFileSet() {
$num = array_push($this->fileSets, new FileSet());
return $this->fileSets[$num - 1];
}
public function setReportType($type) {
$this->reportType = trim($type);
}
public function setReportName($name) {
$this->reportFileName = trim($name);
}
public function setReportDirectory($directory) {
$this->reportDirectory = trim($directory);
}
public function main() {
if (!isset($this->fileToCheck) && count($this->fileSets) === 0) {
$exceptionMessage = "Missing either a nested fileset or the "
. "attribute 'file' set.";
throw new BuildException($exceptionMessage);
}
if (count($this->suffixesToCheck) === 0) {
throw new BuildException("No file suffix defined.");
}
if (is_null($this->reportType)) {
throw new BuildException("No report type defined.");
}
if (!is_null($this->reportType) &&
!in_array($this->reportType, $this->acceptedReportTypes)) {
throw new BuildException("Unaccepted report type defined.");
}
if (!is_null($this->fileToCheck) && !file_exists($this->fileToCheck)) {
throw new BuildException("File to check doesn't exist.");
}
if ($this->reportType !== 'cli' && is_null($this->reportDirectory)) {
throw new BuildException("No report output directory defined.");
}
if (count($this->fileSets) > 0 && !is_null($this->fileToCheck)) {
$exceptionMessage = "Either use a nested fileset or 'file' "
. "attribute; not both.";
throw new BuildException($exceptionMessage);
}
if (!is_null($this->reportDirectory) && !is_dir($this->reportDirectory)) {
$reportOutputDir = new PhingFile($this->reportDirectory);
$logMessage = "Report output directory does't exist, creating: "
. $reportOutputDir->getAbsolutePath() . '.';
$this->log($logMessage);
$reportOutputDir->mkdirs();
}
if ($this->reportType !== 'cli') {
$this->reportFileName.= '.' . trim($this->reportType);
}
if (count($this->fileSets) > 0) {
$project = $this->getProject();
foreach ($this->fileSets as $fileSet) {
$directoryScanner = $fileSet->getDirectoryScanner($project);
$files = $directoryScanner->getIncludedFiles();
$directory = $fileSet->getDir($this->project)->getPath();
foreach ($files as $file) {
if ($this->isFileSuffixSet($file)) {
$this->filesToCheck[] = $directory . DIRECTORY_SEPARATOR
. $file;
}
}
}
$this->filesToCheck = array_unique($this->filesToCheck);
}
if (!is_null($this->fileToCheck)) {
if (!$this->isFileSuffixSet($file)) {
$exceptionMessage = "Suffix of file to check is not defined in"
. " 'suffixes' attribute.";
throw new BuildException($exceptionMessage);
}
}
$this->runPhpLocCheck();
}
protected function isFileSuffixSet($filename) {
$pathinfo = pathinfo($filename);
$fileSuffix = $pathinfo['extension'];
return in_array($fileSuffix, $this->suffixesToCheck);
}
protected function runPhpLocCheck() {
$files = $this->getFilesToCheck();
$result = $this->getCountForFiles($files);

if ($this->reportType === 'cli' || $this->reportType === 'txt') {
$printer = new PHPLOC_TextUI_ResultPrinter;
if ($this->reportType === 'txt') {
ob_start();
$printer->printResult($result);
file_put_contents($this->reportDirectory
. DIRECTORY_SEPARATOR . $this->reportFileName,
ob_get_contents());
ob_end_clean();
$reportDir = new PhingFile($this->reportDirectory);
$logMessage = "Writing report to: "
. $reportDir->getAbsolutePath() . DIRECTORY_SEPARATOR
. $this->reportFileName;
$this->log($logMessage);
} else {
$printer->printResult($result);
}
} elseif ($this->reportType === 'xml') {
$xml = $this->getResultAsXml($result);
$reportDir = new PhingFile($this->reportDirectory);
$logMessage = "Writing report to: " . $reportDir->getAbsolutePath()
. DIRECTORY_SEPARATOR . $this->reportFileName;
$this->log($logMessage);
file_put_contents($this->reportDirectory . DIRECTORY_SEPARATOR
. $this->reportFileName, $xml);
}
}
protected function getFilesToCheck() {
if (count($this->filesToCheck) > 0) {
$files = array();
foreach ($this->filesToCheck as $file) {
$files[] = new SPLFileInfo($file);
}
} elseif (!is_null($this->fileToCheck)) {
$files = array(new SPLFileInfo($this->fileToCheck));
}
return $files;
}
protected function getCountForFiles($files) {
$count = array('files' => 0, 'loc' => 0, 'cloc' => 0, 'ncloc' => 0,
'eloc' => 0, 'interfaces' => 0, 'classes' => 0, 'functions' => 0);
$directories = array();

foreach ($files as $file) {
$directory = $file->getPath();
if (!isset($directories[$directory])) {
$directories[$directory] = TRUE;
}
PHPLOC_Analyser::countFile($file->getPathName(), $count);
}

if (!function_exists('parsekit_compile_file')) {
unset($count['eloc']);
}
$count['directories'] = count($directories) - 1;
return $count;
}
protected function getResultAsXml($result) {
$newline = "\n";
$newlineWithSpaces = sprintf("\n%4s",'');
$xml = '<?xml version="1.0" encoding="UTF-8"?>';
$xml.= $newline . '<phploc>';

if ($result['directories'] > 0) {
$xml.= $newlineWithSpaces . '<directories>' . $result['directories'] . '</directories>';
$xml.= $newlineWithSpaces . '<files>' . $result['files'] . '</files>';
}
$xml.= $newlineWithSpaces . '<loc>' . $result['loc'] . '</loc>';

if (isset($result['eloc'])) {
$xml.= $newlineWithSpaces . '<eloc>' . $result['eloc'] . '</eloc>';
}
$xml.= $newlineWithSpaces . '<cloc>' . $result['cloc'] . '</cloc>';
$xml.= $newlineWithSpaces . '<ncloc>' . $result['ncloc'] . '</ncloc>';
$xml.= $newlineWithSpaces . '<interfaces>' . $result['interfaces'] . '</interfaces>';
$xml.= $newlineWithSpaces . '<classes>' . $result['classes'] . '</classes>';
$xml.= $newlineWithSpaces . '<methods>' . $result['functions'] . '</methods>' . $newline;
$xml.= '</phploc>';
return $xml;
}
}

Hooking the phploc task into Phing

To use the task in your Phing builds simply copy it into the phing/tasks/my directory and make it available via the taskdef task. The next table shows the available task attributes and the values they can take to configure it's behaviour and output. As you will see it also provides the ability to generate reports in a XML format; I chose to implement this feature to have the possibilty to transform the report results into HTML documents by applying for example a XSLT stylesheet. This way they can provide more value to non-technical project members or can be made accessible in a CI system dashboard if desired.

NameTypeDescriptionDefaultRequired
reportTypestringThe type of the report. Available types are cli|txt|xml.cliNo
reportNamestringThe name of the report type without a file extension.phploc-reportNo
reportDirectorystringThe directory to write the report file to.falseYes, when report type txt or xml is defined.
filestringThe name of the file to check.n/aYes, when no nested fileset is defined.
suffixesstringA comma-separated list of file suffixes to check.phpNo

Supported Nested Tags:
  • fileset
The closing buildfile extract shows an example phploc task configuration and is also available at the public GitHub repository. Happy phplocing!
<?xml version="1.0"?>
<project name="example" default="phploc" basedir=".">
<taskdef name="phploc" classname="phing.tasks.my.PHPLocTask" />
<target name="phploc">
<tstamp>
<format property="check.date.time" pattern="%Y%m%d-%H%M%S" locale="en_US"/>
</tstamp>
<phploc reportType="txt" reportName="${check.date.time}-report"
reportDirectory="phploc-reports">
<fileset dir=".">
<include name="**/*.php" />
<include name="*.php" />
</fileset>
</phploc>
</target>
</project>