Tuesday 26 June 2007

Turning a Zend_Log log file into a RSS feed

Whilst touring the web I found an interesting project for turning Apache Web Server log files into RSS feeds. This approach can be adjusted to monitor the maintenance needs of a web application deployed on an assumed productive system. Therefor a XML capable Zend_Log instance will be set up and the resulting log file will be transformed into a RSS feed via a custom Action Helper wrapping a XSLT transformation.

Setting up the XML logger
To reduce the transformation effort a XML logger can be used for accumulating all log entries possibly raised by the application. The following code listing outlines the way to do so in the applications bootstrap file.

...

$writer = new Zend_Log_Writer_Stream('../logs/recordshelf-log.xml');
$writer->setFormatter(new Zend_Log_Formatter_Xml());
$logger = new Zend_Log($writer);
$logger->setEventItem('timestamp', date('D, j M Y H:i:s', time()));

Zend_Registry::set('logger', $logger);
Zend_Registry::set('log', '../logs/recordshelf-log.xml');

...
After setting up the XML logger instance all entries to the log file will look like the following ones, assuming the default format has been kept. It's a bit weird that the resulting log file is not a valid XML document because of a missing XML declaration and a non present root element.
...
<logEntry>
<timestamp>Mon, 25 Jun 2007 22:00:02</timestamp>
<message>Log message 1</message>
<priority>3</priority>
<priorityName>ERR</priorityName>
</logEntry>
<logEntry>
<timestamp>Mon, 25 Jun 2007 22:00:25</timestamp>
<message>Log message 2</message>
<priority>7</priority>
<priorityName>DEBUG</priorityName>
</logEntry>
<logEntry>
<timestamp>Mon, 25 Jun 2007 22:01:30</timestamp>
<message>#0 Exception stacktrace part 1
#1 Exception stacktrace part 2
#2 Exception stacktrace part 3</message>
<priority>3</priority>
<priorityName>ERR</priorityName>
</logEntry>
<logEntry>
<timestamp>Mon, 25 Jun 2007 22:01:30</timestamp>
<message>Log message 3</message>
<priority>3</priority>
<priorityName>ERR</priorityName>
</logEntry>
...
Shifting formats
The desired format transformation can be achieved via a custom Action Helper which simply wraps up a XSLT transformation and can be reused for further 'transformation scenarios' in any Action Controller. It's arguments are the source XML and the XSL stylesheet to apply. The following code shows the Recordshelf_Controller_Action_Helper_Xslt located in the applications custom library.
<?php

require_once 'Zend/Controller/Action/Helper/Abstract.php';

class Recordshelf_Controller_Action_Helper_Xslt extends Zend_Controller_Action_Helper_Abstract {

public function direct(SimpleXMLElement $xml, $xslFile) {

if(!file_exists($xslFile)) {

throw new Zend_Exception("Unable to load xsl stylesheet '{$xslFile}'");

}

$doc = new DOMDocument();
$xslt = new XSLTProcessor();

$doc->load($xslFile);
$xslt->registerPHPFunctions();

// Needed for <pubDate> value in <channel> element
$xslt->setParameter('', 'creationDate', date('D, j M Y H:i:s', time()));
$xslt->importStyleSheet($doc);

$doc->loadXML($xml->asXML());

$this->getResponse()->setHeader('Content-Type', 'text/xml')
->setBody($xslt->transformToXML($doc));

}

}
The XSL stylesheet responsible for the log to RSS transformation is located in an application-root/xsl directory and looks like the following.
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:php="http://php.net/xsl">

<xsl:template match="/">
<rss version="2.0">
<channel>
<title>recordshelf.org maintenance feed</title>
<link>http://recordshelf.org/</link>
<description>RSS representation of the recordshelf.org application log file</description>
<language>en-US</language>

<!-- Get creationDate value set in Action Helper -->

<pubDate><xsl:value-of select="$creationDate"/></pubDate>

<xsl:for-each select="logEntries/logEntry">

<item>
<title>
<xsl:variable name="priority" select="priority"/>

<!-- Choose log priority type as <title> value -->

<xsl:choose>
<xsl:when test='$priority = 0'>Emergency message</xsl:when>
<xsl:when test='$priority = 1'>Alert message</xsl:when>
<xsl:when test='$priority = 2'>Critical message</xsl:when>
<xsl:when test='$priority = 3'>Error message</xsl:when>
<xsl:when test='$priority = 4'>Warning message</xsl:when>
<xsl:when test='$priority = 5'>Notice message</xsl:when>
<xsl:when test='$priority = 6'>Informational message</xsl:when>
<xsl:otherwise>Debug message</xsl:otherwise>
</xsl:choose>

</title>

<!-- Raise exception stacktrace readability -->

<description><xsl:value-of select="php:function('nl2br',string(message))"/></description>
<pubDate><xsl:value-of select="timestamp"/></pubDate>
</item>
</xsl:for-each>
</channel>
</rss>
</xsl:template>
</xsl:stylesheet>
Gluing the Action Controller with the Action Helper
The last step to finally achieve the transformation is adding a Action Controller to the application which delegates the work to the afore crafted XSLT Action Helper. This Action Controller should be access restricted to prevent revealing sensitive data.
<?php

class MaintenanceController extends Zend_Controller_Action {

public function init() {

Zend_Controller_Action_HelperBroker::addPath('Recordshelf/Controller/Action/Helper/',
'Recordshelf_Controller_Action_Helper');

}

public function rssAction() {

// Load XML log file and turn it into a valid XML document
$xmlLog = '<?xml version="1.0" encoding="UTF-8"?><logEntries>';
$xmlLog.= file_get_contents(Zend_Registry::get('log'));
$xmlLog.= '</logEntries>';

// Call Action Helper for the transformation
$this->_helper->xslt(new SimpleXMLElement($xmlLog), '../xsl/logToRss.xsl');

}

...

}
Finally polling the applications health
With everything in place the applications 'maintenance/monitor' RSS feed is now available at http://domain.org/maintenance/rss/ for aggregation in a feed reader. I currently use the FeedReader3 since it allows the definition of custom polling intervals.

feedreader

6 comments:

Anonymous said...

Now this will be very useful.

I will be able to keep track of my server via a feed, thanks!

Anonymous said...

Thank you for this tip. Very useful.
Centralise the access logs of many projects in an rss reader is a good solution.

malandrin

Anonymous said...

Nice Idea,
but I cannot get it to work.
I put it in the bootstrap, a log file is written, but where to put all the other files?

Raphael Stolt said...

Sorry for the delay. In the example here the custom Xslt action helper is located in the applications library directory, i.e. recordshelf/library/Recordshelf/Controller
/Action/Helper. The Xsl file for the transformation of the xml log file is located in a directory called xsl in the applications root directory, i.e. recordshelf/xsl. The action controller i.e. MaintenanceController is located in the applications controllers directory, i.e. recordshelf/application/default/controllers as I'm using the conventional modular directory layout.

I hope this helps you a bit.

Unknown said...

got a few questions:

1. I had to create an empty view file for the feeds url. Should I just disable the view altogether?

2. My feed won't update, it just loads on subscription. New messages don't appear. What might be causing this?

Raphael Stolt said...

Regarding 1. you have to disable the auto-rendering for the rssAction via $this->_helper->viewRenderer->setNoRender(); as this post was written for/with the 0.8 version of the Zend Framework.

Regarding 2. I guess this might be a chaching problem of the used feedreader, although a new timestamp is created on each request as you will see when you call the action directly in the browser.