Entries by argh :: a programmer's best friend

Clay Loveless recently posted a small rant [1] about loner applications, and that inspired me to write up a comment of sorts.

I think that the interoperability Clay wants requires out-of-the-box thinking, but sadly most of the programming world lives in tight boxes. To change that tide would require a lot of dedication and time, both of which I for one am not prepared to invest.

It is really quite sad, but the time and effort I need to get the author(s) of any piece of software to become aware of the non-interoperability with software XY I prefer to invest in getting a solution done - especially since the answers are usually "works for me", "don't need software XY", "it's open source, fix it yourself", "nobody needs that except you", "no time, maybe this winter". I usually try contacting the authors in case I should get lucky, but tend to avoid those kinds of discussions as they lead nowhere.

I think that one of the core reasons for the lack of interoperability is the reprogramming the wheel syndrome. I did that quite a few times myself, and while it may often be because the programmer feels he can do it better, I think there is more to it than just that. These are a few issues I can think of on the spot:

• Lack of knowledge/training in development paradigms
• Mislead prioritizing of featuresets
• Ignorance of existing tools, either willing or unwilling
• Lack of interest/motivation to find out if there are existing libraries for task XY
• Confidence that it is easy and fast to develop on one's own (like database abstractions and authentication layers)
• Unwillingness to conform to existing APIs
• Lack of understanding of existing APIs
• Stressed work environments that do not allow for research of existing tools (or training, for that matter)
• General mistrust of external libraries

I am sure a psychiatrist specializing in modern software developers would be able to find quite a bunch more issues, but I am not sure I want to know about that in such detail after all.


Expectations, Approaches

This is one more issue that I feel compelled to write about, since it is the reason why I have reprogrammed the wheel a few times in the past, and actually still do: different approaches to a problem.

As an example, have you ever wondered why there are really that many CMSs out there? Is it really because someone said "I can do it better"? That may very well be the case, but why did he/she think so? I think it is because that someone had a different idea of how a CMS should work, dismissing the other system because it did not match his expectations. Like the users using a software like a CMS will be more at ease with one or the other system, developers will be more at ease with one or the other library. PEAR's HTML_QuickForm [2] essentially does the same than patForms [3], but in a different way - why does one developer choose the one over the other? The choice is not always based on the featureset.

I know that when I look at a library and it does not work the way I expect it to, I am easily tempted to find another (if possible) or think of writing one of my own. My friends and colleagues have largely succeeded in curing me of the syndrome, like by teaching me how to use wrappers to overcome the awkwardness of using an "alien" API. Now I usually only program a new wheel when there are not really any good libraries for the task at hand, or the library of choice has serious issues.


Real life and productivity vs interoperability

When I started my own business, I knew I would need an application that would help me build the websites for my customers, so I started looking around. There are many great frameworks and CMSs out there, and I took a good look at them. My decisionmaking was based on a few simple rules:

The system had to

• allow complete freedom of layout
• feature a multilingual frontend and administration
• be extensible to add any kind of features
• have enough modularity to allow frequent structure changes
• be secure
• work under PHP4 (most of my customers run on shared hosting)
• be open source for easy adjustment to custom needs
• have a user-friendly and self-documenting interface for endusers
• have an administration sdk to easily add new administration features
• take care of all redundant website building tasks
• allow me to let loose my customers in the administration interface
• allow me to use third-party libraries (interoperability )
• be around for a while

In everyday use, the goals were

• to empower me to keep the quality as high as I want it to
• to allow me to reduce my prices to lessen the gap with the discount offers of the competition
• to give me an edge on development times versus the competition

I don't want to turn this post into a feature comparison, so I will skip that step. After reviewing several frameworks and CMSs, I realized that none would fit what I wanted. Only a custom solution would work, because for the rate of development I targeted I needed a tool that functions on a level I can comprehend. Inspired by patPortal, I planned and started the development of SimpleSite [4]. As a strictly solo project at first and due to the development timeline priorities, interoperability was the least of my worries. Even now, after about a year and a half of constant development it is still lacking in that regard. However, I have made sure that it is interoperability-ready: all the required interfaces and structures are built to allow swapping components as you see fit.

I think that knowing how to prepare for interoperability and adding it to your application development plans is much more important than actually offering it, as it makes the job for other developers that much easier. I also know that it is not always that simple either however - in SimpleSite, the templating engine of choice is patTemplate. While you can use any template engine you like in your modules, the main structure is always handled by patTemplate, and there are related helper methods that only work with patTemplate. It is sometimes really a matter of weighing the development effort and related costs against the added value interoperability would add.


Related links
[1] Clay loveless' "Stop writing loner applications" post
[2] PEAR HTML_QuickForm
[3] patForms
[4] The SimpleSite development portal

If like me, regular expressions are constantly nagging at you because you just can't get the knack of them, you'll be happy to learn that I have finally found that software I have been waiting for all these years: RegexBuddy. Behind this mindbogglingly dull name hides a very nice visual regular expression editor, debugger, tester and optimizer. Regular expression nirvana!

I am a completely lost case regarding regexes, so I have purchased a copy - and I can honestly say that's the most well spent money I ever spent on software (with the exception of Photoshop, I think). The integrated regexp parser tree view shows you exactly what a regexp is made of, what it matches, where you could optimize it as well as correct common errors. It supports Perl-Compatible regexes as well as POSIX extended ones, has integrated PHP syntax highlighting... I'll stop here before anyone thinks I am affiliated with the creators of the software (but you were probably already thinking just that).

As pictures tell more than words, I'll let these screenshots speak for themselves:



The regex wizards out there will probably laugh at this, but I can highly recommend it if you want to learn regular expressions or simply to be able to make some of your own without external help

Yesterday we released the first beta of patForms, following our second alpha release. There has been a lot of activity around patForms lately, and most notably a new lead developer has joined the team: Sven Fuchs. Among other things he has added a complete layer that allows patForms propel integration, and has contributed new renderers.

Quick overview of changes:

• A lot of bugfixes, subpackages more stable
• Propel integration
• New renderers: Smarty, PHPTal and HTML_Template_Flexy
• Form namespaces
• New rules and storages
• Enhanced parser
• Minor enhacements and new features

With all the bugfixes and enhancements in this new version, the whole package has become a lot more stable and along with the new functionality has taken a step more towards making working with forms fun We are planning to make a first stable release in october, so we need your help in weeding out the leftover bugs!

If you are using patForms in your projets, please upgrade and test the new version (please see the section on upgrading further down in this post for additional notes). If you do not know patForms yet, I think this is a good time to discover the projet. With the extensive and comprehensive examples collection, you can start working with it in a breeze!

You can report any bugs or feature requests in our new Trac-powered project manager, where you can also find a documentation wiki.

[1] Download via our PEAR channel server
[2] Download via the traditional download page
[3] Project manager (bugs, feature request, wiki)
[4] Online examples collection

Stephan uploaded the slides for his two talks on the International PHP Conference in Amsterdam yesterday evening. Apparently both sessions were well attended, and the Power Workshop on XML and WebServices he has made along with Tobias Schlitt went quite well

You can have a look at the slides for Stephan's talks right here:

Go OO! - Real-life Design Patterns in PHP5
Component and Event Based Architectures

The examples are not included yet, but we will add them as a separate package soon.

We are currently planning a complete relaunch of the php-tools.net site, and would welcome any feedback on how we can improve it in the process. The new site will have a completely new design and backend (we are still deciding what we will use), but even though we have a good idea of what featureset the new site will need, you can still help. Tell us what you least and most liked about the current site, and what features you missed most: simply send a mail to argh [at] php-tools [dot] net with your ideas / criticism / feature requests.

Thanks in advance from the Team!

Since Greg Beaver invented the PEAR_Server class and Davey Shafik created a handy package to set up your own Channel frontend, we've been itching to make a pat Channel. Stephan installed it on pear.php-tools.net, and after my designer eyes stopped bleeding when I saw the default layout of the beast (don't get me wrong, Davey - I know it's just an example layout ), I made a pat-skin and now it's ready to roll.

For the moment only a few of our packages are available through our channel, but we'll add the others eveventually.

You can read more about the PEAR_Server package on Greg's blog, and more about the Channel frontend package on Davey's blog.

As I have finally had the time to look into the "cannot redeclare xmlrpc_decode()" bug, a new patSpiderizer release (v2.0.9) is now available from our site. This is just a bugfix release, but that one bug was very annoying, and only happened on PHP installations that had the xmlrpc extension included - the function names of our xmlrpc connectivity class simply were the same as those of the extension...

This is probably the last version for the v2 branch (except for bugfixes), as I have started writing up notes for the new v3. This new version will be rewritten from scratch, and synthesize the experience gained with patSpiderizer and its bigger brother XENA PolePosition (German) that I developed for my former employer, Metrix Internet Design GmbH, in one tool. The global concept will bring some fresh wind to the 'cloaking' scene, I think, as it will be the first NOT to cloak

I have added the patTemplate driver to the new Describer class included with patBBCode: this driver uses a patTemplate template file or loaded templates to generate the tag guide, leaving nearly total control in layout. The effect can best be seen by comparing the very simple HTML Driver output with the new patTemplate driver output.

In addition to making it possible to let patBBCode render its tag guide automatically directly in your site's design, the patTemplate driver has a series of options you can set, namely:

• injectContent: whether to inject the content into the loaded templates directly, or return the rendered list as a string. This is very useful if you want to inject the content directly in place into an existing template your application has already loaded.
• sortBy: sorting of the tags list. Possible values are 'tagName' to order by tag names alphabetically, 'attributes' to sort by tags with or without attributes, or set to 'null' to disable completely.
• sortDir: sorting direction. asc|desc
• containerTemplate: sets the name of the template containing the whole tags list
• entryTemplate: the name of the template for each tag
• attributesTemplate: the name of the template with the attributes list
• variablePrefix: a prefix prepended to all variables in the templates - use if some of the standard variable names collide with existing ones in your application

These changes are currently only available in CVS, so you can view the changes in our WebCVS or download the latest CVS snapshot.

Our BBCode parser, patBBCode, ist available as v1.0.0beta. This small library is based on a SAX parser, and retrieves all tags with just one regular expression. The tag transformation is then done by filter objects, or patBBcode itself in case of 'basic' tags without attributes.

A complete documentation is available as well as a series of examples also included in the package for a quick start.

Some are already familiar with patBBCode, but with this release there are a few new things. Filter classes can now trigger their own user errors, e.g. if a vital attribute is missing. The biggest addition, however, is the Describer helper class: it enables you to describe an existing patBBCode object by retieving information about all available tags. This can be used to automatically generate a tag guide which you can retrieve in several output formats by means of a specific Driver object. The following bit of code generates an HTML table with a list of tags and their function:


// create the Describer
$descr =& patBBCode::createDescriber();

// create the driver
$driver =& $descr->createDriver( 'HTML' );

// tell the describer to use the driver
// $descr->setDriver( $driver );

// and display the documentation
echo $descr->describe( $BBCode );


You can view the result here - this is included in the examples collection of the package. For the moment, the HTML driver is very simple, but I intend to add a patTemplate driver which will give you total layout control. Another feature worth noting: the Describer respects the locale you set in the patBBCode object you describe

Comme certains ont peut-être remarqué, j'ai déménagé d'allemagne pour retourner en france. J'en ai profité pour faire de mon blog perso un blog entièrement en français, sur lequel vous pourrez suivre l'actualité de pat sans avoir recours à vos talents de traducteurs en herbe.

(As some might have noticed, I have moved back from germany to france - if you want to try and put your translation talents to the test, you can now follow pat news in french via my own personal blog)

patForms is slowly reaching maturity, and for those of you who have been following patForm's development a little closer, there have recently been a few interesting changes:

- I have entirely rewritten the Date element, which now generates several input fields directly for each token in the specified date format, and can either display as several text input fields or select boxes. The API has changed a bit, so if you are using the Date element in your projets, be careful when upgrading. Please post any bugs you find via bugs.php-tools.net.

- I have reviewed all the patForms elements and rewritten parts to optimize the code, and finish implementing the display and edit attributes - they now work correctly in HTML and Readonly modes.

- Finally, I added a small feature to the examples framework, which is to display the submitted form values after submitting a form.

- Stephan on his side has finished implementing the Group element, and thanks to the new patForms_Collection class which can be used for handling multiple forms, the patForms_Parser can now handle multiple forms in a form template.

- Stephan also added support for external user error codes definitions so you can either define your own for the existing locales, or add new error codes for your own locales.

As it stands, we will release a first beta of patForms this month, along with an article in the international PHP Magazine on easy form management also featuring patForms. You can have a look at the current CVS snapshot via snaps.php-tools.net.

Talking about what can be done with javascript... for the Game Manager I built to calculate the values of starting locations in the Galactic Civilizations PC game, I implemented language switching and skin switching entirely in javascript.

The whole tool is built on objects that tightly communicate amongst themselves - each star system, planet or game is an object and can render itself. The layout control is mostly implemented with stylesheets, the javascript only rendering a very simple html structure. The core is the gameManager class itself - it renders the main navigation elements, and provides a series of helper methods (like the button() method that is used to render a button).

The language switching was not too difficult - I first googled around but the generally used trick with document.write( '[script tag]' ); makes Mozilla reload the page. I found a nicer way to do this: create the script tag with the document.createElement() method, and append it to the document's head tag with document.getElementsByTagName('head')[0].appendChild( tag ); As my language files consist of one file with a list of variables, they simply get replaced when loading the new script. Updating the display is a breeze, I just have to tell the objects to redraw themselves

The skin (stylesheet) switching proved a little more annoying: it works the same way than the javascript include, by appending a link tag to the head - but the existing styles do not get replaced, they are overwritten and extended. Meaning that if in my standard stylesheet, the h1 tag has a padding but not in the second stylesheet, that padding will still be there... Solution: delete all styles before loading the new stylesheet. That's easy to do by accessing the document's styleSheets property, going through each stylesheet and deleting each style rule via document.styleSheets[x].deleteRule( x ); Peter-Paul Koch has an excellent reference on this.

Currently the Game Manager only works with gecko-based browsers like Mozilla or Firefox, as I use some specific methods like the addEventListener() method (to check when the external file has been loaded), and also because I used only the gecko implementation of the styleSheet access methods.

The result matches my expectations: a blazingly fast online tool, that would have taken a lot of time to build with a PHP/Javascript mixture and would not have been this fast. Now I only need to implement the save button and it's ready to really rock Implementing this would take a little time, but basically all I need is already there: if you clicked on one of the View XML buttons in the interface, you have seen that each object can render itself to XML too, so I would just have to submit that to a PHP script for storage, and parse existing XML to rebuild the needed settings... but that's for another blog entry

You can view the whole Game Manager package in my personal WebCVS.

With the current work on patForms, patTemplate and patBBCode, we have started unifying the example collections for all our tools. For this, we have built the patExampleGen class, which manages each collection by building the navigation and tabbed example pages automatically.

Using patExampleGen has several advantages:

- It gives us a comfortable way to manage all the examples for a tool by having all example descriptions in a central location.

- It makes it easier for the developers to view the examples, as you have both the output and highlighted source code for each example directly in the examples interface.

- Depending on the tool, some additional info like the template source code for patTemplate examples is also generated automatically via the extension features of the class that enable you to add your own dumpers.

- The actual source code in the examples is kept at a minimum to be easy to understand by having the exampes framework render any needed output like the frame of the html pages.

- The tabbed interface for each example is also extensible, making it easy to add specific information on a per-example basis (like adding some extra notes, for ex.).

If you want to take a look at it, you can grab a copy of the CVS version of patForms (WebCVS), or patBBCode (WebCVS) and have a look at the examples/patExampleGen folder in the distribution. It is not currently available as a separate tool for download, but may become in the future. We welcome any comments on patExampleGen in our forum!

A recent entry in our patSpiderizer forum initiated me to renew the discussion about the ethics of cloaking. patSpiderizer enables you to create spider pages with contents tailored for a spider's needs, with the motivation to help websites with few indexable content (like flash-sites) to get listed in search engines. At the time of the inital release of patSpiderizer in 2001, about everybody was involved in cloaking, and everyone had his own more or less successful cloaking recipe. Can you remember those pages with a giant keyword block at the bottom with black text on black background? At the time, this worked, as the spiders did not filter those pages out yet.

What has changed since then? Not much - the cloaker/spider war goes on, each side inventing new ways to screw each other. Did you know that some spiders actually check the grammatical structure of your website's content? Forget keyword lists All this nonsense awakens my urge to break a few necks. Imagine all that fake content automatically generated from keyword lists, just to get to the top of search engine rankings until the spiders filter those out too... I would not want to have to write the search algorythms needed to get intelligible search results from that heap of garbage.

patBBCode is our custom pat BBCode parser, with a slightly different syntax than the widely known BBCode. pat BBCode features tags with attributes, empty tags and even tags with custom namespaces. Tag filters enable mapping classes to tags with a very simple to use API for quick development.

patBBCode uses a sax-based parser to parse the tags, so that developers who want to add their own tags need not worry about that part. A filter class only needs to implement the createContent() method, and gets the content and attributes of the tag to work with. A small, custom template class makes it very easy to generate the needed content from any number of tag template files.

Simple tags that need not be mapped to a class are stored in .bbt (patBBCode templates) files used to render the content of the tag, e.g. the tag template for the [B] tag could look like this: <b>%s</b>, where %s stands for the content of the tag.

The first available version of patBBCode still used rather unflexible tag configuration files, and on Stephan's advice I implemented tag configuration readers. The main patBBCode class takes care of parsing the tags and rendering them from the tag template files, while the tag configuration readers are used to determine wether a tag exists and is allowed. Currently two readers are available:

• The AutoFile reader is the default reader, and determines whether a tag exists and is allowed by checking if the template files are available in the filesystem.


• The INI reader reads tag configurations from .ini files, enabling easy switching between tag configurations.

You can have a look at the CVS tree for this project at cvs.php-tools.net, and grab the current CVS snapshot from snaps.php-tools.net.