Lot 49: Greg Beaver's blog - Computing

PHP Advent, xUnit testing with phpt

nospam@example.com (Greg Beaver) — Thu, 10 Dec 2009 11:13:00 -0700

Yesterday, the article I wrote for PHP Advent went live at http://phpadvent.org/2009/testing-with-phpt-and-xunit-by-greg-beaver.

This is a simple, and potentially controversial article about what makes a good testing framework, and an alternative to PHPUnit for PHP users. If you have any feedback, feel free to post comments here.

How to put the FAIL in open source

nospam@example.com (Greg Beaver) — Tue, 17 Nov 2009 07:44:21 -0700

There has been a bit of buzz about the new PHP standards working group (self-named) that has started work on a real cross-framework standard for the naming of classes and namespaces in PHP. The idea is a wonderful one, and at first things looked like they were nothing but good. A group of framework developers met at php|tek, whipped up a quick draft, and after returning home, started a new mailing list, http://news.php.net/php.standards to discuss finalizing the draft discussed at php|tek.

At this point, signs of trouble began to crop up, and the good intentions began to result in a chill on the openness with suggestions of reducing input through moderation and discussions of who was a "member" of the standards group. None other than Rasmus made a plea for more openness, which was answered by a resounding "no thanks" with the creation of the moderated php-standards group at groups.google.com.

Since then, I have made several pleas, off-list and some on, to open the discussion and move things back to the way open source and specifically PHP works. The reasoning is simple: open source thrives on contributions from all levels of hierarchy, not just a chosen few people who have developed prestige and karma by working on large-scale popular frameworks. You don't have to take my word for it. Immediately after the creation of the list, many non-members found immediate flaws in the standard, some of which were fixed, and many which were ignored and are now causing trouble on the moderated list. This, ironically, is what led to the moderated list. All I can conclude is that the people who came up with the standard actually are actively seeking a way to avoid valid input. Harsh? Perhaps. Let's take a look at a recent thread:

http://groups.google.com/group/php-standards/browse_thread/thread/5b85ca398ba90289 epitomizes the closed nature of the standard. I find it appalling.

If you feel strongly that we need a standard, and want one that will actually be useful, you need to contact the people in the projects and demand that they open the process, or it will not happen, and the standard that they come up with will be hobbled by the lack of openness. Nothing is worse than a standard that lacks a mandate and the legitimacy of being both open and well thought-out. Make your voice heard!

How to manage all that noise? Where PHP development and chamber music rehearsal meet

nospam@example.com (Greg Beaver) — Sun, 28 Jun 2009 17:54:34 -0700

Lately, there have been many well-intentioned but I would contend misguided ideas proposed to handle noise on public programmer mailing lists. The premise is that in fact there are very important messages being drowned out in a chorus of irrelevant messages from ill-informed developers. Warnings of inefficiencies have been tossed about quite a bit as the curse of "bike shedding."

To understand what they mean by "bike shedding," one must know the bike shed parable. Once upon a time, a group of people got together to build a nuclear reactor to power the city. Everyone in the town said "great, we need more power," and so they got to work designing the plant. Everything was going great, the design of the reactor sailed by until one day, one of the citizens said "hey, let's build a bike shed for the workers so they don't have to come to work in cars and can get exercise." Everyone agreed until an argument broke out over whether to paint the shed blue or red. Eventually, the whole process broke down, and the plant was never built because they couldn't choose a color to paint the bike shed. The moral of the story: everyone agrees on the complex important things, but the process breaks down over arguments about irrelevant details.

I think we can all agree that this is a tragic parable, illustrating what happens when the process for decision-making breaks down.

When there is a problem of signal-to-noise ratio on a mailing list, the problem has to do with the process for accepting public input, I completely agree with this. However, I am troubled by the implication that having unmoderated mailing lists is the intrinsic problem with open source that must be solved. There is a crucial balance between rewarding quality and being open to the outsider that is the life blood of open source. You could even think of it as the "affirmative action" of the programming world: we recognize that amateur programmers who do not come from the establishment of the temples of programming in academic computer science departments or even computing careers, may have ideas that are better than the most learned highest karma achiever.

For my musician reader: Open source thrives on meritocracy: the idea that those who can do junk have more influence than those who can't do as much junk. Karma is the thing with which these individuals are rewarded. Only those with karma are able to actually make changes to the source code. It doesn't matter who you are, or what your background is, if you demonstrate good coding and community, you get karma.

String quartet rehearsal is similar, the best ideas get more weight regardless of whether the person who has them is playing the melody.

Why am I talking about string quartet rehearsal in a post about moderated programmer mailing lists? I come from a profession (professional chamber music) where "karma" is a fluid object, and unlike the programming world tends to break down when we attempt to nail it down. My string quartet has found that rather than try to decide who will be "the decider" for tough decisions, it's not just friendlier, but is also far more efficient to devote our individual energies to two essential things:

always try to find the bigger picture
always try to understand and respect everyone else's crackpot stupid ideas

Sometimes they are crackpot, but most of the time, the reason they just seemed crackpot because they were outside the boundaries of our collective imaginations.

An example: in our first years together, we occasionally would get into endless arguments in my quartet about whether to play a particular musical section faster or slower (PHP developers: think endless namespace separator arguments). After a long time of arguing about this, it usually turned out that we were asking the wrong question, until one day we would find the right question, which was something like "how long are the phrases?" (PHP developers: the right question turned out to be "how do we solve the ambiguity between static methods and namespaced functions?"). After finding the right question, we stopped arguing nearly as much, and the music sounded much better after we all re-adjusted our thinking.

This illustrates an important distinction: the question of how fast to play was not an irrelevant "bike shedding" detail, it was a crucial part of the solution. We still had to choose a speed to play, just as the nuclear power plant people still would need a place to put their bikes. By changing the question from "How fast?" to "How long is the phrase?" we were actually solving the question of what color to paint the bike shed at the same time we solved the question of how to design access to the nuclear reactor, not ignoring it.

The best solution is for those with the most karma to use this power to redirect unfruitful arguments towards larger questions that can lead to a more natural solution to the problem, not to limit external input and even conflict.

Perhaps the principal designers of the ill-fated hypothetical nuclear power plant would have done better to instead suggest that the bike shed be integrated into the reactor building. This would protect the bikes from the elements, and also require any bike thieves to go through security to get to the bikes - and eliminated the question of what color to paint the shed entirely without ignoring it.

It takes hard work and more than a little gut-wrenching conflict to resolve a truly important, difficult problem, and I for one, am willing to tolerate a little noise to find that gem which would otherwise be missed.

code coverage reporting using PEAR, PEAR2, phar, and sqlite3

nospam@example.com (Greg Beaver) — Mon, 13 Apr 2009 17:10:53 -0700

I've recently become enmeshed in developing Pyrus, PEAR's next generation installer (code coverage is moving up, 20% more coverage than my last blog post). This blog post details my journey in making the process of verifying the correctness of the development via testing and code coverage smoother, exponentially faster and more robust as a result.

One of the problems I found when designing the new code for PEAR 1.4.0 (back in the day) was that it was very difficult to determine whether changes would break things. The main problem revolves around the colossal size of the test suite. Because of the complexity of some of the tests (they must download files from a fake internet and then install them into a fake registry and then verify it actually works), the suite takes between 15 and 35 minutes to run, depending on your hardware configuration and PHP version.

This is a real problem when trying to develop with any kind of flow. If, after every change, one needs to sit through 35 minutes of tests, one will never develop anything of substance. Thus, typically I ran a subset of the tests, develop quickly, and then do a run of the entire test suite to make sure there are no unintended breaks in the other code. In addition, I had to simply assume that new tests I design actually execute the code in question, as it is not always possible to verify this directly in a passing or failing test.

The obvious solution here is to generate a code coverage report and go by that. This, however, still does not fix the problem of accidentally not running tests that are affected by a change, and generating a code coverage report for the entire project is even more time consuming.

What to do?

There are a couple of solutions. One is to set up a continuous integration server, and simply do minimal testing at home, and then commit-and-hope, fixing issues as they appear. Although this is a process that is used at php.net very successfully (see gcov.php.net), it doesn't fit my needs: I need to know right away whether I am testing the code I intend to test, and whether I am breaking anything else.

The solution I am toying with is a step beyond what PHPUnit has to offer currently (yes I am aware of other testing frameworks, for all of you doubters out there, and have nothing against them except that I personally find my development to be more lithe with the phpt test style). PHPUnit offers xdebug code coverage reports, and an option to do some fancy logging, to xml and to a database.

PEAR's run-tests command offers xdebug code coverage raw data, saved as a var_export()ed array, but no reporting. PHP's buit-in run-tests.php runner for phpt tests does not offer xdebug code coverage at all.

Neither of these solves the problems I mentioned above. What I dream of is a program that can automatically determine what files - test suites or code - have been modified, and automatically run tests that are affected by the changes, and then create a coverage report that I can view instantly.

Well, the dream is now a reality. Using a combination of the old and new, it is now possible to do near-instantaneous super-targeted test execution and instant code coverage reporting.

There are 4 simple steps and 1 bigger step involved in making this a reality:

install PHP 5.3 or newer from either the recent RC2 (or stable release, if you're reading this in the future) or from CVS
install PEAR 1.8.0 or newer
svn checkout http://svn.pear.php.net/PEAR2/all
copy all/Pyrus_Developer/src/Pyrus/Developer/CoverageAnalyzer/pear2coverage.phar.php to a web directory

Once you have done these four, you need to grab test-modified.php from http://cvs.php.net/viewvc.cgi/pear-core/test-modified.php?view=co&revision=1.4&content-type=text%2Fplain and change the paths for $codepath and $testpath to point at the project you are testing.

Next, run test-modified.php --norender. This will run the phpt tests, and create a pear2coverage.db sqlite3 database in the tests directory.

Finally, browse to pear2coverage.phar.php in your browser, enter the /full/path/to/pear2coverage.db and voila - a fully working instant coverage report.

This setup is the most involved part. As you develop, when you're ready to run tests, simply re-run php test-modified.php --norender, and the tests will be re-run, re-processed, and you will have a full coverage report available within about a minute's time.

I've been using this to develop Pyrus (located in your all checkout at all/Pyrus), and it's been great.

I have plans to examine the feasibility of processing the XML files that PHPUnit outputs, as this would allow generating coverage reports from phpunit-based tests as well. Unfortunately, I'm not sure the database format PHPUnit supports is simple enough to work for my needs, as it has much more detail (such as which classes/methods are declared in files), in order to support more abstract coverage reporting. Thus, I'm not likely to add support for that into this coverage reporter, unless I can be convinced otherwise.

This also demonstrates the phar extension's power. The web application is entirely contained within a phar archive, but web requests can be made for individual files directly, thus the generated html requests a .css file from within the phar, and it is served up by ext/phar with the correct mime type without any additional programming.

It's an exciting time to be developing in PHP, if you ask me.

Pyrus, PEAR2 and web code coverage report for phpt-based tests

nospam@example.com (Greg Beaver) — Sat, 14 Feb 2009 15:55:29 -0700

It's been a while since I reported on any of my PHP activities, and this update is long overdue. I must apologize to Davey for not doing my 7 things, and to all of the facebook people for whom I didn't write 25 things. I am somehow not at all motivated to do this...

In any case, now that work on ext/phar has shifted primarily to maintenance mode, and namespaces are finally ancient history, I've shifted all of my coding energy to getting Pyrus, PEAR's next-generation installer, ready to ship. Pyrus is basically the PEAR installer completely redesigned for PHP 5.3, and it takes advantage of as many of PHP's new bells and whistles as makes sense. Some of the exciting features:

Full support for .tar, .tgz, .tbz, .zip, and .phar archive formats out of the box, complete with signing via md5, sha1, sha256, sha512, or asymmetric true signature via openssl signature (all from ext/phar, which is built into PHP 5.3)
primary registry is SQLite3-based, which is much more robust, and can be relocated very easily, plus support for XML file-based registry as well as PEAR 1.x's serialize-based registry
redundant registries means ability to reconstruct a corrupted install
configuration is stored with the installed files, no more hunting down errant config files, user preferences are stored in home directory or equivalent
for developers: easy generation of package.xml straight from source code checkout
also for developer: easy generation of channel REST files directly from release tarball, no need to install a package (like the old Chiara_PEAR_Server) or even run PHP on the channel web server.

There are many other great features for users who would like to extend Pyrus, including a seamless API for accessing package.xml which is identical regardless of whether the package.xml is from an .xml file, an archive, or an installed package's registry.

Of course, all of these features only matter if they work, and to move along that path, I've been spending my time unit testing up a storm. Using the run-tests command of the old PEAR installer, I'm writing and refining tests. With the addition of xdebug code coverage (pear run-tests -rx does the trick), it is possible to see where stuff is being executed. Unfortunately, I don't like the existing reporting tools (call me a snob), so I whipped up a very simple coverage analyzer and popped it into the subversion where all of PEAR2 is stored.

When I say simple, I mean here is the script for generating the source (svn link):

<?php
namespace {
function __autoload($c)
{
$c = str_replace('PEAR2\Pyrus\Developer\CoverageAnalyzer\\', '', $c);
include __DIR__ . '/' . $c . '.php';
}
}
namespace PEAR2\Pyrus\Developer\CoverageAnalyzer {
$a = new Aggregator(realpath('../../../../../Pyrus/tests'), realpath('../../../../../Pyrus/src'));
if (file_exists(__DIR__ . '/test')) {
foreach (new DirectoryIterator(__DIR__ . '/test') as $file) {
if ($file->isDot()) continue;
unlink($file->getPathName());
}
} else {
mkdir(__DIR__ . '/test');
}
$a->render(__DIR__ . '/test');
}
?>

Highlighting is a little wonky because this is PHP 5.3+ code, but my blog is running 5.2.

The output is at http://pear.php.net/~greg/coverage/. This is a dumbed-down version of gcov.php.net, which I found extraordinarily helpful when writing the unit tests for ext/phar, but unlike gcov, there are some fantastically helpful features. The short list:

summary of all executed files and their coverage
project coverage (lines executed, lines covered, and the percentage of covered code)
annotated source showing the lines covered with hyperlinks to lists of tests that cover each line
this one excites me: per-test code coverage.

For instance, this report shows the tests that execute line 660 of PackageFile/v2.php, next this report shows the files that the PackageFile_v2/basic/file.phpt test actually executes with their proportional coverage, and finally this report shows

the lines in PackageFile/v2.php that are executed by the file.phpt test.

This last part is extremely useful to me, I have already used it to determine when a test is not executing the lines I expect it should be, and to fix problems such as using isset() instead of array_key_exists(), which would otherwise be very difficult to track down.

As you can also see, only 30% of the source to Pyrus is covered by the current tests, and so there is a long ways to go to full coverage. If any of this work interests you, feel free to contribute, PEAR2 has a much lower barrier to entry than PEAR, and as you can see, we need all the help we can get to bring this baby off the ground.

Music, PEAR, phar, namespaces and Macs. Oh my!

nospam@example.com (Greg Beaver) — Tue, 09 Dec 2008 21:00:17 -0700

Why the tri-partite title? I've been silent for a while on the blog, but the title sums up my life for the past several months. What have I been doing? Aside from some crazy music work learning a blisteringly difficult Elliot Carter String Quartet No. 4 as well as Steve Reich's Different Trains, Bartok's 6th Quartet, bringing back Korngold 3rd Quartet, learning Carter's Figment for solo cello, and performing Bach's 5th Suite for Unaccompanied Cello for the first time, I've also been pretty active in coding. I acquired a new Macbook Pro with VMWare Fusion running Windows XP and Ubuntu, and it has made development super-easy. I also put in a significant effort on the phone getting Barack his electoral vote in the Omaha area, which seems to have paid off. So, here's the long version of what I've been doing.

Music and Macs

first the music

The past month has seen my time consumed learning Elliot Carter's 4th String Quartet for the 100th birthday thing in Boston. Although I have not learned the others, a few veterans have reported that it is the most difficult. I certainly found it to be the most difficult thing I've ever learned by a long shot. It took me almost 15 hours of work to be able to play the first page (only 15 measures of music) at 3/4 of the printed tempo, and the piece is 35 pages in the cello part. To contrast, page 35 of the 10 Celebrated Mozart Quartet cello part is the third movement of 6th string quartet in the book. Carter 4th's cello part is as long as 6 Mozart quartet cello parts. However, it is shorter than some of the 4 page Mozart quartets, because it is frickin fast and there are lots and lots of notes. It is the most holy s*** hard piece I've ever learned, and for a while, I was scared it just wasn't going to be possible.

In addition, one of Carter's big compositional inventions is the metric modulation in which a tempo change has a distinct rhythmic relationship. He uses some pretty normal ones in the first movement (the 16th note becomes the new triplet), but the other movements have fun ones such as the 16th note becoming the new septuplet. Because most of my part is written in septuplets, this can be difficult to do. Speeding up septuplets so that you fit 7 into what used to be the space of 6 septuplets is not an easy thing to learn at all.

...then the mac

...unless you have a metronome that can do the metric modulation. We had been using a web-based metronome to plug into our studio's stereo to rehearse, because it had a more pleasing beat than any of our metronomes have, and it occurred to me that I could probably pretty easily write a metronome that is capable of doing metric modulations on my new Macbook Pro.

So, I took it upon myself to learn Objective-C and the Cocoa framework in Xcode to write a metronome. It took me just over a day to learn Objective-C; it's very easy for a C coder to pick up, and the syntax is pretty simple. Learning Cocoa fully took quite a bit longer, but after 3 days I had a working metronome that could do metric modulations (http://chiaraquartet.net/ChiaraMetronome1.0.0.dmg). It had several flaws, not the least of which was the inability to insert a measure group, no way to save your work, and some odd behavior on the first beat (sometimes it is late and fast).

After a few more days of digging, I had figured out most of the quirks (and discovered bindings and Core Data, which simplified things) and had the interface greatly simplifed (http://chiaraquartet.net/ChiaraMetronome1.2.2.dmg). I added the ability to start at any measure in the piece, and to have lead-in measures (which allow you to start the metronome, sit down, find the beat, and then start playing).

Then, I discovered document-based Core Data, which allowed saving files, resulting in the next generation (http://chiaraquartet.net/ChiaraMetronome2.0.0.dmg).

Finally, not to leave an unturned stone, I added the final features, the ability to specify whether a beat is accented, beated or silent, which allows implementation of uneven compound meters (like 5/8), and the ability to accelerate or decelerate tempo, and we have the final version http://chiaraquartet.net/ChiaraMetronome3.0.0.dmg. This version contains 3 sample metronomes, the complete Elliot Carter 4th quartet we used to practice the tempo changes, a sample demonstrating accelerando/ritardando, and a sample demonstrating how the metronome can be used to practice complex polyrhythms like 6 against 7.

In my potentially humble opinion, the metronome is my crowning glory as a programmer, it is incredible for learning complex tempo relationships for composers like Carter and Bartok who call for it. It's also important to note that this metronome should NEVER be used for learning music by composers such as Brahms or Mozart. If I hear of anyone using it for that, I will hunt you down and make you listen to Beethoven 9 with your eyelids peeled back watching Teletubbies.

Short feature list:

free, released as a beta with no implied license or warranty. I still own the copyright, so you can't use it in your own programming projects or sell it without infringing on the copyright. You can use it for your own practicing, and I will eventually release the source under an open-source friendly license once I have time to research the options.
Please send me any metronome files you create, I'll start a public repository that people can download them from
requires Mac OS X Leopard (version 10.5.0) or newer
universal binary
relative tempo slider can be used to practice under tempo and click up the metronome, preserving tempo relationships at tempo changes
ability to program in endless tempo (0 bars) or a limited number of bars of a tempo
ability to do compound meters, both even and uneven
ability to do accelerando/ritardando, including ones that cross tempo group boundaries using the tempo calculator
ability to start and end accelerando/ritardando mid-bar. No support for mid-beat start/stop, but this can be simulated by doubling the number of beats and making the mid-beats silent. For example: instead of 4 beats at 60 bpm, choose 8 beats, and make 2,4,6 and 8 silent, with double the tempo, 120 bpm, then start the accel on beat 4 to start on the and of 2.
complete undo/redo support
ability to split up a tempo group, insert a new tempo group, or copy a set of tempo groups and paste them (copy button)
ability to play only a limited set of tempo groups, simply shift-click or command-click the tempo groups. This can be used, for instance, to skip a repeat in practicing.
ability to accent all downbeats with a simple click.
start at any bar in the piece.
lead-in bars to help prepare tempo prior to playing.

PEAR

Today, I upgrade pear.chiaraquartet.net to the latest version, 0.19.0, which Brett Bieber has been working hard to perfect. The look is a bit cleaner, and I also replaced the defunct Crtx_Frontend_Chiara (or whatever it was called) with the newly minted Chiara_Pear_Server_Frontend_Web which is a direct port over of the old frontend with bugfixes and visual tweaks.

PEAR and phar

A high priority for PEAR is minting its next-generation installer, Pyrus. Pyrus is based on PHP 5.3+, and thus I expended a tremendous amount of energy on the new phar extension,which will be enabled by default in PHP 5.3. The phar extension provides several must-have features for Pyrus:

ability to run out-of-the-box applications without installing or extracting them (a la Java's jar) in zip, tar or native phar file format
ability to create data archives in tar or zip format and convert between them
ability to automatically handle oddities of running an application inside an uncompressed file
web-based applications without extraction

I'm very pleased this will be built into PHP 5.3, it cuts out oodles of code from Pyrus, and will significantly affect both its performance and memory footprint.

PEAR and namespaces. Oh MY!

Also important for PEAR2 is making it easier to work with the names, so I've been fighting in the namespace wars. For a while there, it was looking like namespaces would go the way they had in the PHP 5 beta cycle: to the trash can. With a little bit of work, we found a solution no one is 100% happy with, but having tried out several possibilities, I must say the solution we chose is something we can be proud of. There have been many complainers, but the irony I have seen is that most assert they never planned to use namespaces in the first place, which begs the question: why flap your trap if you have no plan to use the thing? In any case, you know where I stand on the issue. I wrote the new manual as well as the first draft of the patch that made it possible to end the bickering on internals. A big thanks to Dmitry Stogov and Stanislav Malyshev for their patience, and especially persistence in working out the solution. Also thanks to many others who fielded opinions on-list, on IRC, and through private email. Anyone who contributed a real concern rationally and sometimes forcefully deserves credit for the final implementation, and time will prove that it is a very good one for those who need it.

I am, of course, referring to the now infamous decision to use the backslash (\) as the namespace separator in PHP's implementation of namespaces. I won't go into the rationale, except to say that PHP, unlike other languages, implements a very loose autoload implementation, which makes it impossible to do the most common solution to resolving a name conflict between a static method and a namespaced function - disallow that. In other words, it's not possible to detect the conflict unless autoload is called on every potential namespaced function call, which introduces a potentially exponential hidden performance loss. As Stanislav so eloquently put it, name conflicts have not caused major websites to go down, but performance issues have. Thus, anything that calls autoload where it may be unnecessary is an unacceptable solution, and the only way to safely handle the ambiguity is to eliminate it.

That is what using the backslash does, because the syntax is different between a namespaced function call and a static method. Complain or ridicule if you'd like, the rest of us will simply shut up and code.

This decision has several other benefits too, including the ability to audit someone else's code and figure out what you should grep for, a namespace or a class name. This also translates into better autocomplete in future IDEs, and so on.

Wrap up

So, it's been a long blog post, but a lot has happened in the past several months, and it's time to get cracking on the future. More will follow as time allows.

pear.chiaraquartet.net back online

nospam@example.com (Greg Beaver) — Tue, 04 Nov 2008 22:41:59 -0700

For those of you who were wondering where pear.chiaraquartet.net went, we suffered the same fate as Josh Eichorn's blog and other sites, but all has been restored.

Macbook Pro lives up to expectations

nospam@example.com (Greg Beaver) — Thu, 14 Aug 2008 15:35:30 -0700

Well, it's been a long and practically empty summer for me in terms of coding, what with hardly any internet access and even less time. I did manage to have time to improve the scheduling program I wrote for Greenwood Music Camp where we are in residence, but that was it.

Now, a new leaf has been turned over. The Chiara Quartet has a residency at the University of Nebraska-Lincoln that began in 2005 for a 3-year term. Much to our delight, the residency was renewed for another three years, and converted into a permanent residency. None of this has much to do with programming, but a perk of this job is that every 3 years, we get a new computer to replace the old one. This cycle, I requested a Macbook Pro, my first Mac, as I have been using a PC for years.

I had the tech peoples set up VMWare Fusion for me, and so I am currently running Mac OS X, Ubuntu Linux, and Windows XP simultaneously with noticeable performance gain over any previous computer I have owned, including my custom 64-bit Kubuntu desktop. The screen is beautiful (15 inch Matte) and speakers far superior to other Macs I've heard as well as the Toshiba laptop I will soon return. I was able to very quickly set up a working PHP 5.3 with xdebug and Komodo in Mac and linux, and almost have the windows side compilation of PHP working, which will greatly accelerate my development. The ability to share changes with the shared folders between all 3 boxes will be a tremendous boon to my development, this will be fantastic.

It feels good to drop Windows as the native platform too, it's been a long time coming.

PHP namespaces become much better (Derick take note)

nospam@example.com (Greg Beaver) — Fri, 20 Jun 2008 23:33:08 -0700

Recently noted by Derick Rethans in an informative blog post, there is (now was) a problem in the implementation of namespaces in the soon-to-be-released PHP 5.3. I quote:

Take for example this code:
<?php
use PEAR::Date::Interval as Interval;
?>
In PHP 5.3 this would alias the class Interval in the namespace PEAR::Date to the class Interval. For now, this code would work just fine. However, if PHP would introduce a class "Interval" at some point in the future (and PHP can do this as it owns the global namespace) then the above code would suddenly stop working.

This morning, I realized that this would be very easy to fix, and posted a patch to internals@ with a brief explanation. Dmitry committed a fix to PHP_5_3 and HEAD very quickly. The issue raised by Derick is now obsolete, making namespaces not just more useful, but even more powerful than they were.

For instance, it is possible to "override" an internal class, perhaps to add missing functionality that will be present in a future release or fix a bug:

mydate.php:
<?php
namespace Foo;
class DateTime extends ::DateTime
{
// methods here
}
?>

<?php
include 'mydate.php';
use Foo::DateTime; // this essentially overrides internal DateTime class in the current script
$a = new DateTime('2006-05-04');
?>

Those who were lamenting the implementation of namespaces in PHP should take note: if you have a solution to make them better, the turnaround time between proposal to the list and commit can be on the order of a few hours if the idea clearly solves a major problem.

php|tek PEAR/phar/Pyrus talk slides

nospam@example.com (Greg Beaver) — Thu, 29 May 2008 09:46:43 -0700

I've had a few slide requests, so here is a link to the pdf of my slides that Sean Coates put up on the phparch site

http://tek.phparch.com/mats/slides/greg_beaver-pear_phar.pdf

phar is near at php|tek

nospam@example.com (Greg Beaver) — Mon, 19 May 2008 22:33:38 -0700

OK, so my selection of puns is not exactly "funny" per se, but it is true: php|tek is all about phar and variant spellings of the 4 letter word, such as "pear". If you're in the audience for my talk about smart PHP application deployment using phar, PEAR and Pyrus, come say hello, I'm looking forward to meeting just about everyone I've argu^H^H^Hconversed with over the years, and of course you can also check out a copy of my book, I'll stuff a few into the luggage. In other news, phar is tentatively enabled by default in PHP 5.3, marking the exciting next phase in all of the back-breaking work that has gone into it. Check out the complete manual at http://php.net/phar.

Although I perform all the time as part of my day job as cellist of the Chiara Quartet, this will be a very different kind of performing experience and I'm looking forward to the challenge of revealing the true dynamism and excitement that I see in app deployment stuff I've been working on the past several years.

See you in Chicago!

behold pecl/phar and mighty PHP 5.3, also php|arch and php|tek

nospam@example.com (Greg Beaver) — Wed, 26 Mar 2008 15:58:57 -0700

I've been a bit busy, my last PHP post was in October (!) but I do have very good news. Following on the heels of my first php|arch article about the Phar extension in the January issue (http://phparch.com/c/magazine/issue/65), I've been extremely hard at work on perfecting this extension. As a testament to the strength and excitement surrounding the upcoming 2.0 release of the phar extension, my editor at php|arch became a developer on the phar extension after reading and working on the article (hi Steph).

Today, I released phar version 2.0.0a1, the first release of the newly upgraded extension. This extension is a wholly different animal from the last release, 1.2.3. While maintaining BC with the 1.2.x version of pecl/phar, this version adds unimaginable new facilities. The phar extension has transformed from something of a sideshow to a major player, and is being actively considered for inclusion in the core of PHP 5.3. Major new features in phar 2.0:

Phar now supports tar and zip archives with the same API that is supports phar archives. Think of phar as something like the PDO of archive file formats. All phar archives must contain ".phar" in the filename to be executed
Phar has a front controller for web applications that fully handles MIME types, supports mod_rewrite-like functionality with far more flexibility (a php callback is passed the url, it returns the actual file to load) and also the ability to deny access to portions of the phar that are not web-accessible (like library PHP code)
Phar fully solves the problems with current directory and code loading/file loading that plague all other stream wrappers like PHP_Archive and PHK. This allows running complex applications like phpMyAdmin with no code modification directly from the original .tgz after a rename and the addition of a 20-line stub
Phar supports phar:// stream wrappers in include_path for PHP 5.2+, so your require_once-based applications can be pharred up without code modification
Phar supports creation and modification of data-only tar and zip archives (no executable phar stub) via the PharData class. Now you can use phar to create and even convert between file formats. Unlike the zip extension, phar supports bzip2 compression of files, and has no arbitrary limit on open file handles. Unlike the archive extension in pecl, phar fully supports windows, and is the first PHP extension to deliver read/write support for tar archives on both unix and windows.

In other words, phar 2.0 is the "holy crap" release of phar. The manual (which is currently almost fully updated to the API, but not yet rebuilt) is at http://www.php.net/phar. Another way to get to know phar is through the unit tests, which are included in the pecl tarball.

The current release can be downloaded from http://pecl.php.net/get/phar-2.0.0a1.tgz or installed via pecl install phar and windows dll can be downloaded from http://pecl4win.php.net/ext.php/php_phar.dll although it looks like building is having a few hiccups lately.

There are a few known issues to be worked out. One is compatibility with xdebug, which is a trivial problem involving extension load order. Once we work this out, xdebug will support stepping through the source of files within a phar archive in your IDE, a killer feature for those who need to debug things. In PHP 5.3, phar requires a small patch to support the phar stream wrapper in include_path.

...This brings me to PHP 5.3. I've been working hard with Dmitry Stogov, Marcus Boerger (the other phar lead) and a few other list contributors to perfect a patch that will bring stream wrapper support to include_path. This will mean that you can write a userspace stream wrapper, put its url in your include_path and start using it with relative includes such as what PEAR uses to load its code. This powerful feature will allow you to write your code in one way, and run it from several different locations. Expect to see this feature committed in the next week or so, barring major problems.

Lastly, I'm happy to announce that I will be a speaker at this year's php|tek conference in Chicago. It is the first time I've been able to pull myself away from a crazy-busy quartet schedule to attend a conference, and I look forward very much to meeting as many of you as possible while there. My talk will be on smart PHP application deployment, and covers PEAR, Phar, and the new Pyrus installer for PEAR2 (http://tek.phparch.com/c/schedule/talk/d2s6/1). Unfortunately, the talk is at the same as Josh Eichorn's talk on PEAR2 (http://tek.phparch.com/c/schedule/talk/d2s6/0), so we may need to work out some way to shuttle folks back and forth, we'll have to see .

Progress of the highest order is being made here, expect to hear more about the goings-on behind phar and PEAR2 as time permits.

Microsoft and the .NET saga

nospam@example.com (Greg Beaver) — Mon, 29 Oct 2007 13:23:55 -0700

So for the past half year or so, I have been trying on and off to install security updates for Microsoft .NET 1.1 and 2.0. Each time, it hadn't worked. I've now disabled automatic updates because they always fail on these two old updates.

Today, I thought maybe I would take a real crack at fixing this issue, and so did an extensive google search for any information on installing/uninstalling/repairing a .NET installation. There is, not surprisingly, a huge number of problems that people have experienced, this being a Microsoft component. I finally found a page on the Microsoft site that displays instructions on repairing .NET.

Unfortunately, none of the instructions work. I tried to uninstall the component, and it tells me source files are not available. I try re-installing from source, and it tells me that I can't install. I try manually deleting and it tells me the file is in use (yes, I've disabled anti-virus). Because this is an MS Windows machine, it's virtually impossible to tell which file is using the .NET stuff.

Basically, I'm not allowed to do anything productive with the Microsoft stuff on this computer. This incidentally also means I can't install VS 2005 because it needs a working .NET installation, so I can't debug my pecl extensions on Windows. My next computer is going to be a MacBook Pro, and I'm never looking back.

Working towards Pyrus and PEAR2, check it out or miss out

nospam@example.com (Greg Beaver) — Sat, 22 Sep 2007 22:25:26 -0700

Hello all. It's been a very busy time for me, what with suddenly having a full gaggle of new cello students to teach at UNL as well as a fully loaded performance schedule for the quartet, but I have been finding time to code. My priority has been the push towards PEAR2, which means that in addition to attempting to mediate the discussions of what should change from PEAR to PEAR2, I'm churning out code to turn its installer, Pyrus, from vaporware into actual code.

Along these lines, there has been significant progress. The code has been developed on PHP 5.2, but will most likely target PHP 5.3 with the introduction of namespaces. Pyrus completely revamps the way things are done. By using SPL as well as extensions like XMLReader and XMLWriter, Pyrus manages to pack in far more features inside the code while literally cutting thousands of lines of code. The biggest loser has been the package.xml handling code. More than 2500 lines of code have gone byebye, but the API for reading from and creating package.xml is now as intuitive or more intuitive than simplexml.

In addition, using interfaces, a few design patterns and some clever iterators, adding support for a customized packaging solution is a matter of creating a class with a few lines of code that accepts file contents and where to save them. Pyrus does the hard work of constructing the relative path of files within the archive and pre-processing their contents using file tasks.

The code is simple enough that I wrote a simple channel server with no dependencies except for Pyrus in less than 6 hours (just finished the initial work when my battery ran out today) that allows small channels to work without needing a database or even PHP on the channel server, as all the REST files and releases can be generated and then uploaded from a dev server.

In short, it's an exciting time to be involved in PEAR. If you've been ignoring PEAR, perhaps it is time to take another look?

phar gets a major speed boost with version 1.2.1

nospam@example.com (Greg Beaver) — Wed, 29 Aug 2007 21:15:23 -0700

A few minutes ago, I released phar version 1.2.1. phar is a PHP extension that allows the creation of complete filesystems within a single physical file. In addition, a bootstrap stub written in PHP can be used to run a phar archive as if it were an executable file or shell script. Documentation on its usage and how to create phars is at http://php.net/phar in the PHP manual.

This version of phar fixes a number of issues as documented in my last blog post, as well as a quite few issues that Marcus fixed in relation to setting the bootstrap stub, using phar's phar.extract_list INI parameter with an apache module and others (full list here).

In addition, I ran callgrind on the thing and found a few places it could use some trimming of the fat, and was able to cut down the number of stream operations by a factor of 2x when using phar in the most common use case, reading uncompressed files. In addition, I cut another 2x for phars that have an MD5 or SHA1 signature.

This is quite a good release, looking forward to hearing your stories of using it.