Lot 49: Greg Beaver's blog

PHP Advent, xUnit testing with phpt

nospam@example.com (Greg Beaver) — 2009-12-10T18:13:00Z

Yesterday, the article I wrote for PHP Advent went live at http://phpadvent.org/2009/testing-with-phpt-and-xunit-by-greg-beaver.

This is a simple, and potentially controversial article about what makes a good testing framework, and an alternative to PHPUnit for PHP users. If you have any feedback, feel free to post comments here.

How to put the FAIL in open source

nospam@example.com (Greg Beaver) — 2009-11-17T14:44:21Z

There has been a bit of buzz about the new PHP standards working group (self-named) that has started work on a real cross-framework standard for the naming of classes and namespaces in PHP. The idea is a wonderful one, and at first things looked like they were nothing but good. A group of framework developers met at php|tek, whipped up a quick draft, and after returning home, started a new mailing list, http://news.php.net/php.standards to discuss finalizing the draft discussed at php|tek.

At this point, signs of trouble began to crop up, and the good intentions began to result in a chill on the openness with suggestions of reducing input through moderation and discussions of who was a "member" of the standards group. None other than Rasmus made a plea for more openness, which was answered by a resounding "no thanks" with the creation of the moderated php-standards group at groups.google.com.

Since then, I have made several pleas, off-list and some on, to open the discussion and move things back to the way open source and specifically PHP works. The reasoning is simple: open source thrives on contributions from all levels of hierarchy, not just a chosen few people who have developed prestige and karma by working on large-scale popular frameworks. You don't have to take my word for it. Immediately after the creation of the list, many non-members found immediate flaws in the standard, some of which were fixed, and many which were ignored and are now causing trouble on the moderated list. This, ironically, is what led to the moderated list. All I can conclude is that the people who came up with the standard actually are actively seeking a way to avoid valid input. Harsh? Perhaps. Let's take a look at a recent thread:

http://groups.google.com/group/php-standards/browse_thread/thread/5b85ca398ba90289 epitomizes the closed nature of the standard. I find it appalling.

If you feel strongly that we need a standard, and want one that will actually be useful, you need to contact the people in the projects and demand that they open the process, or it will not happen, and the standard that they come up with will be hobbled by the lack of openness. Nothing is worse than a standard that lacks a mandate and the legitimacy of being both open and well thought-out. Make your voice heard!

Transporting a cello: buy a seat? check it? Definitive answers on how to fly with a cello

nospam@example.com (Greg Beaver) — 2009-09-20T05:17:00Z

One of the perennial problems of travelling with a cello is how to transport it. I have been flying with my cello since 1993, and have many experiences with both checking the instrument and with buying a seat for it. This post will answer those burning questions of what to do to transport your instrument safely from one point to another. First, let's start with the only guaranteed unsuccessful strategy I have encountered:

Don't mail your cello

Although large shops such as Shar can marshall the resources to ship instruments, they do so in very different circumstances from what you will be able to do on your own. The few people I know who have shipped their instruments wound up with multi-hundred to thousand dollar repair bills, which was actually a lucky position to be in. The risk of outright destruction is too great.

When to check a cello?

If your instrument is not a very valuable instrument that can be easily replaced if it is utterly destroyed, then it is probably unwise to buy a seat for the instrument. The hassles of buying a seat are going to outweigh the benefits, I guarantee it.

When not to check a cello?

If your cello is very fine, very valuable, or very difficult to replace with an equivalent instrument either in the value that insurance will pay for it, or in any price range, then you should not check your cello. Here are a list of things that can and have gone wrong with both flight cases and regular luggage:

lost luggage - permanently lost luggage happens more often than you would think
destroyed luggage - this speaks for itself, no?
a destroyed instrument inside a pristine, undamaged case
delayed luggage stored outside in sub-zero temperatures overnight
if you're counting on being able to carry the instrument on board because it will fit into the overhead, be aware that the airline can change the type of aircraft with no notice

The last point should be enough to dissuade even the most confident traveler from checking a valuable cello. If you have a flight that transfers, and your flight is canceled due to weather, airlines very rarely allow you to retrieve luggage from the secured area, and it sits in un-climate controlled storage areas for the entire night. This will render a cello to be far more fragile, and if you survive the night without a heart attack from the stress, your broken cello upon retrieval due to its extra fragility from exposure to the elements will push you over the edge.

OK, I'm going to check my cello, what do I need to know?

Continue reading "Transporting a cello: buy a seat? check it? Definitive answers on how to fly with a cello"

Gearing up for a first Beethoven cycle

nospam@example.com (Greg Beaver) — 2009-09-19T05:13:03Z

This year, beginning in December, the Chiara Quartet will undertake our first Beethoven cycle. This comes a year or two before we had planned to do it, but we were offered the chance by Music in Deerfield in Massachusetts, and a quick survey of our repertoire proved that it would be more than possible, as we already have more than half of the quartets solidly in our repertoire. After little juggling of logistics, we committed to 3 concerts this season, starting in December, and 3 next season. We will also be performing the complete cycle in Cambridge, MA at Harvard, and in Lincoln, Nebraska as part of our inaugural Chiara String Quartet concert series at the University of Nebraska-Lincoln.

Once we had the dates in place, the first big question is what order to play them in. Our ordering is as follows:

Concert 1:

Op. 135 in F Major
Op. 18 No. 4 in C Minor
Op. 59 No. 1 in F Major

Concert 2:

Op. 18 No. 3 in D Major
Op. 132 in A Minor
Op. 59 No. 3 in C Major

Concert 3:

Op. 18 No. 1 in F Major
Op. 95 in F Minor
Op. 127 in E Flat Major

and the following season:

Concert 4:

Op. 18 No. 5 in A Major
Op. 59 No. 2 in E Minor
Op. 130 (with revised finale) in B Flat Major

Concert 5:

Op. 74 in E Flat Major
Op. 18 No. 2 in G Major
Op. 131 in C Sharp Minor

Concert 6:

Op. 18 No. 6 in B Flat Major
Op. 130 (with Grosse Fuge) in B Flat Major

There are several obvious choices going on in this ordering. First is that most concerts contain a quartet from each of the three periods of Beethoven's life. A late quartet sits in every possible concert slot, as does a middle quartet. We will play Op. 130 twice in its entirety with the two possible finales, and the entire cycle ends with the Grosse Fuge. Putting together this order also splits up the pieces we don't yet have in our repertoire between the two seasons. Most concerts mix up major and minor keys as well.

How, you might ask, does one prepare for such a collosal undertaking?

The majority of our preparation for this cycle predates the initial request for us to play one. We've been a full-time professional quartet since 2000, and so this will coincide with our 10th season. Individually, we've played a ton of chamber music, and a lot of Beethoven as part of that.

I first heard a Beethoven cycle live when I was 10 years old, and the Juilliard Quartet performed one at the Wharton Center in East Lansing, Michigan. I had only been playing the cello for 3 years, and so my memory consists of the opening chords of Op. 127, and the hairy legs of the quartet members peeping out between their socks and tuxedo pantlegs. What? I was 10 years old, give me a break. If you haven't heard the Op. 127, its opening is one of the most memorable things Beethoven wrote, so you must come hear us play it live in our final concert this spring. Becca's father was in the Concord Quartet and she literally grew up listening to live Beethoven cycles.

One of the first quartets I played in the "Shürtzaut" quartet (a group of 4 13-year old boys who refused to tuck in their shirts) was the second movement of Op. 18 No. 4, and this was one of the pieces that I played under the coaching of Rostislav Dubinsky of Borodin Quartet fame the following year in his Quadro-quartet at IU String Academy. When the Chiara Quartet was formed at Musicorda in 1993, we played Op. 95 and had a chance to play the second movement with Arnold Steinhardt in his masterclass when our original violinist Rachel was sick. The following year, we coached Op. 59 No. 2 very intensely with Norman Fischer of Concord Quartet fame (and soon to be my teacher at Rice University). That same year, a group of students decided to practice the late quartets for a week and then to read through them. While at Rice, Jonah and I learned Op. 59 No. 1 with Paul Katz of Cleveland Quartet fame as our coach. At Tanglewood in 1999, I played Op. 127 with a group that coached with both Robert Mann and Raphael Hillyer of Juilliard Quartet fame, and this was one of the pieces with which the Chiara Quartet made our Alice Tully debut in 2005. Op. 59 No. 3 was one of the pieces with which we made our Carnegie Hall debut the following year, and one of the pieces that helped us win a prize at the Borciani Competition the same year. Op. 18 No. 3 was one of the pieces we played in the year we won first at the Fischoff Competition. While graduate students at Juilliard, Jonah, Becca, Anna Elashvili and I played the Op. 130 with the Grosse Fuge on her graduation recital after coaching with Robert Mann.

Even with a list this long, the preparation for a cycle is more than just learning the quartets. I played all of the sonatas and variations for cello in 2000 in New York (and will repeat the performance of all the sonatas this year). Julie, Noori and I have been listening to Richard Goode's recording of all the piano sonatas, and in my orchestral days, I played all of the symphonies except for Number 8. We've read biographies of Beethoven, looked at actual manuscripts at Beethovenhaus in Bonn as well as facsimiles online, and heard cycle recordings by the Juilliard, Emerson, Takacs, and many recordings by the Concord, Cleveland, Tokyo as well as performances by Brentano, Borromeo, Biava, Miro, Pacifica and other quartets I'm sure I'm forgetting, as well as student performances of many of the works.

We are extremely excited to embark on this monumental journey and hope that you will join us in the first performance at Harvard on Wednesday, December 2 in Paine Hall, or at Smith College in Northampton on Saturday December 5 in Sweeney Concert Hall on the Music in Deerfield chamber music series. If you're in Nebraska, you can hear us threepeat the performance on Sunday, December 13 in the Sheldon Art Gallery at 7:30 PM

Redesigning the blog the lazy way

nospam@example.com (Greg Beaver) — 2009-09-19T05:04:52Z

After several years with the old template, I decided it was time to upgrade, and am trying out this new template. It squishes the entries quite a bit more than the old one, but has a nicer look/feel. Let me know if you like/dislike it.

Why health care reform is doomed in the United States

nospam@example.com (Greg Beaver) — 2009-09-05T05:36:16Z

I recently remarked on my facebook status that health care reform was doomed in the current intellectual climate, and urged those reading to prove me wrong. Unfortunately, I don't see any evidence that I was wrong yet, perhaps the president's speech on Wednesday will change that view, but I am not holding my breath. There are three excellent articles that capture the essence of my skepticism. Together, they are a whopping 15 pages of prose (that's a lot for news articles). Basically, I have seen three prevailing views on health care reform amongst my friends and my facebook friends:

We have an obligation to fix the system, support universal health insurance coverage (i.e. support the various Democratic proposals)
You're crazy to believe that government-run health care is anything other than an IRS agent with a scalpel and the efficiency of the post office, plus it's a violation of privacy rights to turn over health care to the government.
market-based reforms are the only logical choice (i.e. tax credits for buying insurance, or insurance exchanges, or co-ops, etc.)

Note that I did not include in that list anything about the crazies, because none of my friends are insane. At least not the kind of insanity as represented by Jon Stewart's answer to Barney Frank's question to a constituent at his town hall meeting "On what planet do you spend most of your time?" Stewart's answer: "apparently, a planet where a mixed race president and a gay Jew qualify as Nazis."

No, my friends are rational, primarily college-educated, and mostly split between liberal-leaning musicians and conservative-leaning programmers (that split in itself is interesting to me, but is not relevant to this post). Before I go any further, I should acknowledge that yes, I am a liberal-leaning musician. In my own defense, I am liberal because of my life experiences. I am good friends with someone who was formerly homeless, have been screwed by my health insurance companies, and have lived in poverty in a poor neighborhood, as well as benefiting directly from government student loans. No, I am not a communist, thanks for asking though.

Why is health reform doomed? Not because the crazies are dominating the dialogue, far from it. It is not because the Democratic plan is too expensive, too invasive of privacy, or too government-focused and not enough market-focused. It is because we are asking the wrong questions in order to solve this, and nobody (I'm looking at you, Obama) is correcting us.

I have read large portions of both the contentious H.R. 3200 house bill, as well as nearly the entire H.R. 3400 (a little-known Republican bill that is popular amongst conservatives on facebook as a better way to reform health care). There are some obvious problems in both. Neither fully pays for itself (H.R. 3400 flat out doesn't provide any revenue in the bill, instead relying upon the hope that cutting costs will pay for it). However, the flaws in both bills jump right off of the page when one examines this article by David Goldhill in the September 2009 issue of the Atlantic:

http://www.theatlantic.com/doc/200909/health-care

To summarize the salient points, the health care system does not have any kind of market mechanisms to set prices in place, and this is a direct result of the introduction of comprehensive health insurance in 1954. Unlike other forms of insurance (car insurance, for example), comprehensive insurance pays for the equivalent of oil changes, which both disguises the true cost and encourages severe inflation.

Thus, Goldhill concludes, the problem is not that not everyone has health insurance, it is that we don't pay for health care, our insurers do, and so we have no say in pricing. He also goes on to recommend both mandatory health savings accounts to replace health insurance for predictable expenses such as checkups, the occasional equivalent of a fender-bender for health, and costs associated with aging (with some ideas for managing poverty thrown in), and a mandatory single-payer catastrophic insurance with premiums pinned to age with no other factor taken into account.

At this point, those in the "Obama, get your government out of my [fill-in-the-blank-here]" camp are thinking "single payer? hell no." To you, I suggest reading this editorial:

http://www.nytimes.com/2009/09/03/opinion/03kristof.html

Again, to summarize: government runs several things better than the privately managed things they replaced. The examples in the article are fire departments, police departments, post offices, libraries, and health care (outside the U.S.). I would limit this list to fire fighters and police departments: I doubt that even the most staunch conservative (again, the most staunch sane conservative) would argue that it would be better to outsource firefighting to for-profit corporations. In fact, apparently "In New York City, according to accounts in The New York Times in the 1850s and 1860s, firefighting often descended into chaos, with drunkenness and looting." Wow.

Now as for Goldhill's conclusion of how to solve the problems, there is a flaw in his reasoning as well. To get at the core, we have to examine some of his assertions:

The housing bubble offers some important lessons for health-care policy. The claim that something—whether housing or health care—is an undersupplied social good is commonly used to justify government intervention, and policy makers have long striven to make housing more affordable. But by making housing investments eligible for special tax benefits and subsidized borrowing rates, the government has stimulated not only the construction of more houses but also the willingness of people to borrow and spend more on houses than they otherwise would have. The result is now tragically clear.

This paints a simple cause-and-effect picture that doesn't even begin to capture the problem. His tidy explanation doesn't explain why lenders such as Wells Fargo singled out poor people illegally to net them into sub-prime loans, or why the market of derivatives caused what should have been a localized collapse to poison the entire economy. He also states:

Accidentally, but relentlessly, America has built a health-care system with incentives that inexorably generate terrible and perverse results. Incentives that emphasize health care over any other aspect of health and well-being. That emphasize treatment over prevention. That disguise true costs. That favor complexity, and discourage transparent competition based on price or quality. That result in a generational pyramid scheme rather than sustainable financing. And that—most important—remove consumers from our irreplaceable role as the ultimate ensurer of value.

I don't doubt that pretty much everyone agrees with the description of how health care pricing works. The part that I want to emphasize is in bold (my emphasis, not Goldhill's) In the previous sentence, Goldhill uses this language to describe the doctors/insurers: "They all want to serve patients well. But they also all behave rationally in response to the economic incentives those distortions create."

This language reflects a world view of a businessman who has faith in the power of the free market. Under ordinary circumstances, this is the right view, our history shows that markets very often set the best price, and everyone wins. Unfortunately, the so-called Great Recession we are currently in did not result, as Goldhill seems to be claiming, from people making rational decisions, and markets do not solve all problems. Put on your reading glasses, this next article is the long one:

http://www.nytimes.com/2009/09/06/magazine/06Economic-t.html

This article by Nobel-prize winning economist Paul Krugman is by the guy one could argue is the smartest of the three whose articles I've quoted. Although you absolutely must read the whole article - especially if you have "faith in the market" - I will try to summarize it:

You can't have 100% faith in the market because people are intrinsically irrational, and occasionally the government probably must play a role in stabilizing a market before it spirals into a depression. In short: Keynesian economics was more right than the stuff we've been using to set public policy since the 1970s.

The solution Goldhill proposes for controlling health care costs makes an assumption that people will rationally choose the best price for health care, and bases this assertion upon evidence in fields such as Lasik vision correction, which has had a price drop as the rest of health care prices have skyrocketed. This is a compelling argument, and one I believe is a very good one, but it is flawed for handling all kinds of health care.

People who are under physical or emotional stress very rarely make rational decisions, and when you are having severe chest pains, you are going to do whatever the doctor tells you to do. If the doc says get angioplasty, you get it. If the doc says we're going to open up the chest right now, you do it. Very few people will whip out the "could I see your competitor's prices?" line while clutching their chests in pain. Put simply, emergency care cannot be rationally priced by the consumers of the care. Perhaps emergency care simply cannot have a free market to set prices. This greatly complicates the issue unless one simply decides that the catastrophic insurance would have to cover certain emergency procedures with a deductible, much like what we have now (those of us with insurance, of course).

The non-emergency care portion of health care would respond quite well to a free market, but I would question whether certain procedures wouldn't suffer from the effects of things like positional monopolies (really sick people can't travel to get lower prices).

A bigger flaw is that there is no way to transition from our current system into the new one without horribly disrupting health care for at least some people. The potential for unintended consequences is horrendous, and this will be impossible to navigate politically.

So what kind of questions should we be asking? For one, how can we transform our comprehensive health insurance system into one that does is not comprehensive? After changing this, how do we provide a safety net for people who get too sick to work, for people who are too poor to pay premiums or contributes to HSAs? How can we use government to manage the market breakdowns that will occur with our irrational behavior in things like emergency health care pricing in a market-based health care economy?

As long as the question being asked is simply "How do we provide the existing insurance to everyone?" health care reform doesn't stand a chance.

How to manage all that noise? Where PHP development and chamber music rehearsal meet

nospam@example.com (Greg Beaver) — 2009-06-29T00:54:34Z

Lately, there have been many well-intentioned but I would contend misguided ideas proposed to handle noise on public programmer mailing lists. The premise is that in fact there are very important messages being drowned out in a chorus of irrelevant messages from ill-informed developers. Warnings of inefficiencies have been tossed about quite a bit as the curse of "bike shedding."

To understand what they mean by "bike shedding," one must know the bike shed parable. Once upon a time, a group of people got together to build a nuclear reactor to power the city. Everyone in the town said "great, we need more power," and so they got to work designing the plant. Everything was going great, the design of the reactor sailed by until one day, one of the citizens said "hey, let's build a bike shed for the workers so they don't have to come to work in cars and can get exercise." Everyone agreed until an argument broke out over whether to paint the shed blue or red. Eventually, the whole process broke down, and the plant was never built because they couldn't choose a color to paint the bike shed. The moral of the story: everyone agrees on the complex important things, but the process breaks down over arguments about irrelevant details.

I think we can all agree that this is a tragic parable, illustrating what happens when the process for decision-making breaks down.

When there is a problem of signal-to-noise ratio on a mailing list, the problem has to do with the process for accepting public input, I completely agree with this. However, I am troubled by the implication that having unmoderated mailing lists is the intrinsic problem with open source that must be solved. There is a crucial balance between rewarding quality and being open to the outsider that is the life blood of open source. You could even think of it as the "affirmative action" of the programming world: we recognize that amateur programmers who do not come from the establishment of the temples of programming in academic computer science departments or even computing careers, may have ideas that are better than the most learned highest karma achiever.

For my musician reader: Open source thrives on meritocracy: the idea that those who can do junk have more influence than those who can't do as much junk. Karma is the thing with which these individuals are rewarded. Only those with karma are able to actually make changes to the source code. It doesn't matter who you are, or what your background is, if you demonstrate good coding and community, you get karma.

String quartet rehearsal is similar, the best ideas get more weight regardless of whether the person who has them is playing the melody.

Why am I talking about string quartet rehearsal in a post about moderated programmer mailing lists? I come from a profession (professional chamber music) where "karma" is a fluid object, and unlike the programming world tends to break down when we attempt to nail it down. My string quartet has found that rather than try to decide who will be "the decider" for tough decisions, it's not just friendlier, but is also far more efficient to devote our individual energies to two essential things:

always try to find the bigger picture
always try to understand and respect everyone else's crackpot stupid ideas

Sometimes they are crackpot, but most of the time, the reason they just seemed crackpot because they were outside the boundaries of our collective imaginations.

An example: in our first years together, we occasionally would get into endless arguments in my quartet about whether to play a particular musical section faster or slower (PHP developers: think endless namespace separator arguments). After a long time of arguing about this, it usually turned out that we were asking the wrong question, until one day we would find the right question, which was something like "how long are the phrases?" (PHP developers: the right question turned out to be "how do we solve the ambiguity between static methods and namespaced functions?"). After finding the right question, we stopped arguing nearly as much, and the music sounded much better after we all re-adjusted our thinking.

This illustrates an important distinction: the question of how fast to play was not an irrelevant "bike shedding" detail, it was a crucial part of the solution. We still had to choose a speed to play, just as the nuclear power plant people still would need a place to put their bikes. By changing the question from "How fast?" to "How long is the phrase?" we were actually solving the question of what color to paint the bike shed at the same time we solved the question of how to design access to the nuclear reactor, not ignoring it.

The best solution is for those with the most karma to use this power to redirect unfruitful arguments towards larger questions that can lead to a more natural solution to the problem, not to limit external input and even conflict.

Perhaps the principal designers of the ill-fated hypothetical nuclear power plant would have done better to instead suggest that the bike shed be integrated into the reactor building. This would protect the bikes from the elements, and also require any bike thieves to go through security to get to the bikes - and eliminated the question of what color to paint the shed entirely without ignoring it.

It takes hard work and more than a little gut-wrenching conflict to resolve a truly important, difficult problem, and I for one, am willing to tolerate a little noise to find that gem which would otherwise be missed.

code coverage reporting using PEAR, PEAR2, phar, and sqlite3

nospam@example.com (Greg Beaver) — 2009-04-14T00:10:53Z

I've recently become enmeshed in developing Pyrus, PEAR's next generation installer (code coverage is moving up, 20% more coverage than my last blog post). This blog post details my journey in making the process of verifying the correctness of the development via testing and code coverage smoother, exponentially faster and more robust as a result.

One of the problems I found when designing the new code for PEAR 1.4.0 (back in the day) was that it was very difficult to determine whether changes would break things. The main problem revolves around the colossal size of the test suite. Because of the complexity of some of the tests (they must download files from a fake internet and then install them into a fake registry and then verify it actually works), the suite takes between 15 and 35 minutes to run, depending on your hardware configuration and PHP version.

This is a real problem when trying to develop with any kind of flow. If, after every change, one needs to sit through 35 minutes of tests, one will never develop anything of substance. Thus, typically I ran a subset of the tests, develop quickly, and then do a run of the entire test suite to make sure there are no unintended breaks in the other code. In addition, I had to simply assume that new tests I design actually execute the code in question, as it is not always possible to verify this directly in a passing or failing test.

The obvious solution here is to generate a code coverage report and go by that. This, however, still does not fix the problem of accidentally not running tests that are affected by a change, and generating a code coverage report for the entire project is even more time consuming.

What to do?

There are a couple of solutions. One is to set up a continuous integration server, and simply do minimal testing at home, and then commit-and-hope, fixing issues as they appear. Although this is a process that is used at php.net very successfully (see gcov.php.net), it doesn't fit my needs: I need to know right away whether I am testing the code I intend to test, and whether I am breaking anything else.

The solution I am toying with is a step beyond what PHPUnit has to offer currently (yes I am aware of other testing frameworks, for all of you doubters out there, and have nothing against them except that I personally find my development to be more lithe with the phpt test style). PHPUnit offers xdebug code coverage reports, and an option to do some fancy logging, to xml and to a database.

PEAR's run-tests command offers xdebug code coverage raw data, saved as a var_export()ed array, but no reporting. PHP's buit-in run-tests.php runner for phpt tests does not offer xdebug code coverage at all.

Neither of these solves the problems I mentioned above. What I dream of is a program that can automatically determine what files - test suites or code - have been modified, and automatically run tests that are affected by the changes, and then create a coverage report that I can view instantly.

Well, the dream is now a reality. Using a combination of the old and new, it is now possible to do near-instantaneous super-targeted test execution and instant code coverage reporting.

There are 4 simple steps and 1 bigger step involved in making this a reality:

install PHP 5.3 or newer from either the recent RC2 (or stable release, if you're reading this in the future) or from CVS
install PEAR 1.8.0 or newer
svn checkout http://svn.pear.php.net/PEAR2/all
copy all/Pyrus_Developer/src/Pyrus/Developer/CoverageAnalyzer/pear2coverage.phar.php to a web directory

Once you have done these four, you need to grab test-modified.php from http://cvs.php.net/viewvc.cgi/pear-core/test-modified.php?view=co&revision=1.4&content-type=text%2Fplain and change the paths for $codepath and $testpath to point at the project you are testing.

Next, run test-modified.php --norender. This will run the phpt tests, and create a pear2coverage.db sqlite3 database in the tests directory.

Finally, browse to pear2coverage.phar.php in your browser, enter the /full/path/to/pear2coverage.db and voila - a fully working instant coverage report.

This setup is the most involved part. As you develop, when you're ready to run tests, simply re-run php test-modified.php --norender, and the tests will be re-run, re-processed, and you will have a full coverage report available within about a minute's time.

I've been using this to develop Pyrus (located in your all checkout at all/Pyrus), and it's been great.

I have plans to examine the feasibility of processing the XML files that PHPUnit outputs, as this would allow generating coverage reports from phpunit-based tests as well. Unfortunately, I'm not sure the database format PHPUnit supports is simple enough to work for my needs, as it has much more detail (such as which classes/methods are declared in files), in order to support more abstract coverage reporting. Thus, I'm not likely to add support for that into this coverage reporter, unless I can be convinced otherwise.

This also demonstrates the phar extension's power. The web application is entirely contained within a phar archive, but web requests can be made for individual files directly, thus the generated html requests a .css file from within the phar, and it is served up by ext/phar with the correct mime type without any additional programming.

It's an exciting time to be developing in PHP, if you ask me.

Pyrus, PEAR2 and web code coverage report for phpt-based tests

nospam@example.com (Greg Beaver) — 2009-02-14T22:55:29Z

It's been a while since I reported on any of my PHP activities, and this update is long overdue. I must apologize to Davey for not doing my 7 things, and to all of the facebook people for whom I didn't write 25 things. I am somehow not at all motivated to do this...

In any case, now that work on ext/phar has shifted primarily to maintenance mode, and namespaces are finally ancient history, I've shifted all of my coding energy to getting Pyrus, PEAR's next-generation installer, ready to ship. Pyrus is basically the PEAR installer completely redesigned for PHP 5.3, and it takes advantage of as many of PHP's new bells and whistles as makes sense. Some of the exciting features:

Full support for .tar, .tgz, .tbz, .zip, and .phar archive formats out of the box, complete with signing via md5, sha1, sha256, sha512, or asymmetric true signature via openssl signature (all from ext/phar, which is built into PHP 5.3)
primary registry is SQLite3-based, which is much more robust, and can be relocated very easily, plus support for XML file-based registry as well as PEAR 1.x's serialize-based registry
redundant registries means ability to reconstruct a corrupted install
configuration is stored with the installed files, no more hunting down errant config files, user preferences are stored in home directory or equivalent
for developers: easy generation of package.xml straight from source code checkout
also for developer: easy generation of channel REST files directly from release tarball, no need to install a package (like the old Chiara_PEAR_Server) or even run PHP on the channel web server.

There are many other great features for users who would like to extend Pyrus, including a seamless API for accessing package.xml which is identical regardless of whether the package.xml is from an .xml file, an archive, or an installed package's registry.

Of course, all of these features only matter if they work, and to move along that path, I've been spending my time unit testing up a storm. Using the run-tests command of the old PEAR installer, I'm writing and refining tests. With the addition of xdebug code coverage (pear run-tests -rx does the trick), it is possible to see where stuff is being executed. Unfortunately, I don't like the existing reporting tools (call me a snob), so I whipped up a very simple coverage analyzer and popped it into the subversion where all of PEAR2 is stored.

When I say simple, I mean here is the script for generating the source (svn link):

<?php
namespace {
function __autoload($c)
{
$c = str_replace('PEAR2\Pyrus\Developer\CoverageAnalyzer\\', '', $c);
include __DIR__ . '/' . $c . '.php';
}
}
namespace PEAR2\Pyrus\Developer\CoverageAnalyzer {
$a = new Aggregator(realpath('../../../../../Pyrus/tests'), realpath('../../../../../Pyrus/src'));
if (file_exists(__DIR__ . '/test')) {
foreach (new DirectoryIterator(__DIR__ . '/test') as $file) {
if ($file->isDot()) continue;
unlink($file->getPathName());
}
} else {
mkdir(__DIR__ . '/test');
}
$a->render(__DIR__ . '/test');
}
?>

Highlighting is a little wonky because this is PHP 5.3+ code, but my blog is running 5.2.

The output is at http://pear.php.net/~greg/coverage/. This is a dumbed-down version of gcov.php.net, which I found extraordinarily helpful when writing the unit tests for ext/phar, but unlike gcov, there are some fantastically helpful features. The short list:

summary of all executed files and their coverage
project coverage (lines executed, lines covered, and the percentage of covered code)
annotated source showing the lines covered with hyperlinks to lists of tests that cover each line
this one excites me: per-test code coverage.

For instance, this report shows the tests that execute line 660 of PackageFile/v2.php, next this report shows the files that the PackageFile_v2/basic/file.phpt test actually executes with their proportional coverage, and finally this report shows

the lines in PackageFile/v2.php that are executed by the file.phpt test.

This last part is extremely useful to me, I have already used it to determine when a test is not executing the lines I expect it should be, and to fix problems such as using isset() instead of array_key_exists(), which would otherwise be very difficult to track down.

As you can also see, only 30% of the source to Pyrus is covered by the current tests, and so there is a long ways to go to full coverage. If any of this work interests you, feel free to contribute, PEAR2 has a much lower barrier to entry than PEAR, and as you can see, we need all the help we can get to bring this baby off the ground.

Music, PEAR, phar, namespaces and Macs. Oh my!

nospam@example.com (Greg Beaver) — 2008-12-10T04:00:17Z

Why the tri-partite title? I've been silent for a while on the blog, but the title sums up my life for the past several months. What have I been doing? Aside from some crazy music work learning a blisteringly difficult Elliot Carter String Quartet No. 4 as well as Steve Reich's Different Trains, Bartok's 6th Quartet, bringing back Korngold 3rd Quartet, learning Carter's Figment for solo cello, and performing Bach's 5th Suite for Unaccompanied Cello for the first time, I've also been pretty active in coding. I acquired a new Macbook Pro with VMWare Fusion running Windows XP and Ubuntu, and it has made development super-easy. I also put in a significant effort on the phone getting Barack his electoral vote in the Omaha area, which seems to have paid off. So, here's the long version of what I've been doing.

Music and Macs

first the music

The past month has seen my time consumed learning Elliot Carter's 4th String Quartet for the 100th birthday thing in Boston. Although I have not learned the others, a few veterans have reported that it is the most difficult. I certainly found it to be the most difficult thing I've ever learned by a long shot. It took me almost 15 hours of work to be able to play the first page (only 15 measures of music) at 3/4 of the printed tempo, and the piece is 35 pages in the cello part. To contrast, page 35 of the 10 Celebrated Mozart Quartet cello part is the third movement of 6th string quartet in the book. Carter 4th's cello part is as long as 6 Mozart quartet cello parts. However, it is shorter than some of the 4 page Mozart quartets, because it is frickin fast and there are lots and lots of notes. It is the most holy s*** hard piece I've ever learned, and for a while, I was scared it just wasn't going to be possible.

In addition, one of Carter's big compositional inventions is the metric modulation in which a tempo change has a distinct rhythmic relationship. He uses some pretty normal ones in the first movement (the 16th note becomes the new triplet), but the other movements have fun ones such as the 16th note becoming the new septuplet. Because most of my part is written in septuplets, this can be difficult to do. Speeding up septuplets so that you fit 7 into what used to be the space of 6 septuplets is not an easy thing to learn at all.

...then the mac

...unless you have a metronome that can do the metric modulation. We had been using a web-based metronome to plug into our studio's stereo to rehearse, because it had a more pleasing beat than any of our metronomes have, and it occurred to me that I could probably pretty easily write a metronome that is capable of doing metric modulations on my new Macbook Pro.

So, I took it upon myself to learn Objective-C and the Cocoa framework in Xcode to write a metronome. It took me just over a day to learn Objective-C; it's very easy for a C coder to pick up, and the syntax is pretty simple. Learning Cocoa fully took quite a bit longer, but after 3 days I had a working metronome that could do metric modulations (http://chiaraquartet.net/ChiaraMetronome1.0.0.dmg). It had several flaws, not the least of which was the inability to insert a measure group, no way to save your work, and some odd behavior on the first beat (sometimes it is late and fast).

After a few more days of digging, I had figured out most of the quirks (and discovered bindings and Core Data, which simplified things) and had the interface greatly simplifed (http://chiaraquartet.net/ChiaraMetronome1.2.2.dmg). I added the ability to start at any measure in the piece, and to have lead-in measures (which allow you to start the metronome, sit down, find the beat, and then start playing).

Then, I discovered document-based Core Data, which allowed saving files, resulting in the next generation (http://chiaraquartet.net/ChiaraMetronome2.0.0.dmg).

Finally, not to leave an unturned stone, I added the final features, the ability to specify whether a beat is accented, beated or silent, which allows implementation of uneven compound meters (like 5/8), and the ability to accelerate or decelerate tempo, and we have the final version http://chiaraquartet.net/ChiaraMetronome3.0.0.dmg. This version contains 3 sample metronomes, the complete Elliot Carter 4th quartet we used to practice the tempo changes, a sample demonstrating accelerando/ritardando, and a sample demonstrating how the metronome can be used to practice complex polyrhythms like 6 against 7.

In my potentially humble opinion, the metronome is my crowning glory as a programmer, it is incredible for learning complex tempo relationships for composers like Carter and Bartok who call for it. It's also important to note that this metronome should NEVER be used for learning music by composers such as Brahms or Mozart. If I hear of anyone using it for that, I will hunt you down and make you listen to Beethoven 9 with your eyelids peeled back watching Teletubbies.

Short feature list:

free, released as a beta with no implied license or warranty. I still own the copyright, so you can't use it in your own programming projects or sell it without infringing on the copyright. You can use it for your own practicing, and I will eventually release the source under an open-source friendly license once I have time to research the options.
Please send me any metronome files you create, I'll start a public repository that people can download them from
requires Mac OS X Leopard (version 10.5.0) or newer
universal binary
relative tempo slider can be used to practice under tempo and click up the metronome, preserving tempo relationships at tempo changes
ability to program in endless tempo (0 bars) or a limited number of bars of a tempo
ability to do compound meters, both even and uneven
ability to do accelerando/ritardando, including ones that cross tempo group boundaries using the tempo calculator
ability to start and end accelerando/ritardando mid-bar. No support for mid-beat start/stop, but this can be simulated by doubling the number of beats and making the mid-beats silent. For example: instead of 4 beats at 60 bpm, choose 8 beats, and make 2,4,6 and 8 silent, with double the tempo, 120 bpm, then start the accel on beat 4 to start on the and of 2.
complete undo/redo support
ability to split up a tempo group, insert a new tempo group, or copy a set of tempo groups and paste them (copy button)
ability to play only a limited set of tempo groups, simply shift-click or command-click the tempo groups. This can be used, for instance, to skip a repeat in practicing.
ability to accent all downbeats with a simple click.
start at any bar in the piece.
lead-in bars to help prepare tempo prior to playing.

PEAR

Today, I upgrade pear.chiaraquartet.net to the latest version, 0.19.0, which Brett Bieber has been working hard to perfect. The look is a bit cleaner, and I also replaced the defunct Crtx_Frontend_Chiara (or whatever it was called) with the newly minted Chiara_Pear_Server_Frontend_Web which is a direct port over of the old frontend with bugfixes and visual tweaks.

PEAR and phar

A high priority for PEAR is minting its next-generation installer, Pyrus. Pyrus is based on PHP 5.3+, and thus I expended a tremendous amount of energy on the new phar extension,which will be enabled by default in PHP 5.3. The phar extension provides several must-have features for Pyrus:

ability to run out-of-the-box applications without installing or extracting them (a la Java's jar) in zip, tar or native phar file format
ability to create data archives in tar or zip format and convert between them
ability to automatically handle oddities of running an application inside an uncompressed file
web-based applications without extraction

I'm very pleased this will be built into PHP 5.3, it cuts out oodles of code from Pyrus, and will significantly affect both its performance and memory footprint.

PEAR and namespaces. Oh MY!

Also important for PEAR2 is making it easier to work with the names, so I've been fighting in the namespace wars. For a while there, it was looking like namespaces would go the way they had in the PHP 5 beta cycle: to the trash can. With a little bit of work, we found a solution no one is 100% happy with, but having tried out several possibilities, I must say the solution we chose is something we can be proud of. There have been many complainers, but the irony I have seen is that most assert they never planned to use namespaces in the first place, which begs the question: why flap your trap if you have no plan to use the thing? In any case, you know where I stand on the issue. I wrote the new manual as well as the first draft of the patch that made it possible to end the bickering on internals. A big thanks to Dmitry Stogov and Stanislav Malyshev for their patience, and especially persistence in working out the solution. Also thanks to many others who fielded opinions on-list, on IRC, and through private email. Anyone who contributed a real concern rationally and sometimes forcefully deserves credit for the final implementation, and time will prove that it is a very good one for those who need it.

I am, of course, referring to the now infamous decision to use the backslash (\) as the namespace separator in PHP's implementation of namespaces. I won't go into the rationale, except to say that PHP, unlike other languages, implements a very loose autoload implementation, which makes it impossible to do the most common solution to resolving a name conflict between a static method and a namespaced function - disallow that. In other words, it's not possible to detect the conflict unless autoload is called on every potential namespaced function call, which introduces a potentially exponential hidden performance loss. As Stanislav so eloquently put it, name conflicts have not caused major websites to go down, but performance issues have. Thus, anything that calls autoload where it may be unnecessary is an unacceptable solution, and the only way to safely handle the ambiguity is to eliminate it.

That is what using the backslash does, because the syntax is different between a namespaced function call and a static method. Complain or ridicule if you'd like, the rest of us will simply shut up and code.

This decision has several other benefits too, including the ability to audit someone else's code and figure out what you should grep for, a namespace or a class name. This also translates into better autocomplete in future IDEs, and so on.

Wrap up

So, it's been a long blog post, but a lot has happened in the past several months, and it's time to get cracking on the future. More will follow as time allows.

pear.chiaraquartet.net back online

nospam@example.com (Greg Beaver) — 2008-11-05T05:41:59Z

For those of you who were wondering where pear.chiaraquartet.net went, we suffered the same fate as Josh Eichorn's blog and other sites, but all has been restored.

What do "Batman, the Dark Knight" and Barack Obama have in common?

nospam@example.com (Greg Beaver) — 2008-08-25T03:49:02Z

Last night, I saw "Batman, the Dark Knight" for the first time. Aside from being a great movie and featuring ridiculous good performances from the late Heath Ledger and others, it got me thinking. I've noticed a huge shift in the way movie makers are portraying both reality and fantasy since 9/11. First, there is the obvious increase in the number of movies about governmental intrigue, CIA spy junk, invasion of privacy and other things reflecting and amplifying reality. However, yesterday I noticed that the Joker was referred to as a "terrorist" several times in the movie.

Let's think about this: a comic book villain whose sole motivation is to sow chaos and destroy civilization is being equated directly with people who reap violence in order to achieve a political or military goal. The last time we saw the Joker, he was a brilliant but somewhat goofy character in Jack Nicholson who when killing people turned them into caricatures of death in a grotesquely cute way with exaggerated smiles burnt into the corpses faces. Contrast this to Ledger's Joker who tells conflicting and gory stories of how he got his smile scars before cutting people's faces open with a knife. Nicholson's Joker wants to create chaos, but for him it's all a big joke. Ledger's Joker is just scary. He enjoys chaos because it causes good people to do evil things and even become evil embodied, and says this explicitly in one scene.

The scene that got me thinking about the connection to politics was (cover your eyes if you haven't seen the movie yet) when the massive prisoner seems to be about to blow up the innocents' ferry boat and instead throws the detonator out the window, showing one of the only moments of redemption and hope. What has exploded Barack Obama's political stature from non-existent to presidential nominee of the Democrats is "Hope" as political platform. Go back 8 years to the post-Lewinsky election. Can you imagine any politician even being able to raise the slightest amount of money based on "hope," let alone win the nomination of either party? What gives Barack's message such potency is that it is the extreme opposite to what we see from the Dick Cheney camp. Continuous war against an enemy that has neither a defined name, a defined country, or a clear target is a guaranteed recipe for despair and extreme paranoia. Especially when the only unifying things for this enemy is that they are a few thousand people amongst the 1 billion plus who practice the world's most populous religion, and they are out to get us.

Barack's successful message of hope is a direct extreme opposite of the fear and militarism practiced by the establishment, and thus it is very, very important for us to understand what this could mean to us as citizens. Why? Extremes sometimes balance each other out by pulling towards the middle, but when they remain separate and intensely opposite, they create what we used to know as "extremists," those who cling to the far fringes of an idea where the air is thin, intense and rational thought disappears into distant memory. To be blunt, if you support Barack because he makes you feel hopeful, this is only slightly less scary to me than supporting Bush because he makes you feel safe. These emotions have the same root: escaping fear.

I happen to personally enjoy both feeling safe and feeling hopeful, but more important to me as a voter is how the candidates will handle themselves in day-to-day workings. Do they have the intellect to differentiate between opinion and fact? Can they understand the importance of peer-reviewed science and how to translate this into public policy? How do they handle conflict amongst the people around them, and how do they handle external conflict imposed upon them by circumstance? Do they have the ability to see long-term consequences and understand short-term methods of handling them? Answering these questions for me means looking both at their positions, how they have evolved (especially convenient changes in the past 2 years based on polling), news on how their campaign has been run, and what was important to the candidates before they had any idea of burnishing their public image in order to run for political office.

Returning to the movies, the best example of how our psyche has changed with regards to thrillers comes from the 1998 movie "The Siege" starring Denzel Washington, Annette Benning and Bruce Willis with Tony Shaloub in a supporting role years before he hit it big time with "Monk." Willis plays the evil power-hungry general who takes the law into his own hands and is ultimately punished by the strength of the Constitution. The movie starts out with terrorist bombings in New York City that escalate from a bus to destruction of a building by a car bomb. In a key moment, Washington walks in on an interrogation of an Arab who is sitting naked in a shower room of a Brooklyn football stadium and Willis starts to discuss "alternative protocols" like waterboarding. Washington here delivers his big speech that concludes with "what if what they really want is for us to bend the law a little bit, shred the constitution just a little bit? Because if you torture this man, they will have won. They have already won!" At the time, this seemed to be the most remote, horrendously extreme possibility, that an American would even consider torturing a detainee, and it is this act (and the subsequent execution of the prisoner by Willis) that Washington cites in the arrest warrant at the conclusion of the movie (sorry to spoil another one).

Fast forward to today, where in the Republican primary, all but a couple of candidates publicly supported using torture as an interrogation method - and tried to use it to become the nominee for president, and torture was authorized by the Justice Department, the vice president and the president, none of whom have faced even a trial for their decisions. Is it any wonder hope has been transformed from an ideal into a powerful political tool?

Macbook Pro lives up to expectations

nospam@example.com (Greg Beaver) — 2008-08-14T22:35:30Z

Well, it's been a long and practically empty summer for me in terms of coding, what with hardly any internet access and even less time. I did manage to have time to improve the scheduling program I wrote for Greenwood Music Camp where we are in residence, but that was it.

Now, a new leaf has been turned over. The Chiara Quartet has a residency at the University of Nebraska-Lincoln that began in 2005 for a 3-year term. Much to our delight, the residency was renewed for another three years, and converted into a permanent residency. None of this has much to do with programming, but a perk of this job is that every 3 years, we get a new computer to replace the old one. This cycle, I requested a Macbook Pro, my first Mac, as I have been using a PC for years.

I had the tech peoples set up VMWare Fusion for me, and so I am currently running Mac OS X, Ubuntu Linux, and Windows XP simultaneously with noticeable performance gain over any previous computer I have owned, including my custom 64-bit Kubuntu desktop. The screen is beautiful (15 inch Matte) and speakers far superior to other Macs I've heard as well as the Toshiba laptop I will soon return. I was able to very quickly set up a working PHP 5.3 with xdebug and Komodo in Mac and linux, and almost have the windows side compilation of PHP working, which will greatly accelerate my development. The ability to share changes with the shared folders between all 3 boxes will be a tremendous boon to my development, this will be fantastic.

It feels good to drop Windows as the native platform too, it's been a long time coming.

PHP namespaces become much better (Derick take note)

nospam@example.com (Greg Beaver) — 2008-06-21T06:33:08Z

Recently noted by Derick Rethans in an informative blog post, there is (now was) a problem in the implementation of namespaces in the soon-to-be-released PHP 5.3. I quote:

Take for example this code:
<?php
use PEAR::Date::Interval as Interval;
?>
In PHP 5.3 this would alias the class Interval in the namespace PEAR::Date to the class Interval. For now, this code would work just fine. However, if PHP would introduce a class "Interval" at some point in the future (and PHP can do this as it owns the global namespace) then the above code would suddenly stop working.

This morning, I realized that this would be very easy to fix, and posted a patch to internals@ with a brief explanation. Dmitry committed a fix to PHP_5_3 and HEAD very quickly. The issue raised by Derick is now obsolete, making namespaces not just more useful, but even more powerful than they were.

For instance, it is possible to "override" an internal class, perhaps to add missing functionality that will be present in a future release or fix a bug:

mydate.php:
<?php
namespace Foo;
class DateTime extends ::DateTime
{
// methods here
}
?>

<?php
include 'mydate.php';
use Foo::DateTime; // this essentially overrides internal DateTime class in the current script
$a = new DateTime('2006-05-04');
?>

Those who were lamenting the implementation of namespaces in PHP should take note: if you have a solution to make them better, the turnaround time between proposal to the list and commit can be on the order of a few hours if the idea clearly solves a major problem.

php|tek PEAR/phar/Pyrus talk slides

nospam@example.com (Greg Beaver) — 2008-05-29T16:46:43Z

I've had a few slide requests, so here is a link to the pdf of my slides that Sean Coates put up on the phparch site

http://tek.phparch.com/mats/slides/greg_beaver-pear_phar.pdf