Lot 49: Greg Beaver's blog

I've been a bit busy, my last PHP post was in October (!) but I do have very good news. Following on the heels of my first php|arch article about the Phar extension in the January issue (http://phparch.com/c/magazine/issue/65), I've been extremely hard at work on perfecting this extension. As a testament to the strength and excitement surrounding the upcoming 2.0 release of the phar extension, my editor at php|arch became a developer on the phar extension after reading and working on the article (hi Steph).

Today, I released phar version 2.0.0a1, the first release of the newly upgraded extension. This extension is a wholly different animal from the last release, 1.2.3. While maintaining BC with the 1.2.x version of pecl/phar, this version adds unimaginable new facilities. The phar extension has transformed from something of a sideshow to a major player, and is being actively considered for inclusion in the core of PHP 5.3. Major new features in phar 2.0:

• Phar now supports tar and zip archives with the same API that is supports phar archives. Think of phar as something like the PDO of archive file formats. All phar archives must contain ".phar" in the filename to be executed
• Phar has a front controller for web applications that fully handles MIME types, supports mod_rewrite-like functionality with far more flexibility (a php callback is passed the url, it returns the actual file to load) and also the ability to deny access to portions of the phar that are not web-accessible (like library PHP code)
• Phar fully solves the problems with current directory and code loading/file loading that plague all other stream wrappers like PHP_Archive and PHK. This allows running complex applications like phpMyAdmin with no code modification directly from the original .tgz after a rename and the addition of a 20-line stub
• Phar supports phar:// stream wrappers in include_path for PHP 5.2+, so your require_once-based applications can be pharred up without code modification
• Phar supports creation and modification of data-only tar and zip archives (no executable phar stub) via the PharData class. Now you can use phar to create and even convert between file formats. Unlike the zip extension, phar supports bzip2 compression of files, and has no arbitrary limit on open file handles. Unlike the archive extension in pecl, phar fully supports windows, and is the first PHP extension to deliver read/write support for tar archives on both unix and windows.

In other words, phar 2.0 is the "holy crap" release of phar. The manual (which is currently almost fully updated to the API, but not yet rebuilt) is at http://www.php.net/phar. Another way to get to know phar is through the unit tests, which are included in the pecl tarball.

The current release can be downloaded from http://pecl.php.net/get/phar-2.0.0a1.tgz or installed via pecl install phar and windows dll can be downloaded from http://pecl4win.php.net/ext.php/php_phar.dll although it looks like building is having a few hiccups lately.

There are a few known issues to be worked out. One is compatibility with xdebug, which is a trivial problem involving extension load order. Once we work this out, xdebug will support stepping through the source of files within a phar archive in your IDE, a killer feature for those who need to debug things. In PHP 5.3, phar requires a small patch to support the phar stream wrapper in include_path.

...This brings me to PHP 5.3. I've been working hard with Dmitry Stogov, Marcus Boerger (the other phar lead) and a few other list contributors to perfect a patch that will bring stream wrapper support to include_path. This will mean that you can write a userspace stream wrapper, put its url in your include_path and start using it with relative includes such as what PEAR uses to load its code. This powerful feature will allow you to write your code in one way, and run it from several different locations. Expect to see this feature committed in the next week or so, barring major problems.

Lastly, I'm happy to announce that I will be a speaker at this year's php|tek conference in Chicago. It is the first time I've been able to pull myself away from a crazy-busy quartet schedule to attend a conference, and I look forward very much to meeting as many of you as possible while there. My talk will be on smart PHP application deployment, and covers PEAR, Phar, and the new Pyrus installer for PEAR2 (http://tek.phparch.com/c/schedule/talk/d2s6/1). Unfortunately, the talk is at the same as Josh Eichorn's talk on PEAR2 (http://tek.phparch.com/c/schedule/talk/d2s6/0), so we may need to work out some way to shuttle folks back and forth, we'll have to see .

Progress of the highest order is being made here, expect to hear more about the goings-on behind phar and PEAR2 as time permits.

Here is an interesting example of how screwed up the political process is: Mike Huckabee, Republican of Arkansas, is being heavily criticized for having raised taxes during his term as governor.  For years, the rallying cry of so-called "fiscal conservatives" has been decreasing taxes, under the premise that government is always less efficient than private enterprise at solving problems.

Of course, as with all one-sided issues, the question of taxation versus private enterprise is a canard.  It is more than foolish to blame high taxes for all of the problems in this world.

For example, Huckabee raised taxes approximately $500 million net (once you take into account taxes eliminated) as a governor.  However, at the same time, the state's $200 million budget deficit turned into a $844 million surplus in the same time.  Tax advocates might argue that the increase would be even greater had taxes been lowered, but let's look at the truth of the matter.  Reagan and both Bush presidents cut taxes, and the deficit rose in both cases.  Clinton raised taxes, and the deficit decreased at the same time as the economy flourished.  For citations, see here, here, here, and here.  The last citation is particularly interesting, as it discusses the long-term effects of budgets based on deficit spending (living off the government credit card is another way of thinking of it).  The standard economic effect is that as you borrow, your interest rates climb to compensate.  An interesting side note is the end of the article predicting that interest rates will only rise 1.12% after 2004's record deficit spending.  In fact, the sub-prime mortgage lending crisis we currently find ourselves in was caused by a very slight increase in the interest rate as dictated by the Federal Reserve.  The rate increase itself should have been harmless, but when combined with the obsessive anti-regulatory credo our current administration lives and breathes, we have thousands of unethical predatory adjustable rate mortgages suddenly increasing monthly payment by several hundred dollars, and foreclosures by the thousand affecting the entire economy with a ripple effect.  Another interesting side note is that the only reason our economy continues to function is because China is literally supporting our system by buying huge numbers of government bonds.  The new question nobody is asking is what does this really mean?  The obvious answer is that should China wish to do so, it could squeeze the U.S. government and shift our policies in its favor just by threatening - behind the scenes - to stop this process.  What it really means is that very quietly, and unnoticeably, the control over the U.S. economy and even policy is shifting to Asia.  This is not intrinsically bad, and in a perfect world, allows our economy to function with the help of our benevolent neighbors.  What it also does do is allow a bad apple government to exert undue control over the U.S. from afar, much as a bad apple U.S. government has run amok over the Iraqi people, but in a far more subtle way with profound implications for everything here, including the military and other national security issues.

So, coming back to our main story, Huckabee raised taxes to fix horrendously ill-maintained state highways, to pay for nursing homes for poor seniors, to pay for strapped school districts, supporting tuition breaks for children of illegal immigrants, and improving the state parks system.  In case you've been living in a hole, these are all liberal issues, and yet they all passed with huge bi-partisan support in Arkansas.

So why don't I like him much?  Actually, much of what he says on his issues page resonates with what is really important.  His views that the environment actually kind of matters, that education is a serious priority for providing vectors to move from a poorer life to a richer one, faith as a non-prescriptive but important part of life - all of these sound great.  However, he then proceeds to turn right around and say it is just fine to proscribe that his personal view of homosexuality and faith allows him to legislate the lives of gay people in a highly intrusive and discriminatory way.  He also blindly lumps together Iraq in with Al Qaeda, conveniently ignoring the fact that it was the United State's own imperial hubris that led to the creation of "Al Qaeda in Mesopotamia" - an organization that did not exist prior to 2003.  His demonification of illegal immigrants, pandering to white racists who see "those Mexicans" as a problem and finally have an excuse to get rid of them is pure opportunism, especially considering his actual stance as governor supporting the children of illegal immigrants.

Ultimately, however, this is a candidate who understands that government has a purpose that serves to rectify the natural inequities inherent in the capitalistic system.  I'd never vote for the guy because it takes more than an effective manager to unify a country that has thrived on division under the worst manager in the history of the nation, but it is comforting to see that even social conservatives are starting to recognize that the Bush/Reagan paradigm of economic management and governmental anarchy is the worst thing to ever happen to the United States economy.  It would give me no end of pleasure to be *only* arguing about domestic issues of personal privacy, health insurance, education, and all the other things that really matter to day-to-day living.  With an effective manager at the helm, the United States can finally pull out of this tremendous quandary that is the Bush legacy.

More fuel for the fire - it appears that the CIA, under the explicit direction of the executive branch of the U.S. Government (translate: Bush administration) tortured at least three suspects in interrogation. The "technique" used is waterboarding, a medieval torture technique designed to convince the person experiencing it that they are drowning and about to die a horrible death. This in turn is expected to convince the person being waterboarded to give up information on terrorists.

The Attorney General nominee explicitly says in his testimony before Congress:

He said that while he personally found waterboarding and similar interrogation methods “repugnant,” he could not call them illegal. One reason, he said, was to avoid any implication that intelligence officers and their bosses had broken the law.

“I would not want any uninformed statement of mine made during a confirmation process to present our own professional interrogators in the field, who must perform their duty under the most stressful conditions, or those charged with reviewing their conduct,” Mr. Mukasey wrote, “with a perceived threat that any conduct of theirs, past or present, that was based on authorizations supported by the Department of Justice could place them in personal legal jeopardy.”

That's right - the Department of Justice explicitly authorized waterboarding.  Mukasey is right on one point - the CIA operatives carrying out orders will not be expected to answer to their crime of following orders on this earth, as Congress has passed a law forbidding prosecuting of interrogators following protocols set forward by their bosses.  This, incidentally, was the most common defense by Nazis in the Nuremberg trials after World War II: "I was just following orders when I pulled the trigger and shot men, women and children over a trench they had dug."  This leaves us with only one recourse: prosecuting their bosses, namely the Bush administration people who authorized the torture.

At this point, that leaves us with one recourse - impeachment.  On what grounds can the House impeach?

The Constitution, Article II reads:

Section. 4.

The President, Vice President and all civil Officers of the United States, shall be removed from Office on Impeachment for, and Conviction of, Treason, Bribery, or other high Crimes and Misdemeanors.

Treason is explicitly defined as waging war against the U.S., or aiding an enemy of the U.S., and it must be explicit, so that is out of the question.  Bribery is not a known problem, so that is out as well.  This leaves high crimes and misdemeanors.  What's that?  It's intentionally vague.  For a guideline of precedent, Nixon was indicted for illegal wiretapping, misuse of the CIA, perjury, bribery, obstruction of justice, and other abuses of executive power.

This raises another question: is torture even illegal?

The Bill of Rights reads:

Amendment VIII

Excessive bail shall not be required, nor excessive fines imposed, nor cruel and unusual punishments inflicted.

This, unfortunately, does not discuss interrogation, only punishment. However, this Amendment is quite explicit:

Amendment IV

The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no warrants shall issue, but upon probable cause, supported by oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized.

Unreasonable search of person can be construed clearly to mean interrogation techniques that are unreasonable, as the purpose of interrogation is to search a person's memory for wrongdoing or information on other wrongdoers.  In other words, the Bill of Rights does not explicitly mention torture by name, but it does explicitly forbid its use as an interrogation technique.

The truth is that president Bush has already explicitly authorized warrantless searches of American citizens, and most likely had to approve the torture techniques used to interrogate terror suspects.  Cheney is famous for micro-management, look at his pre-Iraq war intelligence manipulation efforts.  Rumsfeld is obviously directly implicated in these schemes as is Gonzales.  Rove most likely was not involved as his role was purely political horror shows.

The evidence is clear: our president and vice president are guilty of high crimes and misdemeanors, and there is no one in the legislature stepping up to the plate with the guts to propose an article of impeachment.

Some might argue that impeachment is "bad for the country" but the fact is that injustice is far worse.  When a private citizen commits a high crime such as murder, there must be a trial and justice, or it rips a hole in the fabric of society.  For years, the United States has been able to take a high road when dealing with other countries that have perpetrated terrible human rights abuses.  By avoiding punishment for the crimes of the Bush administration in the name of "the good of the country" we will be encouraging unscrupulous leaders of other countries to use the United States's conduct as justification for their own evils, and they would be right.  If we fail to appreciate this dilemma as a nation, then we do are implicitly authorizing foreign powers to torture our soldiers and CIA operatives when they are captured, as well as private citizens who are mistakenly accused of spying or terrorism like the Canadian citizen the United States mistakenly arrested and tortured in Syria.

Once again, this is not a liberal versus conservative issue, it's a basic fact: we don't torture people in the United States.  Those who do it and those who authorize it are criminals.  In fact, it could very well be more of a conservative than a liberal issue, as ignoring this torture implies a kind of relativistic view of morals.  If it is OK for the president of the United States to authorize torture of terrorism suspects, but it is not OK for a police officer to torture a burglary suspect, and not OK for parents to torture their children, we have a moral dilemma that can only be explained by saying that the concept of right and wrong is incorrect, that there are cases where "wrong" is "right."  You don't need me to explain why that is the path to absolute evil.  Please call your representatives in Congress and let them know that you expect them to uphold the law independent of political considerations, and that you want impeachment and justice for the American people to rectify the egregious abuses of executive power by this administration.

I just read in the New York Times that the head of the Consumer Product Safety Commission, Nancy A. Nord, has written to Congress asking that current legislation to double the size of the CPSC be scrapped.  This is in the wake of great ideas like selling children's toys from China that were painted with lead-based paint, putting industrial waste into pet food and toothpaste, and other fantastic situations.

This brilliant idea from Nancy Nord follows the consistent pattern of the Bush administration's goal to eliminate government altogether.  The theory is that by systematically dismantling the government, all of our lives will be simpler, better, and happier.  Unlike other administrations, Bush's has even extended this theory to the military, outsourcing "security" in a warzone to companies like Blackwater.  Perhaps the theory is correct.

After all, for the first time ever, the standard deviation from the median income in the United States is nearly zero - there are no poor people, and no super-rich people!  Our military is spending less money and less time in conflict zones, inspiring other governments to follow the United States' lead and hire mercenary forces to do their dirty work for them!  For the first time, all products and chemicals brought to market are being evaluated for their potential detrimental effects.  Also, surprisingly, every American has full access to health care and private insurance with no danger of bankruptcy from a single hospital stay.  In fact, every small child has guaranteed access to essential care, cutting down dramatically on future health care costs when these children get ill later in life.  Perhaps the next best step is to turn over our interstate highway system to private companies, so that we can truly reap the benefits of lower taxes.

Oh wait, scrap that, for the first time since the late 1800s, we have a huge disparity of the super-rich and the poor, a shrinking middle class, and the U.S. is the only "rich" country in the world experiencing this fact.  The stock market is at an all-time high, and yet millions of new home owners are about to lose their homes.  Private security firms contracted by the U.S. government to do the military's job do so at double the cost or more, with no oversight or accountability, resulting in callous disregard for the lives of civilians in the countries they work in.  The European Union is leading the way in safety of chemicals on the market, forcing U.S. manufacturers like Dow to conform to standards much higher than those in the U.S., and the U.S. dollar continues to drop relative to the Euro, suggesting that these increased safety standards don't hurt economic development at all.  The EU is also leading in environmental regulation.  Health care is obviously a disaster because it is private - compare costs and benefits to any other country with systemic government-sponsored health care for your evidence.

What most self-described conservatives fail to appreciate is the total cost of things.  Taking into account only taxes and regulations is short-sighted at best, and dangerous at worst.  When comparing costs, one must take into account how much things would cost if you have to pay for them yourself.  Health insurance is a good example.  Your employer can take advantage of economy of scale to get you a much lower rate for the same thing than you can get yourself, sometimes hundreds of dollars lower.  The government is on an even larger scale, and can therefore smooth over large differences between groups with minimal pain to those on the extreme edges.

For an obvious example of how government works successfully, the interstate freeway system was entirely government-funded, and has worked with minimal interruption or problems with thousands more miles of concrete than other countries.

The truth is that saving a few bucks now by reducing the size of government has a huge impact on the cost of living, especially for lower-income brackets, resulting in not only worse living conditions for low-income households, but increasing unstable political environments as a result.  Organized crime or corruption/fiefdoms steps in to fill the void of an absent government, much as we have seen in countries like the former Soviet Union.

It's time for Americans to wake up and smell the toxic waste.  Voting "for the other guy" is the knee-jerk reaction, but won't do anything substantive.  You need to contact your current representatives in the government, your senator and representative, and let them know that you understand the importance of government as a service to the people, and oppose dismantling government as a principle.  Take some time out of your day to do some research into how smart people who have studied the effects of government versus privatization have documented the way the world actually works, don't take my word for it.  Be a real citizen already!

So for the past half year or so, I have been trying on and off to install security updates for Microsoft .NET 1.1 and 2.0. Each time, it hadn't worked. I've now disabled automatic updates because they always fail on these two old updates.

Today, I thought maybe I would take a real crack at fixing this issue, and so did an extensive google search for any information on installing/uninstalling/repairing a .NET installation. There is, not surprisingly, a huge number of problems that people have experienced, this being a Microsoft component. I finally found a page on the Microsoft site that displays instructions on repairing .NET.

Unfortunately, none of the instructions work. I tried to uninstall the component, and it tells me source files are not available. I try re-installing from source, and it tells me that I can't install. I try manually deleting and it tells me the file is in use (yes, I've disabled anti-virus). Because this is an MS Windows machine, it's virtually impossible to tell which file is using the .NET stuff.

Basically, I'm not allowed to do anything productive with the Microsoft stuff on this computer. This incidentally also means I can't install VS 2005 because it needs a working .NET installation, so I can't debug my pecl extensions on Windows. My next computer is going to be a MacBook Pro, and I'm never looking back.

Hello all.  It's been a very busy time for me, what with suddenly having a full gaggle of new cello students to teach at UNL as well as a fully loaded performance schedule for the quartet, but I have been finding time to code.  My priority has been the push towards PEAR2, which means that in addition to attempting to mediate the discussions of what should change from PEAR to PEAR2, I'm churning out code to turn its installer, Pyrus, from vaporware into actual code.

Along these lines, there has been significant progress.  The code has been developed on PHP 5.2, but will most likely target PHP 5.3 with the introduction of namespaces.  Pyrus completely revamps the way things are done.  By using SPL as well as extensions like XMLReader and XMLWriter, Pyrus manages to pack in far more features inside the code while literally cutting thousands of lines of code.  The biggest loser has been the package.xml handling code.  More than 2500 lines of code have gone byebye, but the API for reading from and creating package.xml is now as intuitive or more intuitive than simplexml.

In addition, using interfaces, a few design patterns and some clever iterators, adding support for a customized packaging solution is a matter of creating a class with a few lines of code that accepts file contents and where to save them.  Pyrus does the hard work of constructing the relative path of files within the archive and pre-processing their contents using file tasks.

The code is simple enough that I wrote a simple channel server with no dependencies except for Pyrus in less than 6 hours (just finished the initial work when my battery ran out today) that allows small channels to work without needing a database or even PHP on the channel server, as all the REST files and releases can be generated and then uploaded from a dev server.

In short, it's an exciting time to be involved in PEAR.  If you've been ignoring PEAR, perhaps it is time to take another look?

The past 6 years have seen an unprecedented rise in the institutionalization of fear as public policy in the United States.  This is of course primarily due to the attacks of 9/11, but not because of the terrorist attacks.  The institutionalization of fear is entirely due to our political leaders, led at the top by a president but followed by the many lawmakers within the legislative branch independent of political party.  The unprecedented abuse of fundamental clauses of our Constitution such as the right to have no unreasonable search and seizure, the right not to be tortured, and now even the freedom of religion all result from the institutionalization of fear as a policy of government.  This New York Times article (http://www.nytimes.com/2007/09/10/us/10prison.html) details the latest in the long string of evils committed in the name of preventing terrorism.  The article has a great deal of information, but the most important portion of the article is the first paragraph:

Behind the walls of federal prisons nationwide, chaplains have been quietly carrying out a systematic purge of religious books and materials that were once available to prisoners in chapel libraries.

Yes, it really is true, books that might foment radical muslim ideology or other terrorist ideas are being removed from prisons.

I almost don't know what to say.  Prisons already by design remove one of the three rights so often shouted from the hilltop when quoting the American Declaration of Independence:

We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness

Removal of liberty is of course the point of a prison, but removal of the possibility of spiritual enlightenment is a dangerous game to play.  It falls under the category of "medicine is worse than the disease."  It is impossible to decide in advance what will result in a prisoner's transformation into a new person.  The mindset of creating an underclass of people with fewer rights has never resulted in long-term good.  Just ask those whose ancestors have been enslaved, or those who live today in a caste system.  It's not just morally wrong, it's bad for everyone.  This has been known for thousands of years.  The ChNew Testament includes dozens of examples of Jesus reaching out to people universally reviled, whether because of their profession (prostitutes and tax collectors), disease (leprosy) or because they were imprisoned.  Of course, many Christians argue that you should be nice to prisoners because who knows - maybe Christ will return as one of today's underclass just as he was first a lowly carpenter.  The real reason is not so self-serving.  The real reason is that it is wrong to consider other humans to be worth less than your own self-worth, regardless of your reasoning.  This is above religious teachings, above secular laws, and demonstrably bad public policy by a quick scan of the reasons for past governments' demise.

One could of course argue that prisoners are to be punished in prison and violating basic civil rights doesn't matter.  One could argue that all of the changes made to our society are necessary evils such as the warrantless monitoring of hundreds of phone conversations overseas, the re-instatement of guilt by association (providing material support to a group that may engage in terrorism is illegal, even if you were funding charitable efforts of that group like providing food to starving families in Palestinian territories), pre-emptive invasion of countries not threatening any harm to us, holding prisoners without charges because they might someday engage in terrorism if released.  One would be wrong.

All of these policy decisions were based on pre-conceived fears, and not on evidence that they can actually work to prevent the criminal act of violent terrorism.  In fact, the evidence suggests that wasting time on broad dragnets like warrantless surveillence actually decreases the chance of finding a real threat, and instead results in the innocent being treated as guilty without a trial and often the guilty being completely overlooked.  Guilt by association was the hallmark of the McCarthy hearings in Congress.  Not many people remember today the famous "I have here in my hand a list of suspected Communists..." line of Joseph McCarthy.  In retrospect, the threat of communism seems absurd.  In fact, the United States was the one instigating military agression by vastly expanding the nuclear arsenal, developing submarines that could deploy missiles to major population centers in the USSR within 15-30 minutes, and creating NATO to counteract the perceived threat of the Eastern bloc.  The USSR was reacting to the very real threat of the United State's perceived military aggression, which was ironically the result of the United State's fear of communism.  Sound familiar?

Torturing prisoners, removing freedom of religion, unreasonable search and seizure - all of these are not the ultimate problem.  The reason these are enshrined in our Constitution as no-nos for our government is because all of them have been known tools of despots and dictatorships for hundreds of years.  Every freedom in the Constitution is there to guarantee that we do not have a despotic system of government, and the reason is simply because despotism does not work.  Putting too much power in the hands of too few results in far worse decisions being made about the lives of those who are governed.  This isn't rocket science, but it requires the minimal ability of self-recognition on the part of those in power.

I challenge our leaders to ask themselves the real reason why they feel compelled to remove so many essential liberties.  Is it really to free America from the tyranny of terrorism and the fear that it engenders?  Perhaps it is to "temporarily" enslave Americans to fear in order to bring a speedier death to nameless faceless "terrorists" for the greater good of society?  Or maybe it is because the American system of government has failed, and liberty cannot result in true happiness, only total control, total monitoring of the populace for potential threats can ensure the safety of the public.

Sound familiar?  Substitute "bourgeois capitalist" for "terrorist" and "class injustice" for "fear" and you have the exact rationale used by communist dictators for enslaving and murdering their own people in order to bring about the greater good.

Fortunately, in the United States of America, citizens still have the full right of freedom of speech without fear of imprisonment (as long as you don't divulge classified information like how the advisor to the president tried to expose an active duty CIA agent for political reasons), and if our leaders fail us, it is then only up to us to remove them and hope that their replacements will understand why the old guys were kicked out.  The question then, is what are you going to do?  Will you condone the deaths of innocents because your leader has provided you some selfish benefit, or will you do the right thing and let your congresspeople and your senators know what you think about these issues and how your vote will be cast?

The tyranny of fear is all-corrupting, it results in not just fewer liberties but more deaths to innocent people.  The number of civilians killed in Iraq is unconscionable, and far exceeds the number of murders of Iraqi citizens perpetrated by Saddam Hussein.  Focusing all resources on anti-terrorism directly resulted in the unpreparedness of FEMA to deal with Hurricane Katrina, which is empirically a much greater threat to the life and liberty of Americans than terrorism (many more hurricanes hit the US than do terrorists).  Please help America put a halt to the madness of fear-governed policy, and minimally participate in government by calling or emailing your representatives.  It is time for U.S. citizens to turn the empty rhetoric about freedom and democracy so often foisted upon other countries into true freedom and democracy at home.

A few minutes ago, I released phar version 1.2.1. phar is a PHP extension that allows the creation of complete filesystems within a single physical file.  In addition, a bootstrap stub written in PHP can be used to run a phar archive as if it were an executable file or shell script.  Documentation on its usage and how to create phars is at http://php.net/phar in the PHP manual.

This version of phar fixes a number of issues as documented in my last blog post, as well as a quite few issues that Marcus fixed in relation to setting the bootstrap stub, using phar's phar.extract_list INI parameter with an apache module and others (full list here).

In addition, I ran callgrind on the thing and found a few places it could use some trimming of the fat, and was able to cut down the number of stream operations by a factor of 2x when using phar in the most common use case, reading uncompressed files.  In addition, I cut another 2x for phars that have an MD5 or SHA1 signature.

This is quite a good release, looking forward to hearing your stories of using it.

Well it's been the craziest summer of my entire life (see Chiara Quartet website for the skinny). Of course, at the moment my schedule went into overdrive, two major issues "hit the fan" in the PHP world, in that Marcus proposed the phar extension for inclusion in the core of PHP, and Arnaud and I proposed major changes to the PEAR coding standards for the new PEAR2 repository. More on PEAR2 later.

phar is an interesting beast. Right before the summer began, I received a private email from a user who noticed that when trying to package together more than around 270 files into the same phar, his OS X would run out of available file handles to use. This is a problem that Marcus and I had suspected could be an issue based on the reports of alpha testers, but had not yet seen in the wild.

Marcus did a bunch of work on improving the stability of phar earlier in the spring and summer, bringing us to phar version 1.2.0 in late May, but did not have time to look at this complicated issue

So, early this week I sat down and buried myself inside the computer for many hours, finally achieving victory a few minutes ago. The process was arduous for many reasons. First, I needed to write better introspection, so I created a gdb helper file similar to the .gdbinit bundled with PHP that would allow me to browse open phars, open file handles, and a phar file entry very quickly.

Next, I had to track down the source of the problem. It turned out to be in the implementation of flushing to disk, which aggregates all unmodified and modified phar entries and then pushes them into a new phar on the disk, generates the manifest, calculates a hash, yadda yadda. After this act, every single file entry in the phar has an open file pointer for a temporary file containing the file's contents, even the ones that were unused!  This was bad, so I created a simple routine to check for unused files and close them during flush.

Thanks to our voluminous unit tests (with upwards of 90% code coverage according to gcov), I found a large issue in rename, and also discovered that I would need to implement a just-in-time re-opening of the file handle.  At the same time, I discovered a subtle coordination issue with compression where a newly compressed file was being treated as compressed before the compression could actually occur.  After fixing this, I finally had my patch.  Further real-life testing is needed before I trust that this is the version of phar that will be released as 1.2.1, but it's a grand step closer to fully robust.

Perhaps I'm the only person who thinks this is exciting, but it's been an issue since the earliest versions of phar with write support, and now is an issue no more.  This makes the next step of profiling and comparing to other solutions a lot more palatable to me, as I much prefer stability before optimization for all of the obvious reasons.

By the way, for those of you wondering how to make the transition from programming in PHP to programming PHP itself, I can't claim to be an expert, but I can tell you who is.  Sara Golemon.  Her book, Extending and Embedding PHP is an indispensable resource for anyone who wants to either write extensions or dig into the source of PHP itself.  Combine this with the (free) search facility of lxr.php.net and a clever use of grep, and you're well on your way to being a PHP internals coder with very little extra effort.  Sara's book is worth at least twice the cover price, and I recommend it even to those who have no interest in programming extensions, as understanding a bit (or more) of the internals of PHP is worth its weight in gold when you are making design choices for your applications or libraries.

A quick note about an obscure problem peculiar to PHP files that make use of PHP 5's new __HALT_COMPILER(); token.  Prior to PHP 5.2.2, if you included two files that contained the __HALT_COMPILER(); token, you would get a notice that __COMPILER_HALT_OFFSET__ was already defined.

This posed a rather large problem for the phar file format, which makes use of __HALT_COMPILER();.  Essentially, this eliminated any possibility of using phar as a library format.

However, never fear, your trusty Greg is here!

With this patch to PHP 5 and this patch to PHP 6, the problem is eliminated.  This patch will also affect applications like PHK that utilize the same principle.  Most of you won't notice the difference until Pyrus, the next generation of the PEAR Installer, is released.

No. At least, we haven't seen any evidence of this assertion.

However, Anne Midgette of the New York Times wrote a very interesting article encompassing her own opinion of the issue. For the article, she interviewed several people with varying perspectives on the issue including the CEO of Chamber Music America, the artistic directors of Chamber Music at Lincoln Center, several concert presenters, a few young performers and composers, and me . One of the most striking parts of the article is where she documents different people's definitions of the term "Chamber Music" itself - it really does seem like nobody has a clue how to successfully define the term, but is this really any different from other genres? Terms like impressionism, pointillism, expressionism all have vast gray areas where renowned experts disagree on what falls within which category. Is Rock music Pop? Is Hip Hop Pop? Is Funk Hip Hop? You could work yourself into a frothy academic frenzy trying to taxonify art properly and still end up with no real definition that everyone can agree upon.

Her main point is that the term "chamber music" is in fact a huge barrier to audiences appreciating chamber music at all. This is an interesting idea, but I find that it is not entirely accurate. The truth is that most people in the United States have no associations whatever with chamber music, negative or positive. She suggests that calling chamber music something else might reinvigorate the field. To me, this seems to be a decent attempt at a new direction, but I doubt that would have any significant effect on either the profession or on audience size at concerts. The content of the concert, the energy level of the group, the focus of the concert (is it a shut-up-and-sit-down event or can you eat dinner while listening to late Beethoven?) are far more significant than what you call it.

All of these nit-picking concerns aside, I am very happy to see an article challenging some of the odd conventions that exist in the world of classical chamber music, and am honored that we were included in the examples of people looking for other ways of doing what we do. It's very exciting to be able to play anything we would play in a concert hall in a club - dumbing down isn't even a consideration when we're putting together our set lists (set list - you might call them "halves" of a traditional concert, and you'd have the same basic idea. Each set is one half of what we end up playing in the club, as we usually play 2 sets).

As an example of one of the tiny things that we can do which is not possible in a concert hall is when we play in clubs, there are no rules about what we can wear on stage. It's quite literally a dream come true for me, as I have always found that suits and tuxedos are designed explicitly to prevent blood from reaching the brain and to prevent audience members from feeling comfortable. We've done our best within the conventions of the concert hall to relax this feeling, but in the concert hall there is no easy way to duplicate the freedom of just being yourself you have when playing in a club.

Just a quick note, I tried out the Pixy XSS and SQLI Scanner (http://pixybox.seclab.tuwien.ac.at/pixy/index.php) on a few simple PEAR files.  On the first, I got a java exception, on the second it was unable to resolve the simplest of includes (no ability to resolve include_path).

In short, the thing is useless for anything written using PEAR.  Fun!

Shortly after my last blog post, I got a very helpful email from Rob Richards, who offered to help me with my trials and tribulations. He took a quick look at the schema for package.xml, and a sample package.xml and was able to find two small tweaks to make parsing actually work (gasp). This changes everything. I am abandoning the creation of a relax NG schema in favor of the battle-tested xsd. The error messages for xsd validation are far clearer than the rng ones. For instance with this script:

and the <date> tag moved to the wrong location, I get this error:

Line 565: Element '{http://pear.php.net/dtd/package-2.0}date': This element is not expected. Expected is ( {http://pear.php.net/dtd/package-2.0}stability ).

Fantastic! The error actually tells you what the problem is and gives a useful clue on how to fix it - this is *exactly* what I need to strip out the thousands of lines of unnecessary schema validation code in Pyrus (hallelujah).

I still haven't worked out the problems I was having with rng, perhaps another update on this later.

Update: I've found a solution that will work for Pyrus.  See my next blog post.

While working on Pyrus, the next generation installer for PEAR, I've been attempting to figure out how to leverage PHP's excellent new libxml-based XML validation to speed up package.xml validation in PEAR. To my great surprise and annoyance, I am starting to believe that this is the wrong thing to do.

Originally, I designed a W3C Schema for package.xml 2.0 using XMLSpy, but found that libxml is unable to handle the schema whatsoever. Instead, with the release of PEAR 1.4.0 I designed a schema validator based on the array functions of PHP and the unserialized array generated by XML_Serializer's XML_Unserializer class. Because this is not an elegant validator of W3C Schema or Relax NG, I called it "stupidSchemaValidate()". This is used to validate the basic structure of the XML level-by-level, and so requires many function calls.

I never considered using DTD because it can't handle namespaces.

Now that I am working on the PHP 5+ implementation of Pyrus, the first thing I thought I might do is create a Relax NG schema that the PHP libxml can handle. After an entire day of fighting with the thing, I've managed to discover more than 10 simple and valid Relax NG schema that simply don't work with the version of libxml distributed with PHP 5.2.3. In addition, with helpful error messages like "Expecting name, got nothing here," even with the use of libxml_use_internal_errors() I find the error reporting to be excruciatingly useless.

With my hacked-together stupidSchemaValidate(), I get wonderfully clear error messages like "unexpected <channel>, expecting one of <version>, <date> on line 23" which make it not just easy but simple to debug broken package.xml formatting.

Is there anyone out there who has had any luck getting a supposedly smart external validation tool to print remotely helpful error messages on validating a complex .xml with PHP?

It's been a while since my last blog post, I've found myself with more limited free time than usual and have thus had to focus on the really important things.  I was recently elected president of PEAR (I ran unopposed, so it's a durn good thing I was elected), and this has resulted in a flurry of activity.  A lot of energy is going into revamping the PEAR website, http://pear.php.net.  At this stage, most of the changes are not visible to the naked eye, as we are writing unit tests and refactoring gradually.  However, one change will be highly visible.

I'm excited to announce the first two mirrors of pear.php.net are now actively mirroring the installer REST files and actual .tgz files of package releases.  They are http://us.pear.php.net (provided by Joshua Eichorn and bluga.net) and http://de.pear.php.net (provided by Christian Weiske).  The option exists at a future date of mirroring the entire website, but this will not be possible without further changes to the infrastructure.

To try out one of the new mirrors, simply run:

pear config-set preferred_mirror us.pear.php.net

or

pear config-set preferred_mirror de.pear.php.net

Then, continue using the installer normally.  Users in Europe may notice a considerable performance increase by using de.pear.php.net.  Ain't progress great?