Dependencies in PEAR 1.4.0, Part 2

Here's a short list of problems in PEAR 1.3.x that could not be solved without a major rewrite of the dependency system:

1. Dependencies cannot be properly validated on uninstall (this is the most serious flaw)
2. Dependencies on pecl packages can't work properly, as often packages are installed, but a different php.ini is used for the installer and for the live site (CLI installer for a web site, for instance)
3. Stricter versioning of dependencies allowing safe upgrades is not possible without a release of every package involved
4. Splitting a larger package into smaller packages is not possible.
5. There is no way to specify feature sets rather than individual package dependencies (like windows installer full/custom install sets)
6. Uninstalling subpackages with a parent package must always be done by manually listing each package on the command line
   

PEAR 1.4.0 solves all of these problems elegantly (well, at least I think so, but I'm not biased).

Dependencies cannot be properly validated on uninstall

In PEAR 1.3.x, once a package is installed, the only record of other packages that depend on it is in each package's individual registry entry. Sc the only way to verify the validity of an uninstall is to scan each individual package's registry entry for dependencies that match the package being uninstalled. This can be excessively slow and actually precludes the possibility of ever being able to uninstall a subpackage!

Why

A subpackage can be defined by a dependency relationship between the parent package and the subpackage. This relationship can be loose:

A optionally depends on subA
subA requires A

or strict:

A requires subA
subA requires A

What this means is that a subpackage is in fact a circular dependency. In the first case, an uninstall of subA will warn the user that the A package can optionally use the subA package. An uninstall of A will fail, because subA is required. This is expected behavior and works great.

For the second case, an uninstall of subA will fail, because A is installed. An uninstall of A will fail because subA is required. Even this command would fail:

$ pear uninstall A subA

Because both packages require the other package to be uninstalled first in order to properly uninstall!

In PEAR 1.4.0, this restriction is lifted by the new dependency database (PEAR_DependencyDB) and the advanced dependency validator (PEAR_Dependency2). Now, uninstall checking of dependencies is extremely fast, and allows this command to work:

$ pear uninstall A subA

Both of these will still fail, as expected

$ pear uninstall A
$ pear uninstall subA

The reasoning is that A and subA are actually just subdivisions of a single package. An example of this might be splitting up PEAR_Command from the main PEAR package so that updates to the command structure can be released on a separate schedule from the core package. This is NOT a planned change, just in case you were wondering, but it would be possible, to allow greater distributed developing and other worthy goals.

Dependencies on pecl packages can't work properly

In PEAR 1.3.x, we have had more bug reports of "install failed because GD extension not detected" and other similar reports, than just about any other bug. This is caused by one simple dilemma: PEAR is designed to both install extensions to PHP written in C, AND to install user-space packages written in PHP.

The main issue is that when upgrading a PECL extension, if the extension is loaded in memory, it cannot be overwritten by the installer. Fortunately, the -n switch to php forces the ignoring of php.ini, allowing smooth and simple upgrading of existing PECL packages. Unfortunately, this unloading of extensions means that PEAR can't detect these extensions to validate installation of user-space PHP scripts that rely upon them!

There are 2 solutions to this problem. First, there are now two commands, "pear" and "pecl" that can be used to install packages. They do exactly what you expect (pear has no -n switch, pecl does).

Second, the way that an extension dependency is validated has changed. If you wish to depend on a pecl package, you need only add one element to the package dependency, and it will validate against either the package registry, or in-memory extension. Here is the new element:

<package>
 <name>pecl_http</name>
 <channel>pear.php.net</channel>
 <providesextension>http</providesextension>
</package>

This element is a contraction of "provides" and "extension" and tells the installer that if the extension http is in memory, no validation check on the package registry need be made, instead an extension dependency type can be used to validate the extension, and only if the check fails will the package registry be used to validate install. This will allow the best of both worlds - extensions installed but enabled in another php.ini file can still be used for dependencies by validating against the package registry, but in-memory validation will also work.

Incidentally, although outside the realm of dependencies, PEAR 1.4.0 also supports binary PECL packages, which will finally bring windows users inside the fold of extension installers thorugh the pecl command.

Stricter versioning of dependencies allowing safe upgrades is not possible without a release of every package involved

In PEAR 1.3.x, the only way to do strict versioning was to specify an upper limit to package versions that work, using an lt or le dependency.

<dep type="pkg" rel="lt" version="1.5">Foo</dep>

Alternatively, an eq dependency could be used to limit an installed version to a single package

<dep type="pkg" rel="eq' version="1.5">Foo</dep>

This has several obvious flaws. If Foo 1.6 is released and tested with the parent package, the parent package will not allow an upgrade to Foo without an upgrade to the parent package. Suddenly, we've reached the Catch-22 described in the first flaw - neither package can be successfully upgraded.

In essence, suddenly the ONLY way to upgrade either package is through the --force option. This is hardly a good solution.

PEAR 1.4.0 introduces the idea of a suggested version. A package can specify an open-ended dependency that in fact suggests a version of a dependency that should be installed. For example, let's say PEAR can work with XML_RPC 1.1.0 and higher, but there are several crucial bugs fixed in 1.2.0, the dependency could be:

<package>
 <name>XML_RPC</name>
 <channel>pear.php.net</channel>
 <min>1.1.0</min>
 <suggestedversion>1.2.0</suggestedversion>
</package>

At install-time, if the suggested version of XML_RPC is not the installed version, installation/upgrade of PEAR will in fact fail without --force. However, the previous Catch-22 is completely eliminated. First off, a fresh install will always download the suggested version. Second of all, a new tag in the main body of package.xml version 2.0 allows XML_RPC 1.2.1 to be released and specifically state that it is compatible with a range of PEAR versions!

<compatible>
 <name>PEAR</name>
 <channel>pear.php.net</channel>
 <min>1.4.0</min>
 <max>1.4.3</max>
</compatible>

This has the effect of silently allowing XML_RPC 1.2.1 to be installed or upgraded from 1.2.0 along with PEAR. In this way, stricter versioning can be used to stop catastrophic upgrade problems for highly sensitive applications while still retaining the full benefits of a distributed application development model.

Splitting a larger package into smaller packages is not possible.

As packages grow in complexity, often the need to separate out existing files into subpackages becomes apparent. In the current PEAR 1.3.x installer, this is physically impossible, as the file conflict checker will note that the subpackage conflicts with the parent package. PEAR 1.4.0 circumvents this restriction by introducing the <subpackage> dependency type. This dependency type is identical to the <package> tag, except the file conflict rules are relaxed. If the subpackage conflicts with the installed version of a parent package, then the conflict is ignored. As a warning, no checking is done against the newer parent package's file list, and this can unfortunately allow a poorly debugged parent package to shoot itself in the foot. However, since the subpackage dependency is specified inside the newer parent package's package.xml, this is something that can be absolutely controlled by the parent package. It may be possible to validate against the newer package's file list as well, I just haven't found a good way to do it yet (all the information is available at install-time)

There is no way to specify feature sets rather than individual package dependencies

In PEAR 1.3.x, there are two kinds of dependencies, required and optional. This is simple, and pretty effective. PEAR 1.4.0 continues these two kinds of dependencies, but adds a third kind, optional dependency groups. These can be used to define feature sets.

A feature set is a logical grouping of package, subpackage or extension dependencies that defines a particular feature used by the package.

For instance, the PEAR package defines 3 feature sets

1. Web Installer
2. Gtk Installer
3. Remote FTP Installer

The first feature set contains the PEAR_Frontend_Web package, the second the PEAR_Frontend_Gtk package, and the third Net_FTP.

These feature sets are installed by simply appending the feature set like an html anchor:

$ pear install PEAR#remoteinstall

This will either install PEAR and Net_FTP, or if PEAR is already installed, it will install Net_FTP by itself. In other words, it is possible to install feature sets AFTER the primary package has been installed! Even more important, it is now possible to uninstall feature sets, solving the final problem: Uninstalling subpackages with a parent package must always be done by manually listing each package on the command line

$ pear uninstall PEAR#remoteinstall

This will uninstall PEAR and Net_FTP (although this is not really possible - you can't uninstall PEAR!)

Also introduced is the idea of a default dependency group. The default dependency group can be used to supply a list of packages or subpackages that should be installed by default. Advanced users can in fact choose a different install set, and the <package> dependency can specify that the default install set NOT be installed using the <nodefault /> tag:

<package>
 <name>Foo</name>
 <channel>foo.example.com</channel>
 <min>1.2.0</min>
 <nodefault/>
</package>

With these solutions, PEAR has taken one more step towards truly dynamic and secure installation. In addition, the work that the end-user client has to perform has decreased dramatically, in exchange for minimal extra effort from developers. Stay tuned for my next blog post about customizable installer roles and install-time tasks.