synaptics, yum, apt-get, pacman, etc).
While this is not wrong, package management actually has a lot bigger scope than that. I will expound some of these issues which I think is the most important.
What is Package Management, really?
Package management is also known by its other trendy (at the time of writing) name, Application Lifecycle Management. It is the aspect of managing software (nee applications) which are installed in a system - from their creations, installations, updates, and de-installation (or removal) when their duties are over.
Its scope covers from:
- Creation of deliverable software packages
- Making available of the packaged software
- Delivery of the packaged software
- Installation of the packaged software
- Tracking and registration of installed software
- Checking the currency of the installed software
- Notification of available software updates
- Delivery of software updates
- Installation of software updates
- De-installation (removal) of software
- Cleanup and possibly archiving of user-generated data from removed applications.
And less obvious but no less important:
- Dependency checking (whether a software about to be installed depends on something else)
- Dependency resolution (what to do if such dependency exists and the dependent software has not been installed yet, or its version is too old, etc)
- Detect and avoid conflicts of applications in the system, preferably before they are installed together
- Ability to perform installation/removal/updates as a transaction, which means the software is either installed or not, but never partial.
- Tracking and registration of user data bound to the application
And in especially the enterprise environment:
- Central management of software packages installed in various client devices (desktops, laptops, mobile) - this includes remote installation and de-installation (this is a billion-dollar industry in itself).
And that's just to begin with. I may have missed a few others.
Why Package Management is important
Looking at the above aspects, it should be immediately clear why package management is important. It is one of the side-aspects that people often don't appreciate (like the janitorial or secretarial services) - the state of the package management in the system determines, in the long run, the health of the system.
Imagine a system where the package management software ("package manager" for short) cannot always successfully remove that it installed. Over time, cruft and garbage leftover by uninstalled applications will grow boundlessly, consuming precious system resources and slowing the system to the point it becomes unusable. When that happen, the only recourse is to re-install the entire operating system. Sounds familiar?
And like it or not, there is always de-installation process even if you don't uninstall anything - because every application update always involves removal either before or after the the new update is installed. A system with bad package management system will deteriorate over time as more and more updates are applied to it. Sounds familiar?
It is difficult to over-emphasise the need of a good package management.
That being said, there is no perfect package management system. Available package managers (both free and commercial) systems always have deficiencies in one way or another. That is not a reason not to choose a package management system altogether, because obviously some package managers are better than others, even though they aren't perfect.
Package management considerations for FatdogArm
For FatdogArm, the criteria I use will only meet a subset of those functionalities. The reason is simplicity and practicality - unlike many other operating systems; FatdogArm (and Fatdog64 and Puppy Linux in general) can be reset to its original pristine state in less than 1 minute; making mistakes in software installation lest costly than others.
They are grouped into two categories:
- those that belong to the software user (end-user)
- those that belong to the software packager (developer)
The package manager should at least:
- have two versions; one graphical and one for command line use
(so failure in starting graphical desktop does not preclude the use of
the package manager);
- be remote repository aware: that is, it can list available packages
from remote repository and can download/install packages from that
- be able to show list of installed software (which may be different from
the available software from the remote repository);
- be able to show "updates" or newer version of the installed software,
if it is available on the remote repository;
- be able to detect integrity of software downloaded from remote repository;
- be able to install software downloaded by other means (not through the
package manager itself);
- remove installed software.
This is an oft-looked-over part of package management. Not only a package manager needs to provide ease-of-use to the end user; it must also help the developer or a packager to build, package, and deliver the software to them.
In general this means two things:
- A tool to automate the software packaging.
- A simple software distribution method.
Specifically, for FatdogArm, I'm looking for
- An automated method to build package.
- A remote repository which can be published on standard web-server without any need for CGI or other complex requirements (where publishing can be done just by FTP-ing the repository contents).
FatdogArm package manager
After various considerations, I have decided to adopt Slackware's package management software suite ("pkgtools"), complemented with Jason Woodward's excellent slapt-get and gslapt front-ends, as the package manager for FatdogArm.
pkgtools will provide local package management; while the front-ends
provides access to remote repositories.
(Slackware has its own front-end to access remote repository called
slackpkg, but Jason's one is nicer).
pkgtools package format is simple (it's just a tarball), the
package logging format is simple; it comes with tools to build packages
makepkg); and the package manager is text-based which means it can be
used even when the graphical desktop isn't working.
slapt-get adds remote repository access (=listing of available files on repository servers); gslapt adds a nice graphical GUI on top of that. The repository format is simple (just a bunch of packages plus PACKAGES.TXT) which is publishable on any web server; integrity is maintained by using a centralised MD5 checksum and optionally, GPG signing.
As originally noted in BuildingApplications article, I have used paco as a temporary measure to log and capture the software being built. While paco never claims to be a package manager (the description on its homepage has this to say),
Paco was written to solve this problem in a quite simple fashion.
When installing a package from sources, paco wraps the "make install" command (or whatever command or group of commands are needed to install the files into the system), and saves installation information into a text database.
paco is actually a working and a worthwhile package manager in its own.
Not only it can logs the installation, it can also list all previously
installed software (and remove them if asked), as well as create tarballs
of previously installed files (called pacoballs) which can be used to
re-install the same software later (on the same or different machine).
In addition it also has a nice graphical GUI interface (
gpaco) for viewing
installed packages, list the files, install pacoballs, etc.
So why not paco as the package manager? Indeed, I would have gone with paco
were it not for one big gaping hole: paco only handles local files, and it
does not have remote repository access; in a way it is very similar to
pkgtools without slapt-get/gslapt (and I would argue paco is way better
than pkgtools, but I digress).
I could have written my own front-end to enable remote-repository access to
paco; but it is easier to use existing tools.
Fatdog64 (on which FatdogArm is based) comes with its own original package manager called Fatdog Package Manager (FPM) written by me.
It meets the criteria too; and in some respects is better (e.g. FPM has categories; gslapt doesn't; FPM is layered-filesystem aware and can restore original file from the basesfs layer when a package is uninstalled).
So why not FPM instead of pkgtools/slapt-get/gslapt?
- Performance reasons - gslapt, from its virtue of being written in C, is much faster than FPM (written in bash script and gtk-server).
- gslapt's version and dependency checking is more comprehensive.
- slapt-get works in terminal, there is no equivalent console-based UI for FPM.
Bootstrapping FatdogArm package management
As paco already has all the package installation information, it is a matter of converting its logs into pkgtools log format. This is quite straightforward as both formats are simple tarballs adorned with text file descriptions. I wrote a script to do this, you can get it here. The script can automatically generate dependency information too.
After the conversion, all packages originally logged with paco will show up as installed packages in pkgtools/slapt-get/gslapt; and they can be removed and re-installed or updated as needed.
As part of the conversion, we can also create pkgtools tarballs - these will be used for build the initial repository. Once the pkgtools tarball it is a simple matter to use Jason's provided script to build repository for slapt-get/gslapt.
This article concludes the series of FatdogArm porting articles. By the time we reached this step, we already have a working operating system which is good enough for distribution to others.
From here onwards, it is just a matter of maintenance and optimising the system for specific purposes, such as size optimisation, making use of specific hardware features, etc.
There are of course large topics of information that I skip; not because they are not important but because they distract from the main purpose of these articles: illustrating the FatdogArm porting process. I may write them in the future as independent articles.
For now, though, this is The End (but don't forget to read the extras).
- Slackware's pkgtools
- Slackware package management
- Jason Woodward's slapt-get and gslapt
- Script to create slapt-get/gslapt repository
- src2pkg, a tool that can create many different package formats from source