:wc: relfile 00-WikiIndex ---3 Package Management %abs= Package management is the way to manage software packages on the system. For many people, when __package management__ is mentioned what comes to mind is an application which they can use to download, install, and uninstall other software (software like ==synaptics, yum, apt-get, pacman==, etc). While this is not wrong, package management actually has a lot bigger scope than that. I will expound some of these issues which I think is the most important. %= ---4 What is Package Management, really? Package management is also known by its other trendy (at the time of writing) name, __Application Lifecycle Management__. It is the aspect of managing software (nee __applications__) which are installed in a system - from their creations, installations, updates, and de-installation (or removal) when their duties are over. Its scope covers from: # __Creation__ of deliverable software packages # __Making available__ of the packaged software # __Delivery__ of the packaged software # __Installation__ of the packaged software # __Tracking__ and __registration__ of installed software # __Checking the currency__ of the installed software # __Notification__ of available software updates # __Delivery__ of software updates # __Installation__ of software updates # __De-installation (removal)__ of software # __Cleanup__ and possibly __archiving__ of user-generated data from removed applications. . And less obvious but no less important: - __Dependency checking__ (whether a software about to be installed depends on something else) - __Dependency resolution__ (what to do if such dependency exists and the dependent software has not been installed yet, or its version is too old, etc) - __Detect and avoid__ conflicts of applications in the system, preferably before they are installed together - Ability to perform installation/removal/updates as a __transaction__, which means the software is either installed or not, but never partial. - __Tracking__ and __registration__ of user data bound to the application . And in especially the enterprise environment: - __Central management__ of software packages installed in various client devices (desktops, laptops, mobile) - this includes remote installation and de-installation (this is a billion-dollar industry in itself). . And that's just to begin with. I may have missed a few others. ---4 Why Package Management is important Looking at the above aspects, it should be immediately clear why package management is important. It is one of the side-aspects that people often don't appreciate (like the janitorial or secretarial services) - the state of the package management in the system determines, in the long run, the health of the system. Imagine a system where the package management software ("__package manager__" for short) cannot always successfully remove that it installed. Over time, cruft and garbage leftover by uninstalled applications will grow boundlessly, consuming precious system resources and slowing the system to the point it becomes unusable. When that happen, the only recourse is to __re-install__ the entire operating system. Sounds familiar? And like it or not, __there is always de-installation process__ even if you don't uninstall anything - because every application update always involves __removal__ either before or after the the new update is installed. A system with bad package management system will deteriorate over time as more and more updates are applied to it. Sounds familiar? It is difficult to over-emphasise the need of a good package management. ---- That being said, there is no perfect package management system. Available package managers (both free and commercial) systems always have deficiencies in one way or another. That is __**not**__ a reason __**not**__ to choose a package management system altogether, because obviously some package managers are better than others, even though they aren't perfect. ---4 Package management considerations for ''FatdogArm'' For ''FatdogArm'', the criteria I use will only meet a subset of those functionalities. The reason is simplicity and practicality - unlike many other operating systems; ''FatdogArm'' (and Fatdog64 and Puppy Linux in general) can be reset to its original pristine state in less than 1 minute; making mistakes in software installation lest costly than others. They are grouped into two categories: - those that belong to the software __user__ (end-user) - those that belong to the software __packager__ (developer) . :: %%End-user consideration%% The package manager should __at least__: # have two versions; one graphical and one for command line use (so failure in starting graphical desktop does not preclude the use of the package manager); # be remote repository aware: that is, it can list available packages from remote repository and can download/install packages from that repository; # be able to show list of installed software (which may be different from the available software from the remote repository); # be able to show "updates" or newer version of the installed software, if it is available on the remote repository; # be able to detect integrity of software downloaded from remote repository; # be able to install software downloaded by other means (not through the package manager itself); # remove installed software. . %%Developer consideration%% This is an oft-looked-over part of package management. Not only a package manager needs to provide ease-of-use to the end user; it must also help the __developer__ or a packager to build, package, and deliver the software to them. In general this means two things: # A tool to automate the software packaging. # A simple software distribution method. . Specifically, for ''FatdogArm'', I'm looking for # An automated method to build package. # A remote repository which can be published on standard web-server without any need for CGI or other complex requirements (where publishing can be done just by FTP-ing the repository contents). . ---4 ''FatdogArm'' package manager After various considerations, I have decided to adopt Slackware's package management software suite ("__pkgtools__"), complemented with Jason Woodward's excellent slapt-get and gslapt front-ends, as the package manager for ''FatdogArm''. ==pkgtools== will provide local package management; while the front-ends provides access to remote repositories. (Slackware has its own front-end to access remote repository called ==slackpkg==, but Jason's one is nicer). pkgtools package format is simple (it's just a tarball), the package logging format is simple; it comes with tools to build packages (==makepkg==); and the package manager is text-based which means it can be used even when the graphical desktop isn't working. slapt-get adds remote repository access (=listing of available files on repository servers); gslapt adds a nice graphical GUI on top of that. The repository format is simple (just a bunch of packages plus PACKAGES.TXT) which is publishable on any web server; integrity is maintained by using a centralised MD5 checksum and optionally, GPG signing. ---- As originally noted in BuildingApplications article, I have used [[http://paco.sourceforge.net] paco] as a temporary measure to log and capture the software being built. While paco never claims to be a package manager (the description on its homepage has this to say), %wikiseq= After the installation of a source package with "./configure && make && make install", one is usually left with having no idea of what it was installed and where it all went, making it difficult to uninstall the package in the future. Paco was written to solve this problem in a quite simple fashion. When installing a package from sources, paco wraps the "make install" command (or whatever command or group of commands are needed to install the files into the system), and saves installation information into a text database. %= paco **is** actually a working and a worthwhile package manager in its own. Not only it can logs the installation, it can also list all previously installed software (and remove them if asked), as well as create tarballs of previously installed files (called __pacoballs__) which can be used to re-install the same software later (on the same or different machine). In addition it also has a nice graphical GUI interface (==gpaco==) for viewing installed packages, list the files, install pacoballs, etc. So why not paco as the package manager? Indeed, I would have gone with paco were it not for one big gaping hole: paco only handles local files, and it does not have remote repository access; in a way it is very similar to ==pkgtools== without slapt-get/gslapt (and I would argue paco is way better than pkgtools, but I digress). I could have written my own front-end to enable remote-repository access to paco; but it is easier to use existing tools. ---- Fatdog64 (on which ''FatdogArm'' is based) comes with its own original package manager called Fatdog Package Manager (FPM) written by me. It meets the criteria too; and in some respects is better (e.g. FPM has categories; gslapt doesn't; FPM is layered-filesystem aware and can restore original file from the basesfs layer when a package is uninstalled). So why not FPM instead of pkgtools/slapt-get/gslapt? - Performance reasons - gslapt, from its virtue of being written in C, is much faster than FPM (written in bash script and gtk-server). - gslapt's version and dependency checking is more comprehensive. - slapt-get works in terminal, there is no equivalent console-based UI for FPM. . ---4 Bootstrapping ''FatdogArm'' package management As paco already has all the package installation information, it is a matter of converting its logs into pkgtools log format. This is quite straightforward as both formats are simple tarballs adorned with text file descriptions. I wrote a script to do this, you can get it [[../files/paco2pkg.gz] here ]. The script can automatically generate dependency information too. After the conversion, all packages originally logged with paco will show up as installed packages in pkgtools/slapt-get/gslapt; and they can be removed and re-installed or updated as needed. As part of the conversion, we can also create pkgtools tarballs - these will be used for build the initial repository. Once the pkgtools tarball it is a simple matter to use [[http://software.jaos.org/git/slapt-get/plain/FAQ.html#slgFAQ17] Jason's provided script] to build repository for slapt-get/gslapt. ---3 Conclusion This article concludes the series of ''FatdogArm'' porting articles. By the time we reached this step, we already have a working operating system which is good enough for distribution to others. From here onwards, it is just a matter of maintenance and optimising the system for specific purposes, such as size optimisation, making use of specific hardware features, etc. There are of course large topics of information that I skip; not because they are not important but because they distract from the main purpose of these articles: illustrating the ''FatdogArm'' porting process. I may write them in the future as independent articles. For now, though, this is **The End** (but don't forget to read the [[00-WikiIndex#extras] extras]). :: ---- References: - [[+_blank http://slackware.com/config/packages.php] Slackware's pkgtools] - [[+_blank http://www.slackbook.org/html/package-management.html] Slackware package management] - [[+_blank http://software.jaos.org/#slapt-get] Jason Woodward's slapt-get and gslapt] - [[+_blank http://software.jaos.org/git/slapt-get/plain/FAQ.html#slgFAQ17] Script to create slapt-get/gslapt repository ] - [[+_blank http://www.src2pkg.net/] src2pkg, a tool that can create many different package formats from source] .