Wednesday, February 04, 2015

Native IPS Manifests

We used to use pkgbuild tool to generate IPS packages. However recently I started working on internal Solaris SPARC build and we decided to use IPS fat packages for x86 and SPARC platforms, similarly to how Oracle delivers Solaris itself. We could keep using pkgbuild but as it always puts a variant of a host on which it was executed from, it means that we would have to run it once on a x86 server, once on a SPARC server, each time publishing to a separate repository and then use pkgmerge to create a fat package and publish it into a 3rd repo.

Since we have all our binaries already compiled for all platforms, when we build a package (RPM, IPS, etc.) all we have to do is to pick up proper files, add metadata and publish a package. No point in having three repositories and at least two hosts involved in publishing a package.

In our case native IPS manifest is a better (simpler) way to do it - we can publish a fat package from a single server to its final repository in a single step.

What is also useful is that pkgmogrify transformations can be listed in the same manifest file. Entire file is loaded first and then any transformations would be run in the specified order and new manifest will be printed to stdout. This means that in most cases we can have a single file for each package we want to generate, similarily to pkgbuild. There are cases where there are lots of files and we do use pkgsend generate to generate all files and directories, and then we have a separate file with metadata and transformations. In this case pkgbuild is a little bit easier to understand compared to what native IPS tooling offers, but it actually is not that bad.

Let's see an example IPS manifest, with some basic transformations and with both x86 and SPARC binaries.

set name=pkg.fmri value=pkg://ms/ms/pam/access@$(PKG_VERSION).$(PKG_RELEASE),5.11-0
set name=pkg.summary value="PAM pam_access library"
set name=pkg.description value="PAM pam_access module. Compiled from Linux-PAM-1.1.6."
set name=info.classification value=""
set name=info.maintainer value="Robert Milkowski "

set name=variant.arch value=i386 value=sparc

depend type=require fmri=ms/pam/libpam@$(PKG_VERSION).$(PKG_RELEASE)

dir group=sys mode=0755 owner=root path=usr
dir group=bin mode=0755 owner=root path=usr/lib
dir group=bin mode=0755 owner=root path=usr/lib/security
dir group=bin mode=0755 owner=root path=usr/lib/security/amd64      variant.arch=i386
dir group=bin mode=0755 owner=root path=usr/lib/security/sparcv9    variant.arch=sparc

&lttransform file -> default mode 0555>
&lttransform file -> default group bin>
&lttransform file -> default owner root>

# i386
file SOURCES/Linux-PAM/libs/intel/32/    path=usr/lib/security/          variant.arch=i386
file SOURCES/Linux-PAM/libs/intel/64/    path=usr/lib/security/amd64/    variant.arch=i386

# sparc
file SOURCES/Linux-PAM/libs/sparc/32/    path=usr/lib/security/          variant.arch=sparc
file SOURCES/Linux-PAM/libs/sparc/64/    path=usr/lib/security/sparcv9/  variant.arch=sparc

We can then publish the manifest by running:
$ pkgmogrify -D PKG_VERSION=1.1.6 -D PKG_RELEASE=1 SPECS/ms-pam-access.manifest | \
    pkgsend publish -s /path/to/IPS/repo
This would really go into a Makefile so in order to publish a package one does something like:
$ PUBLISH_REPO=file:///xxxxx/ gmake publish-ms-pam-access
In case where there are too many files to list them manually in the manifest, you can use pkgsend generate to generate a full list of files and directories. You need to create a manifest with only package meta data and all transformations (which would put files in their proper locations, set desired owner, group, etc.). In order to publish a package one puts into a Makefile somethine like:
$ pkgsend generate SOURCES/LWP/5.805 >BUILD/ms-perl-LWP.files
$ pkgmogrify -D PKG_VERSION=5 -D PKG_RELEASE=805 SPECS/ms-perl-LWP.p5m BUILD/ms-perl-LWP.files | \
    pkgsend publish -s /path/to/IPS/repo

1 comment:

David said...

I remember building SVR4 packages, which were compatible across NCR UNIX MP-RAS for Intel and Sun SPARC Solaris ~20 years ago. Of course, the same packages would install under Intel Solaris, but the underlying 3rd party software infrastructure was not available.

When I released software, the same identical package would install/update/migrate underlying OS & Application Infrastructure. Life was simpler with standards traversing different Operating Systems, different OEM's, different CPU architectures.