Setting The Environment
Packaging For MX Repository
- Clean MX or MEPIS standard environment – only Debian Stable and MX/MEPIS 12 and M12 community repositories enabled. However, for M12/MX 14, wheezy-backports is allowed to be enabled by default. An M12 build environment is usable to create MX 14 packages. A clean virtual machine (eg Virtual Box environment) is a good substitute for a clean stand alone install. Be aware that many 3D applications cannot run in a virtual machine, including many games and obs-studio. Setting up a shared folder between the virtual machines and the host OS will speed up workflow, though files should be passed though the folder in an unextracted state (e.g., extracted Debian source may get the permissions changed for some files. Passing debs, tarballs, and .dsc files are fine. Also, building a package in a shared folder may have unpredictable results, so move it to another folder within the VM first.) It’s also handy to save a snapshot of your VM in a known clean state, so you can revert to it if necessary–for package installation testing, for example…Also, for M11/M12/MX 14 guests, make sure your user is part of the vboxsf (shared folder) group with Kuser in KDE, or the MX User Manager/Group Membership
- If any portion of your OS environment has been upgraded to testing or unstable, you cannot build for standard MX/MEPIS repositories
- Due to legal considerations, no packages from the Debian-Multimedia repository can be installed, and the repository must be disabled. Exceptions can be made for packages that have no legal (patent) or licensing issues, but just haven’t made it into upstream Debian yet; in that case, they should be ported to our repository.
- The Debian backports repository disabled – they are not always compatible with MX/MEPIS, except you should have it enabled for M12/MX 14. It is up to the maintainer to decide whether to build against the stable libraries, or those from backports. Either are acceptable in M12 and MX. However, consider if you want standard Debian users to be able to use your package–you’ll get a lot of kudos for doing that.
- Clean separate install of MX/M12 on a separate test / build partition
- Where possible we would like package builders / testers / maintainers to be able to use both 32bit and 64bit architectures.
- If your hardware is adequate, MX/M12 installs in virtual machines are useful in compiling and testing packages.
- Basic build packages for MEPIS 12 and MX.:
apt-get install packaging-dev This is a metapackage which makes sure you have all the packaging tools.
- Add in ~/.bashrc (a hidden text file in your home directory) these environment variables:
export DEBFULLNAME=”Your Name” (use your own name!)
export DEBFULLNAME=”Steven Pusser (Stevo)”
These MUST be exactly the same as used in creating your public/private keys in gpg or Kgpg.
These variables are used by “dch -i” and debmake/dh-make when creating debs, and since the package signing gets your name from debian/changelog as added by the “dch -i”, they should match.
- Run in a konsole terminal as a regular user
source ~/.bashrc (~ is shorthand for your home directory)
You will also need to create a public/private key pair. This can be done in the terminal with gpg, or with a user-friendly frontend such as Kgpg, under Keys >> Generate Key Pair. Make sure not to lose your gpg password! You can also export your secret key under one version, then import into Kgpg in another install., or copy your ~/.gnupg folder into another system, such your virtual machines
Debian Packaging Documentation – optional – but should be read along with this document. Gives insights into debian packaging procedures.
apt-get install debian-policy developers-reference
- Debian-policy: the Policy includes explanations of the structure and contents of the Debian archive, several OS design issues, the Filesystem Hierarchy Standard (which says where each file and directory should be) etc. For you, the most important thing is that it describes requirements that each package must satisfy to be included in the distribution. (see /usr/share/doc/debian-policy/policy.html/index.html)
- Developers-reference: for all matters not specifically about the technical details of packaging, like the structure of the archive, how to rename, orphan, pick up packages, how to do NMUs (Non-Maintainers Uploads), how to manage bugs, best packaging practices, when and where to upload etc. (see /usr/share/doc/developers-reference/index.en-us.iso-8859-1.html)
Recommended Reading – Web
- Wikipedia article on the Debian Build Toolchain (http://en.wikipedia.org/wiki/Debian_build_toolchain)
- Debian New Maintainers Guide (http://www.debian.org/doc/maint-guide/ch-start.en.html)
- Debian English Packaging Tutorial (http://wiki.debian.org/BuildingTutorial)
- Debian Building Without Helper (http://wiki.debian.org/Courses2005/BuildingWithoutHelper)
- Ubuntu Packaging Guide Complete: Ubuntu specific but still some good info (https://wiki.ubuntu.com/PackagingGuide/Complete)
Base Procedure – Packaging for Official MEPIS & Community Repos
Packager (part 1)
- Packager uses one of recommended methods (in this procedure) for creating debian package
- Following successful deb creation, Packager tests installing the created deb, and if successful, advises their Maintainer of the availability of the completed package – along with sending him/her the necessary files for the repository (the deb(s), dsc, diff,debian.tar.xx, changes and source files). The extracted source folder where the build took place should always be deleted. If in doubt, take a look at the files listed in the .changes file, those are the only ones that the repository requires. The .build file is a build log–it’s not necessary to upload, but compresses very well.
NOTE: You have to submit the maintainer the “.changes” file for a given package as well as all the files listed in the “Files:” section within the changes file. See “Special_Notes_-_IMPORTANT” below.
Maintainer (part 1)
- On receipt of advice from their packager, the Maintainer downloads the files from the link provided, virus scans them, installs them in a test environment (to ensure working), and if ready for testing, uploads them to the test repository.
- Once the package appears in test, the Maintainer contacts their Packager and advises him/her to post in the original ML thread that their package is available in the Test repository, and to request testing and feedback.
- Testers (from the community) install debs and commence thorough checking of application.
- Any errors / lack of functionality are noted
- Following full testing, feedback is given in the original thread on MEPIS Community forum.
Packager (part 2)
- Packager acts on Tester feedback.
- If package fails successful testing with errors or loss of functionality, errors are corrected by Packager, and files resubmitted for testing.
- If package passes testing with full functionality, Packager’s duties are complete, and Maintainer arranges for final migration of Package from test to main repository.
Maintainer (Part 2)
Once package has passed testing and is declared ready for the repository,
- Package is either sent to Developer certified as correct and ready for the mainr epositorie , OR
- The Maintainer also either adds or edits the entry in the Community Packaging Section of the MX/MEPIS Community website.
- Be sure to reversion packages by simply adding :
- mcr120+1 to the end of the debian version (for 12.0 Community Repository)
- mx140+1 to the end of the debian version (for MX 14 repo only)
- mx150+1 to the end of the debian version (for MX 15 repo only)
- If you are packaging a version of a package that is not in Debian, give it a -0 version, as in 1.2.3-0mx150+1. This ensures an “official” Debian -1 package will take priority if it appears later in the upstream Debian repo or in the official backports repository, depending on pinning and apt-preferences.
- If the version ends in debian1 or ubuntu1 etc, drop the debian or ubuntu minor version number.
- Be very careful not to increase the version number any more than you have to
- The number should be higher than the equivalent debian number and less than the most likely next debian number; .i.e., if you’re jumping ahead of Debian and packaging Kodi 16.0 when upstream Debian only has 15.2, then the versioning should be 16.0-0mx150+1, to leave “room” for a possible rebuild of Debian’s 16.0-1 version as 16.0-1mx150+1. Anything tacked onto the version makes it higher than Debian’s, except the “~”, so that a jessie-backport 1.2.3-4~bpo80+1 version will be lower than the 1.2.3-4 version in upstream Debian.
upstream Debian: 915resolution 0.53-1
MX: 915resolution 0.53-1mx150+1
next MX: 915resolution 0.53-1mx150+2
next Debian: 915resolution 0.53-2
then MX: 915resolution 0.53-2mx150+1
Recommended Package Creation Methods
Basic Package Creation Decision Tree
Attempt in the order shown.
1. Backporting From Upstream Debian. The package is in the Debian pool or stable-backports. A newer version is available but not for Debian Stable. In this case, get the Debian source for the newer version and change the maintainer and minor info to M12/MX. Also note that source code from Debian has been vetted for security and bugs much more than any other sources, making it the first choice.
NOTE – unless you are a “Debian wizard�? avoid using source from Debian ‘experimental’ if possible
- a. Attempt to rebuild packages in MX/M12. If this works, great. Time to test and publish.
- b. Try to resolve compilation issues. Abandon the backport if it needs a newer libglib2.0, gtk-3-dev, or other base system libraries. But if that newer build-dep was added by a patch or something in debian/rules, disabling the patch/reverting rules might fix it.
- c. Try a newer, but not quite so new version from Debian that may be more compatible with MEPIS. Look in the Debian snapshots pool(snapshots.debian.org) if you know exactly what change you’re trying to avoid. Maybe even inspect the change or Debian patch that broke compatibility and attempt to reverse it in the MEPIS version of the latest Debian version.
2. Repackaging from Debian-Backports As a fallback, check the backports pool for a Debian Stable version. Again, change the maintainer and the minor version.
- a. Attempt to repackage.
- b. If that fails, attempt to fix the issues.
- c. You can also find older package sources from upstream at snapshot.debian.org that are still more recent than the ones in Debian stable.
3. Backporting from Ubuntu If still no success, try an Ubuntu version. Source files can be found at packages.ubuntu.com. Be very very careful about introducing Debian incompatibilities. Libraries may have different names, such as for Qt 5. Often you can find sources in a PPA or even a private Debian-compatible repository. If so, double check the licenses of the source to make sure it can be freely redistributed. Also, Ubuntu packages must be put in the correct Debian section: free, contrib, or non-free, not the Ubuntu universe and multiverse sections. (in the first section of the debian/control file)
4. Packaging from Bare Source Tarball If no debianized source version can be found, try creating a debianized version from vanilla source. If there is an older debianized version, try reusing its debian directory.
Details of Each Method
1 Backporting From Upstream Debian (Recommended Method)
- Go to http://packages.debian.org and type the name of the package you want to backport into the “Keyword” text input box, and change the “Distribution” drop-down box to “any”.
- Click the link for the version you want to backport. You can also peruse the Debian changelog to get an idea of what is different between the versions.
- On the right-hand side of the page there is a section titled “Links for <packagename>”. Download the three files under the sub-heading “Download Source Package”. There is normally an .orig.tar.gz, a diff.gz,(or a debian.tar.gz and orig.tar.gz/bz2/xz set) and a dsc file. Sometimes there may not be a .diff.gz file if the package is a Debian native package (Synaptic, for example) and not from an outside source, but everything from here on out is still the same. To download the .dsc, you must right-click and select “Save As”, as normally the browser will open .dsc files as a plain text file. Be careful! Often the directories for clicking to save and right-clicking to save as in your browser are different. Make sure all files are downloaded to the same folder. I usually use a folder in my /home/username/Source directory titled “DEB-packagename-version”.
- Open a console and cd to the folder you downloaded to, or navigate to the folder using your file manager and open a terminal at that location.
- In the terminal, type “dpkg-source -x ” and then hit the tab key. If there is only one .dsc file in the directory (as there should be), then the filename will be entered automatically. Hit enter to issue the command, and the source should be extracted. IMPORTANT: most packages are now using the 3.0 (quilt) source format, which applies patches from /debian/patches when this command is issued. The older formats don’t apply the patches until the build command is run. If you wish to avoid the patching with 3.0, just manually extract the debian and orig tarballs, and put the debian folder inside the source–now the patching won’t happen until the build is started.
- Type “cd ” and hit tab. The newly created folder’s name should be automatically filled in. Hit enter to change to the source folder.
- Type dch -i into the terminal and hit enter. This will open a debian
changelog editor in the console with the time/date stamp and your
maintainer information already entered (as long as you exported as
outlined above in “Setting the Environment”). The debian version of the
package will have been incremented, so change it back to the version in
the previous entry in the changelog and add the appropriate tag to the
end, as follows:
- M12 Community version: foo_1.1-1mcr120+1
- Change “UNRELEASED” to “mepis” or “mx” as appropriate. After the asterisk, type “Repackaged (“Packaged” for native MX packages or one where you’ve done the actual original debianization) for MEPIS 12 community repository/ MX 14/ MX 15″ (as appropriate). Hit Ctrl+X to begin to exit the debian changelog editor. Hit “y” to save changes. Then hit Enter to save the changes to the same file. This will return you to the command line prompt. This file may have to be edited more later, if anything must be changed in the code or /debian directory to get the package to build.
- Type cd debian into the console and hit enter to change to the /debian directory.
- Open the /debian/control file in a text editor. Watch out for any “control.in” or “rules.in” files, you will have to edit those instead, since the build process will convert those to the final control or rules files. Any other “.in” files have the same warning.
- Add “XSBC-Original-“ before the Maintainer, and make a new line above that that reads “Maintainer: yourDEBFULLNAME <yourDEBEMAIL>”, replacing yourDEBFULLNAME and yourDEBEMAIL with the info you exported as outlined above in “Setting the Environment”. Then, create a line after Uploaders: or (if there is no Uploaders: line, XSBC-Original-Maintainer:) that reads, Bugs: mailto: <yourDEBEMAIL>, again replacing yourDEBEMAIL with what you have exported in your “~/.bashrc” file (see “Setting the Environment”). Save and close the control file.
- THE CHANGELOG AND CONTROL FILES ARE VERY VERY STRICT ABOUT SYNTAX. An extra whitespace in the wrong place can result in lots of error messages or even halt the build. All kinds of spaces and tabs are allowed in between items in a list, however, such as build-depends. Be careful…
- Type cd .. into the console and hit enter to return to the source directory.
- Type su -c ‘/usr/lib/pbuilder/pbuilder-satisfydepends’ This will install all build-dependencies listed in the /debian/control file, as long as they are in the repositories.
- This will also autoremove any packages marked as such, so check that your desktop environment is not marked as such with su -c ‘apt-get -s autoremove It is normal for many “-dev” packages to be removable, but watch out for your core desktop packages. Consult the packaging whizzes on how to unmark any packages. This may also downgrade some backported libraries to Debian stable versions, just upgrade them back if you wish to have the newer libraries used for your backport. It’s also possible to just manually install missing build-depends in Synaptic or the command line.
- If this works, awesome! If some build-depend is not found, you may have to look into backporting and installing a newer version or even an entirely new package to meet the build-deps. Sometimes the same libraries are provided by different package names under stable and upstream Debian. If this is the case, edit the control file again with your preferred text editor to reflect the package names in the current M12/MX Debian base. After this is done, return to this step in the original package.
- Type “debuild -sa” for the first build . If the
same source tarball is already present in the repository, for example,
you are building a mx150+2 version because of a bug in the mx150+1
packaging, leave off the -sa. Most packages support
compiling with parallel threading, that is, it will use more than one
core on a multicore CPU for compiling. Some packages will error out
during the compile with this enabled, and it does not help
arch-independent script-based programs, such as pure Python or Perl. To
use parallel builds, add the option -jN to the build
command, where N=number of cores you have plus 1, i.e. “debuild -sa -j5”
for a four core build. Hit <Enter> to begin building the package.
- After the packages have been built, you will be prompted to enter your GPG key passphrase to sign the ancillary packages. You can also sign packages at any time using the “debsign” command, which is provided by devscripts:
Debsign will try and sign the package with the system key assigned to the maintainer (email@example.com) You can also specify the key with the -k argument if necessary:
debsign -k <key ID>
This is handy if you have someone else build the packages on another architecture for you, since you can still sign their packages with your secret key.
The above “first build” procedure is the same on either 32 or 64-bit, it does not matter which architecture it’s done on. For architecture-independent programs, (python or perl programs that build only “*all.deb” files, that’s it–install and test. Arch-dependent compiled programs need to be built on the other platform. For the second build, delete the extracted source folder and copy the build folder you have all the files in to the second build platform and do the same setup as above, except this time the build-command is going to be “debuild -B -jN”
- You should now have all the necessary .debs (and the source, diff/debian or (native packages) .tar.gz/xz, changes, build files) in the top-level folder that contains the Debian source files. Install them as appropriate and test!
- which means only the arch-dependent “B”inary files will be built. The source files won’t be regenerated, either.
- Source packages are also found at http://packages.debian.org. Procedures are the same as in Step 1, except select Search on: “Source package names”
- Get source from http://packages.ubuntu.com or possibly http://www.getdeb.net. Libraries may have been renamed to custom Ubuntu versions.
- Build the package normally for the newest MX version first, since backporting will be easiest for newer releases.
- When building for the older MX/MEPIS versions, start with the same source packages, but before building adjust the /debian/changelog file to show the build is for the older MX/MEPIS release, i.e. mcr120+1
- If they will share a repository (see below), build this next version without the “-sa” like so: debuild for the first architecture and debuild -B for the second architecture so that you don’t get new original source tarballs marked for upload, just the /debian/diffs tarball and new .dsc file. You can still use the -j# switch for building with #-1 cores on a multicore machine. You can also discard the original source tarball after finishing both builds and testing successfully, since it will not be uploading into the repository.
- All MEPIS CR versions and MX 14 share the same repo. MX 15 has a separate repository, so currently all builds there are with -sa for new package versions, but the same procedure will be used when we get multiple versions of MX using that repository. If you are uploading multiple builds for MX 15, keep the MEPIS and MX 15 files in separate folders. If it’s a sizable orig.tar file, you can include just one copy to save bandwidth, but let the repo maintainer know that.
- Also keep this in mind if you’re rebuilding a version of a package that already has its .orig tarball in the respective repository, so you won’t use the “-sa” flag in that case. An example would be doing an mx150+2 rebuild to fix a problem or something missing in the first build, or the various Liquorix kernels that all use the same orig.tar.xz file as a base for a particular kernel release.
- If there’s the problem with checksum conflicts, the repo maintainer generally can force the packages in anyway, but would prefer not to have to do it that way.
Special Notes – IMPORTANT
The packages in the archive areas main, contrib and non-free are grouped further into sections to simplify handling.
The archive area and section for each package should be specified in the package’s Section control record (see Section, Section 5.6.5). However, the maintainer of the Debian archive may override this selection to ensure the consistency of the Debian distribution. The Section field should be of the form:
section if the package is in the main archive area,
area/section if the package is in the contrib or non-free archive areas.
The Debian archive maintainers provide the authoritative list of sections. At present, they are: admin, cli-mono, comm, database, debug, devel, doc, editors, education, electronics, embedded, fonts, games, gnome, gnu-r, gnustep, graphics, hamradio, haskell, httpd, interpreters, introspection, java, kde, kernel, libdevel, libs, lisp, localization, mail, math, metapackages, misc, net, news, ocaml, oldlibs, otherosfs, perl, php, python, ruby, science, shells, sound, tasks, tex, text, utils, vcs, video, web, x11, xfce, zope.
32b / 64b
The ideal is for the package builder to create both 32b and 64b packages for arch-dependendent packages. When this is not possible, the packager must find another package builder to compile the matching package for the other architecture. It is also imperative if builds are split between 2 packagers – that same source and build method is used. It is also permissible to use the openSUSE build service to build your packages, as long as you control the OBS repository and sign the finished packages with debsign. Note that the OBS requires a different versioning format (i.e., “mx150.1” instead of “mx150+1”)