Wiki » SOP Moving RPMs

Moving RPMs between repos

Packages may occasionally need to be moved between package repositories. E.g. from updates_testing to updates, or from updates_testing to release in Cauldron. There are different ways these should be moved, depending on the reason.

In all cases, make sure that all relevant packages are moved at once. This means moving all subpackages as well as any relevant dependencies. Failing to do so can result in a broken package repository. Also, make sure both source and destination hdlists files are updated after to move or urpmi won't see them (also the equivalent dnf metadata files). Moves are performed as root on the host repository.mageia.org (the main build server, duvel as of this writing).

The directory /distrib/bootstrap/ is the working area where changes are made. This area is synced by the hdlist updates scripts to the directory /distrib/mirror/ which is what is rsynced by the mirrors and is seen by end users.

Stable Package Updates

If a package has been updated in a stable distro version due to a security fix or bug fix, use the procedure in pushing updates (using mgaadv). This procedure should be used for almost any move within a stable distro release to ensure the proper user visibility (errors excepted). These moves are always between updates_testing and updates.

However, if something went wrong during that procedure (such as a .src.rpm was accidentally left out of a security advisory and so it didn't get moved like it should), that can be fixed like this:

1. If an RPM was left out of a security advisory, fix that by adding the package to the file in your local copy of ~/mageia-advisories/advisories/ and checking it in.
2. Move the package from updates_testing to updates by running a command like this as root on duvel: mga-move-pkg --sync 9/core/foobar-0.1-1.mga9.src.rpm
3. Run this as root to fix the file ownership: find /distrib/{bootstrap,mirror}/ -user root \! -type l -exec chown schedbot:schedbot {} +
4. The MGSA updated in step 1 won't be public until the next security or bug advisory is pushed. It can be expedited using a procedure TBD.

Stable Package Removal

If a package has been uploaded under updates_testing by error and a developer is requesting removal:

1. First, check that this is a valid request. Was this asked by the packager that built the package or the registered maintainer of the package? Is this a removal from updates_testing (good) or another section (suspect)?
2. Run the mga-move-pkg in dry-run mode to see what files need to be moved: mga-move-pkg --dry-run 9/core/mypackage-99-1.mga9.src.rpm
3. Or, find the RPM files manually under /distrib/bootstrap/distrib/9/*/media/{,debug/}core/updates_testing/ and /distrib/bootstrap/distrib/9/SRPMS/core/updates_testing/ (where 9 is the stable mga release)
4. Delete those RPMs then run as root mga-hdlists 9 core updates_testing (or as relevant).
5. Run this as root to fix the file ownership: find /distrib/{bootstrap,mirror}/ -user root \! -type l -exec chown schedbot:schedbot {} +

Cauldron Normal Package Moves

These are less critical than moves in a stable distribution since there are not many guarantees about Cauldron and the formal move procedure needed for stable releases isn't needed. These moves usually result from a packager making a mistake while building, such as building in the wrong section. Follow this procedure:

1. Evaluate the request. Was this asked by the packager that built the package or the registered maintainer of the package? Sanity check it to make sure it makes sense. For example, a request to move a package from Nonfree to Core should only be done if the package belongs in Core, especially that the license allows it.
2. Run a command of this form:

mga-move-pkg --sync cauldron/core/foobar-0.1-1.mga10.src.rpm
find /distrib/{bootstrap,mirror}/ -user root \! -type l -exec chown schedbot:schedbot {} +

This will move the SRPM and all its binary RPMs from Cauldron's updates_testing to release and move the older version, if one exists, from release to /var/lib/schedbot/old (a.k.a. effectively deleting it) with a user confirmation required first.

On a released distro (not Cauldron), this will move from updates_testing to updates instead. On a released distro (not Cauldron), you can also add --backport to move from backports_testing instead.

Usage: mga-move-pkg/mga-adv-move-pkg [--dry-run] [--sync] [--no-confirm] [--backport] <release>/<section>/<pkg> [[--backport] <release>/<section>/<pkg>...]
--dry-run     Just display commands and don't run them
--sync        Update hdlists after move
--no-confirm  Don't require manual confirmation for each move (only valid if full SRPM names given)
--backport    The following package is in subsection "backports" (default "updates")

To perform a bulk move operation where the SRPMs are specified with a glob (e.g. moving all Plasma packages), you can do something like this:

cd /distrib/bootstrap/distrib/cauldron/SRPMS/core/updates_testing/
mga-move-pkg --sync $(ls *package-1.2.3.4-*.src.rpm | sed s@^@cauldron/core/@)
find /distrib/{bootstrap,mirror}/ -user root \! -type l -exec chown schedbot:schedbot {} +

NOTE: the pushing updates procedure involves running update_mga-advisories without arguments which switches to user mga-advisories, which runs the command mgaadv process which starts by running mga-adv-move-pkg which in turn runs mga-move-pkg. So this procedure happens under the hood there.

Cauldron Unusual Package Moves

Sometimes packages need to be moved between other sections besides updates. However, sometime it should not be done with a move. For example, moving from updates_testing to tainted_testing should be done by deleting from updates_testing, bumping the release and rebuilding in tainted_testing. This is because "tainted" becomes part of the release name embedded in the RPM during the build. If another unusual move is needed, follow this procedure:

1. Evaluate the request. This is even more important here because it's not a usual case. Specifically, consider if it's a move in or out of tainted. TBD
2. Find the RPMs under /distrib/bootstrap/distrib/cauldron/*/media/core/updates_testing/ (or the relevant location)
3. Manually move the files, making sure all architectures, the SRPM and all subpackages are removed.
4. If packages are being removed from the release subsection, copy ALL the equivalent older version files back as replacements.
5. Run mga-hdlists cauldron core updates_testing or as relevant for ALL sections that experienced a change of packages.
6. Run this as root to fix the file ownership: find /distrib/{bootstrap,mirror}/ -user root \! -type l -exec chown schedbot:schedbot {} +

Cauldron Package Removal

This usually happens when a packager mistakenly builds the wrong or bad version of a package, or finds that it isn't quite ready for release.

1. Evaluate the request. Will deleting the package break the repository (is it a dependency of something else)? Is there a prior version still available in the repository that would be used in its place? Was this asked by the packager that built the package or the registered maintainer of the package?
2. In most cases, the requester will want to delete the latest version of a package and restore an older version. Make sure the old packages are still available under /var/lib/schedbot/old/
3. Find the RPMs under /distrib/bootstrap/distrib/cauldron/*/media/core/updates_testing/ (or the relevant location). If the RPMs to remove are indeed from updates_testing (or backports_testing), then you can use the "Raw Commands" output of mga-move-pkg --dry-run 9/core/foobar-0.1-1.mga9.src.rpm to find the RPM paths given the SRPM names (add --backport as appropriate).
4. Delete package(s) manually, making sure all architectures, the SRPM and all subpackages are removed.
5. If this is the release subsection, copy ALL the equivalent older version files back as replacements.
6. Run mga-hdlists cauldron core updates_testing or as relevant for the removed section.
7. Run this as root to fix the file ownership: find /distrib/{bootstrap,mirror}/ -user root \! -type l -exec chown schedbot:schedbot {} +

Restoring old RPMs

A move using mga-move-pkg results in the old RPMs being stored in /var/lib/schedbot/old for a time. If a request comes in to restore one of those RPMs (usually in conjunction with deleting a newer version that's not working), it can be handled like #Cauldron Unusual Package Moves, copying from that old location.