From Mageia wiki
Jump to: navigation, search

Updates advisories are managed using mgaadv and SVN. This page explains how to set up and configure a system, and create or amend an update advisory.

What are Advisories?

Any proposed package update needs to include an advisory, which is the information which will be included in the update announcement that is intended for users and system administrators, describing the issues that have been fixed by the update as well as any other steps they may need to take besides just installing the updated packages. Advisories will usually (or always, in the case of security updates) include references, which are links to web pages which provide even more information about the issues. The advisory will first identify the name of the source package(s) for which security issues are being fixed. Then it will include descriptions of the security vulnerabilities, including CVE identifiers, if applicable. Information that users or system administrators may need to know about the update follows next. Finally, the references are listed. Also, the names and versions of the updated packages need to be listed in the bug report so that the QA Team knows what needs to be tested and the sysadmins will know which packages need to be pushed as updates once the testing has completed.

The descriptions of the security issues usually follow a format that is similar to that used on the CVE website, which identifies the affected software and version range, the type of security vulnerability, and the affected portion of the software's functionality or code. The type of vulnerability can include the type of software flaw (double-free, use-after-free, buffer over-read, buffer overflow, integer overflow, null pointer dereference, divide-by-zero, reachable assertion, etc) and the impact of the issue (denial of service, information disclosure, remote code execution, privilege escalation, etc). These descriptions can be taken from the CVE entries themselves, other vendors' advisories, other vendors' bug or security tracker entries, or upstream or third-party advisories or announcements for the issues. Sometimes we have to piece the available information together and write the descriptions ourselves. In cases where very little or no information about the vulnerabilities is available, the advisory is written in a more generic manner, usually just indicating that the package has been updated to a certain newer version and that it fixes bugs and/or security issues.

The references will include links to the CVE entries on the CVE website; other vendors' advisories, especially if they are the source of information that we used to write our descriptions or fix packages, or if they include useful additional information; upstream advisories, and third-party advisories if appropriate. If we updated the software to a newer version, we also include links to the release announcements, release notes, or change logs for the new versions if possible.


To see what they look like, the current on-line list of them is at https://svnweb.mageia.org/advisories/ ; GoTo a recent page and click the Rev column to see the actual current text of individual advisories. Equally, once you have set up the advisory system on your box,

$ cd ~/mageia-advisories/advisories
$ ls
$ less <filename>

will show them locally. (The extra final ID... line is not our affair). Here is a typical security example:

type: security
subject: Updated python-yaml packages fix security vulnerability
CVE:
 - CVE-2014-9130
src:
  4:
   core:
     - python-yaml-3.10-5.1.mga4
description: |
  Jonathan Gray and Stanislaw Pitucha found an assertion failure in the way
  wrapped strings are parsed in Python-YAML, a YAML parser and emitter for
  Python. An attacker able to load specially crafted YAML input into an
  application using python-yaml could cause the application to crash.

  This issue is similar to CVE-2014-9130, but the assertion was independently
  implemented in Python-YAML.
references:
 - https://bugs.mageia.org/show_bug.cgi?id=14917
 - http://advisories.mageia.org/MGASA-2014-0508.html
 - https://www.debian.org/security/2014/dsa-3115
ID: MGASA-2015-0004

SVN access

As a prelude, install SVN.
To be able to add or edit advisories, you need to have commit access to the svn server. So you must be in the mga-qa-committers and mga-qa, or mga-packagers-committers ldap group. This is done by sysadmins. The steps shown below should be sufficient, and not require reference to Packagers_ssh.

Creating an SSH key

First of all you will need to create an RSA SSH authentication key. Install the openssh package if it is not already installed.

  • Do it as your normal local user, no need for root.
  • Give a passphrase. There is no way to recover a lost passphrase, so make sure to select one that is both strong and easily remembered by you.
  • Call the key 'mageia' rather than the default 'id_rsa'. It separates it from your regular ssh key which you might want to use elsewhere for passwordless logins etc. e.g.
$ ssh-keygen -t rsa -f .ssh/mageia

This will generate 2 files:
1) ~/.ssh/mageia, which contains the protocol version 2 RSA authentication identity of the user; the private key. This file must not be readable for other users, as it will be ignored by ssh-agent if this condition is not satisfied:

$ chmod 600 ~/.ssh/mageia

2) ~/.ssh/mageia.pub, which contains the protocol version 2 RSA public key for authentication. It is this file's contents that you upload.

Uploading your SSH key

You need to upload your public SSH key to the mageia server before you are allowed to connect to the SVN. This is done through https://identity.mageia.org/. When connected with your account, add the attribute sshPublicKey with the content of your public key ~/.ssh/mageia.pub.

Warning!
The above procedure is need to be done by sysadmins, contact with them or ask for help in qa-discuss@ml.mageia.org (you need to be subscribed)

Once your SSH key has been added, it can take up to ten minutes for the key to be active on the server.

To check that your ssh key has been correctly uploaded, try to connect with ssh to svn.mageia.org

 $ ssh <MageiaUsername>@svn.mageia.org :

If your ssh key is accepted, you should see something like the following messages :-

Enter passphrase for key '/home/<mylocaluser>/.ssh/mageia':
You tried to execute:
Sorry, you are not allowed to execute that command.
You are member of the following groups :
mga-users mga-shell_access mga-board mga-council mga-qa
mga-qa-committers mga-iso-testers
Connection to svn.mageia.org closed.

This means that your ssh key was correctly uploaded to the server.

The following command should then work :

$ svn ls svn+ssh://svn.mageia.org/svn/advisories/

It is recommended to add a config for ssh, in ~/.ssh, so that it always automatically associates the MageiaUser with svn.mageia.org. Create ~/.ssh/config with this inside it:

Host svn.mageia.org
IdentityFile /home/<localUser>/.ssh/mageia
Hostname svn.mageia.org
User lewyssmith
Compression yes
CompressionLevel 9
ServerAliveInterval 120
ForwardX11 no
ForwardAgent yes

Host *
ForwardX11 yes
ForwardAgent yes

The file should be chmod 600 <localuser> so it is only accessible to you, same as for the private keys, or ssh will complain.

You can test it with the command you did before and leave the remote username off this time. It *should* be automatic to use it for that host now. eg:

$ ssh svn.mageia.org
Enter passphrase for key '/home/mylocaluser/.ssh/mageia':
You tried to execute:
Sorry, you are not allowed to execute that command.
You are member of the following groups :
mga-users mga-shell_access mga-board mga-council mga-qa
mga-qa-committers mga-iso-testers
Connection to svn.mageia.org closed.

Advisories Installation & Configuration

Installation

First create the advisories directory in advance, to prevent mgaadv initqaconf from failing. It will need to be able to accommodate perhaps 50Mb.

mkdir ~/mageia-advisories/
mkdir ~/.mga-advisories

Or it can be put anywhere and use a symlink in the home directory pointing to the chosen location:

$ mkdir -p <path-to-where-you-want-the-advisories/>
$ ln -s <actual-path> ~/mageia-advisories

Then install the package mga-advisories and python3-yamllint:

# urpmi mga-advisories python3-yamllint

Configuration

Configuration of mgaadv is done using the command mgaadv initqaconf :

$ mgaadv initqaconf

This should open a text editor with something like this :

---
advisories_dir: /home/boklm/mageia-advisories/advisories
mode: qa
out_dir: /home/boklm/mageia-advisories/html
status_dir: /home/boklm/mageia-advisories/status

You can change this if you you want :

  • advisories_dir: the directory where advisories infos are (a checkout of the advisories svn repository)
  • out_dir: the directory where html pages will be saved
  • status_dir: directory where status of the advisories is saved

After saving the file and closing the text editor it should checkout the advisories svn in the advisories_dir directory.
If you want to edit the configuration later, the file is .mga-advisories/mga-advisories.conf in your home directory.

By default it will save advisories in the directory mageia-advisories under your home directory; typically:

$ tree -d mageia-advisories
mageia-advisories
├── advisories
├── html
└── status

It is the 'advisories' sub-directory that actually contains them (or will).

You may want to subscribe to qa-commits mailing list to see e-mails detailing commits in our repository, but this will create a lot of e-mail traffic. You will also get occasional emails when they commit mageiasync updates. https://ml.mageia.org/l/info/qa-commits

Advisory template

The guts of creating advisories is an editing template, pre-formatted and primed with key information. This is at /usr/share/mga-advisories/tmpl/newadvisory.adv, and issued thus:

type: [% type %]
[% IF type == 'security' -%]
subject: Updated [% name %] packages fix security vulnerability
CVE:
 - CVE-2014-XXXX
[% ELSE -%]
subject: Updated [% name %] packages fix [something]
[% END -%]
src:
  5:
   core:
     - [% name %]-1.0-1.mga5
description: |
  Advisory text to describe the update.
  Wrap lines at ~75 chars.
references:
 - https://bugs.mageia.org/show_bug.cgi?id=[% bugnum %]

It is worth adapting it to your personal preferences, preserving the exact formatting e.g

  • If we are currently maintaining just one Mageia release, put that after 'src:'.
  • If we are currently maintaining 2 releases, put in a 'src' stanza for each one. It is easier to delete the unused stanza (an update for just one release), than to duplicate and modify it. It also serves as a reminder about updates to two releases, e.g.
...
src:
  5:
   core:
     - [% name %]-1.0-1.mga5
  6:
   core:
     - [% name %]-1.0-1.mga6
description: |
...

Creating (or modifying) an Advisory

Note:
There is a new tool written by papoteur, mga-advisor, that can import references, SRPMs with their new NVRs and CVEs and does in more ways help to avoid typos and wrong indentation.

Overview

The process of uploading a new advisory involves first changing into ~/mageia-advisories/advisories, syncing your collection of files with the server (svn up) then creating the advisory (eg. mgaadv new security 17780 graphite2), copy/paste info from bugzilla into the editor it opens & format it to YAML indents and 75ish characters line wrap, save the file and exit the editor. That will have created a local .adv file (eg. 17780.adv) in which you then tell svn to track (svn add 17780.adv) and do a quick check with a diff (svn diff), which shows differences local compared to the server, to make sure it's the only thing different from the server and as a last check before uploading. Then if all is OK commit it, or check it in, to the server (svn ci -m "Add sec adv 5 graphite2 mga#17780").

Always start

  • Be logged into the bug in Bugzilla:

- it has all the necessary information, normally in just one Comment.
- so that you can update the bug to note the advisory..

  • Put yourself into your local advisories directory, and synchronise it with the the remote one:
$ cd ~/mageia-advisories/advisories/
$ svn up

Create a new Advisory

Creating a new advisory is done with the command mgaadv new :

$ mgaadv new [type] [bugnum] [name]
  • Replace [type] with the type of advisory (security or bugfix).
  • Replace [bugnum] with the bugzilla number of the bug used to track the update.
  • Replace [name] with the basic name of the SRPM, before -version and .src.rpm. Watch out for SRPM names which include '-'.

If >1 SRPM is involved, just cite the main one and adapt subsequent editing accordingly.

It should open a text editor with something like this :

type: bugfix
subject: Updated [package] package fixes [something]
CVE:
 - CVE-xxxx-yyyy
src:
  2:
   core:
     - something-1.0-1.mga2
  3:
   core:
     - something-1.0-1.mga3
description: |
  Advisory text to describe the update.
  Wrap lines at ~75 chars.
references:
 - https://bugs.mageia.org/show_bug.cgi?id=12345

The format of the file is YAML (see the wikipedia page for basic syntax). The format is very precise, and getting it exactly right constitutes the bulk of the work.
- Use spaces rather than tabs to indent.
- Leave a blank line, without a space in it, at the end of the advisory file. SVN needs this for some reason. If you forget, you will see a message about it in svn diff and the commit email.

You should edit the following infos :

  • subject: a one line summary of the advisory. This is already filled in. For security advisories, a common adaptation is to pluralise vulnerability/ies.

- [package] is the basic SRPM name only.
- [something] is a minimal summary of what is fixed for bugfixes.

  • CVE: if it is a security update, a list of CVE identifiers, one per line. This is often the trickiest editing job; the time to do a security advisory depends largely on the number of CVEs. They can be given in:

- The bug title.
- The Advisory description, per paragraph.
- The Advisory description as a list at the end.
- The 'references' CVE URLs, the end of these.
If there are just a few, it is easiest to pick them out individually from the description or bug title.
If the description ends with a list of CVEs, copy them en gros from there and split them up.
If there are many scattered throughout the description, the bug 'references' normally contain a contiguous list of all CVE URLs. Easiest then is to copy this list en gros, and edit each line to leave just the terminal CVE.
It is worth cross-checking the bug CVEs between the Description and References URL list, to make sure one is not omitted (count), or that a number is not inconsistent.

  • src: the list of SRPMs, in a tree of the mageia release and media as shown. This is nominally filled in, *do* correct it. Omit the '.src.rpm' suffixes. Most Advisory errors arise from the source tree not being right.

Note that:
- if the update relates to nonfree or tainted SRPMs, rather than core, adapt the 'src' tree appropriately. Rarely, an update can include both 'core' and 'tainted' SRPMs, and hence both trees under the release.
- all SRPMs include after the version the Mageia release 'mgax' which must correspond with the release tree.
- for core srpms, the name ends with the release '.mgax'.

src:
  6:
   core:
     - srpm-1.0-1.mga6

- for nonfree or tainted SRPMs the name ends with the media, which must correspond with the media tree, e.g.

src:
  5:
   nonfree:
     - srpm-1.0-1.mga5.nonfree
  6:
   tainted:
     - srpm-1.0-1.mga6.tainted
  • description: a longer description of the update. Text should be wrapped at <= 75 chars (Kwrite is the editor normally invoked, which shows the column number). Preserve the 2 leading spaces on non-blank lines (Kwrite's auto-indent helps here).
  • references: a list of URLs. The references have the bug as the first link; that is how it starts out, so just add the extra ref's below it. Do *not* add CVE URLs here, as they are added automatically in the advisory. Just copy the contiguous list of 'reference' URLs. Be careful about the final blank line if auto-indent adds spaces; check that the cursor moves immediately from the end of the last reference URL to the start of the next [last] line.

When you finished editing the advisory, simply Save the file, quit the editor. Its name is <bugnum>.adv in the advisories directory. Beware editors that automatically create a backup copy of the file [bugnum].adv~ ; these are a nuisance.

Check the syntax of the new advisory

Run the yamllint program on the new advisory file to look for basic syntax errors (like bad indentation):

yamllint -d '{extends: default, rules: {document-start: {present: false}}}' <bugnum>.adv

If your advisory was created with mga-advisories older than ver. 0.28 (which uses a slightly different, but correctly working, indentation) then it is better to use this command:

yamllint -d '{extends: default, rules: {document-start: {present: false}, indentation: {indent-sequences: consistent}}}' <bugnum>.adv --no-warnings

Add the new advisory to SVN

Then add the new file to SVN:

$ svn add <bugnum>.adv
A         <bugnum>.adv

Finish as in Final Steps below.

Modifying an existing advisory

This will already exist in .../advisories/<bugnum>.adv . Changes are purely contextual: to correct a malformed advisory, revise the text, add CVEs or references etc. Simply edit your local copy according to the rules above, then finish as follows.

Final steps

  • Review the end result, and changes, before you commit:
$ less <bugnum>.adv
$ svn diff

Use of 'less' gives a nice clean view of what the advisory looks like. It will NOT show the final blank line
'svn diff' should show (after the 1st line) entirely '+' lines for a new advisory, changed lines for an amended one. It will warn "\ No newline at end of file" if that is the case.
If you see anything wrong, edit the advisory to correct it; then repeat the commands above.

  • Once satisfied that the Advisory is correct, commit it thus (ci = check in):
$ svn ci -m '<commit message>'
Adding         xxxx.adv
Transmitting file data .
Committed revision xxx.

where 'commit message' is free format, conventionally something like:
[Add|Correct|Update] [security|bugfix] advisory [MageiaRelease(s)] [short SRPM name(s)] mga#[bugnum]

For example (assuming it's an advisory for the 'xmoto' package with bug number 12345):

[New advisory for a Mageia 5 security update]
$ svn ci -m 'Add security advisory M5 xmoto mga#12345'

[New bugfix advisory Mageia4 & Mageia5]
$ svn ci -m 'Add bugfix advisory M4/M5 xmoto mga#12345'

[Corrected security update Mageia4 advisory]
$ svn ci -m 'Correct security advisory M4 xmoto mga#12345'

[Revised advisory for Mageia4 & 5 bugfix update]
$ svn ci -m 'Update bugfix advisory M4/M5 xmoto mga#12345'
  • Once the advisory is commited, update the bug:-

- Add the word 'advisory' to the Whiteboard;
- UNtick the 'Add me to the CC list' (unless you really want to receive all subsequent bug comments & changes);
- Possibly add a Comment that the advisory is done, and from which Comment(s) it was taken. This should not be necessary in simple cases: Bugzilla automatically notes the Whiteboard change.

Other commands

$ mgaadv list

Lists all advisories, with their IDs. It takes a long time.

$ mgaadv show <advisoryID>

shows a text version of the advisory, but in a different format. It also takes a long time.

Testing the advisory

This is very optional. You can use mgaadv mksite command to generate the advisories web site to check that your new advisory is displayed correctly:

$ mgaadv mksite

Then open a web browser to view the out_dir directory defined in mgaadv configuration, conventionally /home/<localuser>/mageia-advisories/html .

Publishing advisory

This is the final step, once the commited Advisory is in place and deemed correct.
Publishing is done by assigning an ID to an advisory. In the future, it is planned that mga-advisories moves the updated packages to the updates repository automatically, but it is not ready yet. For now, this part still needs to be done by the sysadmin team.

Adding old advisories

If someone has a lot of time, it's possible to add old advisories from https://wiki.mageia.org/en/Support/Advisories/.

To add already published advisories, the process is the same, with the following differences in the advisory file :

  • set the ID manually
  • set the no_mail option so that an email is not sent for this already published advisory
  • set pubtime to the publication time of the update, as epoch value

Example :

no_mail: 1
pubtime: 1339329600
ID: MGASA-2012-0112

To know the epoch value of a date, you can use the following command (in this example for 2012-06-10) :

$ date +"%s" -d '2012-06-10 13:00'