From Mageia wiki
Revision as of 12:36, 30 October 2017 by Ngompa (talk | contribs) (Identify which architecture and Mageia release you use: Update table to mga5/mga6)
Jump to: navigation, search


Drakconf multiflag.png
Other languages
English ; Français ;


What is an update

An update is a new package. It can be a security update, a bug fix of an existing package, the addition of a missing feature or a new package, not available earlier.


Before it is validated by the QA team it is referred to as an Update Candidate as it is just a potential candidate for update until it has been through the QA process and been approved for pushing to our users as an official Mageia update.


The process of the QA team testing the update candidate, feeding back any issues to the packager, retesting when the packager builds new packages to fix any issues they want to include and generally making sure update candidates are OK before being accepted as an official Mageia update is refered to as validation.


Where to find update candidates

To prevent regressions, update candidates are pushed to a testing repository waiting for the QA Team to validate them. A corresponding update request on Mageia's Bugzilla is used to coordinate the validation process.


An update request looks like any other bug report except it is asking for the QA team to test update candidates and it will be assigned to us. It is not unusual for an existing bug report to become an update request, where the update candidate has been built to fix the issue being reported there.


You can find update requests of packages waiting for validation here.


How to validate an update

If you have packaged the update candidate then you may now test on one arch to help to speed things up but you must give a detailed testing procedure. Validating the update must be left for a QA team member.


Identify which architecture and Mageia release you use

In order to correctly categorize your QA work, it is necessary to know which architecture and Mageia release you use. To find out, enter

$ cat /etc/release 

in a terminal or open the file /etc/release in any text editor. Then compare the output to the following table:


Content of /etc/release Architecture Release (Version) Whiteboard Keyword
Mageia release 5 (Official) for i586 i586 5 MGA5-32-OK
Mageia release 5 (Official) for x86_64 x86_64 5 MGA5-64-OK
Mageia release 6 (Official) for i586 i586 6 MGA6-32-OK
Mageia release 6 (Official) for x86_64 x86_64 6 MGA6-64-OK




Warn other testers so we can avoid duplicating work

In order not to do the same job as other people from QA, duplicating the work, add a message to the update request on Bugzilla when starting work on it. Just something simple to let others know you are testing it.

An example of the type of message to leave:

Testing i586, MGA4.

Note: "i586" (Architecture) and "MGA4" (Release) must reflect what's present on your system.


We don't always do this step as there are usually more bugs than people so we don't tend to bump into one another but it can be used if the testing is likely to take some time or if it is something popular and easy to test.

On large, important or complex packages such as the Kernel, KDE, Firefox, etc. it can be useful to have many people testing simultaneously. That speeds up the QA process and also ensures it is more thorough. It can still be useful to let people know you are also testing the update. It is never a bad thing if more than one person tests the same thing though.


Test the update candidate

Preparation

  1. Boot the correct installation and install all the officially released updates. Check the “Product”, “Version” and “Platform” in the header of the bug report to find what it should be, e.g. Mageia 4 x86_64. Also note that some bugs may contain updates for both releases so also check the advisory from the packager. You should have all Testing medias disabled at this stage.

    The advisory is a description of why the package is being updated and what has been done to it. It is there so that we know what to test for. It is later uploaded to SVN by somebody who has access and made public on mageia's webpage, as well as on the updates-announce mailing list when the update candidates are pushed as an update.

    It should also list the RPMs being updated as well as their SRPMs.

  2. Try to reproduce the bug before installing the updated packages from the testing repository. This is for thoroughness as it proves you are able to verify the bug is fixed after the update. It also shows you what to expect if it isn't.
    If you are unable to reproduce the bug, without suitable hardware for instance, you should still carry out testing of the update to ensure there have been no new bugs introduced. You can also leave a message asking if somebody else is able to fully test, for example, somebody with the correct hardware. Common sense applies here. It can be useful to send an email to the discuss or dev mailing list asking for people with the correct hardware to assist with testing.

    If you find the tag "has_procedure" in the Whiteboard field of the bug report, then there is a procedure available to assist you - either a recipe for testing basic functionality of the package or (mostly with security fixes) a way to replicate or provoke the issue. This can be anything, e.g. a 'bad' input file or a patch.


Installing the Update Candidate

  1. Enable the relevant testing repository in your sources. See the Enabling the Testing media page for instructions how to do this.

  2. Install the relevant updates, preferably with MageiaUpdate. Don't install everything from the Testing media, just the ones you are about to test. Ensure you select all the related packages, including any required libs. (Tip: check the version numbers) If you install the update via urpmi, don't forget to install the updated libs too.

  3. Disable the testing repository again

  4. Make sure you now have the update candidate installed. See How to find a source RPM#Search among installed packages by name for instructions.


Testing

  1. If this is a security update you will see the CVE numbers listed in the advisory, they should also be in the bug summary at the top of the page and packagers often remember to set the Component to Security too. Attempt to find a Proof of Concept (PoC) online if one is not listed on the bug advisory. There are various websites dedicated to security. A popular one is securityfocus.com. It allows you to search using the CVE number and often gives useful information or Proof of Concept code which can be used to show the vulnerability is indeed fixed. If none is available then just test for any regressions and state in your comment that you couldn't find any PoC's to save others from having to search. Carry out normal testing (look out for regressions).

  2. If it is a bugfix update, check that the bug is fixed by following the process hopefully provided in the update request. You usually need to do some research for yourself, but nobody can be expected to learn every package. The QA Tips and Tricks page may give you some ideas how to test the package(s). You can also find testing procedures by looking in our testing procedures category or add them if they are not already there. If you have investigated the matter but still do not know how to test it then you can leave a message asking for more information on how to do so.

  3. Do some quick tests to check that the update does not break anything else. For example, test desktop effects or DRI still work as expected if the update is about Xorg.


Reporting

  1. If your testing found a problem then feedback your findings by leaving a comment on the bug; and add 'feedback' to the Whiteboard: this will cause the update to be greyed in the list. The packager may request further information from you or need further testing to track down the problem so be prepared to follow it up. Include the release and architecture you were testing with.
    For example..

    Testing Mageia 4 i586.
    I found a problem...etc.


    Note: "Mageia 4" (Release) and "i586" (Architecture) must reflect what's present on your system.

  2. If everything is OK then leave a comment to say that you have completed testing and add a keyword to the Whiteboard. The Whiteboard keywords help us keep track of when a bug is ready to be validated.
    For example..

    Comment: Testing complete Mageia 4 i586 for the srpm php-5.3.13-1.1.mga2.src.rpm
    Whiteboard: MGA4-32-OK

    Note: "Mageia 4" (Release), "i586" (Architecture) and "MGA4-32-OK" (Whiteboard Keyword) must reflect what's present on your system.

  3. Check if the update candidate is ready for validation Validate it if it is (see below)


Note:
  • MageiaUpdate will only see update candidates if the correct Testing media is enabled and selected as an update media, check this with edit-urpm-sources.pl or in Mageia Control Centre. Refer to the Enabling the Testing media page for further information.
  • You may also need to update the media with urpmi.update "media name". For example urpmi.update "Core Updates Testing"
  • Never set "Release" media as update media when using edit-urpm-sources.pl, only "Updates" media.




Validate the update if it is ready

All the actions described here relate to the Bugzilla entry.

  1. Check if the update candidate is ready for validation Ideally it is only ready for validating when it has been successfully tested on both architectures on all releases it covers. As some bugs contain updates for more than one Mageia release, each must be completed for both i586 and x86_64 before it can be validated. However, when QA is hard pressed, and the update is not major (like kernels, or a widespread application), an OK for just one architecture is permitted. You can most easily see this on madb where it will show a green circle next to each Mageia Version when it is ready to go.
  2. Add a validation Comment to let people know the update has been properly tested on both architectures and, if necessary, all releases.
  3. If necessary, add a list of the Source RPM's (SRPM's). Nowadays, this is usually already given in the bug with the packages involved. Also, the bug header field RPM Package may show it. For further information on finding the SRPM for a package, see the How to find a source RPM page.
  4. Cite the advisory Either copy/paste it, or (more likely) say which Comment it can be found in.
  5. Add the validated_update keyword into the Keywords text box. This will automatically add sysadmin-bugs@ml.mageia.org into the 'CC' list so that the SysAdmin Team are aware it is ready to be pushed.
  6. Click on Save
Note:
You will normally see that somebody mentions that the advisory has been uploaded. You need not worry about this. Team leaders and people with relevant permissions are now uploading the advisories as yaml text files to svn where they are then used, when the update is pushed, to update the advisories website and updates-announce mailing list.

They are uploaded before sysadmin push the update and, once completed, denoted by a star (*) in the first field on madb and the 'advisory' tag in the whiteboard field of the update request on bugzilla.

Only some QA members can upload the advisories so if you see that one is needed you can also leave a comment to say so, to ensure that it isn't forgotten.

Please DO NOT add the advisory tag to the whiteboard entry if the advisory has not been added to svn. If you're not clear on what adding the advisory to svn means, then do not add the whiteboard tag!




What to do if...

...the update request is not following the updates policy

If the update request is missing too much information like the advisory, SRPM or steps to reproduce a bug then leave a message asking the packager to follow the updates policy.

For example:

Please follow the updates policy and provide an advisory with your update request.
https://wiki.mageia.org/en/Updates_policy#Maintainer_.28or_any_interested_packager.29
Please reassign to QA when you have had a chance to look at this.
Thank you.


Then change the status to ASSIGNED and reassign the bug to the person who asked for QA to validate it.


...the update candidate does not fix the issue or creates another one

Simply reply in the update request by giving all the information about the bug or regression and which release and architecture you were testing on. Be prepared to follow up with further testing or information as requested.

If there is no response after a week has passed, reassign the bug back to the packager and change the status to ASSIGNED. Also add qa-bugs@ml.mageia.org as a CC contact on the bug so that we can respond to the packager when they catch up with it.


...you need help or guidance or are unsure of anything whatsoever

Please just ask! The only silly question is the one we never ask because we feel it is a silly question. You can either ask on IRC in #mageia-qa on irc.freenode.net or on the qa-discuss mailing list. If there is nobody available in #mageia-qa then you can always try in #mageia-dev too.


More information

Please refer to the QA Team portal for further information on the Mageia QA Team.


  • Mageia's full updates policy.
  • The QA Tips and Tricks page should provide you with some inspiration when performing testing.
  • QA-Discuss mailing list: qa-discuss@ml.mageia.org
  • QA-Bugs mailing list: qa-bugs@ml.mageia.org
  • You can also use Mageia App Db to see pending update candidates and perform various useful comparisons.
  • Sophie is a bot on IRC providing much useful information on Mageia packages.
    Try typing :help in #mageia-qa or :more packagename