If you just joined the team, what happens next?
Firstly - Welcome to the team!
If you see gaps in the documentation where we can make improvements, that's fantastic so do say so. It has been written by over-burdened, battle-hardened, tired and decrepit QA testers who may have missed out something which seems obvious to them but may not be so obvious to you!
If you've never contributed to an open source project before you may feel as though you should wait to be issued work to do. The opposite is true though, so please dig in and get started! Do what you can and don't be too worried about making a mistake.
We're generally relaxed and friendly in the QA team but always busy! The QA team is responsible for testing two main things. Our daily task is to test all package updates to stable Mageia releases before they are pushed as updates for everybody to install. We call this process validation or validating updates. Stable releases are all Mageia versions other than Cauldron or any which have reached their end of life and are no longer supported. We also test all new ISOs before they are released to the mirrors.
There is a lot to take in when you first begin testing but please don't feel intimidated by anything. People are all different and have different levels of experience and understanding, that is normal. We have always actively encouraged people to ask questions, however silly you may think they are. We don't believe there is such a thing as a silly question, so please don't hesitate to ask about anything you need help or guidance with.
Please try to attend the weekly team meetings which take place on IRC in #mageia-qa on irc.freenode.net every Thursday
When validating updates we follow the QA process for validating updates. It is a step-by-step process which will lead you through all the steps necessary to find the update candidates awaiting your attention, configure your medias, use bugzilla, perform the testing and document your findings.
Of course, each package being updated is different. It is our responsibility to work out how best to test them all. Luckily though we have tested many packages before and bugzilla can be searched to see how something was tested the last time it was updated. This search is made easy by clicking on the 'Bugzilla' link on the right-hand side of the madb list page. We also began documenting some of these procedures here on the wiki, although we've not made much progress in doing so. You can search those pages by clicking the 'Wiki' link on that same madb list page. There are also some tips to help you along the way.
Don't worry if you are confused already! It can seem intimidating at first, please don't be though. We've all started somewhere and all need to ask for help when we get stuck. Nobody could possibly know everything. We can help you to get started and are also here to answer your questions.
It is a good idea to try and have a Mageia installation for each of the stable releases in each architecture (i586/x86_64). They can be in virtual machines (eg. VirtualBox) or on real hardware. Most updates can be tested perfectly well in a VM. There are a few which do need a real hardware installation though, such as new kernels.
Our second responsibility is to test new ISOs before they are released. Alpha/Beta/RC ISO releases occur roughly once per month during the build-up towards the next stable Mageia release.
We have started to create a wiki page for this too so you can find a good basic explanation there of how we collaborate on this and what to expect.
Pre-release ISOs are held on a development server, with limited bandwidth, so we have to limit access to it to those who are actually taking part in the pre-release testing. When ISOs are subsequently released they are pushed onto the public mirrors and are then available to anybody for wider testing.
ISOs are only released to the public after they have all been tested and judged to be suitably OK. The level of OK depends on how far we are along the development path. For early Alpha releases our expectations are that they will boot. In later Beta releases, we expect them to be a lot nearer completion and the RC should be close to bug-free.
It's a good idea to begin with a complete set of the latest ISOs, you can download these from your favorite mirror beforehand. You will find the latest alpha/beta ISOs in the Cauldron directory. They can then be renamed appropriately and used as a basis to begin the synchronization process later, which helps to speed things up.
We use rsync to synchronize the ISOs to our local machine, which only downloads parts which have changed. This means that when there are rebuilds, and there are usually plenty of rebuilds, the synchronization process is normally quite quick. We have tools which make synchronizing the ISOs simple, ISO testing rsync tools. They also perform md5sum and sha1sum checks on the ISOs and offers to dump one onto a USB stick for you.
We make use of a collaborative pad, which everybody can edit at once, to keep notes of what we are testing and the results of our tests. It's important that we also create bug reports on bugzilla when we find an issue too, we then note the bug numbers on the pad along with our notes. The pad is just for our benefit and not generally read by packagers or devs.
If, during our tests, we find a severe issue which is likely to block that particular release then we also email qa-discuss about it, CC'ing the person who built the ISO and generally make some noise about it.
Martin (martinw) currently builds the classical installer based ISOs and the live ISOs. He emails qa-discuss when he has built the ISOs with details and a link to the pad so we can start testing them.
Rsync login details are sent via private email to all those who have added their details to the list of QA ISO testers along with details of how to easily rename previous ISOs before you begin. The login details can be added at the top of the dorsync script.
Don't worry, we all know what it's like when you first get started. There is a lot to take in and it is easy to become overloaded with information and start to feel intimidated. Please do not be intimidated by anything you encounter, we are here to help you :)
It's worth restating that in the QA team we do not believe in silly questions. There is no such thing. Only ones we didn't ask out of fear we would look silly. Please don't hesitate to ask if there is anything at all you need help or guidance with. You won't be judged and won't look silly!
The qa-discuss mailing list is always available to discuss QA matters and ask for any help. There is also the IRC channel, #mageia-qa on irc.freenode.net, where you can often get instant assistance and chat with other members of the team. It is useful to collaborate and compare notes!