From Mageia wiki
< User:Morgano
Revision as of 11:23, 14 January 2021 by Morgano (talk | contribs) (Swap file: more than one)
Jump to: navigation, search
this page is a draft.
It requires improvements. If you want to improve it, simply log in and click on the Edit tab.

Please remove this {{Draft}}template, when you're sure the page is complete and correct.


View the other draft pages, or other pages to improve and maintain.
Synopsis:
Notes on getting the best out of Mageia Live run from removable storage ( USB-stick / SD card / CompactFlash... ) with persistence partition which store system updates, settings, and user files - optionally encrypting it all.

PAGE REVIEW TIME !

Quickstart

This page is long - but don't fear, it is basically easy:

Download a Mageia Live ISO, plug in a modern USB stick in your computer, start Mageia IsoDumper, select the stick, select the ISO, checkmark persistence and optionally encrypt, go! Done.

The rest of this page helps you improve your experience in various ways.

You have no tool to create persistence?

If you do not already run Mageia, you do not have IsoDumper to create the persistence partition.

Then use another tool to create a normal Live of Mageia 8 or later without persistence, boot it and add the persistence partition and reboot, then optionally a swap file and/or other partitions.

Advantages of persistence

  • Store your work, logins, settings.
  • Faster easier boot - no questions of locale, time, network, etc. configured - you can surf the net, reach your cloud... quickly and you can configure autostart of i.e web browser, so just plug in, turn on...
  • Have your printer and other devices and services installed, i.e file syncing.
  • Add additional program of choice.
  • Functionality updates to programs.
  • Security updates i.e. to the system and your web browser.
  • As it no longer uses the RAM file system, it stands more chance of working on machines with low amounts of RAM. (Like a conventionally installed system.)
  • If you really need to use one of the proprietary drivers, it doesn't have to be rebuilt each time you reboot. This is particularly important if you need the Broadcom wl driver, because that often only works after a reboot.

Encryption

From Mageia 8 onwards you can select to have all persistence encrypted!

When the Live system boots it asks for a key. Also when you plug in the storage to a computer running Mageia or similar competent system you will be asked for key to be able to read or write your files. Without correct key the persistent content is not readable.

Advantages

You know everything you do is stored encrypted: network setup, your files, logins used by any programs, your documents,... (Even the added/updated program binaries...)

Drawbacks

  • Even when system install an updated kernel, it can still only boot the original.
  • In case of file system havoc (accidentally pulling it out, another system overwrites it, bad device), it requires specialists to recover content - while there exist several tools to recover non overwritten content from then non encrypted default ext4 formatted persistence.

Conclusion

Selecting to have IsoDumper make persistence encrypted is a both easy and secure solution. However, in some cases you may want to use a folder/file level encryption solution instead. An alternative is to do some Custom partitioning and have a separate encrypted partition only for some user files.

Availability

  • Mageia 8: All Live ISO works with persistence both with and without encryption.
  • Mageia 7.1: All Live ISO works with persistence. No encryption. 32 bit need fixing after updating.
  • Mageia 6.1 64 bit only, unofficial, not much tested. Persistence work but at least have the following shortcomings: 1) performs file system checks every boot due to unclean shutdown, but all writes should have been flushed, so should be OK. 2) you can install kernel updates, but the live system will always boot from the original kernel. In total it works without any tweaking for me on Dell Precision M4400 (but not Thinkpad T400) including after full update.

Alternative methods

Mageia can be installed to removable device like you would to a fixed disk.

You can spin a custom Live ISO that contains updated packages, additional programs per your taste, omit programs and languages you don't need... It may too be used with persistence per this wiki page.

Another way to make a different type Mageia Live USB.

Related pages

Release notes: Mageia 7, Mageia 8

Persistence with memory devices

IsoDumper put the ISO on removable storage and optionally the persistent partition, optionally encrypted.

You can use dd to install Live ISO to a flash drive as non persistent live, and then use Diskdrake.

Diskdrake is useful for adding persistence to an existing nonpersistent Live system, or making custom partitioning.

Why most tools including Diskdrake in Mageia 7 can not see the Live partitions and content: mga#27778#c6

Choosing storage

For the persistence, it is very important that the storage is fast at many small writes. Especially for package management, but also some programs can be affected. Helpful links:

Storage speed test results -- See if your device is there - or maybe add it.

How to test storage speed. -- Mageia have the tools packaged.

Choosing ISO

We offer four Live versions of each Mageia release: Three different preinstalled DE (Desktop Environment): Plasma, Gnome, and Xfce. All three in 64 bit, plus Xfce also in 32 bit version.

Thanks to persistence you can when the system is booted up add any DE Mageia have, so the choice is about taste, functions, compatibility, speed, and using less space.

Which Desktop

I suggest Xfce, regardless of what you are going to use later.

Because all updates above the pre-installed version take space like they were newly installed! That is because space, where any preinstalled package resides, is read-only.

So if you choose Plasma, which is big, and then there are lots of updates to Plasma, there will be a lot of wasted space occupied by the old versions in the large Plasma live ISO.

Anyway, even if you would want to normally use Plasma or Gnome, having xfce as a fallback is good, and to use on computers with USB2 / slow CPU / low amount of RAM.

Install another DE later (if you want) like Gnome and/or Plasma or another DE we support, after updating the system.

32 (i586) or 64 bit (x86_64)

64 bit runs slightly faster - use if you never need to run a 32-bit computer using this device.

32 bit is the most computer compatible. Is also slightly smaller, so saves a *tiny* bit storage space, RAM, and load time.

Recipe for updated Live

How to create an updated Live system from scratch.

  1. Choose storage device.
  2. Choose ISO.
  3. Use IsoDumper to put the ISO on the storage, checkmark persistence, and optionally encryption if Mageia 8 and later. ( Alternatively perform custom partitioning. )
  4. Boot it up. This first boot you select language etc.
  5. Configure wifi internet connection.
  6. Open a console terminal to enter the following commands:
  7. Become root user, in /root: su -
  8. Remove unneeded localisations: remove-unused-packages /!\Caution/!\ Deselect to remove drivers, and let it work. For more information on this, and possibly more things to remove, see Removing packages.
  9. Add (*) standard repositories, either from an automatically selected mirror: urpmi.addmedia --distrib --mirrorlist OR specify your favourite mirror: urpmi.addmedia --distrib http://ftp.acc.umu.se/mirror/mageia/distrib/7.1/i586 (edit URL to suit!).
  10. Update: urpmi --auto-up --auto
  11. If Mageia 7.1, run again remove-unused-packages as the updated version rinse more.
  12. If you are non-English but like to have English locale too, re-add it: urpmi locales-en autocorr-en hunspell-en hyphen-en mythes-en
  13. If you use Mageia 7 32 bit, read Updated Mageia 7 32 bit Live fail booting. You can right now before rebooting Fix Mageia 7 32 bit update boot problem.
  14. Reboot. If the menu option to reboot do not work, issue in a terminal: reboot.

After a successful reboot, do your custom settings, install other DE, other DM, useful programs. Maybe give "Live" a password, and make a guest user. Before and/or after, it may be a good idea to back up the memory folder.

(*) Keep the two default Live repos that are built into the ISO. They contain the necessary packages to install proprietary drivers and bootloaders. You can list their content using urpmq --list --media 'Live'. They are good to keep in case you need drivers to get connected. "Removing" what is in ISO stored repo does not save space anyway.

Custom Live ISO

For those interested, it is possible to spin custom/updated ISOs.

It can be much reduced in size just because you do not need to include all translations - that change alone on the Live Xfce ISO will save over 800 MB. You can omit programs you do not need, include them you do need, use another desktop, preconfigure desktop background...

Also custom ISOs support persistence.

See Building the Live ISOs, using draklive2, a mastering tool that creates an ISO image containing a Mageia live system (or 'Live ISO' for short). The image is a hybrid ISO, allowing it to be used with both optical media and a variety of flash storage.

Persistence tricks

Basics

In the persistent partition, you find the folder memory. It contains any changes from the plain Live ISO: settings, documents, system logs, changes, new programs and updates... In its subfolder /home you find your user(s) subfolder(s) and documents like on a normal install. The other subfolders contain changes to the initial system, in conventional layout, and while you can i.e reach memory/etc/ you should use the normal path /etc/ etc.

If memory is missing or renamed it will be automatically created at boot.

Altering persistence

/!\ This should not be done while persistence is in use, so either boot the Live skipping using persistence, or plug the Live storage into another system to edit it.

To back up everything (updates, settings, documents...), you can copy the memory folder. To be sure to get everything, and keep date and ownership! One way is to use the command cp with the switch -a. Example plug in Live your normal Mageia system, mount the mgalive-persist partition, and in a console enter su -, cp -a run/media/Your_Computer_User/mgalive-persist/memory/ /where/you/want/it/ Adapt links as suitable - the "run/media/Your_Computer_User/" part is from using Dolphin to mount it.

Maybe you want to take a backup after you have first configured and updated the storage, before adding your personal choice of programs and documents.

Moving, duplicating

Everything

Any Mageia Live system based on the exactly same Live ISO file can exchange the whole memory folder.

So you can copy, replace, sync memory between such devices. (Just remember the systems should not be running, and copy everything and keep ownership, time, etc.) Note that you may want to somehow omit/remove any large swap file if you have such - to save space - and recreate it on the target.

This facilitates moving to larger, smaller, different type storage devices, make preconfigured Lives to family and friends, and with custom partitioning you can add another partition, choose another file system, choose encryption or not...

User files

Between Live systems based on different ISOs, you can safely only move memory/home. (Normal procedures of eventually different user configuration applies.)

Swap file

Even just a few hundred megabytes of swap makes big difference on systems with small RAM. Also, simply having swap available enables kernel to work a bit more efficiently.

Adding swap in the form of a file instead of a partition have advantages:

  • It can easily be implemented without repartitioning.
  • You can more easily remove it if you need the space - while running. And easily recreate a different size.
  • As Live can not boot with LVM, this is the way to have swap encrypted without an extra key entry.

Recipe, to execute on the running Live

1. Create a swap file - in this example 1GiB (1MiB 1ki times): as root: dd if=/dev/zero of=/swapfile bs=1M count=1k, mkswap /swapfile, chmod 0600 /swapfile.

2. Kernel can not use swap on loop device, so we must let fstab mount the persistence partition, and then the swap file in it. So create a mount point for the persistence partition, as root: mkdir /mnt/mgalive-persist.

/swapfile is in the persistence partitions filesystem addressed as /memory/swapfile. So full path will be /mnt/mgalive-persist/memory/swapfile.

UUID is the most reliable way to identify partitions. To see the UUID of your persistence partition, issue: blkid | grep "mgalive-persist".

3. Based on above, add two lines to /etc/fstab (Use your specific UUID, and also adjust "ext4" if you have chosen another file stystem type!):

UUID=a0406603-49e3-4908-820d-65736dd55f24 /mnt/mgalive-persist ext4 defaults 0 0
/mnt/mgalive-persist/memory/swapfile none swap sw 0 0

4. Reboot.

5. Example commands to verify it works, no need to be root: swapon, free -h, grep -i swap /proc/meminfo.


Remove / change size

Remove it: as root swapoff -a, rm /swapfile.

Create a new per step 1, any size you like. If you want to use it right away, as root swapon -av activates all configured and available swap.

If you want to wait with creating swap again, you can leave fstab as is; it will boot without swap file with no delay.

You may have more than one swap file, so you can quickly reduce total swap whenever needed without removing all swap: Create them like in step 1 but use a different name (i.e "/swapfile2") and add one line to fstab similar to last one in step 3. To activate/deactivate one of them any time issue swapon/swapoff with full path, i.e: swapon -av /mnt/mgalive-persist/memory/swapfile2. swapon lists all active swap including path, size and current size in use.


( work in progress )

So far the above is only tested on default Mageia Live persistent filesystem type ext4, and without encryption. To do:

  • Hibernation.
  • For encrypted ext4.
  • For F2FS and btrfs (may need to set special file type) with encryption and plain.

Custom partitioning

Very seldom needed, but for special needs, you can configure partitions manually in many ways. Here is the game plan:

  • Mageia Live automatically use a partition labelled mgalive-persist as storage. It can be of any supported file system type - note that Mageia 8 support F2FS, but not Mageia 7.1. When created by IsoDumper, file system is ext4.
  • You can create any other partition you like, i.e a swap partition, and/or a partition formatted suitable for sharing space when the device is plugged into another operating system.
  • CAUTION: Make sure you do not set a UUID or partition label identical to something else on the running system, or any system it will be attached too or booted on, as that may confuse mounting and use of filesystems.
    • Exception: Always name persistent partition "mgalive-persist"; Live only looks for "mgalive-persist" partition on the device it booted from.
  • For any LUKS encrypted partition, you will at boot be asked for the key for each. mgalive-persist can be encrypted.
  • LVM is not supported.

Example:

One partition formatted to share content with other system types, persistence with optional encryption, optional swap.

  1. Choose ISO and storage device.
  2. Write the live ISO to the device using IsoDumper. Do not enable persistence.
  3. If you are not already, make sure you are booted on Mageia 8 (Mageia 8 Live is OK) for next step; (*1)
  4. Start Diskdrake: (*2) to create the persistence partition;
    • Size: Leave empty space at end of the storage for any swap or partition to share files (see last steps below).
    • Type: ext4 or F2FS. (*3)
    • Optional: select to encrypt, input (and memorise!) encryption key.
    • Important: Set the label to mgalive-persist.
  5. Exit Diskdrake. (*4)
  6. Now boot on the new Live system; this first time you choose language, accept license, select keyboard...
  7. Remove unneeded localisation (this also is to verify if your storage is fast), see Recipe for updated Live.
  8. Configure network, configure repositories, update the system.
  9. Reboot - it should now boot up to desktop no questions asked (except optional encryption key)
  10. Launch Diskdrake, and, for what you reserved space:
    • Create a swap partition. (*5) Note: Unless you want to hibernate, you may prefer to add a swap file instead.
    • Create an NTFS or FAT32 partition for sharing with some systems. Format it and label it suitably, i.e FAT32-share. Select a mount point, i.e /media/FAT32-share (*6)
    • Exit Diskdrake, letting it save changes to fstab.
A booted Mageia 8 Live screenshot of Diskdrake and fdisk -l showing custom partitioning: a FAT32 partition in addition to the encrypted ext4 mgalive-persist.

(*1) About the version of the running system

  • Early support limitation: Max total four partitions can be created by Diskdrake in Mageia 8 beta 2; drakxtools-18.37.
  • If you use another Live as tool, do NOT use one created from the same ISO as your target Custom Live - see Caution. So i.e boot on Live Plasma if you base your Custom system on Live Xfce.
  • If target is a Live Mageia 8 or 32 bit Mageia 7.1: you can instead boot the Live system and start Diskdrake in it to perform the actions on itself, with some limitations. If so create the persistence, reboot, and then create the other partitions similarly to rest of this instruction. But it is more reliable to follow the suggested workflow per the numbered steps.

(*2) About partitioning

  • The suggested order of partitions makes it easier to extend the persistence partition if needed.
  • Use expert mode in Diskdrake (button down right) to be able to access more functions and file system choices like F2FS.
  • /!\ In this step if Diskdrake ask say no to save changes to fstab, as that may render your running system unbootable!

(*3) F2FS is optimised for flash storage so works a bit faster and with less wear. If you instead prefer to better be able to access the files when plugged into another unix system, or retrieve files if the filesystem breaks, more systems and tools support ext4. Both use journal for resilience. The many options to tweak them, and alternative filesystems are out of scope of this document. You may need to install f2fs-tools if you want F2FS.

(*4) Do not exit if Live is Mageia 7.1 64 bit, because there Diskdrake can not work on its own partitions. Instead, use the running Diskdrake for optionally creating swap and the sharing partition. Do NOT let it save fstab (as it would be on the runnign system). You later need to edit fstab manually on the booted target system to use these partitions, an update grub config for hibernation.

(*5) Swap is useful on computers with little RAM, or if you want to hibernate your live system. If you encrypt anything else it is good to enrypt swap too. You will at boot be asked for key for each encrypted partition, so a tip is to use same keyword. Needless to say, you should use swap only on a fast and large storage device. You may want to set low swappiness, which is out of scope for this article - do a web search.

(*6) default fstab options are OK: default,nofail,umask=000 0 0 (any user, skip if it have problem (accidentally deleted or whatever)).

Note

  • Never change anything concerning mgalive-persist while it is in use.

Plugging it into other systems

  • There seem to be no problem using the added Microsoft style partitions.
  • Like Mageia, also other systems let users easily access the ESP partition, but don't alter things there!
  • /!\ Warning Some Microsoft Windows (R) versions pop up a dialog asking to format a disk, the "mgalive-persist", "to make it usable" - Don't!...

Alternative tools

You can also use dd, fdisk and mkfs. The usage is out of scope for this article, but an example is here. Be very careful as it i easy to make any mistake and destroy your running system!

Install your Live

You can install a copy of the Live system, with or without your changes, to a computer fixed storage or to another removable storage. You have full control over partitioning like if you used classic installer. If you want to remove or add packages, you do that when booted in the new system.

See Install to removable device, Live installer.

Boot options

The boot menu main items are described here.

Below we explain some more advanced options.

Also see how to set up kernel options.

Original kernel

If [F1] is shown, you can use it to toggle between booting the original kernel in the Live iso, or the updated one in persistence. (The text after [F1] toggles.)

Boot command line

You can change the kernel boot command line: Press e, use arrow keys to navigate into the line beginning with "$linux". It is a long line so usually two or more rows of text.

General examples

Remove the words splash quiet to see text output how booting proceeds.

For other options see How to set up kernel options. Some may help to solve graphics or other hardware problems.

Skip using persistence

How: Add mgalive.overlay=ram to the boot command line. This is a Mageia Live special, supported from Mageia 8 beta 2 onwards.

When: Maybe some update made something not working on some computer so you want to try this Mageia Live ISO like it was before any updates. Or you want to backup or restore the persistence memory folder, or alter the mgalive-persist partition. Or you want to install to computer or another storage using Live installer without any changes you have made.

Even when booting without using persistence, the partition is available to mount so you can reach your files.

Tips

Running Live on weak computers

  • Using persistence save on RAM compared to using a ramdisk which otherwise is used.
  • Use a lightweight desktop such as Xfce, or one of our even lighter desktops. Note that you can have several desktops installed and choose at each login which to use.
  • Use light weight programs. i.e Falkon for web browsing, and Gwenview for browsing-cropping-resizing photos. (Even Xfce by default use the heavy Firefox and Gimp, respectively)
  • Have a swap to free some RAM - add a swap file or partition: Custom partitioning. A few hundred MB is enough unless you want to hibernate.

Caution

Same ISO -> Identical UUIDs and labels

Naturally. It have complications:"

Do not use two USB sticks created from the same ISO file at the same time, as they will have identical root volume labels and UUIDs, and GRUB and the Linux kernel won't know which one to use.

Known problems

Reformatting (reusing) a Live system

Live ISOs are (hybrid) ISOs.

The ISO filesystem signature confuse many tools when you want to reformat a storage you have used for Live.

What works:

  • IsoDumper can format one partition, and then other tools can work if you need more.
  • Updated Diskdrake / Mageia 8 and later installer when you select "Clear all" to or use whole drive.
  • Command line tools, i.e use wipefs to wipe the iso9660 partition.

Gnome on Mageia 8

Graphics and network: see Mageia 8 Errata.

Updated Mageia 7 32 bit live fail booting

Fully updating a 32 bit Live persistent installation will make it fail booting. mga#27638

Workaround to boot: Use original kernel.

Booted up (or before rebooting after updating), you can perform the fixes below.

Fix Mageia 7 32 bit update boot problem

  1. Boot up with persistence working (by workaround above or before reboot after update)
  2. Open a console terminal to enter the following commands
  3. Become root user: su -
  4. Get the updated file: wget https://bugs.mageia.org/attachment.cgi?id=12009 -O mgalive-root.sh
  5. Make it executable: chmod +x mgalive-root.sh
  6. Put it in place: mv mgalive-root.sh /usr/lib/dracut/modules.d/90mgalive/ - confirm overwriting
  7. Before executing it, update the following line according to the latest kernel installed!
    dracut -f /boot/initrd-5.7.19-desktop586-3.mga7.img 5.7.19-desktop586-3.mga7

Timeout removing unused packages

On a slow device and ISO prior to Mageia 8 beta 2: mga#27580 urpmi killed by timeout while removing localisations - rpm database get irreparably broken.

Workarounds: Either of

  • Do not use a slow storage - if you hit this problem you would want to upgrade the storage anyway...
  • Update to the fixed drakxtools before using remove-unused-packages so it do not time out.
  • Do not use remove-unused-packages until after full update, or remove localisations manually a few at a time (tedious)!

To do in this document

Discussion and Documenting

Let Martin and QA review facts, let docteam review form.

Goal: This page is to fill the need in https://wiki.mageia.org/en/Docteam_Todo_list#Else_to_do_5 , point 4: persistent live ISO, pointing to forum topic [1], extending to [2], [3]

Note this is four closely related pages: When all content is revised, copy contents to make new final wiki pages - all four pages at once!

Use the following names:

I believe all internal links are set for that already.


- At that point also check the other points in Update below, if we can update something first.

When this page is moved and official

Update

Watch and verify when progressing

Fixed in Git

  • mga#27864 - Mageia 8 beta 2 diskdrake and installer stumble on MBR of our own Mageia Live - Fixed in git after beta 2, to be OK in next ISOs
  • mga#27862 - Diskdrake fail to create more than four partitions on a Live storage. - Fixed in git after beta 2, to be OK after next drakxtools update

Testing Done

(Plus a lot not documented...)

Custom partitioning

  • The solution with added FAT32 and swap works great, but Diskdrake have multiple issues depending on workflow

Mga 8b2

  • X86_64 encr pers OK: Xfce, Plasma, GNOME
  • i586 Test OK Xfce i586 Live with encrypted persistence on Thinkpad T43 and T400, Dell Precision M6300 and Dimension M4400. (round 1 was failing with any persistence, round 2 OK)
  • X86_64 on sloooow storage: timeout test removing locales: OK fixed.

Mga 8b1

  • Mga8b1 x86_64 xfce with and without encryption: reboot OK Thinkpad T400
  • Mga8b1 i586 persistence FAIL BOOT Thinkpad T43 - both encrypted and plain.

Mga7.1

  • x86_64:Updates without reboot problem OK (fast storage) Thinkpad T400
  • i586 reboot fail, workaround fix documented above.
  • On slow storage: remove-unused-packages get timeout 10min -> rpm database crash! Proactive workaround documented above.

Mga6.1

  • Mga6.1 x86_64 Xfce persistence: OK - including full update on Dell precision M4400. (Fail booting on Thinkpad T400 - probably something incompatible - fails before persistence is used.)
  • Mga6.1 i586 Live fail booting on Thinkpad T43 and T400, different failure on Dell Precision M6300 and Dimension M4400.

Testing To do

See also chapter Update

Solve, Describe

  • Problem appear when you plug in a persistent live into a running system and want to read or write user files: users that have changed their UID and GID from default (Like I always do, so family members all have different) can not access it (Live user UID is 1000)

Author

  • Speedier boot on same machine: Hibernate using swap, Skip hardware detection? systemctl mask mandriva-everytime.service

Procedure verifications

  • When Mageia 8 RC is out do custom partitioning per procedure, many partitions a) from another Mga8 Live; b) on a Live itself.
  • Test custom partitioning per example interactively editing. i.e Diskdrake really works? (see Update above re mga7 Diskdrake not working for all)
    • workaround mga7 diskdrake for mga8 Live, and can 7.1 Live manage adding partitions on booted media (boot without persistence)? Only works when booted with persistence?? Other quirks - add one part at a time. Better do from a booted mga8!
    • swap file: Can it be made to hibernate to it?
  • Test memory backup to computer instruction and restore to new storage per example interactively editing
  • Custom partitioning: verify the FAT32 works with XP, W7 OK, (W10 OK), and some device. Do some system want to "fix" (destroy) any filesystem? Yes old W10 but not updated ?!
  • Test SD card slot in Laptops that have such - can it boot there (BIOS issue) or do it need USB adapter?
  • Align persistent partition and other partition(s) to erase blocks! Stripe, alignment, Ext2: use 4 instead w journal turned off, http://blog.mcnalu.net/faster-usb-sticks.html, Erase block: https://forums.linuxmint.com/viewtopic.php?t=275082 https://en.wikipedia.org/wiki/Flash_memory
  • KDE partitionmanager support ExFAT ? mga7 updated, Mga 8. Other file systems?
    • Do IsoDumper align partition to *erase* blocks? Improve: assume worst (big blocks?)

"Installing to disk" from desktop

  • Is it successful to install to other removable storage? Stumble on swap file?