• How does a Clonezilla USB boot? - Was: One Rescue USB does it all ...

    From Java Jive@21:1/5 to Java Jive on Sat Jul 6 19:42:37 2024
    XPost: alt.os.linux, alt.windows7.general

    On 01/07/2024 13:25, Java Jive wrote:

    On 24/06/2024 11:01, Java Jive wrote:

    .... still can't get a bootable result.  Any further help with
    this would be welcome.

    Given sector-by-sector images of ...

        W98 console mode emergency boot floppy (RawWriteWin)
        W98 console mode emergency boot USB - entire disk (dd if=/dev/sdx)
        W98 console mode emergency boot USB - partition (dd if=/dev/sdx1)

    .... I've now obtained the results of tests in the following table  -
    which probably need a fixed font to display properly  -  details notes
    are appended ...

                      Floppy  Disk  Partition
    Ubuntu 22 mount:    Y      N       Y   (USB reversed compared to rest)
    Ubuntu 22 qemu:     Y      Y       N Ventoy:             N      N       N Clonezilla:         Y      Y       N
    Grub2 - loop:       N      N       N
    Grub2 - memdisk:    Y      N*      N

    *  This was particularly disappointing, because this, or the partition equivalent, is what I'd most like to achieve as the alternatives that do
    work are more convoluted, and given the previous patterns of results,
    once the floppy image worked in this arrangement, I expected the hard
    disk image to work as well, but sadly it didn't :-(

    If anyone can suggest a way of getting either the disk or partition
    image to boot directly with Grub2, I'd be very grateful.

    [snip]

    Next, CloneZilla USB stick, with SysLinux.cfg entries as follows:

    Floppy.img

        label EmergencyW98
        # MENU DEFAULT
        # MENU HIDE
        MENU LABEL Emergency W98 Boot Disk (Floppy)
        # MENU PASSWD
        kernel memdisk
        append initrd=/live/Floppy.img
        TEXT HELP
        Boot Windows '98 emergency console
        ENDTEXT

    Boots!


    USB-Disk.img

        label EmerDellGhostImg
        # MENU DEFAULT
        # MENU HIDE
        MENU LABEL Emergency W98 Boot - Dell Diags & Legacy Ghost (disk)
        # MENU PASSWD
        kernel memdisk
        append initrd=/live/USB-Disk.img
        TEXT HELP
        Boot Windows '98 console with Dell Diagnostics
        for Latitude D610 & Precision M6300 & Ghost (2003)
        ENDTEXT

    Boots!

    I confess I'm struggling somewhat with all this ...

    For one thing, I'm finding it difficult to know if my image of the W98 Emergency USB is valid or not. I have an old one, and a new one, as
    follows ...

    The original USB made several years ago booted fine. As still quoted
    above, the old image made from it boots from a Clonezilla USB by adding
    it to its SysLinux menu, which suggests that there's nothing wrong with
    it, but it won't boot anywhere else that I've tried.

    However, when I launched GPartEd with the USB still plugged in
    accidentally, I noticed messages concerning the partition being 'bigger'
    than the actual disk, and thought: "That's why the [expletive deleted]
    image won't boot, there's something wrong with its layout!"

    So I remade the entire USB. I copied off the files, and using GPartEd
    itself deleted the single partition, made a new single FAT32 partition occupying the available space, copied the files back in the mandatory
    (at one time, but perhaps not now) order - IO.SYS, MSDOS.SYS, the rest
    - booted from my one remaining working W98 floppy and did a SYS to
    copy the PBR and ensure the system files were present in the correct
    order in the directory, and ensured that the remade disk could boot,
    which it could. I then remade the image from the remade USB, the image referred to above as 'new'.

    However, this new image NO LONGER boots from the same Clonezilla USB as
    did the old! Copy the old, supposedly flawed (according to GPartEd),
    image back overwriting it, and that one boots! WTF???!!!

    But neither image will boot anyhow or anywhere else. The source USB
    boots fine, has done for years and still does after being remade, and
    the old supposedly flawed image boots via Clonezilla, but neither will
    boot by any other method I've tried. By contrast, now that I've
    discovered how to do it, a RawWriteWin image of an Emergency W98 boot
    floppy boots absolutely everywhere that I've tried it, no trouble at all.

    So, putting aside for the moment the problem of trying to directly load
    the emergency W98 USB image from Grub2, given that the only practical
    way I've found so far of booting it successfully was adding it as an
    option to the SysLinux menu of a Clonezilla USB, I've been investigating
    trying to boot Clonezilla directly off a hard disk partition, and I'm
    baffled, because I can't work out how the devil Clonezilla is booting
    itself off a USB, so that I can reproduce it on a HD.

    My knowledge of traditional booting methods is pretty good, or at least
    was but perhaps is now getting rusty. Some years ago, I published a
    page on my site detailing how PCs booted off a legacy MBR hard disk:

    https://www.macfh.co.uk/JavaJive/PCHardware/PCBootProcess.html

    Briefly, each sector is 0x200 (512) bytes long, and there's a chain
    loading system from sector to sector to load an OS.

    A) PC BIOS in RO non-volatile memory ...

    B) ... Loads the first sector of the first hard disk, the Master Boot
    Record (MBR), and checks for two bytes at offset 0x1FE (510) which must
    be 0x55 0xAA identifying it as boot code. If these are found the BIOS
    jumps to run the code in this sector. This code reads a partition table
    at 0x1C0 (448), looking for the first partition marked active and then
    if one is found ...

    C) ... Loads the first sector of that partition, the Partition Boot
    Record (PBR), again checks for the same two bytes at offset 0x1FE, and
    again if found jumps to run the code there. The code in the PBR then
    loads the operating system in that sector in an OS-dependent way.

    Often the code in PBRs, and perhaps also some MBRs, begins with just two
    bytes which form a jump past some data to the beginning of the booting
    code proper further on in the same sector. Hence the first byte is
    often 0xEB for a relative branch jump, and the second byte determines
    the destination of the branch, so if the first two bytes are EB 58 as in
    the example I have before me, the Program Counter will be 2 after
    reading this code, so the jump is to, and the booting code proper begins
    at, offset 0x5A.

    Comparatively, I'm not so clear on the modern system, having only vague anecdotal knowledge, but there seems to be a requirement to find files
    of one or more certain standard names in a directory off the root called
    Boot. BootMgr seems to be one such possible file.

    However, examining a Clonezilla USB which boots successfully yields no
    method known to me of booting!

    There are at least three possible candidate definitions for the first
    menu which is displayed:
    Grub2: /boot/grub/grub.cfg
    IsoLinux: /syslinux/isolinux.cfg
    SysLinux: /syslinux/syslinux.cfg

    These three menus are all very similar, so it's not immediately obvious
    which is actually being displayed, but they're just text files, so one
    can edit them to make them unique in some way to reveal themselves. As
    I've already given the answer upthread, it doesn't need a spoiler alert
    to reveal that, at least on my systems I have here, it's the SysLinux
    menu, however I think some more modern systems may use the Grub2 menu.

    But, in either case, *HOW* do we even get that far, that's the mystery!

    1) To begin with the legacy options, the obvious first places to look
    are the MBR & PBR (beware line wrap in some of what follows) ...

    I've examined the MBR of a working Clonezilla USB, but am not going to reproduce it here, as there appears to be nothing out of the ordinary
    about it, at least as far as I can tell without actually disassembling
    it. However, it's worth noting that it looks nothing like the MBR of a
    HD with Grub2 installed, so it's a reasonable assumption that grub is
    not directly loaded from the MBR as normally it would be.

    By contrast there *IS* something out of the ordinary, but not in a
    helpful way, in the PBR of a working Clonezilla Partition:

    00000000 eb 58 90 53 59 53 4c 49 4e 55 58 00 02 08 c2 10
    |.X.SYSLINUX.....|
    00000010 02 00 00 00 00 f8 00 00 3f 00 ff 00 00 08 00 00
    |........?.......|
    00000020 00 98 1e 00 9f 07 00 00 00 00 00 00 02 00 00 00
    |................|
    00000030 01 00 06 00 00 00 00 00 00 00 00 00 00 00 00 00
    |................|
    00000040 80 01 29 b5 6b 34 f2 43 4c 4f 4e 45 5a 5f 33 31
    |..).k4.CLONEZ_31|
    00000050 32 20 46 41 54 33 32 20 20 20 fa fc 31 c9 8e d1 |2 FAT32 ..1...| 00000060 bc 76 7b 52 06 57 1e 56 8e c1 b1 26 bf 78 7b f3
    |.v{R.W.V...&.x{.|
    00000070 a5 8e d9 bb 78 00 0f b4 37 0f a0 56 20 d2 78 1b |....x...7..V
    .x.|
    00000080 31 c0 b1 06 89 3f 89 47 02 f3 64 a5 8a 0e 18 7c
    |1....?.G..d....||
    00000090 88 4d f8 50 50 50 50 cd 13 eb 62 8b 55 aa 8b 75
    |.M.PPPP...b.U..u|
    000000a0 a8 c1 ee 04 01 f2 83 fa 4f 76 31 81 fa b2 07 73
    |........Ov1....s|
    000000b0 2b f6 45 b4 7f 75 25 38 4d b8 74 20 66 3d 21 47 |+.E..u%8M.t
    f=!G|
    000000c0 50 54 75 10 80 7d b8 ed 75 0a 66 ff 75 ec 66 ff
    |PTu..}..u.f.u.f.|
    000000d0 75 e8 eb 0f 51 51 66 ff 75 bc eb 07 51 51 66 ff
    |u...QQf.u...QQf.|
    000000e0 36 1c 7c b4 08 e8 e9 00 72 13 20 e4 75 0f c1 ea |6.|.....r.
    .u...|
    000000f0 08 42 89 16 1a 7c 83 e1 3f 89 0e 18 7c fb bb aa
    |.B...|..?...|...|
    00000100 55 b4 41 e8 cb 00 72 10 81 fb 55 aa 75 0a f6 c1
    |U.A...r...U.u...|
    00000110 01 74 05 c6 06 46 7d 00 66 b8 08 20 00 00 66 ba |.t...F}.f..
    ..f.|
    00000120 00 00 00 00 bb 00 80 e8 0e 00 66 81 3e 1c 80 8e
    |..........f.>...|
    00000130 c1 80 6a 75 74 e9 f8 02 66 03 06 60 7b 66 13 16
    |..jut...f..`{f..|
    00000140 64 7b b9 10 00 eb 2b 66 52 66 50 06 53 6a 01 6a
    |d{....+fRfP.Sj.j|
    00000150 10 89 e6 66 60 b4 42 e8 77 00 66 61 8d 64 10 72
    |...f`.B.w.fa.d.r|
    00000160 01 c3 66 60 31 c0 e8 68 00 66 61 e2 da c6 06 46
    |..f`1..h.fa....F|
    00000170 7d 2b 66 60 66 0f b7 36 18 7c 66 0f b7 3e 1a 7c
    |}+f`f..6.|f..>.||
    00000180 66 f7 f6 31 c9 87 ca 66 f7 f7 66 3d ff 03 00 00
    |f..1...f..f=....|
    00000190 77 17 c0 e4 06 41 08 e1 88 c5 88 d6 b8 01 02 e8
    |w....A..........|
    000001a0 2f 00 66 61 72 01 c3 e2 c9 31 f6 8e d6 bc 68 7b
    |/.far....1....h{|
    000001b0 8e de 66 8f 06 78 00 be da 7d ac 20 c0 74 09 b4 |..f..x...}.
    .t..|
    000001c0 0e bb 07 00 cd 10 eb f2 31 c0 cd 16 cd 19 f4 eb
    |........1.......|
    000001d0 fd 8a 16 74 7b 06 cd 13 07 c3 42 6f 6f 74 20 65
    |...t{.....Boot e|
    000001e0 72 72 6f 72 0d 0a 00 00 00 00 00 00 00 00 00 00
    |rror............|
    000001f0 00 00 00 00 00 00 00 00 fe 02 b2 3e 18 37 55 aa
    |...........>.7U.|

    The thing to note is that the only clear text strings are 'SYSLINUX', 'CLONEZ_312', 'FAT32', and 'Boot error'. It's possible that some
    further text is encrypted in some way, but I would judge that to be
    extremely unlikely for open source material - the only times I've
    encountered that previously were attempts to copy protect games that
    were supplied on floppies - so, unless someone disassembles the actual
    code and finds differently, I think we must assume that the text we can
    see is actually all there is.

    The second string is the disk label, the third is the name of the
    partition format, and the fourth is obviously an error message if
    something goes wrong, so the loader can only be looking for a file,
    presumably in the root directory, called SYSLINUX. However, there's no
    such file, only a directory called syslinux, and although files called
    syslinux exist, they are in sub-directories three levels down. Perhaps unsurprisingly therefore, when trying to boot the trial version of
    Clonezilla copied to a HD partition, I use in grub.cfg 'chainloader +1',
    which should load and run that PBR, I get that exact message: 'Boot
    error'! Further, if I make a binary dump of the original USB PBR into a
    file, say, PBR.bin, copy it to the HD trial version root, change that
    message to, say, 'B00t error', and then alter the grub.cfg entry to read 'chainloader /PBR.bin, I then get 'B00t error' as expected. This may
    seem a waste of time, but it shows that what I think is happening *IS*
    actually happening, always valuable when you are uncertain and confused,
    as most definitely I am just now!

    So it seems certain that, somehow or other, normal Clonezilla on a USB
    is *NOT* being booted via its PBR, because there is no file called
    SYSLINUX off the root, only a directory and there doesn't seem to be a
    way of selecting a particular file in that directory, and anyway, as
    described below, the menu looks different than when booted directly from
    the USB.

    What else might be happening?

    2) I can find 9 files likely to be candidates for involvement in
    loading Clonezilla to boot into the SysLinux menu ...

    /ldlinux.sys
    /syslinux/isolinux.bin
    /utils/linux/x86/extlinux
    /utils/linux/x86/syslinux
    /utils/linux/x64/extlinux
    /utils/linux/x64/syslinux
    /EFI/boot/bootia32.efi
    /EFI/boot/bootx64.efi
    /EFI/boot/grubx64.efi

    ... so I tested those:

    menuentry "Clonezilla #1 (/ldlinux.sys)" {
    savedefault
    set root=(hd0,msdos2)
    syslinux_source /ldlinux.sys
    syslinux_configfile /syslinux/syslinux.cfg
    }
    ...
    menuentry "Clonezilla #9 (/EFI/boot/grubx64.efi)" {
    savedefault
    set root=(hd0,msdos2)
    syslinux_source /EFI/boot/grubx64.efi
    syslinux_configfile /syslinux/syslinux.cfg
    }

    They all work identically - they display the SysLinux Menu (but as
    console text without any background image or special fonts), and from
    there will run Clonezilla and boot the W98 Emergency Floppy Disk image,
    but, unlike a Clonezilla USB booted normally, the Emergency W98 USB
    image doesn't boot, giving the same MEMDISK error as previously posted:
    MEMDISK 6.03 20171018 Copyright 2001-2014 H. Peter Anvin et al
    MEMDISK: No ramdisk image specified!

    The differences in the menu and subsequent behaviour suggest to me that
    none of these are how Clonezilla is booted normally from a USB.

    3) Next there are countless files in ...
    /boot/grub and subdirs
    ... but the only one that seems promising is ...
    /boot/grub/efi.img
    ... which has a dummy PBR at its start, containing the message ...
    This is not a bootable disk. Please insert a bootable floppy and
    press any key to try again.
    ... which is the exact message I get if I chainload it from grub!

    So HTF does Clonezilla boot itself from USB, into that highly
    distinctive menu, and how can one reproduce that on a HD?

    Notes:

    ! The grub2 'chainloader' command expects a boot sector or a file
    thereof, and if the boot signature 55 AA is not present at offset 1FE,
    it aborts with:
    error: invalid signature.
    Press any key to continue...

    ! The memdisk pseudo-kernel expects a genuine disk image, and if not
    given one variously aborts, crashes out, or hangs, for example at:
    Loading boot sector... booting...

    https://wiki.syslinux.org/wiki/index.php?title=MEMDISK

    --

    Fake news kills!

    I may be contacted via the contact address given on my website:
    www.macfh.co.uk

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Java Jive@21:1/5 to Java Jive on Sun Jul 7 17:47:43 2024
    XPost: alt.os.linux, alt.windows7.general

    As some of what I wrote in my last post turned out to be incorrect, I'll
    top post the solution here, and those who are sufficiently interested to
    want to learn from my mistakes can read the corrections appended below it:

    WRT Clonezilla 3.1.2-22 and getting it to boot from a HD exactly as it
    does from a USB. As previously described, Clonezilla themselves don't
    actually tell you how to do this ...
    https://clonezilla.org/livehd.php
    ... the method they give boots not into the initial menu, but straight
    into the first choice from that menu. However, Clonezilla is a syslinux system, and there are other sources of information about syslinux that
    are more helpful, for example this, the relevance of which will become apparent:
    https://wiki.archlinux.org/title/syslinux

    WARNING: The following overwrites both the MBR of the entire disk and
    the PBR of Clonezilla's partition, so if you have other bootable
    partitions &/or grub installed to choose between them, all this will be
    lost, so make any backups necessary for restoration before running the following command!

    As long as you don't mind wrecking your multi-boot system and rebuilding
    it afterwards, which may be why Clonezilla don't tell you about this
    option, you can run a program to get it to boot off a HD exactly as it
    does off a USB. In both USB and HD copies, there is a bash script ...
    /utils/linux/makeboot.sh
    ... which can be run to make the installation bootable in exactly the
    same way from either medium. Probably it would be needed for a USB only rarely, because programs that write the ISO image to the USB seem
    generally to take care of things pretty well, but for a HD installation,
    it seems to be the easiest way to get everything to work as it should.

    However, if your Clonezilla HD partition is formatted FAT32 like a USB,
    then you can't just run the file, because FAT32 doesn't support the x attribute, and by default modern linux systems tend to mount FAT32
    partitions translating the attributes of all files as non-executable.
    The easiest way around this is to run the script through bash ...
    sudo bash <mounted path>/utils/linux/makeboot.sh /dev/sdxy
    ... where sdxy is the designation of the Clonezilla disk and partition,
    in my case ...
    sdb2

    Now when you boot from that hard disk, it will boot straight into the
    normal brightly coloured Clonezilla menu with large fonts, and will
    behave seemingly exactly as if it had been booted off USB, even, hooray, booting my Emergency W98 USB image successfully.

    Then you can reinstall grub to get everything else back. For the
    Clonezilla option, preserving what has been won, add to the file (in
    Ubuntu, other distros may differ) ...
    /etc/grub.d/40_custom
    ... the following ...
    menuentry {
    savedefault # If you've enabled saving the last grub choice
    set root=(hdx,y) # Where x & y are the CZ disk and partition #s
    chainloader +1
    }

    As this form of grub entry was tried before, without success, it is
    pertinent to ask why it works now, but not before? From the ArchLinux
    source above it seems to be because the PBR needs to know the beginning
    sector of the file ldlinux.sys - it seems to ignore the actual file
    system, and goes straight wherever it was told, at the time that it was installed, that that file begins. If the file is moved or the CZ file
    system is copied to another place, this direct linkage is lost, and the
    boot fails with the message I played around with earlier: "Boot error"

    This explains several things:

    + The initial failure of the attempt to use the chainloader command -
    the file was no longer to be found at the sector designated in the PBR.

    + The absence of any path names in the PBR, 'SYSLINUX' seems to be just
    a generic system name describing the bootloader type for those who
    happen to know, which, until I read the above link, I did not.

    Corrections to my previous post follow ...

    On 06/07/2024 19:42, Java Jive wrote:

    [variously snipped for brevity]

    So, putting aside for the moment the problem of trying to directly load
    the emergency W98 USB image from Grub2, given that the only practical
    way I've found so far of booting it successfully was adding it as an
    option to the SysLinux menu of a Clonezilla USB, I've been investigating trying to boot Clonezilla directly off a hard disk partition, and I'm baffled, because I can't work out how the devil Clonezilla is booting
    itself off a USB, so that I can reproduce it on a HD.

    My knowledge of traditional booting methods is pretty good, or at least
    was but perhaps is now getting rusty.  Some years ago, I published a
    page on my site detailing how PCs booted off a legacy MBR hard disk:

    https://www.macfh.co.uk/JavaJive/PCHardware/PCBootProcess.html

    I've amended this with the correction below.

    Briefly, each sector is 0x200 (512) bytes long, and there's a chain
    loading system from sector to sector to load an OS.

    A) PC BIOS in RO non-volatile memory ...

    B) ... Loads the first sector of the first hard disk, the Master Boot
    Record (MBR), and checks for two bytes at offset 0x1FE (510) which must
    be 0x55 0xAA identifying it as boot code.  If these are found the BIOS
    jumps to run the code in this sector.  This code reads a partition table
    at

    ... actually two bytes earlier at 0x1BE (446), the reason being: each partition record is 0x10 (16) bytes long and there are 4 of them, and
    the boot flag is at 0x1FE (510), and ...
    0x1FE (510) - 0x40 (64) = 0x1BE (446)

    looking for the first partition marked active and then
    if one is found ...

    C) ... Loads the first sector of that partition, the Partition Boot
    Record (PBR), again checks for the same two bytes at offset 0x1FE, and
    again if found jumps to run the code there.  The code in the PBR then
    loads the operating system in that sector in an OS-dependent way.

    See the description above for why the following is as it is:

    By contrast there *IS* something out of the ordinary, but not in a
    helpful way, in the PBR of a working Clonezilla Partition:

    00000000  eb 58 90 53 59 53 4c 49  4e 55 58 00 02 08 c2 10 |.X.SYSLINUX.....|
    [...]
    00000040  80 01 29 b5 6b 34 f2 43  4c 4f 4e 45 5a 5f 33 31 |..).k4.CLONEZ_31|
    00000050  32 20 46 41 54 33 32 20  20 20 fa fc 31 c9 8e d1  |2 FAT32 ..1...|
    [...]
    000001d0  fd 8a 16 74 7b 06 cd 13  07 c3 42 6f 6f 74 20 65
    |...t{.....Boot e|
    000001e0  72 72 6f 72 0d 0a 00 00  00 00 00 00 00 00 00 00 |rror............|
    000001f0  00 00 00 00 00 00 00 00  fe 02 b2 3e 18 37 55 aa |...........>.7U.|

    The thing to note is that the only clear text strings are 'SYSLINUX', 'CLONEZ_312', 'FAT32', and 'Boot error'.

    The first string seems only to describe the PBR type ...

    The second string is the disk label, the third is the name of the
    partition format, and the fourth is obviously an error message if
    something goes wrong.

    The following is FALSE! See description above.

    So it seems certain that, somehow or other, normal Clonezilla on a USB
    is *NOT* being booted via its PBR, because there is no file called
    SYSLINUX off the root, only a directory and there doesn't seem to be a
    way of selecting a particular file in that directory, and anyway, as described below, the menu looks different than when booted directly from
    the USB.

    I'll leave the following in even though they're correct because, if when
    trying things out you see an error message such as the following, it's
    useful to know the likely cause:

    Notes:

    !  The grub2 'chainloader' command expects a boot sector or a file
    thereof, and if the boot signature 55 AA is not present at offset 1FE,
    it aborts with:
            error: invalid signature.
            Press any key to continue...

    !  The memdisk pseudo-kernel expects a genuine disk image, and if not
    given one variously aborts, crashes out, or hangs, for example at:
            Loading boot sector... booting...

    https://wiki.syslinux.org/wiki/index.php?title=MEMDISK

    --

    Fake news kills!

    I may be contacted via the contact address given on my website:
    www.macfh.co.uk

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)