• debian-installer: how to exactly specify partition sizes?

    From Andy Smith@21:1/5 to All on Fri Jul 11 01:00:01 2025
    Hi,

    Let's say I have an existing machine that was installed using the text
    mode trixie RC1 debian-installer, and I want to exactly copy the
    partition layout of this machine on the next machine.

    Probably the best long term way would be to record the partitioning
    details in a preseed file and load that.

    But let's say I want to do it by hand. How?

    The manual partitioner accepts B, MB, GB and the binary equivalents as
    units. However, there is not any value I can specify that replicates an existing disk layout.

    For example, given this:

    $ head -4 /proc/partitions
    major minor #blocks name

    259 1 7501465944 nvme1n1
    259 2 487424 nvme1n1p1

    You'd expect that specifying "499122176B" (that's 487424 * 1024) would
    result in a partition of exactly 487424 1024-byte blocks being created,
    right? Nope, it creates one 498073600 bytes long which is exactly 1 MiB (1048576 bytes) less than what I asked for.

    Also trying to use MB or MiB never does result in anything that exactly
    matches the other existing partition layout.

    I can make the first partition match by manually adding 1048576 bytes
    and asking for that. But when I try the same trick with the next
    partition, again it gives me something slightly off.

    Why does it alter the value I specify? The other machine was installed
    using debian-installer so if there is some alignment thing going on,
    well, the original one was acceptable to debian-installer so it
    shouldn't need to alter the values I specify.

    Presumably I could replicate it by remembering the value I typed in when installing the first one, since hopefully the changes it makes are deterministic. But I don't remember.

    Any ideas? Perhaps I can get sfdisk into the installer environment then
    I could paste the output of sfdisk -d into sfdisk at a shell prompt.

    It feels wrong that the installer doesn't give me n amount of bytes when
    I ask for it with "nB".

    Thanks,
    Andy

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Felix Miata@21:1/5 to All on Fri Jul 11 01:30:01 2025
    Andy Smith composed on 2025-07-10 22:52 (UTC):

    Let's say I have an existing machine that was installed using the text
    mode trixie RC1 debian-installer, and I want to exactly copy the
    partition layout of this machine on the next machine.

    Probably the best long term way would be to record the partitioning
    details in a preseed file and load that.

    But let's say I want to do it by hand. How?

    The manual partitioner accepts B, MB, GB and the binary equivalents as
    units. However, there is not any value I can specify that replicates an existing disk layout.

    For example, given this:

    $ head -4 /proc/partitions
    major minor #blocks name

    259 1 7501465944 nvme1n1
    259 2 487424 nvme1n1p1

    You'd expect that specifying "499122176B" (that's 487424 * 1024) would
    result in a partition of exactly 487424 1024-byte blocks being created, right? Nope, it creates one 498073600 bytes long which is exactly 1 MiB (1048576 bytes) less than what I asked for.

    Also trying to use MB or MiB never does result in anything that exactly matches the other existing partition layout.

    I can make the first partition match by manually adding 1048576 bytes
    and asking for that. But when I try the same trick with the next
    partition, again it gives me something slightly off.

    Why does it alter the value I specify? The other machine was installed
    using debian-installer so if there is some alignment thing going on,
    well, the original one was acceptable to debian-installer so it
    shouldn't need to alter the values I specify.

    Presumably I could replicate it by remembering the value I typed in when installing the first one, since hopefully the changes it makes are deterministic. But I don't remember.

    Any ideas? Perhaps I can get sfdisk into the installer environment then
    I could paste the output of sfdisk -d into sfdisk at a shell prompt.

    It feels wrong that the installer doesn't give me n amount of bytes when
    I ask for it with "nB".

    I can't help with what you actually asked for, because I never use any OS installer's partitioner to add, remove, resize, or anything else to do with partition table writes that isn't merely about UUIDs, LABELs or filesystem formats. I use OS installers for nothing more than formatting or naming, if required, or assigning existing partitions to mount points. All partitioning I do
    I have been doing the same way since last century, using text mode partitioner from http://www.dfsee.com/ that writes fully compatible tables whether run from its binaries for DOS, OS/2, Windows, Linux, or (as I hear it) MacOS (where I boot
    Linux to partition). Using DFSee gives me exactly what I desire on a consistent and reliable basis, and I get logs in the process whose selected sections serve as
    catalogs of what I have installed where on the hundred or more HDDs and SSD scattered about the premises.
    --
    Evolution as taught in public schools is, like religion,
    based on faith, not based on science.

    Team OS/2 ** Reg. Linux User #211409 ** a11y rocks!

    Felix Miata

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Pascal Hambourg@21:1/5 to Andy Smith on Sat Jul 12 11:40:02 2025
    On 11/07/2025 at 00:52, Andy Smith wrote:

    You'd expect that specifying "499122176B" (that's 487424 * 1024) would
    result in a partition of exactly 487424 1024-byte blocks being created, right? Nope, it creates one 498073600 bytes long which is exactly 1 MiB (1048576 bytes) less than what I asked for.

    Also trying to use MB or MiB never does result in anything that exactly matches the other existing partition layout.

    I can make the first partition match by manually adding 1048576 bytes
    and asking for that. But when I try the same trick with the next
    partition, again it gives me something slightly off.

    Why does it alter the value I specify? The other machine was installed
    using debian-installer so if there is some alignment thing going on,
    well, the original one was acceptable to debian-installer so it
    shouldn't need to alter the values I specify.

    Debian installer's partitioning tool, partman, is based on libparted. By default, it applies parted's "optimal" alignment constraint. On disks
    with 512-byte sector size, it aligns partition start and end positions
    on 1-MiB (2048 sectors) boundaries. Additionally, it reserves a gap at
    the end of the disk to avoid ambiguity about whether MD RAID 0.90 or 1.0 metadata belong to either the entire disk or the last partition.

    Free space and partitions are identified as start-end intervals in
    bytes. When creating a partition, partman specifies in which free space interval, the partition type (primary|logical), the position (beginning|end|full) and the desired size in bytes (irrelevant if the
    position is "full").

    The partitioning log is recorded in /var/log/partman during the
    installation or in /var/log/installer/partman in the installed system.
    Here is an example:

    /lib/partman/free_space/50new/do_option: IN: NEW_PARTITION =dev=sda Primary ext2 50105906176-112857186303 full 62800000000

    50105906176-112857186303 is the free space start-end interval in bytes
    (~62.75 GB).
    "full" means that the new partition must use the entire free space.
    The desired size was rounded (62.8 GB) from the exact free space size by
    the user interface.

    parted_server: Read command: NEW_PARTITION
    parted_server: command_new_partition()
    parted_server: Note =dev=sda as changed
    parted_server: Opening outfifo
    parted_server: requested partition with type Primary
    parted_server: requested partition with file system ext2
    parted_server: add_primary_partition(disk(312581808),97863098-220424191)

    312581808 is the disk size in sectors (160 GB).
    97863098-220424191 is the desired partition start-end interval in
    sectors, calculated from the free space start-end interval, desired
    partition size and position (beginning|end|full).

    parted_server: OUT: OK


    parted_server: OUT: 3 50106204160-112857186303 62750982144 primary ext2 /dev/sda3

    Here is the actual resulting partition interval and size in bytes. The
    start position had to be adjusted to align on the next 1-MiB boundary.
    The end position was already aligned.

    It is important to note that reported free space intervals do not take
    the alignment constraint into account, so the resulting partition size
    may be at worst 2-MiB smaller than the desired size. This is blatant
    with the first partition at the beginning of the disk: the reported free
    space begins right after the partition table (at sector 2 with MSDOS, 34
    with GPT), so partman will request to create a partition in interval 34-(34+desired size), then will align the start sector to 2048. This
    results in a partition being 1-MiB smaller than expected.

    The free space after the first partition is aligned, so the next primary partition sizes should not be altered if they are multiple of 1 MiB. On
    MSDOS, logical partition size is reduced by 1 MiB because of the nested extended partition table before each logical partition.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andy Smith@21:1/5 to Pascal Hambourg on Sat Jul 12 14:10:01 2025
    Hi Pascal,

    On Sat, Jul 12, 2025 at 11:38:20AM +0200, Pascal Hambourg wrote:
    Debian installer's partitioning tool, partman, is based on libparted. By default, it applies parted's "optimal" alignment constraint. On disks with 512-byte sector size, it aligns partition start and end positions on 1-MiB (2048 sectors) boundaries. Additionally, it reserves a gap at the end of the disk to avoid ambiguity about whether MD RAID 0.90 or 1.0 metadata belong to either the entire disk or the last partition.

    Thanks for the detailed explanation!

    I'm still a little confused why partman alters supplied sizes that come
    from something that was previously installed by partman. I was giving it
    sizes that were 1-MiB aligned so I expected the first one to start 2048
    sectors from the start and go on for the exact amount of bytes
    specified, since those values were already aligned.

    The partitioning log is recorded in /var/log/partman during the installation or in /var/log/installer/partman in the installed system.

    Is there any way to use this log to feed in to another instance of
    partman to replicate what was done before?

    Or can it just be used to make a preseed that would exactly replicate
    it?

    Reading all you've given here, I now kind of understand what it does and
    why, but it still doesn't sound possible to easily replicate what was
    already done in another partman.

    It is important to note that reported free space intervals do not take the alignment constraint into account, so the resulting partition size may be at worst 2-MiB smaller than the desired size. This is blatant with the first partition at the beginning of the disk: the reported free space begins right after the partition table (at sector 2 with MSDOS, 34 with GPT), so partman will request to create a partition in interval 34-(34+desired size), then will align the start sector to 2048. This results in a partition being 1-MiB smaller than expected.

    The free space after the first partition is aligned, so the next primary partition sizes should not be altered if they are multiple of 1 MiB. On MSDOS, logical partition size is reduced by 1 MiB because of the nested extended partition table before each logical partition.

    I did notice that the first partition I asked for by exact byte count
    ended up 1 MiB smaller than I expected so I was able to replicate what
    I wanted by asking for an extra 1MiB, however then I saw this didn't
    hold true for the next partition. Perhaps if I had done the adjustment
    for the first partition but then not for any others I would have been
    able to do it?

    i.e. given existing partman-installed system:

    $ head -6 /proc/partitions
    major minor #blocks name

    8 16 1875374424 sdb
    8 17 487424 sdb1
    8 18 976896 sdb2
    8 19 1952768 sdb3

    I could ask for exact sizes:

    $ echo "487424*512 + 1024" | bc
    249562112
    $ echo "976896*512" | bc
    500170752
    $ echo "1952768*512" | bc
    999817216

    Anyway, I will look into preseed as I should probably do things that way
    if I want them to be replicated.

    Thanks,
    Andy

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Pascal Hambourg@21:1/5 to Andy Smith on Sun Jul 13 09:40:01 2025
    On 12/07/2025 at 14:03, Andy Smith wrote:

    I'm still a little confused why partman alters supplied sizes that come
    from something that was previously installed by partman. I was giving it sizes that were 1-MiB aligned so I expected the first one to start 2048 sectors from the start and go on for the exact amount of bytes
    specified, since those values were already aligned.

    Partition creation involves two steps:

    1) Calculate the desired partition start and end from the free space
    range start and end, and desired position and size. This step does not
    take alignement constraints into account. Basically:

    switch(position)
    "full":
    partition start = free space start
    partition end = free space end
    "beginning":
    partition start = free space start
    partition end = free space start + desired size
    "end":
    partition start = free space end - desired size
    partition end = free space end

    2) Request the partition creation with the calculated start and end.
    This step enforces alignment constraints, so the resulting start and end
    may be independently adjusted to satisfy alignment constraints. Of
    course this may affect the resulting partition size.

    When creating the first partition at the beginning of the disk with a
    GPT partition table, the free space start is 17 KiB. So the calculated partition start will be 17 KiB, and the calculated partition end will be (desired size + 17 KiB). Optimal alignement constraint will
    independently adjust the actual start to 1 MiB and the actual end to
    desired size, so the resulting size will be (desired size - 1 MiB).

    The partitioning log is recorded in /var/log/partman during the installation >> or in /var/log/installer/partman in the installed system.

    Is there any way to use this log to feed in to another instance of
    partman to replicate what was done before?

    You can retrieve desired partition sizes in NEW_PARTITION command lines.
    If you use them you should get the same resulting sizes.

    Or can it just be used to make a preseed that would exactly replicate
    it?

    Do you mean with partman-auto/expert_recipe ?
    Partition size unit in partman-auto recupes is decimal MB, so it may be
    hard to replicate partition sizes which were given in binary units.

    I did notice that the first partition I asked for by exact byte count
    ended up 1 MiB smaller than I expected so I was able to replicate what
    I wanted by asking for an extra 1MiB, however then I saw this didn't
    hold true for the next partition. Perhaps if I had done the adjustment
    for the first partition but then not for any others I would have been
    able to do it?

    I guess so, for primary partitions. Logical partitions would be 1 MiB
    smaller than requested.

    i.e. given existing partman-installed system:

    $ head -6 /proc/partitions
    major minor #blocks name

    8 16 1875374424 sdb
    8 17 487424 sdb1
    8 18 976896 sdb2
    8 19 1952768 sdb3

    I could ask for exact sizes:

    $ echo "487424*512 + 1024" | bc
    249562112
    $ echo "976896*512" | bc
    500170752
    $ echo "1952768*512" | bc
    999817216

    Sizes in /proc/partitions are in KiB, not 512-byte sectors.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andy Smith@21:1/5 to Pascal Hambourg on Sun Jul 13 17:00:01 2025
    Hi Pascal,

    On Sun, Jul 13, 2025 at 09:30:59AM +0200, Pascal Hambourg wrote:
    On 12/07/2025 at 14:03, Andy Smith wrote:
    i.e. given existing partman-installed system:

    $ head -6 /proc/partitions
    major minor #blocks name

    8 16 1875374424 sdb
    8 17 487424 sdb1
    8 18 976896 sdb2
    8 19 1952768 sdb3

    I could ask for exact sizes:

    $ echo "487424*512 + 1024" | bc
    249562112
    $ echo "976896*512" | bc
    500170752
    $ echo "1952768*512" | bc
    999817216

    Sizes in /proc/partitions are in KiB, not 512-byte sectors.

    Oh, yeah. That was a copy/paste from some earlier failed experiment. :)

    Is the above theory sound though for a GPT label? i.e. first one 487424
    * 1024 + 1024, second and further ones just multiply by 1024. Last one
    expect a bit at the end reserved for possible MD metadata.

    (I don't actually care about the last one as the last one is usually
    always "the rest of the available device however partman wants," for
    me.)

    Thanks,
    Andy

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Pascal Hambourg@21:1/5 to Andy Smith on Sun Jul 13 18:00:01 2025
    On 13/07/2025 at 16:54, Andy Smith wrote:

    Is the above theory sound though for a GPT label? i.e. first one 487424
    * 1024 + 1024, second and further ones just multiply by 1024. Last one
    expect a bit at the end reserved for possible MD metadata.

    Yes, it should do the trick. Or divide by 1024 (and add 1 to the first
    one) and input the size as MiB, as partitions created by partman are
    expected to be aligned on 1-MiB boundaries.

    NB: It looks like the first three partitions were created with input
    sizes 500 MB, 1 GB and 2 GB (decimal).

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Pascal Hambourg@21:1/5 to Andy Smith on Sun Jul 13 18:50:01 2025
    On 13/07/2025 at 16:54, Andy Smith wrote:

    first one 487424 * 1024 + 1024

    Correction: 487424 * 1024 + 1048576

    You need to add 1 MiB, not 1 KiB.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)