• Bug#1109499: [debian-mysql] Bug#1109499: bacula-director-sqlite3: fails

    From Carsten Leonhardt@21:1/5 to [email protected] on Fri Aug 1 00:00:01 2025
    Hi Otto,

    Otto Kekäläinen <[email protected]> writes:

    I skimmed through https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1109499 quickly and my initial thought here is that Bacula (or any other program) should not rely on dpkg maintainer scripts to upgrade the database as there isn't any guarantees about what the initial state is, and if anything fails it's really hard to restart or recover. Any failures happening during a
    dpkg run in general will block the upgrades/installs/uninstalls for the
    whole system as apt will refuse to do any new actions until the pending
    dpkg package action has completed successfully.

    Instead I would recommend to do the upgrade as part of the systemd/init service startup sequence. If it fails it's much easier to restart and debug and the failure will only affect one single service, not the whole OS
    package management system. Doing an upgrade check and upgrading the
    database as part of the startup sequence will also behave correctly if the user is for example restoring an old database from backups and trying to start it with a new version of Bacula.

    the bacula packages switched to using dbconfig-common in 2006 and that's probably about as long as I've been a user of them. Since then, there
    haven't been any of the problems you describe (at least as far as I am
    aware - I only became maintainer of bacula in 2016). I think we can say
    that the current method for upgrading using dbconfig-common is well
    tested and in no need of change.

    On the other hand, I concur with Adrian that starting the dist-upgrade
    with a stopped MariaDB-server may break all packages that need to do
    database changes during their upgrades. Paul is the maintainer of dbconfig-common, maybe he knows more of its failure mode.

    Regards

    Carsten

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Adrian Bunk@21:1/5 to All on Fri Aug 1 15:30:01 2025
    On Thu, Jul 31, 2025 at 03:03:23PM -0700, Otto Kekäläinen wrote:
    When there are failures, it's not bad when this is immediately visible
    to the administrator.

    If the server is not fully dedicated to just running Bacula, then
    preventing dpkg/apt from doing anything is a bit overkill just for the
    sake of notifying the user.

    It is the only reliable way to notify the user, and appropriate when
    something major failed.

    Abort with a good error message, and the user knows that something failed
    and has a good starting point for searching for searching for a solution.

    Really bad are silent failures where you only notice some time afterwards
    that something isn't working, and then have to start debugging what the
    problem might be.

    In the case of a Backup server, the classical "really bad" would be that
    the user discovered it is not running when he needs to restore from a
    backup that was not created.

    The postinst will restart the service, which might start the migration
    in the background.

    The migration might take a long time.

    There is likely a recommendation to reboot when the package management system has finished upgrading all packages.

    The administrator reboots, not being aware that a migration is still running in the background.

    There are many ways to communicate things to the system admin, but if
    they fail, the service would still restart the database schema
    migration etc immediately after restart when the service file would
    try to start Bacula and notice that upgrade wasn't completed.

    Rather "would still TRY to restart the migration".

    We have to really really really try hard to avoid rebooting while any
    kind of upgrade or migration is ongoing, sice this might create really
    nasty breakages.

    Can you 100% guarantee that it will never ever cause a problem when
    I unplug the power of the machine at any time during mariadb-upgrade?

    Doing an upgrade check and upgrading the
    database as part of the startup sequence will also behave correctly if the
    user is for example restoring an old database from backups and trying to start it with a new version of Bacula.

    How do you restore anything from backups when the database of your
    backup server is broken?

    We are now talking about Bacula's own database schema, right? For
    example, if the hardware was broken and the sysadmin is installing
    Trixie on a new host and installs the latest Bacula but takes the
    Bacula database from the old Bookworm server, the Bacula schema would
    never migrate if it only happens when triggered by dpkg.

    Whatever the user does manually is outside the scope of what we
    have to support when upgrading.

    If it is part
    of the service startup, it would always do the upgrade of the database
    scheme when Bacula starts and it detects the stuff on the disk is of
    an older format than what the running binary wants to use.

    Far more important is that both before and after the postinst of
    mariadb-server the database server is available.

    The scripts of other packages can assume that when a program or service
    is configured to use a database server, then this database server is
    available during the upgrade.

    Stopping the database server before the upgrade can cause problems.

    Still running mariadb-upgrade instead of a running database server after
    the postinst can cause problems.

    And if connecting to the configured database fails, this is a serious
    error where aborting from the postinst is appropriate.

    There is also a certain irony that you want users to manually stop the
    MariaDB server before the upgrade due to fear of an unclean shutdown
    when running the old version, which guarantees that every MariaDB-using
    service that gets configured before mariadb-server has an unclean
    shutdown with the old version due to non-availability of the database.

    It would cause far less problems if the release notes would instead
    document that in this rare error situation of an unclean MariaDB
    shutdown it is required to downgrade to the bookworm mariadb-server
    for error recovery.

    cu
    Adrian

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Adrian Bunk@21:1/5 to All on Fri Aug 1 22:30:01 2025
    Hi Otto,

    after thinking more about it, I wonder if the MariaDB problem described
    in the Release Notes exists at all:

    You are already stopping the server in the preinst,
    and abort when that failed.

    cu
    Adrian

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)