1. Forum
  2. Usenet
  3. LINUX.GENTOO.USER

  • [gentoo-user] Recovering mail

    From Peter Humphrey@21:1/5 to All on Mon Jun 2 12:00:01 2025
    Greetings,

    My user's .maildir has overfilled and so I've lost some emails. However, I have a cron job running every five minutes to rsync .maildir to /mnt/mailbu. The question is: can I just copy mailbu over .maildir?

    The mail chain is fetchmail (POP3) > postfix > dovecot > kmail (IMAP4).

    --
    Regards,
    Peter.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Stefan Schmiedl@21:1/5 to All on Mon Jun 2 14:10:01 2025
    ------ Original Message ------
    From "Peter Humphrey" <[email protected]>
    To [email protected]
    Date 02.06.2025 11:52:34
    Subject [gentoo-user] Recovering mail

    Greetings,

    My user's .maildir has overfilled and so I've lost some emails. However, I have
    a cron job running every five minutes to rsync .maildir to /mnt/mailbu. The >question is: can I just copy mailbu over .maildir?

    There might be some duplicates depending on e.g. the message state,
    which is encoded in the filename.

    Try rsync -avn /mnt/malibu/ ~/.maildir for a dry-run that should
    tell you what would be copied. If the results look reasonable
    stop dovecot and kmail, run again without the -n, start dovecot.

    At least that's what I would do to fix my maildir.

    Good luck,
    s.



    The mail chain is fetchmail (POP3) > postfix > dovecot > kmail (IMAP4).

    --
    Regards,
    Peter.




    <html><head>

    <style id="signatureStyle" type="text/css"><!--#x9fdf1401839041e #xe61c1979f7dd4eaeb9a72a2370c0e757 p
    {margin: 0px;}
    #x9fdf1401839041e #xe61c1979f7dd4eaeb9a72a2370c0e757
    {font-family: "Segoe UI"; font-size: 12pt;}
    #x9fdf1401839041e #xe61c1979f7dd4eaeb9a72a2370c0e757 p
    {margin: 0px;}
    </style><style id="css_styles" type="text/css"><!--blockquote.cite { margin-left: 5px; margin-right: 0px; padding-left: 10px; padding-right:0px; border-left: 1px solid #cccccc }
    blockquote.cite2 {margin-left: 5px; margin-right: 0px; padding-left: 10px; padding-right:0px; border-left: 1px solid #cccccc; margin-top: 3px; padding-top: 0px; }
    a img { border: 0px; }
    li[style='text-align: center;'], li[style='text-align: center; '], li[style='text-align: right;'], li[style='text-align: right; '] { list-style-position: inside;}
    body { font-family: Consolas; font-size: 11pt; }
    .quote { margin-left: 1em; margin-right: 1em; border-left: 5px #ebebeb solid; padding-left: 0.3em; }
    a.em-mention[href] { text-decoration: none; color: inherit; border-radius: 3px; padding-left: 2px; padding-right: 2px; background-color: #e2e2e2; }
    ._em_placeholder {color: gray; border-bottom: 1px dotted lightblue;} ._em_placeholder:before{color:gray; content: '{{ ';} ._em_placeholder:after{color:gray; content: ' }}';}
    </style></head>
    <body><div>------ Original Message ------</div><div>
    <div>From "Peter Humphrey" &lt;<a href="mailto:[email protected]">[email protected]</a>&gt;</div>
    <div>To <a href="mailto:[email protected]">[email protected]</a></div>
    <div>Date 02.06.2025 11:52:34</div>
    <div>Subject [gentoo-user] Recovering mail</div></div><div x-em-quote=""><br /></div>
    <div id="x25588eb4fe99460" class="plain"><blockquote cite="4994515.GXAFRqVoOG@cube" type="cite" class="cite2">

    <div class="plain_line">Greetings,</div>
    <div class="plain_line"> </div>
    <div class="plain_line">My user's .maildir has overfilled and so I've lost some emails. However, I have</div>
    <div class="plain_line">a cron job running every five minutes to rsync .maildir to /mnt/mailbu. The</div>
    <div class="plain_line">question is: can I just copy mailbu over .maildir?</div></blockquote><span><div id="x25588eb4fe99460" class="plain"><span><br /></span></div><div id="x25588eb4fe99460" class="plain"><span>There might be some duplicates depending
    on e.g. the message state,</span></div><div id="x25588eb4fe99460" class="plain"><span>which is encoded in the filename.</span></div><div id="x25588eb4fe99460" class="plain"><br /></div><div id="x25588eb4fe99460" class="plain"><span>Try rsync -avn /mnt/
    malibu/ ~/.maildir for a dry-run that should</span></div><div id="x25588eb4fe99460" class="plain"><span>tell you what would be copied. If the results look reasonable</span></div><div id="x25588eb4fe99460" class="plain"><span>stop dovecot and kmail, run
    again without the -n, start dovecot.</span></div><div id="x25588eb4fe99460" class="plain"><span><br /></span></div><div id="x25588eb4fe99460" class="plain"><span>At least that's what I would do to fix my maildir.</span></div><div id="x25588eb4fe99460"
    class="plain"><span><br /></span></div><div id="x25588eb4fe99460" class="plain"><span>Good luck,</span></div><div id="x25588eb4fe99460" class="plain"><span>s.</span></div><br /></span><blockquote cite="4994515.GXAFRqVoOG@cube" type="cite" class="cite2"><
    div class="plain_line"><br /></div>
    <div class="plain_line"> </div>
    <div class="plain_line">The mail chain is fetchmail (POP3) &gt; postfix &gt; dovecot &gt; kmail (IMAP4).</div>
    <div class="plain_line"> </div>
    <div class="plain_line">--</div>
    <div class="plain_line">Regards,</div>
    <div class="plain_line">Peter.</div>
    <div class="plain_line"> </div>
    <div class="plain_line"> </div>
    <div class="plain_line"> </div>
    <div class="plain_line"> </div>
    </blockquote></div>


    </body></html>

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Peter Humphrey@21:1/5 to All on Mon Jun 2 17:40:02 2025
    On Monday, 2 June 2025 13:03:54 British Summer Time Stefan Schmiedl wrote:
    ------ Original Message ------
    From "Peter Humphrey" <[email protected]>
    To [email protected]
    Date 02.06.2025 11:52:34
    Subject [gentoo-user] Recovering mail

    Greetings,

    My user's .maildir has overfilled and so I've lost some emails. However, I >have a cron job running every five minutes to rsync .maildir to >/mnt/mailbu. The question is: can I just copy mailbu over .maildir?

    There might be some duplicates depending on e.g. the message state,
    which is encoded in the filename.

    Try rsync -avn /mnt/malibu/ ~/.maildir for a dry-run that should
    tell you what would be copied. If the results look reasonable
    stop dovecot and kmail, run again without the -n, start dovecot.

    At least that's what I would do to fix my maildir.

    Well, that's recovered all the missing ones (an awful lot of them, anyway),
    but I now have thousands of duplicates to find and delete.

    Good luck,

    Thanks :)

    --
    Regards,
    Peter.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael@21:1/5 to All on Mon Jun 2 17:48:22 2025
    On Monday, 2 June 2025 16:34:12 British Summer Time Peter Humphrey wrote:
    On Monday, 2 June 2025 13:03:54 British Summer Time Stefan Schmiedl wrote:
    ------ Original Message ------
    From "Peter Humphrey" <[email protected]>
    To [email protected]
    Date 02.06.2025 11:52:34
    Subject [gentoo-user] Recovering mail

    Greetings,

    My user's .maildir has overfilled and so I've lost some emails. However, >I
    have a cron job running every five minutes to rsync .maildir to >/mnt/mailbu. The question is: can I just copy mailbu over .maildir?

    There might be some duplicates depending on e.g. the message state,
    which is encoded in the filename.

    Try rsync -avn /mnt/malibu/ ~/.maildir for a dry-run that should
    tell you what would be copied. If the results look reasonable
    stop dovecot and kmail, run again without the -n, start dovecot.

    At least that's what I would do to fix my maildir.

    Well, that's recovered all the missing ones (an awful lot of them, anyway), but I now have thousands of duplicates to find and delete.

    Good luck,

    Thanks :)

    The 'doveadm deduplicate' should be able to remove duplicates, or you can
    check this mail-deduplicate purpose written utility:

    https://github.com/kdeldycke/mail-deduplicate

    Dry-run will allow you to check what may be deleted, before you let it loose
    on your maildir.
    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCAAdFiEEXqhvaVh2ERicA8Ceseqq9sKVZxkFAmg91dYACgkQseqq9sKV ZxlQhg//UKcM6eO3cx/rbevBwPoeUmXg/mKPpEJMMadC8VB5GfxDwrC8Z53mvdq4 knVFLimjchAUr7JMjzknoOH/VUNc9JaHcy6djLkj1x3gDyKfO1+VzVEXcnw9nKDp 94PtksNdScO0HddlnnlYXDsgJidUiTNpqjcNiUe8brskL1dB7kFkOLVbAYmZv6k/ ZpYnH+owQSjfX1gIUpaf1ydKV+QBuYPpbnRXJsuHIiDkQpmNvFF9iWaNUy3dJ7fW bZygU2LKwuhmSR2eho5Jba8eACbEnhLcsASuQ+Tzxtw8SYuGRIENR+LF3W2yRN5L pFru50KNNax2YYFZRgrdWpLea5+G5B+CrRDB23n46K/LIPsfB2BJqkHzQOQLmFM+ McVZsFklbhfxszBbjEWrcWIQdPysI+mn1IsJINNEtTtKB0J0rQSqPJ2SQNFYZt1r B1JZjg8tJX4sF+cCIu+twR0gF0G2th29RY00gupZHy2M2mM3PJZWrQUBMHWNWegI GN6WBzHfu721Z9dWW10ICyi+tFHb+U9qUGejj4nHqbwhVaet4poW4wpxqYjldnSR AroDAVz8i2w8PMNEWA+k05eo7+PAJ/ZNKTEF06Y46ll0aqUdybPJmYrSmFn0rF5S 2iHVIzq3+uBGMbBreu2rnDg1//pSWQyvnZMS9tndZXNiPMyZoc4=
    =KN0Q
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Peter Humphrey@21:1/5 to All on Thu Jun 5 00:50:01 2025
    On Wednesday, 4 June 2025 23:15:55 British Summer Time Frank Steinmetzger wrote:
    Am Mon, Jun 02, 2025 at 04:34:12PM +0100 schrieb Peter Humphrey:
    On Monday, 2 June 2025 13:03:54 British Summer Time Stefan Schmiedl wrote:
    ------ Original Message ------
    From "Peter Humphrey" <[email protected]>
    To [email protected]
    Date 02.06.2025 11:52:34
    Subject [gentoo-user] Recovering mail

    Greetings,

    My user's .maildir has overfilled and so I've lost some emails. However, Ihave a cron job running every five minutes to rsync .maildir to /mnt/mailbu. The question is: can I just copy mailbu over .maildir?

    There might be some duplicates depending on e.g. the message state,
    which is encoded in the filename.

    Try rsync -avn /mnt/malibu/ ~/.maildir for a dry-run that should
    tell you what would be copied. If the results look reasonable
    stop dovecot and kmail, run again without the -n, start dovecot.

    At least that's what I would do to fix my maildir.

    Well, that's recovered all the missing ones (an awful lot of them,
    anyway), but I now have thousands of duplicates to find and delete.

    If you dare, you can use mutt. Go to the mail directory, then press Shift-D to mark mails for deletion base on a search expression. Use the edxpression ~= to mark duplicate mails. It will not mark the original mail.

    I'll post the story of what I actually did tomorrow. Interesting idea. I haven't used mutt in a couple of decades.

    --
    Regards,
    Peter.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Steinmetzger@21:1/5 to All on Thu Jun 5 00:20:01 2025
    Am Mon, Jun 02, 2025 at 04:34:12PM +0100 schrieb Peter Humphrey:
    On Monday, 2 June 2025 13:03:54 British Summer Time Stefan Schmiedl wrote:
    ------ Original Message ------
    From "Peter Humphrey" <[email protected]>
    To [email protected]
    Date 02.06.2025 11:52:34
    Subject [gentoo-user] Recovering mail

    Greetings,

    My user's .maildir has overfilled and so I've lost some emails. However, I >have a cron job running every five minutes to rsync .maildir to >/mnt/mailbu. The question is: can I just copy mailbu over .maildir?

    There might be some duplicates depending on e.g. the message state,
    which is encoded in the filename.

    Try rsync -avn /mnt/malibu/ ~/.maildir for a dry-run that should
    tell you what would be copied. If the results look reasonable
    stop dovecot and kmail, run again without the -n, start dovecot.

    At least that's what I would do to fix my maildir.

    Well, that's recovered all the missing ones (an awful lot of them, anyway), but I now have thousands of duplicates to find and delete.

    If you dare, you can use mutt. Go to the mail directory, then press Shift-D
    to mark mails for deletion base on a search expression. Use the edxpression
    ~= to mark duplicate mails. It will not mark the original mail.

    --
    Grüße | Greetings | Salut | Qapla’
    Please do not share anything from, with or about me on any social network.

    “Mankind must put an end to war or war will put an end to mankind.”
    – John F. Kennedy

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEVbE9o2D2lE5fhoVsizG+tUDUMMoFAmhAxZsACgkQizG+tUDU MMrSlxAAwcqm+bWiBapMeOVoQ3llSLvE/7W19lfQv6s9pa/Z+UcDJkX5/B6SUuXr 5bMmEMJdUNgFlUOqjuoL3nvwTbsXEf4U4aOFOdQ7icazHOy4wBEw/pI/1L4J88fV id4BHVBHAyrlsXauCGEyyH1RA0vVduMC81wZGF9Wycp4Ed1xjRCcXnuhZiaHOOrS MsCImJGIURd0RlK8OfhUr0NSOyyloC1Xoa81d02PCZmZlPDBAcOU2P88OKKNeM/K VzwK4t2eauU89Qv5xq8JIHJQqDXbFylhh3CI1tiueIbkjwfH0/8biXF6WGl1hfUV 2YcQFtsxsGLZopdWZRsGjeHiGRYpeo4vaV0QMHt22UM8rmfrSsHkPt2iDhTVeXwA ij5AzHtQfWr/RWDPT5ZdvF6+R8CAacS0k3rkBJgo3izKFNTy1cU2L4QiuxMbN9zs ZQiXv00z46ip74C3t6bMUU4YKrxGow466gsG7hCo02TEVpmwBv8eBBcXETEWgCVK Beq2+N+lTNRkyHHRMScHVsoyGu+adVfTUioh+8zhCwKo4iyVqTObdhtdwmXbWxl2 XRHircXnc9bhllRDTk8LW3XafzbpqWPW2DYcbha9/SEwo04hR4DQDRyuE4npNTUq 0NPvNPfGgiDNPwTdBP8JeZwH65nhC0uqQTXBkfs3ETofd
  • From Peter Humphrey@21:1/5 to All on Thu Jun 5 13:50:02 2025
    On Wednesday, 4 June 2025 23:15:55 British Summer Time Frank Steinmetzger wrote:
    Am Mon, Jun 02, 2025 at 04:34:12PM +0100 schrieb Peter Humphrey:
    On Monday, 2 June 2025 13:03:54 British Summer Time Stefan Schmiedl wrote:
    ------ Original Message ------
    From "Peter Humphrey" <[email protected]>
    To [email protected]
    Date 02.06.2025 11:52:34
    Subject [gentoo-user] Recovering mail

    Greetings,

    My user's .maildir has overfilled and so I've lost some emails. >However, I
    have a cron job running every five minutes to rsync .maildir to >/mnt/mailbu. The question is: can I just copy mailbu over .maildir?

    There might be some duplicates depending on e.g. the message state,
    which is encoded in the filename.

    Try rsync -avn /mnt/malibu/ ~/.maildir for a dry-run that should
    tell you what would be copied. If the results look reasonable
    stop dovecot and kmail, run again without the -n, start dovecot.

    At least that's what I would do to fix my maildir.

    Well, that's recovered all the missing ones (an awful lot of them,
    anyway),
    but I now have thousands of duplicates to find and delete.

    If you dare, you can use mutt. Go to the mail directory, then press Shift-D to mark mails for deletion base on a search expression. Use the edxpression ~= to mark duplicate mails. It will not mark the original mail.

    In the end, I had to recover /mnt/mailbu from the latest backup, dated
    Saturday last, so I lost a couple of days of mails; small beer compared with the hundreds I had lost (mails, not days).

    The process was, first, to stop fetchmail, postfix and dovecot on the server and
    kmail on the desktop, then squirrel the current .maildir away somewhere, then restore the mail backup, /mnt/mailbu, and copy that over .maildir. Then start dovecot and kmail and use kmail to remove duplicates (nearly 6000 of them, as the backup spanned the period before duplicates became less of a problem, and the time before I set up a POP to IMAP converter; kmail is weak with POP because apparently none of its developers have a POP3 account for testing).

    I could then overwrite the offline backup of /mnt/mailbu, the same as it had been but without anything like as many duplicates (nothing's perfect).

    After that I had to take kmail through all its 50 folders, removing lots more duplicates and moving mails to their current folders, which have changed over the 20 years. My filters were behaving oddly too, which I found was due to having three copies of them in the filter file I'd exported. Don't ask me how that happened. Mysteries unsettle me but I had to take that one on the chin.

    Remember that the mail backup contained every mail I'd received since March 2005; some 17000 of them.

    Finally, I tidied everything up by saving the current state, together with the archive in /mnt/mailbu.

    Who'd have thought email could be so complicated to manage?

    --
    Regards,
    Peter.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)