tl;dr: We need more docs about best practices to handle translations as
a package maintainer
My fellow developers,
there are two things in being a DD that I truly despise. The one is
keeping the machine readable debian/copyright up to date, the other is
handling of translations, regardless of whether it's po-debconf, manpage translations or program translations (when I am also the upstream).
I might rant about debian/copyright when I blow my fuse next time, but
today it's going to be translations. For me, it seems impossible to
support translator's work without putting a significant burden of
additional work to put on oneself. Especially when one uses version
control and does not do all development in Mast^wdebian/latest, dealing
with translations is a nightmare when it comes to merge. As the Newbie¹
DD I am, I keep running into either nightmare merges, or unnecessarily
fuzzed or even destroyed translations, in all cases feeling even more
stupid and incompetent when some translator points out my mistakes. I
have felt being sent back and forth between different workflows (all of
them wrong) by following random advice without being able to find
authoritative explanation.
I might have a fundamantal misunderstandig of procedures, but all
documentation one finds, including Chapter 8 in the Developer's
Reference (which links to a document by Tomohiro KUBOTA which is no
longer there), elaborate on how one would do actual translations, but
doesn't go as far as giving best practice documentation about what a
package maintainer is supposed to do to make translation blend into a
normal packaging workflow without being a nuisance ("put them into the
po directory and build the package" doesn't fit a modern packaging
workflow using version control).
My example package is adduser, but I think that my questions might apply
to other packages as well. adduser has both its program messages and the
manual pages translatable, the latter being done with po4a. I am aware
that there are also translations for debconf templates, but adduser
doesn't have those (any more). I think the problems that show with
debconf template translations are similar to the pain one feels with documentation and program translations. I actively avoid using debconf
in my packages because I don't want to go through the pain that handling translations causes, and many parts of Debian consider it bad practice
to use debconf. But that's an entirely different rant.
Handling translations does hurt when the sources are stored in version
control because I constantly end up having changes to pot and po files
in commits where they don't belong, or uncommitted changes that prevent
me, for example, from doing rebases. I have tried to build a workflow
that doesn't hurt me as package maintainer as much, but it has turned
out that this doesn't work because many translators don't care.
Please don't take me wrong, I know this is a rant, and I know that
you'll notice that I am typing this with my fists clenched. But my time
and my nerves and my mental health as as important as it is to be nice
to my translators. I do care, but sometimes it's really hard to maintain
a straight and friendly face while cursing our tools and docs inside.
Whenever I am angry about something in Debian, I start writing docs. So
I try this here, but here I don't know enough to be really helpful. I
hope that this rant will start a positive discussion with actual results
that I could pour into a Wiki page that might actually help with the
pain I am feeling, assuming that many other maintainers feel as well.
Let me try to summarize what I have understood regarding translations
and what my problems are with that.
(1)
When writing software, docs or debconf templates, the respective author
marks certain strings as translateable. There is a number of conventions
to do so which are language dependent. Let's assume that has been done
the right way, there are docs about this.
(2)
There is some point in the development process when it is time to ask
for translations. Translators need a POT file which contains all the translatable strings, and they make a PO file from that which contains
the actual translation.
(3)
Some program (xgettext for program translations, po4a for manual pages
and some podebconf tool for debconf templates) is used to pull the
translatable strings from the source code and to create a POT file.
xgettext doesn't even try to create a meaningful header and overwrites
whatever one has written into the previous version of the POT file, so a wrapper is already needed to have a header that translators can fill
(which they usually don't do).
For Adduser's program translations, my call to xgettext is:
xgettext --keyword=mtx --keyword=gtx --omit-header -o "$TEMP_FILE" --from-code=UTF-8 -L perl adduser deluser $(find . -name "*.pm")².
TEMP_FILE then gets the generated header prepended to result in
adduser.pot.
I have seen this being done in debian/rules' clean target which, in
in-tree builds, causes the POT file to be changed as well and I don't understand at which step of the packaging process it would be a good
idea to commit that POT file. If I build my package out of tree (like I
do out of tradition of svn-buildpackage, I have gbp configured to use ../build-area), the POT file ends up newly generated in the source
package but never gets updated in git. Adduser had POT files from 2022
in git until just recently because I just never noticed. There is no
lintian check and no check inside tracker.d.o for this.
In other packages, there is a dedicated m4 macro to call xgettext which
doesn't make things easier to understand.
(4)
Then,
msgmerge --update --backup=none --no-fuzzy-matching "${PO_FILE}" "${POT_FILE}" is called for every existing PO file. This doesn't move the header from
the POT file to the existing PO file so stupidities like "# COPYRIGHT
THE PACKAGE CREATOR" just never get fixed because the translators don't
seem to care.
If a po file for a language already exists during this step, the already existing translation gets merged into the new po file. In some
circumstances that I have not understood yet, the translation gets
"fuzzied", which I have been told causes a lot of unnecessary and
repeated work for the translators which I am supposed to avoid by doing
manual work myself which I don't understand. Not doing this work is
condemned as "not being nice to translators".
Basically the same applies for this step than for the POT generation
step, with the additional hardship that the PO files are generated,
being written to by a program AND STILL contain a significant part of
human work. I never know how much work of other people I am destroying
by calling msgmerge out of line. In which stage of package build do I do msgmerge? Do I commit the merged po files, when do I commit them, what
do I do with them during git merge when a feature branch is merged?
(5)
podebconf-report-po is used to generate the calls for translation. One
message is sent to this mailing list with the pot file attached, and for
each existing po file, the translators listed in that file get an
individual mail with just the respective po file attached.
If the msgmerge step is forgotten, they get an already translated po
file that doesn't match the pot and therefore is useless.
In theory, for an already existing package, the POT file is not needed,
right?
(6)
Depending on the age of the existing translations, about half of the
messages I send to individual translators are going to bounce. Am I
supposed to report that to
[email protected] as a followup to the
general translation request so that new translators can take up the
outdated translatorless translations? Or am I supposed to send the
general translation request to debian-i18n last so that I can explicitly mention the translatorless languages there?
(8)
When a translator does a translation, they send me a new po file
containing the actual translation. If it's a new language, they start
with the pot file that hopefully has the correct header, and if it's an existing language, they start with the old po file, which almost always
has a historically grown header that is in more or less dire need of streamlining and cleaning. They either take the PO/POT file from the
e-mail attachment, use a package the pulled from the archive³ or they
pull the PO/POT file from git.
They usually don't bother about the header or copyright, so things like
package name, licenseª, Project-ID-Version and PO-Revision-Date are
often questionable, unclear, just plain wrong or cause extra work to
package maintainer because, for example, a different license was chosen
than the actual package is licensed under either out of incompetence or
not caring.
Am I supposed to fix those headers in the po file myself? Am I supposed
to ask the translator to fix the headers? Or am I supposed to just
ignore all of that and just accept whatever I get sent? I often feel
like a smart-mouthed know-it-all when I ask a translator to improve the
headers of their PO file.
(9)
I then commit the po file the translator sent me to version control
(10)
And then I eventually release the package.
In theory, it would probably be good to do all that regeneration when
preparing a package for release. Why don't we have a debian/rules target
like debian/rules prepare-release that might be useful for that? How
could we protect us against uploading a package with outdated POT/PO
files? People make mistakes.
How am I supposed to handle the unavoidable differences between git
branches, that are probably easier to solve when I am just merging a
feature branch but can be a major pain when merging suite branches like experimental, stable, unstable where translation work has already been
done?
There must be some smarter method when merging to mas^wdebian/unstable
than (1) move away all po files, (2) merge, (3) ignore all merge
conflicts in po files, (4) regenerate POT, (5) restore po files moved
away in step 1, (6) msgmerge, (7) do a dedicated translation commit
(one? or one per file?).
I have caused enough breakage in adduser in the last weeks and have
wasted enough time of both translators and myself. For the time being, I
am halting all my efforts to be "nice to translators" to avoid breaking
more things and to keep the chaos in my package trees and version
control low. You can read from my git histories pretty well that I don't
know zilch about what I am doing.
This has to stop as long as being nice on translators means multiplying
my own degree of unthanked busy work.
This happens through a crucial part of adduser development since we are
nearing the freeze, but first I need to build knowledge that I should
have built 25 years ago but noone bothered to document. I really don't
know how translations in Debian have come up to THIS point in the
absence of serious docs. Maybe my fellow DDs are smarter than I am with
all tools involved.
Thanks for reading up to this point. Writing this message alone has cost
me three hours of my time that I'd rather have put in productive
packaging work, and a sleepless night. You know, when I blow a fuse, I
rant, and then I start writing docs. I guess when I put the result of
this discussion in a wiki page, it should be under i18n, right? I am
inclined to put on
https://wiki.debian.org/I18n a dedicated chapter
titled "for package maintainers", probably between "Keyboard input infrastructure support" and "Meetings" as this is a matter beyond
interna of the translation teams and the i18n effort. Am I on the
correct track with that?
Ich habe fertigǂ. Thanks in advance,
Marc Haber
¹ I have only been a DD for a bit more than two decades
² adduser has strings that get used in both translated and untranslated
form, making sure that messages written to the console are translated
and messages written to syslog are written in English to make handling
bug repors easier
³ I have received translations that were obviously done against the POT
file from stable.
ª I have received translations that placed the translation under the
same copyright as $SOME_OTHER_PACKAGE.
ǂ
https://en.wikipedia.org/wiki/Giovanni_Trapattoni#In_popular_culture
H4sIAAAAAAAA/81b7XPaSNL/vn/FbPZcceoQIN5xNnfLOk5ClWO7DL6t3F4qO0gD6Cw0rF7i cH/9dfdIIwEaAa489Vw+OCBrft3T0+/T/olx100iEbJ1KBchX7E45EHk89iTwQ8/sUu53oTe Yhmz88tX7NwL4NWYbQQPGQ9cxpN4KUO2FKF4BW9Pl17E5p4vGPzvelEcerMkFi5LAhdIxEvB Ir4SzPccAUiMR/RMs8CdR74QdUB6N76fTNnoYfrh9p79fPVxNL7+ZfT27f3VZPK3Gvt0NbqH t35YRQvPZS9e4Acghp9e3IXy38KJrbFr/UOEEWzjQhNo1+1O918BvHUv1jKMrY8IYP2aLCJr KvV7v6SMRHVXzDwe1GW4QOTbqXUZChKN9ZbH4oK1mq2u1WxbzR6z7Ytm/69Nu9kkAne31r34 6kX5u8i09fHWejtiH24vPo7/+s/bmyt695pHsTVN5S5DeFUmIbtBSf28gY91seKe/4v4xldr X9Qdufpbui5YJMCnNRV8dcE+btK9M/zOfo7h5/6iSxnEIgB6mzUwFYtvcWPtcy94zZwlDyMR v3mYvrMG2+8ib3MRWleBI10vWFywwcyL1Ub9JOS+9U6Gq+iCBWv6Gr1pvWbq45vzgP34htmv XuPrP2khX7TsfnaAt4G/YaGUMVvxDb7AOKMDA91ahDJZs1gq7dlEsVjViwe+BdlrMlf49Nke DjL4G8nkGk8tYtz35RMoJJ/HAB+AiKMKtKFGazV7W8zKQCBz8ZNUIBmwGayvASZr4XhzxEiB EIF5AewQzGYlXWFGGTQzFHikRPPkxUtihIeLZAXHFaH18QCMLiJKHuxXrkVIimuEbtutDHoK krYsQq/BBzhv9REtHp57bi7OULBVEiew+w0T3xwQlvfVzH572C3SWErYt+uFABHFbCaQaT6L pA8+A5xBvDQCddr9H36qMdiUb81B83hcxFVy+eMsegnHAgbrIm/gjaIaA3fEluB2ONCdg0aD uNj78ds6u/rmxbBRM8VO+zkUGdFSaqveqqDQex4FOBU48ACsZ4fS4U117fzz0K4gDzJSxFG1 UvqgsbDSDN5rGQDBHhEPob6CY+MzP1V/wcDRLAQ7c60zl51TEPgy+TT5gq9b7HqUf31VQbej P3ft5nEyfeJKgg769woz7vSbBfCuAXzkopMs4p/jDs7cV6xeN2MP+vqz3WrZz9EGI3i3Ofgu p1E4icpT6LZbufMcVFkrOXraiiuFOgXaSwV0L5dTs2kXhNa2NdFOp59/7h6pBsdy0LGLHHSP 2l3BcjhbidUM49ucfldBqFsgZNvV+pbTgnBZ2FWFznV77aOYL/Oju04H3z7sc7q9/jMoqihX 9NoPx3jtbr/Kigp8l9qShsfMxUBgYDp8SI3AsoDLBprMmkOYO8HhPZCZYY43yp9UWNtgoD/3 mrn/6xc+D+z8nUGrc9QhHOsWe03b7F0eTvP1D9u+vnLfvTxpuRGgIcB3mq2kKQpwDykk/H4B eYk5+emBPE7AoZi74o/iC73zBRJeiemWiCsoDKqNd08Zzx8Oh4xeu1WNGoinU8NQr9PO9WQw rNATnb+dRQzqFJanmzuGZLafXif35YNh61nEHB68VOmj44goEu4FvFNB0hgMCjIrOQbyQMd5 1X6zXXADD5Ore5XCoNqk3+5vH+7YTCIkaRTYhCODubdI0jxdFdMqkxdhKEMztWHn2LhQwfPA 7lea8MlurGDHlTY8aO0Ki4QzeRPIGkkLRffGsvekJ1breFNnI7W/QAgXc+116K14uEHfSYf1 o5FuT9MdQ5kbBtxXgoY9wdd1KND3w5ZDKM+wWoQSeuYFdDpGzH7nNHs8YIuDfv80vPw7iLwS eWCyghziJUKmrl+pv3IfRszh8CTL2rcqelappsO2rqPvRLjyIuywQKIXgCcwr9GtAC/4yn34 UDhLTMLSetYI0NFldxKIb+B5UCJzsIEkFDWMkEvcngvFvBmiVQWx5lH05KreGW0qWJiRdKwq rpol0abGYtB8vuBeYF7d25VF1jfArFEJwrxYC/IplGQcwIAMs7Aow/wJmM9mDVv0AkeGIWzX 3xhhjR65ICrAS0J0kq7AEKAIzkO5SmVnBtdtpmkmnL+z3zeNm8/MuGaQuwbVovQCxRjqS7of APnUCCpAhv1TrAGEHyXrtS/wKMAVNSB4xiFXthEddN9201gwTkRMnuzPRMacwe+2qYISJFD1 gBUcFSfsZndoDBRppojUKusau9mzjwHJgj6IW8LhqSQ9Al34E1iuSkiBQPuIZGILdyf5Z8hJ LBOHbJt6c3pFFd2huSD4jYcBdU9NPOj6cyZ8tK+07UkHg0kP9tsAx0lCrIH8jZZUBUN9U1Z1 WSXlSg3om4qeS5n4LmZjKmqUIWNyVgFtTDjxJgKZRWcXKcM/zKgxo4Rvj2QKf8y9wH1Jjlgl jkawVvN5pXJZf45KZTMlu/UMShV4re6B1t6Ze0Jjz261u+ZUsdCUTpHnMgkqLLXdNOVMZ1Hj jFLgIK3JsP4Pa6QCNarIUKAr3wseK+Db/YIFTiXkrhKDHsT/ReLFG8VqABEw9Bzwu5CrwqHh WtXYf1p6zpJuOljhXygiscL8FxbORBhh6AvEAuzpa/4MrZWO/cDdgN3u6L74gyZMTXHlC3Cf oPs+3TmAEGSCpEFinqxC7bYMqDPki60k8Bcvob5ot9hsE8Nv4eR9ESwqOu+AOiyT5zqUwNMK RJc5LLrV0JuIYh7GaRdnT5ouj5Y1vC+KYJ+LoEZXK57vitR64hwHiqQYIjg832DIAmJ7aHOJ 8iY/60hwozXM9la8xiKf6KAMYTUcLGx5zR1BN18csoww2gOD9MVPVIGOr+L++Cy1agjgoHsV N0h2u985QVjREt0nbjECg07vh+Z7LPkQz4HVGhjDwkMPQ7erEaQlyJ7SCuQwJmGqCwKUsFBs 7+HtHg+9zM4hzroCvCPkXrMNG19dXbFJ7DK72WzXbavVbNqv6uydDPfwQNprMISZ52vzmoC5 cbbiGEupTgeXgJz/RbmdSOU+MoSQvgcH+0ApCdiFOnAtMIjR56DZ9OwPyyIzw58WqfpLjJQq n93DhLg622DeqOI6CC4G66e6+5X5MAf6MEephiEryBWlI7CaSmLNXgVSz5wk3PmCR7jfmDxe bko8zlIRqLfFIvF5CK4f6lSqgUqOQbUT4Py+epxWgTfdajJA7hd6yDNIMhXkHowW7Iy7mWDT nhhIMRQ+/6aE6CyF84jGBdE+I72HdhbtdzqMYuoYL1gmwoeUAoWBXQZKBrIOBHIFPyEtMLeA 7E57cBD5vRnZjGu889riuPEc7K4xTt6LlfxKBejxOVy3OcivR4xdRI18XGnQtfNLn07/ILvH NdPsbsuYwfIER1M4m4zfV2WVXWPzFdQzGw75qqZF0t7hiwrt6baKIRCquejAvIIGfAH+EiQU U7YJZpAN1xTKS4yD5ABCMceZGUnrGkkUNiIIU6LhSqeRstLQCPWDPLe7+mi67a2ohMabTf/g /E4ogHE5j5+A2msqOxzIEcCP6IEejMgQSRqwmZV0vfmGqMPDfM4HnNcqyhz2+5sH9l4EIoT8 6g5yF89h1/kI0BqfQHjCOENAuOQdcjFJuYAwA8gkntcszQWz42plRFJEiu/nXA2n4PRM6q1e UdD3eZyvLZzLrhCKw0tpi3Mp15Quxbj7J8/3MY0Cec4Tn+7DCOe38fTD7cOUjW4+sd9G9/ej m+mn1xQDZRIzgT0Laies1j62rmFz4AFiHeY/Xt1ffoA1o1/H1+PpJ9zKu/H05moyYe9u79mI 3Y3up+PLh+vRPbt7uL+7nVyB654IkYmZUMpFXWMFJcKMSAZWOogVNd7fXdeoHqKkUGloVd/e 7naGJTb0u2Ul8MhzP+PHuRdGceE7pF/Z163A8LsaKimuWmyvKnzN7mXo5x4OMGJR30SNr0S0 BHQLTot+7i1AQWDvK/2f3s+qVvoSSEsVs/R4n+B20vE5f6bj5d4aUC4Muq6VNcs+bz315cIL ShgN5lT50Mt/Jp5QzIIuz2Sknrpiliy2V+Kx6AcjmqzKcgf9K/qRHSGIi5zWLv3COSrR7x9G 4SB3VxeP4HuI/TtIQwmjUJCXyYI2Vqas47eHlPU7sZxzkE18/Y+wo0VIosqfb/FqVKf/c66K k1D7J4v3SIyukbZWBaqXojMeCOPFZxps2ynqkcNeq3yiMcSMR+wlCZQBVow16vmZ/M4DeB1j n91VzCVp9WNZigQZSy3/in6JGmbZ8N6MO49AGWDmYFwJVYwY4fH2DAG9ACpBsFQqqTBfeqmn chmWrw4WdxitKRLWWJgECMbXsbUQsV6OK83b6fePHZg5Yb4kHzaynzFspCZaMmWluQc1MXCY 3rB4fQkx/2Z8855GeKn/829sXPAZRn8QLywSsToyUo60Embn6F2bUE8/RGqWkioqzKmw+QXK Ty1vD2s54MzLtEpnW09L6efzLEESqaJuPKfDBT+KmAD3xPGaB7BrkNPBOaWbUEV6ftNJ7hcL SxGCEiGndaj95XpN1xbyiaDSnGbJKZcHeaO0kUFIs7hTdo2Uiazd1O0upadfyPVT6UhfgbUv qs87ExALg5LbsBxL28a1lI/UIQYgtRrLfFL4RmqBJWWGBjJOV6nesLodg/ytAWmi01hRB6Os YawBjTM2OaDjg1M7HrHbqbg0wZ2S40p3m13C1FBfsM3CiAJbSy+IzWKovJippEENCpB6SSWm 0fs9YykXoDEuwU2hJuNwB2SwdLNZIRDj9NWvmlHqQGttEKmKqYI7qlKITnOwV7EqoKpF+VC7 XuSE2LOcVS5rdQqfTe6R/JZyWcLB0gSs8tFTVpkRoe1xv87ulNtmf+BvXmr/YWahayILjn2B LSjlN/ObknRERXUt8dJnydfrkiErTaF35Aj1idPMOX7/JHyaH/nRiDboPWdo1ZwddIbPG0Kt QtR1PIYbjAm4LPV1+uIwSzO8UA/HlE+i6/7AkfdP+pzyadZ8GxXoveO7TcR7dZ9I4xrtBqJV FugONnlytEFhOXEGp4K/Ug319Fp+N4X7/+v0aMY7g5J9/56lNpRBF5LF4nedLdJDFTn3MnT1 2MJkJi3V0idRMp973xiw9Hk7k99FqMrqdwoxlqfOxap1V+z5gmy/aeW2s2+8UbG8uUWW/9lU b+zyW8JlXlUAwe9E6SCpgjBK64d9KZxUUO3WcazckfBDVVB3mMeynrFn/asqRFKlgUicBHqo EK9fqdz6j1fmpPBP8hqxiOKo4XuzxkhZyBQfXFJbq75eQRS1q2YUyHHh1b9u+mGrxk39WsD+ cjeafihxvSmxnI7db1ZF6x1fDvk99enFnCd+yZV9CbzJWToqYaKUdPtSJ02aLmgzx2xh0Nt/ 2OoOSh4aY6Ie/Fjj3xCmOSFejkL6dcQujX979BA8Qr2RX1Slgw/xqRRaxmQ8TTxx8mHn/uQw aHt4wtkfhusc4JHOei2lv33Eh4GNc5lj3xcLyLMJFOf1lHxRGMYZrj34tnEw5m2y9j0HB4Ho BvOZZ9du9g/iZ3+e9gz0TsVfcKTSmMMSl/ygGjtUA8ccWy2qbbDTBcbMKG0fpH9F6wWJKMtd S9gZPJ+dwtA4MLZAR5bmD1APqFyDNIcjW/hCSH8GfQRX3VbJw16vSv3VwCYwqoaq1eCma07j S4j2yohW2hyYGs3VYqTC6QfQ7dNoGrtHheIHa8e0BKtTHZSe8JEU2lUUHj3fVxMXBfZPw+8e sQNq+EQxj5PoZArmAvWE0HqQivm629FUyCH60nnMHOKPh2Ht5j63ckYzRfoPmAlTp+d6phpL 3TD2N8dQ0U3B3zipHnWliFVqQzjY8NIJT0lVsw/Zah6URyTEI7OOk0N+h+1sn5qW5xEg7cNn lATFAyprb5XADg/Cqs7ZkdDXcrFA1cZUo2vyr756iQaxNxF8060OGh9DyVBMiXY82/5fG/0X QJBKVlVDAAA=
--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)