Dear mentors,
while updating SCTK package I enabled the execution of the test suite
which was previously disabled. The tests are working fine on x86_64 architecture, but a couple of them are failing on i386.
After investigation [1] I found out that tests are failing because they
rely on the assumptions that, when a and b have the same double value:
1) "a < b" is false;
2) "a - b" is 0.0.
Unfortunately, these assumptions do not always hold on i386 as the behavior
of those operators seem to change according to compilation flags (e.g., optimization or debug flags) or the surrounding instructions (e.g., if a function is invoked nearby or not).

I think the reason for this behavior is probably related to gcc
options -mfpmath
[2] and -ffloat-store [3]. Using -ffloat-store option indeed seems to fix
the issue.

However I am wondering:
1) if enabling -ffloat-store is an acceptable solution;
2) if this is a compiler bug (I always expect approximated results with
double arithmetics, but the above assumptions seem very basic to me, considering that they cover the special case where a and b are equal);
3) what is the most appropriate solution.

Do you have any suggestion?

I just asked the same question upstream, but I would appreciate your
opinion as well.

Bests,
Giulio

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981030#39
[2]
https://gcc.gnu.org/onlinedocs/gcc-10.3.0/gcc/x86-Options.html#x86-Options
[3] https://gcc.gnu.org/onlinedocs/gcc-10.3.0/gcc/Optimize-Options.html#Optimize-Options

<div dir="auto"><div dir="auto">Dear mentors,</div><div dir="auto">  while updating SCTK package I enabled the execution of the test suite which was previously disabled. The tests are working fine on x86_64 architecture, but a couple of them are failing
on i386.</div><div dir="auto">After investigation [1] I found out that tests are failing because they rely on the assumptions that, when a and b have the same double value:</div><div dir="auto"><div dir="auto">1) &quot;a &lt; b&quot; is false;</div><div
dir="auto">2) &quot;a - b&quot; is 0.0.</div><div dir="auto">Unfortunately, these assumptions do not always hold on i386 as the behavior of those operators seem to change according to compilation flags (e.g., optimization or debug flags) or the
surrounding instructions (e.g., if a function is invoked nearby or not).</div><div dir="auto"><br></div><div dir="auto"><span style="white-space:pre-wrap">I think the reason for this behavior is probably related to gcc options</span> <span style="white-
space:pre-wrap">-mfpmath [2] and</span><span style="white-space:pre-wrap"> -ffloat-store [3].</span> <span style="white-space:pre-wrap">Using -ffloat-store option indeed seems to fix the issue.</span></div><div dir="auto"><span style="white-space:pre-
wrap"><br></span></div><div dir="auto"><span style="white-space:pre-wrap">However I am wondering:</span></div><div dir="auto"><span style="white-space:pre-wrap">1) if enabling -ffloat-store is an acceptable solution;</span></div><div dir="auto"><span
style="white-space:pre-wrap">2) if this is a compiler bug (I always expect approximated results with double arithmetics, but the above assumptions seem very basic to me, considering that they cover the special case where a and b are equal);</span></div><
div dir="auto"><span style="white-space:pre-wrap">3) what is the most appropriate solution.</span></div><div dir="auto"><br></div><div dir="auto"><span style="white-space:pre-wrap">Do you have any suggestion?</span></div><div dir="auto"><span style="
white-space:pre-wrap"><br></span></div><div dir="auto"><span style="white-space:pre-wrap">I just asked the same question upstream, but I would appreciate your opinion as well.</span></div><div dir="auto"><span style="white-space:pre-wrap"><br></span></

<div dir="auto"><span style="white-space:pre-wrap">Bests,</span></div><div dir="auto"><span style="white-space:pre-wrap">Giulio</span></div><div dir="auto"><br></div></div>[1] <a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981030#39"

target="_blank" rel="noreferrer">https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981030#39</a><div dir="auto"><div dir="auto">[2] <a href="https://gcc.gnu.org/onlinedocs/gcc-10.3.0/gcc/x86-Options.html#x86-Options" target="_blank" rel="noreferrer">
https://gcc.gnu.org/onlinedocs/gcc-10.3.0/gcc/x86-Options.html#x86-Options</a></div><div dir="auto">[3] <a href="https://gcc.gnu.org/onlinedocs/gcc-10.3.0/gcc/Optimize-Options.html#Optimize-Options" target="_blank" rel="noreferrer">https://gcc.gnu.org/
onlinedocs/gcc-10.3.0/gcc/Optimize-Options.html#Optimize-Options</a></div></div></div>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Giulio Paci wrote:

3) what is the most appropriate solution.

As I understand it, floating point values should not be compared
without some kind of accuracy/precision factor. Zero idea about the
best reference for how to do it correctly, but here is a random one:

https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/

--
bye,
pabs

https://wiki.debian.org/PaulWise

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEYQsotVz8/kXqG1Y7MRa6Xp/6aaMFAmGe8wAACgkQMRa6Xp/6 aaM8sxAAg3SqBfoVFj5zoslv9PwQfQfRp12U8YpTLXJdezTFwKkGYcwMUy6AAuIx TpykCFXbk054pXsOHagqu88jdZzfkRc0Njg+479UCBmpSPrL+r8/fQOZxWXu4ITh xzE/liuVv/CNF9siyV+qxQS65CoEKm8E8bGDAP7fFamSX7EZlrBhUBUpYqE9TbQG p2JCF8ij/pRJCL2Vbw5XOnfvekqTKbe9pZ+sSVBZIIhHMEWwl3mGS812n4kedMSp zI17uJlylVpIm3WtTQ6M4oECGBUd2ajjAxniJOSdmrLA2kNyPaNgSLACpeXcU6Ob 06hARcf+vbKb3VdEw7rVI1GDkB1kS5tvO9KZCRW25KTc2/awB395djTfGRUWstO3 /BctNN9z33hZZOkgVhVWDv1VeLRnJixqwf247+nzIAji8Wnp6FwwBi3CUJjCKMzo UH4zYBjcLcoONLeCDuGBjEXVOD8jfr8NN1slDQSM+XYeGcwmRvmj/c4MKCSE3WVI ZUmDYcRVZxIFVxg0V3gYX/hZNSFfIIuEDV+Z2DXvqDIxf6ClRhUAA/rjTXZZlEva uAcOpl9tKJIrKgbVyDhUdKb6hdA/l08TAjD+hcj8M3RmAr28/VDf7s2ie8o3olUO xAYmkfSNUpULTx2+agy2pcxfS9xztrz3a1ehFGrTqFNY87hyHkg=
=GzpC
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, Nov 24, 2021 at 06:38:07PM +0100, Giulio Paci wrote:

Dear mentors,
while updating SCTK package I enabled the execution of the test suite
which was previously disabled. The tests are working fine on x86_64 architecture, but a couple of them are failing on i386.
After investigation [1] I found out that tests are failing because they
rely on the assumptions that, when a and b have the same double value:
1) "a < b" is false;
2) "a - b" is 0.0.

What do they actually test, why do they use these assumptions?

--
WBR, wRAR

-----BEGIN PGP SIGNATURE-----

iQJhBAABCgBLFiEEolIP6gqGcKZh3YxVM2L3AxpJkuEFAmGfQSEtFIAAAAAAFQAP cGthLWFkZHJlc3NAZ251cGcub3Jnd3JhckBkZWJpYW4ub3JnAAoJEDNi9wMaSZLh qIIP/3s+XVnD0/NunzKDNVqWdubQR2Ce5MUgapxaFDuaKwR5mT/N0GfCp6ro88Pa ZH2ODuYCceBLCgLc6qSlLPJIYUes49mSWzoO1HOKfzQ97J8x+X5t/goLBzppakvr vC3jDDl+sqeI4ifuEm93tifZ5N9jRkURZOEMYtNgwF2vuMg3HyKWzdfzzyY2jmuo HPEsd7BBPdYtOjpURE7apv1AIosu2YrpveilNeNvxnO6/IJizzEXMLzm72Stae6P jFy9tn2WaK02ySh2Ow0LWNT/GRoGeomNMYms/cHDLGdWhHj3o/FkgPQ8HcR6ifCD xw1ni5vPPIeqpj2YrIMlTlUt8INd/zI0XA12GUbkScjfogTjAu+8ocFRpZ60sBVz 7/ARiM2pECghiWpZutpXq3TZTKYF/5OSgfms0HcvIYZvkrBTCJ3+0IG0TeprbXE3 qmfugEr3wyNLWaC2fIdGZX+DKErv8BVf3eOjET9sR5jZt5eOFiwiTCQm113phZNg qceyUh3PVEBbI07xqugiXm/xiOjQWZiY8oTACtxDZIHMXDRqSTxtNMejGqzgL60q g+EBc3GjH/QkiAuk4pv1pT+Keqe3hizQzcJ5q+p7vQcWGe6HnztbmDH5ceRWDX4C JQj79L0Hkv1JPAyHwrMSQUgW9leLT4HifmtdXElJi7vL+tYR
=748J
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, Nov 25, 2021 at 01:13:20PM +0100, Giulio Paci wrote:

The double values refer to timing information. The specific format,
known as CTM, stores information in seconds in decimals (e.g. "30.66" seconds) from the beginning of the stream.
The failing tool reads this information into double variables

Sounds like it just shouldn't read this data into a float type but use
some fixed-point data type instead.

--
WBR, wRAR

-----BEGIN PGP SIGNATURE-----

iQJhBAABCgBLFiEEolIP6gqGcKZh3YxVM2L3AxpJkuEFAmGff8AtFIAAAAAAFQAP cGthLWFkZHJlc3NAZ251cGcub3Jnd3JhckBkZWJpYW4ub3JnAAoJEDNi9wMaSZLh U7oP/2mNUp5BQ7pw60UzE33RFPMSfvPplv0Uz7JlsBbjRzYXkuk6l8cbE8CSi1JC SEjG5yN4x5xxPV0mHhUj03Z0Zn+gjvy2tzA7yTSZtzwcR9oi/Upq3VbGHGFGj8H4 QEori7uvVjxmb6ndQ4gpTmVwAglM8/Oput36rnvgPa1ULVGo0YhdfPmz9wEsCIMX 9MWKhcq1xiCavFp9DtwF1sRo184j/VWPDNrrimUVGwKDnNgbl7Q1daapxpj6FlQt oNX50f9gcDnspMLIgUEr2azRaLoMa7VfiIZDcbjAgsgNUdS9ND+FULIY8ITdsfqN Bvqc2qy5H6D9DQmIL5dwZDDDPkLBsOz3ZK+yR5bE5Xd+IaeAheFMsj7GN3TAzKPm TQNss4p5C+1+r291aXO4pp5RPkRxbO1HwQa1XSe/DLX6CaH3sJqd3SWhT5ExwdRl Q6p84hLWfBazr8oc+GOxL5tQO5JKWmW9qr+1cJcFjyeC6ixyh4lfzG8PxbQYIa9t FADS5qAvOf1/E/NNi28UqBhSUJMNokpwjRQh2XGvUWMC7nfqDmbCt8C/J3ocf0HU Rh6Xe66IIPuDQ2PeNGiPykIv/S69a+gP8X6dCjQwFr+FsTegEjzknAti47PAGg+1 CUYz8Ij2iC5zeY7/fBCud4HFelbnzEmv6Fbiqe2KM3HBKHWL
=XaeD
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Il gio 25 nov 2021, 13:21 Andrey Rahmatullin <[email protected]> ha scritto:

On Thu, Nov 25, 2021 at 01:13:20PM +0100, Giulio Paci wrote:

The double values refer to timing information. The specific format,
known as CTM, stores information in seconds in decimals (e.g. "30.66" seconds) from the beginning of the stream.
The failing tool reads this information into double variables

Sounds like it just shouldn't read this data into a float type but use
some fixed-point data type instead.

This is also my opinion (and already suggested upstream), although it would make the code a little bit less readable.

Moreover I am still wondering if the compiler behavior is correct in this
case and why it is so unstable. Apart from this corner case (which in my opinion should also work), I have not seen bad assumptions about double arithmetics in the rest of the failing tool.

Bests,
Giulio

<div dir="auto"><div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Il gio 25 nov 2021, 13:21 Andrey Rahmatullin &lt;<a href="mailto:[email protected]">[email protected]</a>&gt; ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Thu, Nov 25, 2021 at 01:13:20PM +0100, Giulio Paci wrote:<br>
&gt; The double values refer to timing information. The specific format,<br> &gt; known as CTM, stores information in seconds in decimals (e.g. &quot;30.66&quot;<br>
&gt; seconds) from the beginning of the stream.<br>
&gt; The failing tool reads this information into double variables<br>
Sounds like it just shouldn&#39;t read this data into a float type but use<br> some fixed-point data type instead.<br></blockquote></div></div><div dir="auto"><br></div><div dir="auto">This is also my opinion (and already suggested upstream), although it would make the code a little bit less readable.</div><div dir="auto"><br></div>
<div dir="auto">Moreover I am still wondering if the compiler behavior is correct in this case and why it is so unstable. Apart from this corner case (which in my opinion should also work), I have not seen bad assumptions about double arithmetics in the
rest of the failing tool.</div><div dir="auto"><br></div><div dir="auto">Bests,</div><div dir="auto">Giulio</div><div dir="auto"><br></div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #
ccc solid;padding-left:1ex">
</blockquote></div></div></div>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, Nov 25, 2021 at 8:54 AM Andrey Rahmatullin <[email protected]> wrote:

On Wed, Nov 24, 2021 at 06:38:07PM +0100, Giulio Paci wrote:

Dear mentors,
while updating SCTK package I enabled the execution of the test suite which was previously disabled. The tests are working fine on x86_64 architecture, but a couple of them are failing on i386.
After investigation [1] I found out that tests are failing because they rely on the assumptions that, when a and b have the same double value:
1) "a < b" is false;
2) "a - b" is 0.0.

What do they actually test, why do they use these assumptions?

SCTK is a toolkit to evaluate speech recognition (and other related
tasks) tools performance.
These tools usually read audio streams and produce simple text files
containing the transcriptions and time information (relative to the
stream) to synchronize the transcription to the stream. These files
are very similar to video subtitles files.
The SCTK compares two textual files (usually one is a manually created
file and the other is created by an automatic tool) to score how
different these outputs are.
The tests are checking that SCTK produces the same score reports when
provided with the same input files.

The double values refer to timing information. The specific format,
known as CTM, stores information in seconds in decimals (e.g. "30.66"
seconds) from the beginning of the stream.
The failing tool reads this information into double variables and, to
simplify, it compares "up to when the timings in one file is less than
the timings in the other files. If it exceeds or is the same, it
checks the difference".

In this kind of application you are not usually going beyond what you
can store uncompressed on a filesystem in PCM. So, even assuming audio
samples of 1 byte, int64 should be a reasonable type to store timings
(in samples, rather then seconds). But I understand that doing so
would complicate the logic of the tool, especially since it is very
unlikely that math approximation would be an issue. To be honest I did
not expect the corner case above would fail since it is comparing a
value against another value that should just be the same.

I have uploaded simplified code that showcase the issue and some of
the instabilities [1]. The code seems to behave as if the last value
is different from the other 3, supposed equal values.

[1] https://pastebin.com/embed_js/T3g560UV

Bests,
Giulio

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, Nov 25, 2021 at 01:45:49PM +0100, Giulio Paci wrote:

Moreover I am still wondering if the compiler behavior is correct in this case and why it is so unstable.

It's correct when you don't care about the amount of precision, and it's unstable for the reasons described in gcc(1) for the options you
mentioned, e.g. "This option prevents undesirable excess precision on
machines such as the 68000 where the floating registers (of the 68881)
keep more precision than a "double" is supposed to have. Similarly for
the x86 architecture. For most programs, the excess precision does only
good, but a few programs rely on the precise definition of IEEE floating point.", as in different circumstances the temporary values will have or
not have the x87 80-bit precision, leading to different calculation
results.

--
WBR, wRAR

-----BEGIN PGP SIGNATURE-----

iQJhBAABCgBLFiEEolIP6gqGcKZh3YxVM2L3AxpJkuEFAmGfhzktFIAAAAAAFQAP cGthLWFkZHJlc3NAZ251cGcub3Jnd3JhckBkZWJpYW4ub3JnAAoJEDNi9wMaSZLh lVMP/jizCnq2M07sjlwahqomMVASRLgv+d7BougsVAXQGTrrbKbxrbDrbUQXU/JC UDOZ6dwoGIhXx7iXPdYjyOmhqx7U6YdE4D7ajDzPr3XxwjOxm4lXbbv2kUhkjfTe 1o18EZNr8AUIl5e+SCzba65Grp1mw3plaXLxDf5un0LgJtQmOGBCZVQ6cI51Mvct mjMLcBEof+Ck+P2e5L0E23EZHwTxuIcb3JT7uSBQpdemkSlhg0CH/kK1JbXiQ3j+ Qp58lcooe92uXin8K4qzoh6H0TA+ooUtW0Tyb1usIku+l4yKGrz7mlrai0c3FRrf KMtrhnJ1bjXFGA49ImO6FccMBQqTha2e8Sx6aIxu6SSj8bv1nYogrrOpArsfKNAU IHs6v+8RcJD1k+FZrTf3OYn4tfCrR59/+rzKhj2HS8+odsXhGDJ1WBzXQwYnpyin UdaT7QJPOmM0Dez36bbtstY5OPU4ibODgg/fPMtAJ4V39mJgQ0aCP75u9Tm1pidy nIMKD6Hm3IQ9wNDn88bdljRdoxJeD7O+HOWcldepusoWFS841iaMoyplvBG5UUTc O1uiSmuQQLTdIaoZpigVDpWV+SdAiGOMfnx4KZ9zFi3OZ23teNxWTGf/Mjyz2VN+ ChiqQCBu7GizO7LSegg2GauY6LaRTa6imvZvF8UWVAk4PiDx
=FbPs
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Paul,

On Thu, Nov 25, 2021 at 3:24 AM Paul Wise <[email protected]> wrote:

Giulio Paci wrote:

3) what is the most appropriate solution.

As I understand it, floating point values should not be compared
without some kind of accuracy/precision factor. Zero idea about the
best reference for how to do it correctly, but here is a random one:

https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/

Thanks for this link.
It is a very great resource and summarizes very well what I already
know about double/float and much more.

Since the test case is dealing with timings, I think the most related
article is [1].
However even after reading that article it seems to me that in this
case it should be reasonable to expect stable behavior of those
operators.

I have uploaded simplified code that showcase the issue and some of
the instabilities [2]. The code seems to behave as if the last value
is different from the other 3, supposed equal values.
I am not even sure what I am seeing in the debugger, since most of the
values are optimized out (and I am not so skilled with debuggers).

[1] https://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/ [2] https://pastebin.com/embed_js/T3g560UV

Bests,
Giulio

<div dir="auto"><span style="font-size:12.8px">Hi Paul,</span><br style="font-size:12.8px"><br style="font-size:12.8px"><span style="font-size:12.8px">On Thu, Nov 25, 2021 at 3:24 AM Paul Wise &lt;</span><a href="mailto:[email protected]" style="text-
decoration-line:none;color:rgb(66,133,244);font-size:12.8px">[email protected]</a><span style="font-size:12.8px">&gt; wrote:</span><br style="font-size:12.8px"><span style="font-size:12.8px">&gt; Giulio Paci wrote:</span><br style="font-size:12.8px"><span
style="font-size:12.8px">&gt;</span><br style="font-size:12.8px"><span style="font-size:12.8px">&gt; &gt; 3) what is the most appropriate solution.</span><br style="font-size:12.8px"><span style="font-size:12.8px">&gt;</span><br style="font-size:12.8px"><
span style="font-size:12.8px">&gt; As I understand it, floating point values should not be compared</span><br style="font-size:12.8px"><span style="font-size:12.8px">&gt; without some kind of accuracy/precision factor. Zero idea about the</span><br style=
"font-size:12.8px"><span style="font-size:12.8px">&gt; best reference for how to do it correctly, but here is a random one:</span><br style="font-size:12.8px"><span style="font-size:12.8px">&gt;</span><br style="font-size:12.8px"><span style="font-size:
12.8px">&gt; </span><a href="https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/" style="text-decoration-line:none;color:rgb(66,133,244);font-size:12.8px">https://randomascii.wordpress.com/2012/02/25/comparing-
floating-point-numbers-2012-edition/</a><br style="font-size:12.8px"><br style="font-size:12.8px"><span style="font-size:12.8px">Thanks for this link.</span><br style="font-size:12.8px"><span style="font-size:12.8px">It is a very great resource and
summarizes very well what I already</span><br style="font-size:12.8px"><span style="font-size:12.8px">know about double/float and much more.</span><br style="font-size:12.8px"><br style="font-size:12.8px"><span style="font-size:12.8px">Since the test
case is dealing with timings, I think the most related</span><br style="font-size:12.8px"><span style="font-size:12.8px">article is [1].</span><br style="font-size:12.8px"><span style="font-size:12.8px">However even after reading that article it seems to
me that in this</span><br style="font-size:12.8px"><span style="font-size:12.8px">case it should be reasonable to expect stable behavior of those</span><br style="font-size:12.8px"><span style="font-size:12.8px">operators.</span><br style="font-size:12.
8px"><br style="font-size:12.8px"><span style="font-size:12.8px">I have uploaded simplified code that showcase the issue and some of</span><br style="font-size:12.8px"><span style="font-size:12.8px">the instabilities [2]. The code seems to behave as if
the last value</span><br style="font-size:12.8px"><span style="font-size:12.8px">is different from the other 3, supposed equal values.</span><br style="font-size:12.8px"><span style="font-size:12.8px">I am not even sure what I am seeing in the debugger,
since most of the</span><br style="font-size:12.8px"><span style="font-size:12.8px">values are optimized out (and I am not so skilled with debuggers).</span><br style="font-size:12.8px"><br style="font-size:12.8px"><span style="font-size:12.8px">[1] </
span><a href="https://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/" style="text-decoration-line:none;color:rgb(66,133,244);font-size:12.8px">https://randomascii.wordpress.com/2012/02/13/dont-store-that-in-a-float/</a><br style="font-
size:12.8px"><span style="font-size:12.8px">[2] </span><a href="https://pastebin.com/embed_js/T3g560UV" style="text-decoration-line:none;color:rgb(66,133,244);font-size:12.8px">https://pastebin.com/embed_js/T3g560UV</a><br style="font-size:12.8px"><br
style="font-size:12.8px"><span style="font-size:12.8px">Bests,</span><br style="font-size:12.8px"><span style="font-size:12.8px">Giulio</span></div>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)