• tcl/tk 8.6.12 (built by VS 2022) exec command extremely slow

    From [email protected]@21:1/5 to All on Mon May 1 02:00:19 2023
    I have this very strange problem, and is wondering if anyone has this experience.

    I use Visual Studio 2022 to build tcl/tk 8.6.12 with some customized command bulit into tcl. But I have been doing this for quite a while without any problem.

    For tcl shell 8.6.12 and 8.6.13(with tk86t.dll loaded), use exec to run an external programs will take more than 3 times of executipn time than using tcl 8.6.0 (from star to end of the external program).

    Is there anything I should be aware of for the exec command in tcl 8.6.12 ?

    Also I found there is an error during the bulding of 8.6.13, which is related to
    "git rev-parse HEAD"
    It doesn't seem to affect the final outcome of tcl and tk, so I just ignored it. What is this "git rev-parse HEAD" for anyway ? Should I be cautious about this ?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From S-Y. Chen@21:1/5 to Francois Vogel on Mon May 1 09:11:40 2023
    On Monday, May 1, 2023 at 11:18:54 PM UTC+8, Francois Vogel wrote:
    Le 01/05/2023 à 11:00, [email protected] a écrit :
    I use Visual Studio 2022 to build tcl/tk 8.6.12 with some customized command bulit into tcl. But I have been doing this for quite a while without any problem.

    For tcl shell 8.6.12 and 8.6.13(with tk86t.dll loaded), use exec to run an external programs will take more than 3 times of executipn time than using tcl 8.6.0 (from star to end of the external program).
    Did you by any chance switch from Win10 to Win11? I have seen something resembling your report when I did (there was no delay on Win10 Tcl/Tk
    built with VS 2022). A 5 seconds (timeout?) delay got introduced in my
    app. for no obvious reason (Tcl code unchanged). I didn't go to the
    bottom of this problem though, I resolved it in quick and dirty mode by
    a judiciously placed update in my script.
    Also I found there is an error during the bulding of 8.6.13, which is related to
    "git rev-parse HEAD"
    It doesn't seem to affect the final outcome of tcl and tk, so I just ignored it. What is this "git rev-parse HEAD" for anyway ? Should I be cautious about this ?
    I'm building with VS2022 on Win11 and I'm not seeing an error. Could you show how exactly you are building (show the commands you output), along
    with the the exact error?
    From a quick look, "git rev-parse HEAD" is used in makefile.vc to put
    the commit hash of the bleeding edge (i.e. HEAD) of the current branch
    in the manifest.uuid file. Sounds like we need git in the path for this
    to work, perhaps that's the issue you are seeing.

    Regards,
    Francois

    No not Win 11. I built and run it in Win 10. And it is not only a delay problrm. If the same external program run for 200 seconds in exec of 8.6.0, it will run about 700 seconds in 8.6.12/8.6.13. So I beleive there is something to do with how exec handle
    the external programs in the OS (or the opposite way).

    I use the following building commands (I need multi-thread in tcl. I remembered I got it from some old version building instruction of tcl)

    nmake -nologo -f makefile.vc OPTS=threads MSVCDIR=IDE MACHINE=AMD64 -a
    nmake -nologo -f makefile.vc OPTS=dll MSVCDIR=IDE MACHINE=AMD64

    The error message related to git is like
    (begin)
    git rev-parse HEAD >>C:\...\Tcltk_8_6_13\tcl\win\..\manifest.uuid
    'git' is not an internal or external command, excutable program or batch file NMAKE : fatal error U1077: 'git rev-parse HEAD >>C:\...\Tcltk_8_6_13\tcl\win\..\manifest.uuid:
    return code '0x1'
    (end)

    Regards
    Chen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Francois Vogel@21:1/5 to All on Mon May 1 17:18:49 2023
    Le 01/05/2023 à 11:00, [email protected] a écrit :
    I use Visual Studio 2022 to build tcl/tk 8.6.12 with some customized command bulit into tcl. But I have been doing this for quite a while without any problem.

    For tcl shell 8.6.12 and 8.6.13(with tk86t.dll loaded), use exec to run an external programs will take more than 3 times of executipn time than using tcl 8.6.0 (from star to end of the external program).

    Did you by any chance switch from Win10 to Win11? I have seen something resembling your report when I did (there was no delay on Win10 Tcl/Tk
    built with VS 2022). A 5 seconds (timeout?) delay got introduced in my
    app. for no obvious reason (Tcl code unchanged). I didn't go to the
    bottom of this problem though, I resolved it in quick and dirty mode by
    a judiciously placed update in my script.

    Also I found there is an error during the bulding of 8.6.13, which is related to
    "git rev-parse HEAD"
    It doesn't seem to affect the final outcome of tcl and tk, so I just ignored it. What is this "git rev-parse HEAD" for anyway ? Should I be cautious about this ?

    I'm building with VS2022 on Win11 and I'm not seeing an error. Could you
    show how exactly you are building (show the commands you output), along
    with the the exact error?
    From a quick look, "git rev-parse HEAD" is used in makefile.vc to put
    the commit hash of the bleeding edge (i.e. HEAD) of the current branch
    in the manifest.uuid file. Sounds like we need git in the path for this
    to work, perhaps that's the issue you are seeing.

    Regards,
    Francois

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Francois Vogel@21:1/5 to All on Mon May 1 19:10:22 2023
    Le 01/05/2023 à 18:11, S-Y. Chen a écrit :
    No not Win 11. I built and run it in Win 10. And it is not only a delay problrm. If the same external program run for 200 seconds in exec of 8.6.0, it will run about 700 seconds in 8.6.12/8.6.13. So I beleive there is something to do with how exec
    handle the external programs in the OS (or the opposite way).

    OK, so it's not the same problem I have seen. Sorry.

    The error message related to git is like
    (begin)
    git rev-parse HEAD >>C:\...\Tcltk_8_6_13\tcl\win\..\manifest.uuid
    'git' is not an internal or external command, excutable program or batch file NMAKE : fatal error U1077: 'git rev-parse HEAD >>C:\...\Tcltk_8_6_13\tcl\win\..\manifest.uuid:
    return code '0x1'
    (end)

    Seems like you don't have git installed.
    TIP #599 has introduced a dependency on fossil or git to get the commit
    id of the thing being compiled. The error message you get should
    indicate both fossil and git are missing from your environment. If it's
    the case: how did you check out the source code? If it's not the case:
    please file a bug report here: https://core.tcl-lang.org/tcl/tktnew

    Regards,
    Francois

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Christian Werner@21:1/5 to Francois Vogel on Mon May 1 09:38:50 2023
    Francois Vogel schrieb am Montag, 1. Mai 2023 um 17:18:54 UTC+2:

    ... (there was no delay on Win10 Tcl/Tk
    built with VS 2022). A 5 seconds (timeout?) delay got introduced in my
    app. for no obvious reason (Tcl code unchanged) ...

    Francois,

    5 seconds rings an alarm bell in my head et voila I found this weird snippet

    https://www.androwish.org/home/file?ci=tip&name=jni/tcl/win/tclWinPipe.c&ln=977-998

    Laziness prevented me from writing a ticket in the Tcl core repo. So I believe that those 5 seconds may be caused by some Win32 non-console process
    without a GUI and thus without implicitly making a message queue which can
    be waited on by the parent process. But that's only wild guesswork on an even wilder process management design, to put it mildly.

    BR,
    Christian

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Christian Werner@21:1/5 to Francois Vogel on Mon May 1 10:26:10 2023
    Francois Vogel schrieb am Montag, 1. Mai 2023 um 19:18:37 UTC+2:

    Thanks for the pointer (any reason you point into Androwish instead of
    Tcl?) and thanks for the idea...

    since it's not a standard feature but another private and long forgotten hack by me.

    BR,
    Christian

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Francois Vogel@21:1/5 to All on Mon May 1 19:18:32 2023
    Le 01/05/2023 à 18:38, Christian Werner a écrit :
    Francois Vogel schrieb am Montag, 1. Mai 2023 um 17:18:54 UTC+2:

    ... (there was no delay on Win10 Tcl/Tk
    built with VS 2022). A 5 seconds (timeout?) delay got introduced in my
    app. for no obvious reason (Tcl code unchanged) ...

    Francois,

    5 seconds rings an alarm bell in my head et voila I found this weird snippet

    https://www.androwish.org/home/file?ci=tip&name=jni/tcl/win/tclWinPipe.c&ln=977-998

    Laziness prevented me from writing a ticket in the Tcl core repo. So I believe
    that those 5 seconds may be caused by some Win32 non-console process
    without a GUI and thus without implicitly making a message queue which can
    be waited on by the parent process. But that's only wild guesswork on an even wilder process management design, to put it mildly.

    Thanks for the pointer (any reason you point into Androwish instead of
    Tcl?) and thanks for the idea. I'll have to figure out how this relates
    to the 5 sec issue, and even more why there was a change of behavior
    when switching from Win10 to Win11.

    Regards,
    Francois

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Christian Werner@21:1/5 to Francois Vogel on Mon May 1 10:34:23 2023
    Francois Vogel schrieb am Montag, 1. Mai 2023 um 19:22:16 UTC+2:

    ... Deserves a ticket in Tcl repo, isn't it?

    Yes, indeedly, see

    https://core.tcl-lang.org/tcl/tktview/147b28d0436d4267a3d95082339f78980144b004

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Francois Vogel@21:1/5 to All on Mon May 1 19:22:11 2023
    Le 01/05/2023 à 19:18, Francois Vogel a écrit :
    Le 01/05/2023 à 18:38, Christian Werner a écrit :
    Francois Vogel schrieb am Montag, 1. Mai 2023 um 17:18:54 UTC+2:

    ... (there was no delay on Win10 Tcl/Tk
    built with VS 2022). A 5 seconds (timeout?) delay got introduced in my
    app. for no obvious reason (Tcl code unchanged) ...

    Francois,

    5 seconds rings an alarm bell in my head et voila I found this weird
    snippet


    https://www.androwish.org/home/file?ci=tip&name=jni/tcl/win/tclWinPipe.c&ln=977-998

    Laziness prevented me from writing a ticket in the Tcl core repo. So I
    believe
    that those 5 seconds may be caused by some Win32 non-console process
    without a GUI and thus without implicitly making a message queue which
    can
    be waited on by the parent process. But that's only wild guesswork on
    an even
    wilder process management design, to put it mildly.

    Thanks for the pointer (any reason you point into Androwish instead of
    Tcl?) and thanks for the idea. I'll have to figure out how this relates
    to the 5 sec issue, and even more why there was a change of behavior
    when switching from Win10 to Win11.

    Scrolling down a bit I now see what you meant, the link was slightly off-position I think, it must rather be:

    https://www.androwish.org/home/file?ci=tip&name=jni/tcl/win/tclWinPipe.c&ln=1066-1081

    Correct?

    Deserves a ticket in Tcl repo, isn't it?

    Regards,
    Francois

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From S-Y. Chen@21:1/5 to Francois Vogel on Mon May 1 20:03:00 2023
    On Tuesday, May 2, 2023 at 1:10:28 AM UTC+8, Francois Vogel wrote:
    Le 01/05/2023 à 18:11, S-Y. Chen a écrit :
    No not Win 11. I built and run it in Win 10. And it is not only a delay problrm. If the same external program run for 200 seconds in exec of 8.6.0, it will run about 700 seconds in 8.6.12/8.6.13. So I beleive there is something to do with how exec
    handle the external programs in the OS (or the opposite way).
    OK, so it's not the same problem I have seen. Sorry.
    The error message related to git is like
    (begin)
    git rev-parse HEAD >>C:\...\Tcltk_8_6_13\tcl\win\..\manifest.uuid
    'git' is not an internal or external command, excutable program or batch file
    NMAKE : fatal error U1077: 'git rev-parse HEAD >>C:\...\Tcltk_8_6_13\tcl\win\..\manifest.uuid:
    return code '0x1'
    (end)
    Seems like you don't have git installed.
    TIP #599 has introduced a dependency on fossil or git to get the commit
    id of the thing being compiled. The error message you get should
    indicate both fossil and git are missing from your environment. If it's
    the case: how did you check out the source code? If it's not the case: please file a bug report here: https://core.tcl-lang.org/tcl/tktnew

    Regards,
    Francois

    No I didn't install fossil or git. I just pull out the code archive from SourceForge.

    Is the problem of slow exec related to this ? Should I install fossil/git and recompile it again ?

    Regards
    Chen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Francois Vogel@21:1/5 to All on Tue May 2 22:10:31 2023
    Le 02/05/2023 à 05:03, S-Y. Chen a écrit :
    No I didn't install fossil or git. I just pull out the code archive from SourceForge.

    OK. No bug then.

    Is the problem of slow exec related to this ?

    With 99% chances it's not.

    Regards,
    Francois

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From S-Y. Chen@21:1/5 to Francois Vogel on Tue May 2 19:37:19 2023
    On Wednesday, May 3, 2023 at 4:12:03 AM UTC+8, Francois Vogel wrote:
    Le 02/05/2023 à 05:03, S-Y. Chen a écrit :
    No I didn't install fossil or git. I just pull out the code archive from SourceForge.
    OK. No bug then.
    Is the problem of slow exec related to this ?
    With 99% chances it's not.

    Regards,
    Francois

    Any idea what may cause the slow-down of exec ? Maybe I just take out all the "WaitForInputIdle" command and try again ?

    Regards
    Chen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ralf Fassel@21:1/5 to All on Wed May 3 10:51:39 2023
    * "[email protected]" <[email protected]>
    | I have this very strange problem, and is wondering if anyone has this experience.

    | I use Visual Studio 2022 to build tcl/tk 8.6.12 with some customized
    | command bulit into tcl. But I have been doing this for quite a while
    | without any problem.

    | For tcl shell 8.6.12 and 8.6.13(with tk86t.dll loaded), use exec to
    | run an external programs will take more than 3 times of executipn time
    | than using tcl 8.6.0 (from star to end of the external program).

    Could you check whether a stock precompiled 8.6.13
    (eg. https://sourceforge.net/projects/magicsplat/files/magicsplat-tcl/)
    shows the same slow-down as your version?

    A difference of 700s vs 200s as you noted elsethread sounds very
    severe. I have difficulties to imagine what exec would do to cause this.

    R'

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Christian Gollwitzer@21:1/5 to All on Wed May 3 10:56:43 2023
    Am 03.05.23 um 10:51 schrieb Ralf Fassel:
    * "[email protected]" <[email protected]>
    | I have this very strange problem, and is wondering if anyone has this experience.

    | I use Visual Studio 2022 to build tcl/tk 8.6.12 with some customized
    | command bulit into tcl. But I have been doing this for quite a while
    | without any problem.

    | For tcl shell 8.6.12 and 8.6.13(with tk86t.dll loaded), use exec to
    | run an external programs will take more than 3 times of executipn time
    | than using tcl 8.6.0 (from star to end of the external program).

    Could you check whether a stock precompiled 8.6.13
    (eg. https://sourceforge.net/projects/magicsplat/files/magicsplat-tcl/)
    shows the same slow-down as your version?


    Good idea.

    A difference of 700s vs 200s as you noted elsethread sounds very
    severe. I have difficulties to imagine what exec would do to cause this.


    My guess would be the antivirus scanner. Maybe there is something in Tcl
    which prompts the scanner to check the exec in one version but not the
    other. Also, caching can be an issue, so one should make sure that the shortened time does not occur only on the second run over the same files.

    Christian

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ralf Fassel@21:1/5 to All on Wed May 3 10:58:53 2023
    * "[email protected]" <[email protected]>
    | I use Visual Studio 2022 to build tcl/tk 8.6.12 with some customized
    | command bulit into tcl. But I have been doing this for quite a while
    | without any problem.

    Same compiler, 8.6.13 here, with no such problem observed.

    | For tcl shell 8.6.12 and 8.6.13(with tk86t.dll loaded), use exec to
    | run an external programs will take more than 3 times of executipn time
    | than using tcl 8.6.0 (from star to end of the external program).

    Another idea would be to wrap the external program in a batch file which
    logs the start and end times around the call, so you could see whether
    it is indeed 'exec' or the external program taking different runtimes in
    both circumstances.

    @echo off
    time /t < nul > timelog.txt
    your command goes here
    time /t < nul >> timelog.txt
    REM EOF

    R'

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From S-Y. Chen@21:1/5 to Ralf Fassel on Wed May 3 09:35:43 2023
    On Wednesday, May 3, 2023 at 4:59:09 PM UTC+8, Ralf Fassel wrote:
    * "[email protected]" <[email protected]>
    | I use Visual Studio 2022 to build tcl/tk 8.6.12 with some customized
    | command bulit into tcl. But I have been doing this for quite a while
    | without any problem.
    Same compiler, 8.6.13 here, with no such problem observed.
    | For tcl shell 8.6.12 and 8.6.13(with tk86t.dll loaded), use exec to
    | run an external programs will take more than 3 times of executipn time
    | than using tcl 8.6.0 (from star to end of the external program).
    Another idea would be to wrap the external program in a batch file which logs the start and end times around the call, so you could see whether
    it is indeed 'exec' or the external program taking different runtimes in both circumstances.

    @echo off
    time /t < nul > timelog.txt
    your command goes here
    time /t < nul >> timelog.txt
    REM EOF



    I will take some time to test. Please wait for my result.


    Chen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From et99@21:1/5 to S-Y. Chen on Wed May 3 12:24:21 2023
    On 5/3/2023 9:35 AM, S-Y. Chen wrote:
    On Wednesday, May 3, 2023 at 4:59:09 PM UTC+8, Ralf Fassel wrote:
    * "[email protected]" <[email protected]>
    | I use Visual Studio 2022 to build tcl/tk 8.6.12 with some customized
    | command bulit into tcl. But I have been doing this for quite a while
    | without any problem.
    Same compiler, 8.6.13 here, with no such problem observed.
    | For tcl shell 8.6.12 and 8.6.13(with tk86t.dll loaded), use exec to
    | run an external programs will take more than 3 times of executipn time
    | than using tcl 8.6.0 (from star to end of the external program).
    Another idea would be to wrap the external program in a batch file which
    logs the start and end times around the call, so you could see whether
    it is indeed 'exec' or the external program taking different runtimes in
    both circumstances.

    @echo off
    time /t < nul > timelog.txt
    your command goes here
    time /t < nul >> timelog.txt
    REM EOF



    I will take some time to test. Please wait for my result.


    Chen

    I would suggest you post your code that does the exec, just in
    case. Apologies if you did, I didn't closely read all the replies.

    If it is a virus check, does the .exe file with right click properties
    show unblock? When it does on my win 10, it still runs it but does the
    virus check (or something that takes many seconds).

    Can you substitute your executable with something like

    exec cmd /c dir >nul:

    and see if there's a difference. The dir should go quite fast.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ralf Fassel@21:1/5 to All on Thu May 4 11:37:10 2023
    * "S-Y. Chen" <[email protected]>
    | On Wednesday, May 3, 2023 at 4:59:09 PM UTC+8, Ralf Fassel wrote:
    | > Another idea would be to wrap the external program in a batch file which
    | > logs the start and end times around the call, so you could see whether
    | > it is indeed 'exec' or the external program taking different runtimes in
    | > both circumstances.
    | >
    | > @echo off
    | > time /t < nul > timelog.txt
    | > your command goes here
    | > time /t < nul >> timelog.txt
    | > REM EOF
    | >
    | >

    | I will take some time to test. Please wait for my result.

    Better make those 'time' commands

    time < nul > timelog.txt

    (without the /t) since non-interactive /t only shows minutes (no
    seconds) on my Windows 10.

    I don't know any simple command line program like Unix' 'date' on
    Windows, so I'd suggested 'time'. The redirection from < nul is
    necessary since time is interactive by default.

    HTH
    R'

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From S-Y. Chen@21:1/5 to Ralf Fassel on Thu May 4 09:18:55 2023
    On Thursday, May 4, 2023 at 5:39:17 PM UTC+8, Ralf Fassel wrote:
    * "S-Y. Chen" <[email protected]>
    | On Wednesday, May 3, 2023 at 4:59:09 PM UTC+8, Ralf Fassel wrote:
    | > Another idea would be to wrap the external program in a batch file which | > logs the start and end times around the call, so you could see whether
    | > it is indeed 'exec' or the external program taking different runtimes in | > both circumstances.
    | >
    | > @echo off
    | > time /t < nul > timelog.txt
    | > your command goes here
    | > time /t < nul >> timelog.txt
    | > REM EOF
    | >
    | >

    | I will take some time to test. Please wait for my result.
    Better make those 'time' commands

    time < nul > timelog.txt

    (without the /t) since non-interactive /t only shows minutes (no
    seconds) on my Windows 10.

    I don't know any simple command line program like Unix' 'date' on
    Windows, so I'd suggested 'time'. The redirection from < nul is
    necessary since time is interactive by default.

    HTH
    R'

    I found the problem. It was not caused by tcl. I will explain here.

    Using the precompiled tcl does help a lot.
    The external program I called with exec is an engineering computation program. I call that program by throwing a script file to the program, which will open en existing data file to process.

    Using the precompiled tcl produced two different results. One with normal time and one with about 3 times the span. Be aware that these are two data files but supposedly with totally the same contents (at lease from the users' point of view). The same
    results were re-produced by my own version of tcl/tk 8.6.13.

    I was able to pin down that, even these two data files do have the same contents, one of them does take longer time to open and process in the engineering program. From my experience there should be corruption in its data structure. So I am working to
    figure out the problem, but that's another story.

    Precompiled tcl did take longer time to run with tk86t.dll loaded though.

    Sorry for the trouble, and thank you for all the help.

    Regards
    Chen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)