1. Forum
  2. Usenet
  3. COMP.LANG.TCL

  • limit on threads?

    From saitology9@21:1/5 to All on Wed Aug 2 17:46:49 2023
    Is there a limit on the number of threads you can create in an
    interpreter? Out of no where, I started getting error messages saying
    that the interpreter "can't create a thread". The man page did not say anything on this.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich@21:1/5 to [email protected] on Wed Aug 2 22:26:05 2023
    saitology9 <[email protected]> wrote:
    Is there a limit on the number of threads you can create in an
    interpreter? Out of no where, I started getting error messages
    saying that the interpreter "can't create a thread". The man page
    did not say anything on this.

    Tcl, at least on Linux, seems to impose no limit itself. I tried
    creating a huge number not so long ago as a test, and the 'limiting
    factor' was the memory consumed by each thread. Eventually the process
    would use up all memory in the machine, and become a target for the
    Linux OOM killer.

    The host OS, however, can enforce a limit if it wants, in which case
    Tcl would eventually hit that limit if you create enough threads.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From saitology9@21:1/5 to Rich on Wed Aug 2 19:57:28 2023
    On 8/2/2023 6:26 PM, Rich wrote:

    The host OS, however, can enforce a limit if it wants, in which case
    Tcl would eventually hit that limit if you create enough threads.


    Thanks for the info. This was on Windows 10. It must have been low
    memory issue for me.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich@21:1/5 to [email protected] on Thu Aug 3 01:46:42 2023
    saitology9 <[email protected]> wrote:
    On 8/2/2023 6:26 PM, Rich wrote:

    The host OS, however, can enforce a limit if it wants, in which case
    Tcl would eventually hit that limit if you create enough threads.


    Thanks for the info. This was on Windows 10. It must have been low
    memory issue for me.

    Linux, 1100 empty threads (just thread::create and nothing more), top
    reports 12.6g of virtual memory used.

    That works out to about 12mb per empty thread.

    Naturally if they are doing something they will each be larger.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Joerg Mertens@21:1/5 to Rich on Thu Aug 3 15:27:02 2023
    Rich <[email protected]d> writes:

    Linux, 1100 empty threads (just thread::create and nothing more), top
    reports 12.6g of virtual memory used.

    That works out to about 12mb per empty thread.

    This seems like a lot, so I tested it with an interactive tclsh. After
    loading the thread package top shows 5560k. Each thread::create adds
    about 1000k. Do you have some other packages loaded in your program?

    Joerg

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich@21:1/5 to Joerg Mertens on Thu Aug 3 14:29:10 2023
    Joerg Mertens <[email protected]> wrote:
    Rich <[email protected]d> writes:

    Linux, 1100 empty threads (just thread::create and nothing more), top
    reports 12.6g of virtual memory used.

    That works out to about 12mb per empty thread.

    This seems like a lot, so I tested it with an interactive tclsh. After loading the thread package top shows 5560k. Each thread::create adds
    about 1000k. Do you have some other packages loaded in your program?

    Joerg

    Just a blank interactive tclsh (64 bit).

    $ rlwrap tclsh
    % package require Thread
    2.8.7
    % for {set i 0} {$i < 1000} {incr i} {thread::create}
    %

    And top reports, for the tclsh:

    PID NI VIRT RES SWAP S %CPU %MEM TIME+ COMMAND
    30010 0 11.8g 273760 0 S 0.0 1.1 0:02.84 tclsh

    Only 273760 resident, but tclsh has allocaated 11.8g.

    Just a blank tclsh (launched within rlwrap) shows as:

    31029 0 6864 4636 0 R 0.0 0.0 0:13.47 tclsh

    269k resident per thread -- but the virtual space itself balloons much
    more.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Joerg Mertens@21:1/5 to Rich on Thu Aug 3 17:37:58 2023
    Rich <[email protected]d> writes:

    Joerg Mertens <[email protected]> wrote:
    Rich <[email protected]d> writes:

    Linux, 1100 empty threads (just thread::create and nothing more), top
    reports 12.6g of virtual memory used.

    That works out to about 12mb per empty thread.

    This seems like a lot, so I tested it with an interactive tclsh. After
    loading the thread package top shows 5560k. Each thread::create adds
    about 1000k. Do you have some other packages loaded in your program?

    Joerg

    I didn't take into account that I did the test on OpenBSD. I also
    wouldn't have thought the difference in memory management is so big.

    Just a blank interactive tclsh (64 bit).

    $ rlwrap tclsh
    % package require Thread
    2.8.7
    % for {set i 0} {$i < 1000} {incr i} {thread::create}
    %

    Same here, except the Thread package version is 2.8.8

    And top reports, for the tclsh:

    PID NI VIRT RES SWAP S %CPU %MEM TIME+ COMMAND 30010 0 11.8g 273760 0 S 0.0 1.1 0:02.84 tclsh

    Only 273760 resident, but tclsh has allocaated 11.8g.

    My top output:

    PID PRI NICE SIZE RES STATE WAIT TIME CPU COMMAND
    98511 3 0 966M 475M idle ttyin 0:05 0.00% tclsh8.6

    SIZE is the total size of the process here. Should be similar to
    linux's VIRT column.

    Just a blank tclsh (launched within rlwrap) shows as:

    31029 0 6864 4636 0 R 0.0 0.0 0:13.47 tclsh

    269k resident per thread -- but the virtual space itself balloons much
    more.

    Yes, that's the main difference.

    Joerg

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)