• When running xclip in a function, it cannot return to the main script.

    From [email protected]@21:1/5 to All on Tue Apr 5 03:12:40 2022
    On Ubuntu 20.04.3 LTS, I check the self-compiled git master version of xclip with the following bash script:

    ```
    #! /bin/bash
    myclip() {
    xclip -selection clipbord -in <<< "hello world"
    echo x
    return
    }
    set -x
    while true; do
    y="$(myclip)"
    done
    ```

    It will be blocked there forever, as shown below:

    ```
    $ bash myclip.sh
    + true
    ++ myclip
    ++ xclip -selection clipbord -in
    ++ echo x
    ++ return
    ^C
    ```

    In other words, when running xclip in a function, it cannot return to the main script.

    Please refer to the related discussion here [1].

    [1] https://github.com/astrand/xclip/issues/134#issue-1192854776

    Regards,
    HZ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Helmut Waitzmann@21:1/5 to All on Tue Apr 5 21:38:35 2022
    "[email protected]" <[email protected]>:
    On Ubuntu 20.04.3 LTS, I check the self-compiled git master version
    of xclip with the following bash script:

    ```
    #! /bin/bash
    myclip() {
    xclip -selection clipbord -in <<< "hello world"
    echo x
    return
    }
    set -x
    while true; do
    y="$(myclip)"
    done
    ```

    It will be blocked there forever, as shown below:


    ```
    $ bash myclip.sh
    + true
    ++ myclip
    ++ xclip -selection clipbord -in
    ++ echo x
    ++ return
    ^C
    ```

    In other words, when running xclip in a function, it cannot return
    to the main script.

    Please verify your assumption by calling xclip directly without
    using a shell function:

    (
    while sleep -- 1 && date -- '+%Y-%m-%dT%T%z'
    do
    y="$(
    printf '%s\n' 'hello world' |
    xclip -selection clipbord -in
    )"
    done
    )

    which – according to your statement – should print the current time roughly once in a second, because there is no shell function
    involved.  Does it do so?


    Please refer to the related discussion here [1].


    (I prefer Usenet over WWW.)  The assumption mentioned in

    [1] https://github.com/astrand/xclip/issues/134#issue-1192854776

    though, that the culprit is the forking xclip, is correct, but the
    remedy "nohup" won't help.  Try

    (
    while sleep -- 1 && date -- '+%Y-%m-%dT%T%z'
    do
    y="$(
    printf '%s\n' 'hello world' |
    xclip -selection clipbord -in > /dev/null 2>&1
    )"
    done
    )

    which is the same as above except that the standard output and
    standard error file descriptors of the "xclip" invocation are
    redirected to "/dev/null".  Does it loop?  If it does, can you
    explain, why?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Chris Elvidge@21:1/5 to [email protected] on Tue Apr 5 21:58:20 2022
    On 05/04/2022 11:12, [email protected] wrote:
    On Ubuntu 20.04.3 LTS, I check the self-compiled git master version of xclip with the following bash script:

    ```
    #! /bin/bash
    myclip() {
    xclip -selection clipbord -in <<< "hello world"
    echo x
    return
    }
    set -x
    while true; do
    y="$(myclip)"
    done
    ```

    It will be blocked there forever, as shown below:

    ```
    $ bash myclip.sh
    + true
    ++ myclip
    ++ xclip -selection clipbord -in
    ++ echo x
    ++ return
    ^C
    ```

    In other words, when running xclip in a function, it cannot return to the main script.

    Please refer to the related discussion here [1].

    [1] https://github.com/astrand/xclip/issues/134#issue-1192854776

    Regards,
    HZ


    In a terminal:

    myclip() {
    xclip -selection clipbord -in <<< "hello world"
    echo "X"
    }

    myclip note: it returns almost immediately
    X

    now do edit / paste

    hello world


    --
    Chris Elvidge
    England

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Helmut Waitzmann@21:1/5 to All on Tue Apr 5 23:57:07 2022
    "[email protected]" <[email protected]>:
    On Ubuntu 20.04.3 LTS, I check the self-compiled git master version
    of xclip with the following bash script:

    ```
    #! /bin/bash
    myclip() {
    xclip -selection clipbord -in <<< "hello world"
    echo x
    return
    }
    set -x
    while true; do
    y="$(myclip)"
    done
    ```

    It will be blocked there forever, as shown below:


    […]

    In other words, when running xclip in a function, it cannot return
    to the main script.

    Modify the "myclip" function as shown here:


    myclip()
    {
    printf '%s\n' 'hello world' |
    xclip -selection clipboard -in > /dev/null
    printf '%s\n' x
    return
    }

    Then repeat your example from above with that modified "myclip"
    function.  Does it block?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [email protected]@21:1/5 to Helmut Waitzmann on Tue Apr 5 19:42:19 2022
    On Wednesday, April 6, 2022 at 5:57:34 AM UTC+8, Helmut Waitzmann wrote:
    "[email protected]" <[email protected]>:
    On Ubuntu 20.04.3 LTS, I check the self-compiled git master version
    of xclip with the following bash script:

    ```
    #! /bin/bash
    myclip() {
    xclip -selection clipbord -in <<< "hello world"
    echo x
    return
    }
    set -x
    while true; do
    y="$(myclip)"
    done
    ```

    It will be blocked there forever, as shown below:

    […]
    In other words, when running xclip in a function, it cannot return
    to the main script.
    Modify the "myclip" function as shown here:


    myclip()
    {
    printf '%s\n' 'hello world' |
    xclip -selection clipboard -in > /dev/null
    printf '%s\n' x
    return
    }

    Then repeat your example from above with that modified "myclip"
    function. Does it block?

    No, it works as expected:

    $ cat myclip.sh
    #! /bin/bash
    myclip()
    {
    printf '%s\n' 'hello world' |
    xclip -selection clipboard -in > /dev/null
    printf '%s\n' x
    sleep 1
    return
    }
    set -x
    while true; do
    y="$(myclip)"
    done

    $ bash myclip.sh
    + true
    ++ myclip
    ++ printf '%s\n' 'hello world'
    ++ xclip -selection clipboard -in
    ++ printf '%s\n' x
    ++ sleep 1
    ++ return
    + y=x
    + true
    ++ myclip
    ++ printf '%s\n' 'hello world'
    ++ xclip -selection clipboard -in
    ++ printf '%s\n' x
    ++ sleep 1
    ++ return
    + y=x
    + true
    ++ myclip
    ++ printf '%s\n' 'hello world'
    ++ xclip -selection clipboard -in
    ^C

    Regards,
    HZ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [email protected]@21:1/5 to Helmut Waitzmann on Tue Apr 5 19:35:44 2022
    On Wednesday, April 6, 2022 at 3:38:47 AM UTC+8, Helmut Waitzmann wrote:
    "[email protected]" <[email protected]>:
    On Ubuntu 20.04.3 LTS, I check the self-compiled git master version
    of xclip with the following bash script:

    ```
    #! /bin/bash
    myclip() {
    xclip -selection clipbord -in <<< "hello world"
    echo x
    return
    }
    set -x
    while true; do
    y="$(myclip)"
    done
    ```

    It will be blocked there forever, as shown below:


    ```
    $ bash myclip.sh
    + true
    ++ myclip
    ++ xclip -selection clipbord -in
    ++ echo x
    ++ return
    ^C
    ```

    In other words, when running xclip in a function, it cannot return
    to the main script.
    Please verify your assumption by calling xclip directly without
    using a shell function:

    (
    while sleep -- 1 && date -- '+%Y-%m-%dT%T%z'
    do
    y="$(
    printf '%s\n' 'hello world' |
    xclip -selection clipbord -in
    )"
    done
    )

    which – according to your statement – should print the current time roughly once in a second, because there is no shell function
    involved. Does it do so?

    No, it doesn't:

    $ (
    while sleep -- 1 && date -- '+%Y-%m-%dT%T%z'
    do
    y="$(
    printf '%s\n' 'hello world' |
    xclip -selection clipbord -in
    )"
    done
    )
    2022-04-06T10:31:24+0800
    ^C


    Please refer to the related discussion here [1].

    (I prefer Usenet over WWW.) The assumption mentioned in

    [1] https://github.com/astrand/xclip/issues/134#issue-1192854776

    though, that the culprit is the forking xclip, is correct, but the
    remedy "nohup" won't help. Try

    (
    while sleep -- 1 && date -- '+%Y-%m-%dT%T%z'
    do
    y="$(
    printf '%s\n' 'hello world' |
    xclip -selection clipbord -in > /dev/null 2>&1
    )"
    done
    )

    which is the same as above except that the standard output and
    standard error file descriptors of the "xclip" invocation are
    redirected to "/dev/null". Does it loop?

    Yes. It does:

    $ (
    while sleep -- 1 && date -- '+%Y-%m-%dT%T%z'
    do
    y="$(
    printf '%s\n' 'hello world' |
    xclip -selection clipbord -in > /dev/null 2>&1
    )"
    done
    )
    2022-04-06T10:32:39+0800
    2022-04-06T10:32:40+0800
    2022-04-06T10:32:41+0800
    2022-04-06T10:32:42+0800
    2022-04-06T10:32:43+0800
    2022-04-06T10:32:44+0800
    2022-04-06T10:32:45+0800
    2022-04-06T10:32:46+0800
    ^C

    If it does, can you explain, why?

    I still don't understand

    HZ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Helmut Waitzmann@21:1/5 to All on Wed Apr 6 21:20:28 2022
    "[email protected]" <[email protected]>:
    On Wednesday, April 6, 2022 at 3:38:47 AM UTC+8, Helmut Waitzmann wrote:
    "[email protected]" <[email protected]>:
    On Ubuntu 20.04.3 LTS, I check the self-compiled git master
    version of xclip with the following bash script:

    ```
    #! /bin/bash
    myclip() {
    xclip -selection clipbord -in <<< "hello world"
    echo x
    return
    }
    set -x
    while true; do
    y="$(myclip)"
    done
    ```

    It will be blocked there forever, as shown below:


    ```
    $ bash myclip.sh
    + true
    ++ myclip
    ++ xclip -selection clipbord -in
    ++ echo x
    ++ return
    ^C
    ```

    In other words, when running xclip in a function, it cannot
    return to the main script.

    I still don't understand


    I'll try to explain:


    The various variants of your example I gave in the messages with
    the Message‐Ids <[email protected]> and
    <[email protected]> show:  It's not the
    function that causes the blocking.  The script blocks if and only
    if "xclip" is invoked without redirecting its standard output away
    from feeding the command substitution "$( … )".  Whether that
    occurs inside of a function or just inline makes no difference.


    So, what's going on, there?


    When the shell performs a command substition


    "$( some commands )",

    it first creates and opens a pipe by means of the system call
    "pipe()". (A pipe is a kernel object which can be written into at
    one end and read from at the other end.  See the manual pages
    pipe(7) and pipe(2).)

    Then the shell forks a process that will execute the subshell
    environment: the command line inside the "$( )".

    The forked subshell environment will write to the created pipe
    while the outer shell, i. e. the shell that has forked the subshell
    environment, will continue reading data from the created pipe and
    collecting it until it reaches an end‐of‐file condition.  When it
    has reached the end‐of‐file condition, it will wait by means of one
    of the "wait" family of system calls (see the manual page wait(2))
    for the subshell environment to terminate.

    It puts the collected data it has read from the pipe into the
    parameter list of the command.  In your example that command is the
    variable assignment:

    y="$( … )"

    Therefore, the read data will get assigned to the shell variable
    "y".

    The exit status of the variable assignment will be the exit status
    as obtained from the "wait" system call, i. e. it will be the exit
    status of the command substitution.

    The key to understanding why your example blocks is to understand
    what an end‐of‐file condition in a pipe is:

    When reading from a pipe, there are (at least) three conditions to
    be distinguished:

    (1) A writer has written something to the pipe.  Then upon reading,
    the reader will get that data, which will be removed from the pipe.

    (2) The pipe is empty but the writing end is still open for
    writing, i.e. nothing has been written to it since all data that
    may have been written to it have been read.  Then upon reading, the
    reader will block and wait for further data to arrive as it does
    not know whether the writer eventually will write further data.

    (3) The writer may have written some (or may not have written any)
    data to the pipe and then closes the writing end of the pipe.  When
    there is no writer left to write to the writing end of the pipe,
    then upon reading, the reader will get the data (if there is one)
    like in (1) and then it will get an end‐of‐file condition.

    The end‐of‐file condition tells the reader (in your use case: the
    outer shell) that there will no further data arrive in the pipe (as
    no one has it open for writing any more).

    In the use case of the command substitution the outer shell will
    know that it has read all data to be used in the command
    substitution.  It will close its reading end of the pipe and then
    wait for the subshell environment to terminate.

    Now what's going on, when the subshell environment starts the
    "xclip" command which will fork a subprocess?

    Without redirection of its standard output, that is, the blocking
    use cases, "xclip" will have its standard output file descriptor be
    the writing end of the pipe.  When "xclip" forks its subprocess, the
    subprocess will of course inherit the writing end of the pipe.  =>
    There are two processes which might write to the pipe.

    Now, if the "outer" "xclip" process terminates, its writing end of
    the pipe will implicitly get closed, but the writing end of the pipe
    hold by the forked "xclip" subprocess will remain open.

    As written above in (2) and (3) the reader of the pipe won't get
    an end‐of‐file condition as long as the forked "xclip" subprocess
    neither closes its standard output (Of course it will not:  No
    program is expected to close its standard output.) nor terminates.

    Therefore the outer shell will wait for the subshell environment
    to send an end‐of‐file condition to the pipe until the cows come
    home.


    Now on the other hand, if "xclip" is called with its standard
    output file descriptor redirected from the writing end of the pipe
    to for example "/dev/null", then its forked subprocess will inherit
    the redirected standard output file descriptor.

    No dangling writing end of the pipe will stay open and the outer
    shell will finally get its longed‐for end‐of‐file condition when
    reading from the pipe.

    That scenario is not specific to "xclip".  Here is an example using
    "sleep" in the background.  Note:  "sleep" won't write anything to
    its standard output, and of course it won't even bother closing it:

    (
    while var="$(
    sleep -- 2 &
    date -- '+%Y-%m-%dT%T%z'
    )"
    do
    printf 'command substitution ran at %s\n' "$var"
    date -- '+command substitution ended at %Y-%m-%dT%T%z'
    echo
    sleep -- 1
    done
    )

    The output shows that the command substitution lasts for 2 seconds
    in spite of the "sleep" command being run in the background,
    because the end‐of‐file condition for the reader is delayed until
    that two‐seconds‐sleep terminates and implicitly gets it standard
    output file descriptor, which refers to the writing end of the
    pipe, closed.

    If the standard output of the "sleep" command is redirected from
    the writing end of the pipe to (for example) "/dev/null" then the
    command substitution returns almost immediately, as can be seen
    here:

    (
    while var="$(
    sleep -- 2 > /dev/null &
    date -- '+%Y-%m-%dT%T%z'
    )"
    do
    printf 'command substitution ran at %s\n' "$var"
    date -- '+command substitution ended at %Y-%m-%dT%T%z'
    echo
    sleep -- 1
    done
    )

    May I add my refrain?  Knowing linux or unix greatly helps
    understanding the shell.  Without that knowledge one will have a
    hard job doing so.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [email protected]@21:1/5 to Helmut Waitzmann on Wed Apr 6 18:18:14 2022
    On Thursday, April 7, 2022 at 3:25:41 AM UTC+8, Helmut Waitzmann wrote:
    "[email protected]" <[email protected]>:
    On Wednesday, April 6, 2022 at 3:38:47 AM UTC+8, Helmut Waitzmann wrote:
    "[email protected]" <[email protected]>:
    On Ubuntu 20.04.3 LTS, I check the self-compiled git master
    version of xclip with the following bash script:

    ```
    #! /bin/bash
    myclip() {
    xclip -selection clipbord -in <<< "hello world"
    echo x
    return
    }
    set -x
    while true; do
    y="$(myclip)"
    done
    ```

    It will be blocked there forever, as shown below:


    ```
    $ bash myclip.sh
    + true
    ++ myclip
    ++ xclip -selection clipbord -in
    ++ echo x
    ++ return
    ^C
    ```

    In other words, when running xclip in a function, it cannot
    return to the main script.

    I still don't understand

    I'll try to explain:
    [snipped]
    May I add my refrain? Knowing linux or unix greatly helps
    understanding the shell. Without that knowledge one will have a
    hard job doing so.

    This is really not an easy problem to analyze without the deep understanding of the corresponding Linux or Unix underlying communication mechanism. Thank you for your in-depth and systematic analysis and nice examples.

    Regards,
    HZ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From [email protected]@21:1/5 to [email protected] on Wed Apr 6 18:31:27 2022
    On Thursday, April 7, 2022 at 9:18:16 AM UTC+8, [email protected] wrote:
    On Thursday, April 7, 2022 at 3:25:41 AM UTC+8, Helmut Waitzmann wrote:
    "[email protected]" <[email protected]>:
    On Wednesday, April 6, 2022 at 3:38:47 AM UTC+8, Helmut Waitzmann wrote:
    "[email protected]" <[email protected]>:
    On Ubuntu 20.04.3 LTS, I check the self-compiled git master
    version of xclip with the following bash script:

    ```
    #! /bin/bash
    myclip() {
    xclip -selection clipbord -in <<< "hello world"
    echo x
    return
    }
    set -x
    while true; do
    y="$(myclip)"
    done
    ```

    It will be blocked there forever, as shown below:


    ```
    $ bash myclip.sh
    + true
    ++ myclip
    ++ xclip -selection clipbord -in
    ++ echo x
    ++ return
    ^C
    ```

    In other words, when running xclip in a function, it cannot
    return to the main script.

    I still don't understand

    I'll try to explain:
    [snipped]
    May I add my refrain? Knowing linux or unix greatly helps
    understanding the shell. Without that knowledge one will have a
    hard job doing so.
    This is really not an easy problem to analyze without the deep understanding of the corresponding Linux or Unix underlying communication mechanism. Thank you for your in-depth and systematic analysis and nice examples.

    Dear Helmut Waitzmann,

    I noticed the information of the following contributor [1]:

    Helmut Waitzmann [email protected]

    I don't know if this is you.

    [1] https://github.com/landlock-lsm/tar/blob/master/THANKS#L209

    Regards,
    HZ

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)