• (bash) Which characters are magic inside of double quotes?

    From Kenny McCormack@21:1/5 to All on Sat Nov 25 14:34:43 2023
    I can think of only two: $ and `

    Am I forgetting anything?

    --
    The book "1984" used to be a cautionary tale;
    Now it is a "how-to" manual.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Lew Pitcher@21:1/5 to Kenny McCormack on Sat Nov 25 14:59:27 2023
    On Sat, 25 Nov 2023 14:34:43 +0000, Kenny McCormack wrote:

    I can think of only two: $ and `

    Am I forgetting anything?


    From the bash(1) manpage:

    "Enclosing characters in double quotes preserves the literal value of
    all characters within the quotes, with the exception of $, `, \, and,
    when history expansion is enabled, !."

    So the list is:
    $
    `
    \
    !


    --
    Lew Pitcher
    "In Skills We Trust"

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kenny McCormack@21:1/5 to [email protected] on Sat Nov 25 16:36:16 2023
    In article <ujt24f$2qv42$[email protected]>,
    Lew Pitcher <[email protected]> wrote:
    On Sat, 25 Nov 2023 14:34:43 +0000, Kenny McCormack wrote:

    I can think of only two: $ and `

    Am I forgetting anything?


    From the bash(1) manpage:

    "Enclosing characters in double quotes preserves the literal value of
    all characters within the quotes, with the exception of $, `, \, and,
    when history expansion is enabled, !."

    So the list is:
    $
    `
    \
    !

    Thanks. Luckily, I don't have to worry about !. But I think \ is a dark corner. It sometimes needs to be escaped; sometime not. It is weird. We
    had a discussion (started by me) a while back about this and the conclusion
    was as stated (that it is sort of a dark corner).

    The overall context of this question is embedding scripts (for various interpreted languages) inside shell scripts. For most languages, such as "AWK", you just put it all inside single quotes and everything is fine,
    Most languages use double quotes internally for string delimiting, and it
    all just works. And if you need a single quote, you can usually write \047.

    But there are two languages where the string delimiter is single quote (usually) and for those languages, it ends up being more convenient to
    enclose the script (in the shell script) with double quotes. But then you
    have to worry about the fact that double quotes are not "strong" quotes.
    Hence, this thread.

    The two languages I am referring to are "VIM" and "SQL". Both of these languages are defined in terms of using single quotes as the string
    delimiter, but both sortof/kindof/maybe also accept double quotes. So, you
    can sometimes get away with using double quotes, but sometimes not.

    --
    The people who were, are, and always will be, wrong about everything, are still calling *us* "libtards"...

    (John Fugelsang)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Helmut Waitzmann@21:1/5 to All on Sun Nov 26 04:33:34 2023
    [email protected] (Kenny McCormack):

    [The question: Which characters are magic inside of double
    quotes?]


    The overall context of this question is embedding scripts (for
    various interpreted languages) inside shell scripts. For most
    languages, such as "AWK", you just put it all inside single
    quotes and everything is fine, Most languages use double quotes
    internally for string delimiting, and it all just works. And if
    you need a single quote, you can usually write \047.


    But there are two languages where the string delimiter is single
    quote (usually) and for those languages, it ends up being more
    convenient to enclose the script (in the shell script) with
    double quotes. But then you have to worry about the fact that
    double quotes are not "strong" quotes. Hence, this thread.


    The two languages I am referring to are "VIM" and "SQL". Both of
    these languages are defined in terms of using single quotes as
    the string delimiter, but both sortof/kindof/maybe also accept
    double quotes. So, you can sometimes get away with using double
    quotes, but sometimes not.



    But you can always do it like this:


    For every script to be embedded inside a shell script, procede
    using the following rules:


    In the script to be embedded, replace each single quote by a
    single quote, a backslash, and 2 single quotes (or insert before
    each single quote a single quote, a backslash, and a single quote
    or insert after each single quote a backslash and 2 single
    quotes).


    After this has been done, prepend the whole script to be embedded
    with a single quote and append a single quote.


    Embed the result.


    That's it.  It will work with any script to be embedded into a
    shell script regardless the language of the script to be
    embedded.


    The advantage of this procedure:  It can be done using a
    search‐and‐replace command in your favorite text editor, as well
    as by a "sed" or "awk" command invocation or even by a shell
    script.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kenny McCormack@21:1/5 to [email protected] on Sun Nov 26 07:30:06 2023
    In article <[email protected]>,
    Helmut Waitzmann <[email protected]> wrote:
    ...
    For every script to be embedded inside a shell script, procede
    using the following rules:


    In the script to be embedded, replace each single quote by a
    single quote, a backslash, and 2 single quotes (or insert before
    each single quote a single quote, a backslash, and a single quote
    or insert after each single quote a backslash and 2 single
    quotes).

    Yes, this works. But it is ugly. In the past, I've used the trick of replacing ' with '"'"' - but as you say, replacing it with '\'' saves one character.

    But it is just unaesthetically pleasing. In the instant case, I found it easier to go with " instead of ' and to "hope for the best". It all worked
    out fine, as the embedded script didn't have any of the "magic" characters
    in it.

    The process described above, of replacing the ' with a 4 or 5 character replacement string, is not too bad for AWK, where ' is rare, but gets ugly
    with either of the mentioned languages (VIM and SQL), where the ' is
    common.

    --
    The whole aim of practical politics is to keep the populace alarmed (and hence clamorous
    to be led to safety) by menacing it with an endless series of hobgoblins, all of them imaginary.

    H. L. Mencken

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ben Bacarisse@21:1/5 to Helmut Waitzmann on Sun Nov 26 21:27:37 2023
    Helmut Waitzmann <[email protected]> writes:

    [email protected] (Kenny McCormack):

    [The question: Which characters are magic inside of double quotes?]

    The overall context of this question is embedding scripts (for
    various interpreted languages) inside shell scripts. For most >
    languages, such as "AWK", you just put it all inside single > quotes
    and everything is fine, Most languages use double quotes > internally
    for string delimiting, and it all just works. And if > you need a
    single quote, you can usually write \047.

    But there are two languages where the string delimiter is single quote
    (usually) and for those languages, it ends up being more convenient to
    enclose the script (in the shell script) with double quotes. But then you
    have to worry about the fact that double quotes are not "strong"
    quotes. Hence, this thread. The two languages I am referring to are "VIM"
    and "SQL". Both of these languages are defined in terms of using single
    quotes as the string delimiter, but both sortof/kindof/maybe also accept
    double quotes. So, you can sometimes get away with using double quotes,
    but sometimes not.

    But you can always do it like this: For every script to be embedded inside
    a shell script, procede using the following rules: In the script to be embedded, replace each single quote by a single quote, a backslash, and 2 single quotes [alternatives elided...]

    That works well when there is a "build process" for the script that
    contains the embedded script because, as you say, it's easy to automate.
    But I find it fragile when the embedded script might be edited and hard
    to read when the script needs to be inspected. Lots of '\''...'\''s get
    in the way.

    My preference is for here documents. You can choose a clear and
    staggeringly unlikely delimiter, and the text between the start and the
    end is taken literally (if you 'quote-the-end-delimiter').

    This works best for standard input, but when I need a single string
    argument I put "$(cat in front and )" at the end. Not pretty, perhaps,
    but it keeps all of the mess away from the actual embedded script:

    printf "{%s}\n" "$(cat <<'---END-OF-EMBEDDED-SCRIPT---'
    ...script here...
    ---END-OF-EMBEDDED-SCRIPT---
    )"

    --
    Ben.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Helmut Waitzmann@21:1/5 to All on Mon Dec 11 12:40:47 2023
    Ben Bacarisse <[email protected]>:
    Helmut Waitzmann <[email protected]> writes:

    But you can always do it like this:  For every script to be
    embedded inside a shell script, procede using the following
    rules:  In the script to be embedded, replace each single quote
    by a single quote, a backslash, and 2 single quotes
    [alternatives elided...]

    That works well when there is a "build process" for the script that
    contains the embedded script because, as you say, it's easy to automate.
    But I find it fragile when the embedded script might be edited and hard
    to read when the script needs to be inspected. Lots of '\''...'\''s get
    in the way.


    Yes, that's true.  In this case, I either include the script to
    be embedded in its original form, each line of it preceded by


    #


    in the shell script as well, that is, save it as a shell comment
    for a later source of re-editing.


    Or, when the embedded script is to be to re-edited, construct a
    shell command to be run:  Start that command with the shell
    command line fragment


    printf '%s' \


    then copy and paste the embedded script, which is to be
    re-edited, into the following lines, either in a shell script
    file or use the features of your favourite editor to run a shell
    command line.  Then run that shell script file (resp. the shell
    command line).


    The standard output will be the original script to be embedded.


    My preference is for here documents. You can choose a clear and
    staggeringly unlikely delimiter, and the text between the start and the
    end is taken literally (if you 'quote-the-end-delimiter').

    This works best for standard input, but when I need a single string
    argument I put "$(cat in front and )" at the end. Not pretty, perhaps,
    but it keeps all of the mess away from the actual embedded script:

    printf "{%s}\n" "$(cat <<'---END-OF-EMBEDDED-SCRIPT---'
    ...script here...
    ---END-OF-EMBEDDED-SCRIPT---
    )"


    When using here documents, I tend to use


    cut -d . -f2- <<-EOF
    .These are the lines of the script to be embedded,
    .each of them prepended with the "cut" delimiter.
    .Even indented lines like the following
    . indented line
    .will have their indentation preserved.
    EOF


    rather than just "cat", prepending each line of the script to be
    embedded with a non-whitespace delimiter (for example a period): 
    That allows me to neatly indent the here document using TAB
    characters according to the indentation level of its context in
    the shell script and also frees me from having to worry about
    looking for a suitable here document delimiter: "EOF" (for
    example) will always work, as it is different from any line
    starting with zero or more white space characters followed by a
    period.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to Helmut Waitzmann on Mon Dec 11 15:48:27 2023
    On 11.12.2023 12:40, Helmut Waitzmann wrote:

    When using here documents, I tend to use

    cut -d . -f2- <<-EOF
    .These are the lines of the script to be embedded,
    .each of them prepended with the "cut" delimiter.
    .Even indented lines like the following
    . indented line
    .will have their indentation preserved.
    EOF


    rather than just "cat", prepending each line of the script to be
    embedded with a non-whitespace delimiter (for example a period): That
    allows me to neatly indent the here document using TAB characters
    according to the indentation level of its context in the shell script
    and also frees me from having to worry about looking for a suitable here document delimiter: "EOF" (for example) will always work, as it is
    different from any line starting with zero or more white space
    characters followed by a period.

    Ksh users may use '<<#' as in

    cat <<# EOF
    These are the lines of the script to be embedded.
    Even indented lines like the following
    tab-indented line
    space-indented line
    will have their indentation preserved.
    EOF


    "If # is appended to <<, then leading spaces and tabs will
    be stripped off the first line of the document and up to an
    equivalent indentation will be stripped from the remaining
    lines and from word." [man ksh]

    (Note: by "word" the 'EOF' is meant here. It can also be
    indented if desired.)

    Janis

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)