• Talking to a third-party application

    From Luc@21:1/5 to All on Fri Apr 14 13:19:47 2023
    I want Tcl to talk to an application. This application has no
    communication mechanisms. It isn't designed to talk at all except
    that it can write data to the disk. It can also read from the disk.

    So I did a little kludge. The application keeps writing to /dev/shm
    while Tcl keeps reading that file. Sometimes, Tcl writes to another
    file that the application can read. OK, they are talking.

    But it's an ugly hack and I am struggling with the idea of keeping
    tabs on what has been read, I mean, what lines/messages have been
    properly delivered and must be removed from the queue, besides the
    problem of deleting content from a file that is being written to all
    the time. And I know that Tcl is very good at sockets, which seems
    interesting but I'm not sure it would be useful in my current
    situation. Would it?

    What approach would you recommend to make my communication
    system smoother?

    --
    Luc


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich@21:1/5 to Luc on Fri Apr 14 19:32:02 2023
    Luc <[email protected]d> wrote:
    I want Tcl to talk to an application. This application has no
    communication mechanisms. It isn't designed to talk at all except
    that it can write data to the disk. It can also read from the disk.

    So I did a little kludge. The application keeps writing to /dev/shm
    while Tcl keeps reading that file. Sometimes, Tcl writes to another
    file that the application can read. OK, they are talking.

    But it's an ugly hack and I am struggling with the idea of keeping
    tabs on what has been read, I mean, what lines/messages have been
    properly delivered and must be removed from the queue, besides the
    problem of deleting content from a file that is being written to all
    the time. And I know that Tcl is very good at sockets, which seems interesting but I'm not sure it would be useful in my current
    situation. Would it?

    What approach would you recommend to make my communication system
    smoother?

    Best solution: fix/update the "other application" so it can
    communicate. Although, presumably, you are asking this question
    because you can't modify this "other application".

    Can the 'other application' be asked to read/write to/from standard
    input and output? If yes, then one of the pipe open method (read about filenames that begin with a | character in the open manpage) or a pipe
    channel (read about the 'pipe' subcommand in the 'chan' man page) with
    some 'exec magic' might work.

    If it insists on "a file" can a Unix named pipe file be substituted for
    the file it expects? Read the "mknod" Linux manual page for how to
    create a named pipe.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Luc@21:1/5 to Rich on Fri Apr 14 21:26:01 2023
    On Fri, 14 Apr 2023 19:32:02 -0000 (UTC), Rich wrote:

    Can the 'other application' be asked to read/write to/from standard
    input and output?

    No. I was hoping someone would come up with some kind of trick that would
    make some kind of "virtual" pipe or something. Or something. I don't know.


    If it insists on "a file" can a Unix named pipe file be substituted for
    the file it expects? Read the "mknod" Linux manual page for how to
    create a named pipe.

    Yes, I can have named pipes. I've had them, but have forgotten what it
    was like. Anyway, is there any specific Tcl angle to this approach or is
    it just named pipes being named pipes?

    Tcl sockets do not fit my case, I suppose?


    --
    Luc


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich@21:1/5 to Luc on Sat Apr 15 03:08:47 2023
    Luc <[email protected]d> wrote:
    On Fri, 14 Apr 2023 19:32:02 -0000 (UTC), Rich wrote:

    Can the 'other application' be asked to read/write to/from standard
    input and output?

    No. I was hoping someone would come up with some kind of trick that would make some kind of "virtual" pipe or something. Or something. I don't know.

    You could possibly inject a custom shared library that would intercept
    the open libc call and the new 'open()' call could then do whatever you
    wanted.

    If it insists on "a file" can a Unix named pipe file be substituted
    for the file it expects? Read the "mknod" Linux manual page for how
    to create a named pipe.

    Yes, I can have named pipes. I've had them, but have forgotten what
    it was like. Anyway, is there any specific Tcl angle to this
    approach or is it just named pipes being named pipes?

    Named pipes are just files, they open/read/write/close just like files.

    Tcl sockets do not fit my case, I suppose?

    Sockets are for networking, not files.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Gerald Lester@21:1/5 to Luc on Fri Apr 14 21:55:43 2023
    On 4/14/23 19:26, Luc wrote:
    On Fri, 14 Apr 2023 19:32:02 -0000 (UTC), Rich wrote:

    Can the 'other application' be asked to read/write to/from standard
    input and output?

    No. I was hoping someone would come up with some kind of trick that would make some kind of "virtual" pipe or something. Or something. I don't know.


    If it insists on "a file" can a Unix named pipe file be substituted for
    the file it expects? Read the "mknod" Linux manual page for how to
    create a named pipe.

    Yes, I can have named pipes. I've had them, but have forgotten what it
    was like. Anyway, is there any specific Tcl angle to this approach or is
    it just named pipes being named pipes?

    Just named pipes being named pipes.

    Tcl sockets do not fit my case, I suppose?

    Not without modifying the "other application".

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ted Nolan @21:1/5 to [email protected] on Sat Apr 15 04:40:14 2023
    In article <[email protected]>, Luc <[email protected]d> wrote: >On Fri, 14 Apr 2023 19:32:02 -0000 (UTC), Rich wrote:

    Can the 'other application' be asked to read/write to/from standard
    input and output?

    No. I was hoping someone would come up with some kind of trick that would >make some kind of "virtual" pipe or something. Or something. I don't know.


    If it insists on "a file" can a Unix named pipe file be substituted for
    the file it expects? Read the "mknod" Linux manual page for how to
    create a named pipe.

    Yes, I can have named pipes. I've had them, but have forgotten what it
    was like. Anyway, is there any specific Tcl angle to this approach or is
    it just named pipes being named pipes?

    Tcl sockets do not fit my case, I suppose?


    There is a tcl-fuse package (https://wiki.tcl-lang.org/page/tcl-fuse).
    In theory you could have the "other application" read & write its files
    from a virtual filesystem served by tcl-fuse, in which case you would
    have Tcl procedures called whenever it tried to read or write, and could
    serve up whatever seemed appropriate for input, and do whatever you wanted
    with the output.
    --
    columbiaclosings.com
    What's not in Columbia anymore..

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Luc@21:1/5 to Ted Nolan on Sat Apr 15 22:47:23 2023
    On 15 Apr 2023 04:40:14 GMT, [email protected] (Ted Nolan ) wrote:

    There is a tcl-fuse package (https://wiki.tcl-lang.org/page/tcl-fuse).
    In theory you could have the "other application" read & write its files
    from a virtual filesystem served by tcl-fuse, in which case you would
    have Tcl procedures called whenever it tried to read or write, and could serve up whatever seemed appropriate for input, and do whatever you wanted with the output.


    A-ha! There it is.

    That is ingenious and might be just what the doctor ordered to help me
    manage the queues. There is a learning curve ahead for me, but it sounds
    like it will be a very good fit.

    This is why it's good to ask.

    Thank you, all.

    --
    Luc


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From saitology9@21:1/5 to Luc on Sun Apr 16 19:29:48 2023
    On 4/14/2023 12:19 PM, Luc wrote:
    I want Tcl to talk to an application. This application has no
    communication mechanisms. It isn't designed to talk at all except
    that it can write data to the disk. It can also read from the disk.

    ...

    What approach would you recommend to make my communication
    system smoother?


    The sanest approach, to me, is to simply contact the vendor of that
    application and ask them to add the features you want. Even if they
    don't right away, they may do so in the future on their own which would
    mean you wasted your time with your own kludge, or they may change the application in other ways that render your kludge inapplicable.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Luc@21:1/5 to Ted Nolan on Mon Apr 17 18:26:00 2023
    On 15 Apr 2023 04:40:14 GMT, [email protected] (Ted Nolan ) wrote:

    There is a tcl-fuse package (https://wiki.tcl-lang.org/page/tcl-fuse).
    In theory you could have the "other application" read & write its files
    from a virtual filesystem served by tcl-fuse, in which case you would
    have Tcl procedures called whenever it tried to read or write, and could serve up whatever seemed appropriate for input, and do whatever you wanted with the output.

    Sorry to bother you all with this but I am struggling too much to make any decent progress.

    If I understand it correctly, I need to be able to write files in the
    virtual file system so I can only use FTP or MK4 because all others are read-only. But I don't see how FTP would work since it's not really FTP,
    so I guess I will have to use MK4. Correct so far?

    Now, am I supposed to create a new MK4 VFS? An empty one?
    And how do I connect that to tcl-fuse? I ran some commands and fuse shows
    that a directory is mounted, but it complains that it is "not connected
    on the other end." How do I connect the two things?

    Can someone please advise with more detail?

    --
    Luc


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ted Nolan @21:1/5 to [email protected] on Mon Apr 17 22:00:10 2023
    In article <[email protected]>, Luc <[email protected]d> wrote: >On 15 Apr 2023 04:40:14 GMT, [email protected] (Ted Nolan ) wrote:

    There is a tcl-fuse package (https://wiki.tcl-lang.org/page/tcl-fuse).
    In theory you could have the "other application" read & write its files
    from a virtual filesystem served by tcl-fuse, in which case you would
    have Tcl procedures called whenever it tried to read or write, and could
    serve up whatever seemed appropriate for input, and do whatever you wanted >> with the output.

    Sorry to bother you all with this but I am struggling too much to make any >decent progress.

    If I understand it correctly, I need to be able to write files in the
    virtual file system so I can only use FTP or MK4 because all others are >read-only. But I don't see how FTP would work since it's not really FTP,
    so I guess I will have to use MK4. Correct so far?

    Now, am I supposed to create a new MK4 VFS? An empty one?
    And how do I connect that to tcl-fuse? I ran some commands and fuse shows >that a directory is mounted, but it complains that it is "not connected
    on the other end." How do I connect the two things?

    Can someone please advise with more detail?


    Again it's been several years since I did a minimal use of tcl-fuse, but
    you have got to supply Tcl procedures to do everything a filesystem
    does. For instance, if somebody does an "ls" on the virtual filesystem,
    your procedure has got to say what files appear to be in it.

    The fusevfs.tcl file in the examples directory is probably the
    place to start. IIRC, I used the code there as a starting point.

    Not sure what mk4 is, but I don't see FTP coming into it anywhere.
    --
    columbiaclosings.com
    What's not in Columbia anymore..

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich@21:1/5 to Luc on Mon Apr 17 22:12:27 2023
    Luc <[email protected]d> wrote:
    On 15 Apr 2023 04:40:14 GMT, [email protected] (Ted Nolan ) wrote:

    There is a tcl-fuse package (https://wiki.tcl-lang.org/page/tcl-fuse).
    In theory you could have the "other application" read & write its files
    from a virtual filesystem served by tcl-fuse, in which case you would
    have Tcl procedures called whenever it tried to read or write, and could
    serve up whatever seemed appropriate for input, and do whatever you wanted >> with the output.

    Sorry to bother you all with this but I am struggling too much to make any decent progress.

    If I understand it correctly, I need to be able to write files in the
    virtual file system so I can only use FTP or MK4 because all others are read-only. But I don't see how FTP would work since it's not really FTP,
    so I guess I will have to use MK4. Correct so far?

    Now, am I supposed to create a new MK4 VFS? An empty one?
    And how do I connect that to tcl-fuse? I ran some commands and fuse shows that a directory is mounted, but it complains that it is "not connected
    on the other end." How do I connect the two things?

    Can someone please advise with more detail?

    The manpage provides a basic example (https://tcl-fuse.sourceforge.net/#EXAMPLE) that you should look over.

    Once you create the proper code, you'd have the code create a 'mount'
    at the location where you want the virtual filesystem to appear (note
    the line near th end of the example beginning with 'hello' That
    creates the 'example' vfs in the filesystem at examples/mnt (both of
    which need to exist in the current directory where you run this code).

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Luc@21:1/5 to Rich on Mon Apr 17 21:10:18 2023
    On Mon, 17 Apr 2023 22:12:27 -0000 (UTC), Rich wrote:

    The manpage provides a basic example (https://tcl-fuse.sourceforge.net/#EXAMPLE) that you should look over.

    Once you create the proper code, you'd have the code create a 'mount'
    at the location where you want the virtual filesystem to appear (note
    the line near th end of the example beginning with 'hello' That
    creates the 'example' vfs in the filesystem at examples/mnt (both of
    which need to exist in the current directory where you run this code).


    So it seems that the VFS in this case never really contains anything
    because it doesn't even exist as a file. It is a very different animal
    from a loop device for example. So it gets mounted somewhere in the "host" filesystem and if someone asks to see the content of the directory, it
    really pretends to have a file named "hello" and if someone attempts to
    open the file, it spits out some text as if it was the content of the file
    that didn't, doesn't and will never exist, not too differently from a magician's assistant sitting under the table and passing items to the
    magician during the act, or those videos with the head of a dog and the
    hands of someone pretending to be the dog's hands.

    So far, I am correct. Right?

    Now, I ran the example and opened the directory. Sure enough, there was
    the "hello" file. I opened a terminal session and ran this:


    $ tcl/editor/fuse> echo touch > hi.txt
    An error occurred while redirecting file 'hi.txt'
    open: Function not implemented

    $ tcl/editor/fuse> touch hi.txt
    touch: setting times of 'hi.txt': Function not implemented


    So why is "open" not implemented? The code specifies a proc for "Open."
    I do notice the different case, but the create command line is:

    fuse create hello -getattr Getattr -readdir Readdir -open Open -read Read -write Write

    * Note that I also have a Write proc. I added it. But it only returns,
    doing nothing.

    So I fully expect any "open" hook (or whatever it's called) to be handled
    by the "Open" proc. But it says "not implemented." What does that really
    mean?

    Then 'touch' doesn't work because, again, "not implemented." What exactly
    is "not implemented" and how do I "implement" it? My first reaction to
    this would be to write a "Touch" proc, but the same problem with "Open"
    tells me that I must be entertaining some wrong concept again (the first
    time was mistaking Tcl VFS for some kind of loop device).

    More light, please?

    --
    Luc


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich@21:1/5 to Luc on Tue Apr 18 00:59:41 2023
    Luc <[email protected]d> wrote:
    On Mon, 17 Apr 2023 22:12:27 -0000 (UTC), Rich wrote:

    The manpage provides a basic example
    (https://tcl-fuse.sourceforge.net/#EXAMPLE) that you should look over.

    Once you create the proper code, you'd have the code create a 'mount'
    at the location where you want the virtual filesystem to appear (note
    the line near th end of the example beginning with 'hello' That
    creates the 'example' vfs in the filesystem at examples/mnt (both of
    which need to exist in the current directory where you run this code).


    So it seems that the VFS in this case never really contains anything
    because it doesn't even exist as a file.

    Correct. Which is why it is a (V)irtual file system. The items inside
    don't exist, except as a sideefect of the FUSE callbacks.

    It is a very different animal from a loop device for example.

    Yes, although it is similar in one way. A loop device redirects
    accesses to some other place. The FUSE layer /redirects/ accesses into function calls within the code you write.

    So it gets mounted somewhere in the "host" filesystem and if someone
    asks to see the content of the directory, it really pretends to have
    a file named "hello" and if someone attempts to open the file, it
    spits out some text as if it was the content of the file that didn't,
    doesn't and will never exist, not too differently from a magician's
    assistant sitting under the table and passing items to the magician
    during the act, or those videos with the head of a dog and the hands
    of someone pretending to be the dog's hands.

    That's one way to describe it, and it is a pretty good overall
    description. The functions in the code are the magicians assistants,
    handing out what data needs to be handed out to make the /illusion/
    complete.

    So far, I am correct. Right?

    Yep, overall on the right track.

    Now, I ran the example and opened the directory. Sure enough, there
    was the "hello" file. I opened a terminal session and ran this:

    $ tcl/editor/fuse> echo touch > hi.txt
    An error occurred while redirecting file 'hi.txt'
    open: Function not implemented

    $ tcl/editor/fuse> touch hi.txt
    touch: setting times of 'hi.txt': Function not implemented


    So why is "open" not implemented? The code specifies a proc for "Open."

    Open *is* implemented, only a very restricted 'open'. Note this if
    clausse in the 'open' proc in the example:

    if {"RDONLY" ni [dict get $fileinfo flags]} {
    return -code error -errorcode [list POSIX EACCES {}]
    }

    The example code only implements an open that allows "RDONLY" (Read
    Only) access to the virtual files.

    I do notice the different case, but the create command line is:

    fuse create hello -getattr Getattr -readdir Readdir -open Open -read Read -write Write

    * Note that I also have a Write proc. I added it. But it only returns,
    doing nothing.

    Yes, the original example code did not have a 'write' -- and an empty do-nothing write proc won't allow you to 'write' anything. And with
    the 'open' proc rejecting any open other than a read-only open, most
    operations will still be rejected, even with a write proc installed.

    Also, the fuse functions name Open/Read/Write correspond rather closely
    to the low level raw libc file interface, not the Tcl commands of the
    same name.

    So I fully expect any "open" hook (or whatever it's called) to be handled
    by the "Open" proc. But it says "not implemented." What does that really mean?

    Exactly what it says. The example "open" proc has no implementation to
    allow for a 'read-write' or a 'write only' or an 'append' open.

    Then 'touch' doesn't work because, again, "not implemented." What exactly
    is "not implemented" and how do I "implement" it?

    You have to add code to the open proc to handle the cases you want to implement, and return error codes for the cases you don't implement.
    Then you'd eventually need a write proc to handle the actual writes to
    a virtual file, when you get to that point.

    My first reaction to this would be to write a "Touch" proc, but the
    same problem with "Open" tells me that I must be entertaining some
    wrong concept again (the first time was mistaking Tcl VFS for some
    kind of loop device).

    'touch' is a command, it's implementation is in the form of a set of
    calls to the low level libc file procs. At it's most basic level,
    'touch' would be an 'open' with at least mode CREAT to create a file if
    it does not exist, followed by a call to update the file timestamp (I
    forget the libc name of this function right now), followed by a close
    of the open file. You get the 'not implemented' error because the
    example open rejects anything but RDONLY access modes, and 'touch' is
    opening the file in a (likely) WRONLY or RDWR modes.

    Add some puts to stderr in each of the example procs so you can follow
    the call flow and you'll see more of what is happening under the hood.

    More light, please?

    Hopefully the above is slightly illuminating.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Christian Gollwitzer@21:1/5 to All on Tue Apr 18 07:59:45 2023
    Am 18.04.23 um 02:10 schrieb Luc:
    $ tcl/editor/fuse> echo touch > hi.txt
    An error occurred while redirecting file 'hi.txt'
    open: Function not implemented

    $ tcl/editor/fuse> touch hi.txt
    touch: setting times of 'hi.txt': Function not implemented


    Rich answered a plan what you will have to do to make it work. There is
    some serious work to do before you reach your goal of remote-controlling another application via a VFS. I don't want to discourage you, but maybe
    if you reveal what application you want to control, then maybe someone
    can suggest another way or a similar program which does the same, but is
    more easily scriptable.

    Happy hacking,

    Christian

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Luc@21:1/5 to Christian Gollwitzer on Tue Apr 18 16:41:31 2023
    On Tue, 18 Apr 2023 07:59:45 +0200, Christian Gollwitzer wrote:

    Rich answered a plan what you will have to do to make it work. There is
    some serious work to do before you reach your goal of remote-controlling another application via a VFS. I don't want to discourage you, but maybe
    if you reveal what application you want to control, then maybe someone
    can suggest another way or a similar program which does the same, but is
    more easily scriptable.

    Happy hacking,

    Christian


    "Some serious work"?

    All I want is something to tell me (or rather tell my code) that
    "something wants to write {this content} into this_file" and *maybe*
    write some kind of response to another file.

    In either case, "this_file" is like an address that indicates who is
    supposed to receive the message and {this content} is the message.
    That's all.

    I am beginning to give up on this idea. It's becoming too low level
    or C-ish for my taste. I like high level. I like Tcl.


    --
    Luc


    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)