• data pumping and non-blocking write to channels

    From Andreas Leitgeb@21:1/5 to All on Thu Jan 27 18:20:27 2022
    I've got a Tcl script that does "data pumping" - i.e. read avaliable
    data from one channel, and write it to another channel. Actually, also
    vice versa: if the other channel becomes readable, I forward it to the
    first channel, and there are deeper details, that prevent me from
    using programs like "socat".

    The relevant part is, that all channels have fileevents set for readable,
    and the channels are all set to non-blocking, and any chunk read from one channel is immediately written to the other one (with some modifications
    on the data).

    It may happen that more data gets dumped onto a channel (e.g. serial
    device, or slow network connection) than the channel's OS-buffers
    can accept. In that case, the write syscall fails with EAGAIN and
    tcl seems to keep them until some time later - e.g. until next "puts",
    and a simple "flush" on the channel doesn't seem to cause a retry.

    In the meantime, I made sure that a certain pattern written to the
    channel will be safely filtered out at the receiving end, so every
    second I "puts" a special mini-packet to make sure that tcl's internal
    buffers eventually get drained to the OS.

    Now for the questions:

    Are my observations "defined behaviour" ?

    Or would it be reasonable for tcl, on receiving the EAGAIN to schedule
    a proper re-flush for a bit later?

    Or would it be reasonable to allow me some inspection on "flushed-but- not-yet-written" data right after the "puts", and a means to re-flush
    it (without current necessity of adding extra dummy data to the stream)?

    I could also maintain an output buffer, and set up fileevent writeable
    to puts the buffer piecewise into the channel, and remove the writable
    event whenever the buffer is empty. But without inspection into the OS's buffer size, I might still end up writing too much, and all communication getting stuck again.

    Is my solution with special "ping" packets the only one available right
    now?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)