• Most Straight-Up Way to Retrive COMPLETE JPGs in OpenCV Environment

    From c186282@21:1/5 to All on Thu May 29 01:27:52 2025
    MOST of my JPGs come from SECURITY CAMS. They get refreshed, by
    the cam, about once per second. Good enough.

    But, problem, it's kinda pure CHANCE as to whether I get
    a COMPLETEL jpg. The cams may update at any moment.

    The RESULTS are too-often incomplete images.

    Now software like Motion does find the proper
    beginning/end of JPGs ... but it's kinda high
    CPU. RTSP ... same, but even more CPU and
    a LOT more latency. I have 8 cams now, complete
    ring around the property, Motion will use TOO
    much CPU. RTSP, a good minute or more behind
    events unless you're using a major CPU. Do
    NOT want to afford/use major CPU for this.
    More like N-95/N-100 level. Abandoned the PIs,
    just NOT quite enough.

    This has bugged me for YEARS.

    I like to use OpenCV (python) for this stuff.
    Good lib. Really 'C' ... the Python is just
    the execution infrastructure. No real advantage
    to going all-'C' except obscurity.

    Clearly though, OpenCV isn't entirely oriented
    towards securing COMPLETE jpgs. It acts on
    WHAT'S THERE at the moment. MIGHT be complete,
    or maybe not.

    As best I can tell, kinda I have to read the
    jpgs byte-by-byte - accommodating/restarting
    when the start bytes (too few IMHO) are detected -
    and building the complete JPG in a buffer. Ugly.
    DO-able, but ugly.

    Fortunately my bandwidth/refresh is SLOW,
    it's all low-end wi-fi at best. If I needed
    25+ frames per - forget it. This means the
    CPU I have has a *chance* at doing what
    I want.

    ANYway - does anybody here have better experience
    with this particular kind of issue ... good links
    at least ? Most of my searches don't turn up
    anything really useful/do-able.

    This crap is beyond 'theoretical', it involves
    real external devices and such.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich@21:1/5 to [email protected] on Thu May 29 15:15:39 2025
    c186282 <[email protected]> wrote:
    MOST of my JPGs come from SECURITY CAMS. They get refreshed, by the
    cam, about once per second. Good enough.

    But, problem, it's kinda pure CHANCE as to whether I get a COMPLETEL
    jpg. The cams may update at any moment.

    The problem here is a classic race condition. Your 'file reading'
    program is not synchronized with the 'file writing' program (the cam)
    and so it is opening files the camera has not yet finished writing.

    Some ideas for solutions:

    Can you configure the camera to write to separate jpeg files (i.e.
    0000.jpeg, 0001.jpeg, etc.)? If yes, then you could ignore the
    "newest" file (as it might still be getting written) and only open the
    "second newest" and older files instead.

    If the camera always writes to the same file (this is a poor design,
    but if it is what the camera does and you can't change it you have
    little recourse) then what filesystem does the camera write to? Is it
    a Linux filesystem? If yes, you /might/ be able to use an inotify
    event of "file closed" to synchronize your reader to the writer such
    that the reader only reads the file just after the camera closed it
    (assuming the camera wrote a complete file, then you'd get a 1 second
    chance to read the file whole). If you go this route I'd suggest you
    simply do a straight "file copy" when the inotify event triggers, then
    open and process that copy. That gives you "1 second" to complete the
    copy (fesable) and your "after processing" can take more than 1 second
    without worry at that point.

    Third idea, if your cameras provide a "web" api to retreive the current
    image, maybe retreving an image via that method to use might give you a
    fully valid image.

    Clearly though, OpenCV isn't entirely oriented towards securing
    COMPLETE jpgs. It acts on WHAT'S THERE at the moment. MIGHT be
    complete, or maybe not.

    Not really OpenCV's fault when the file it is reading is incomplete.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andy Burns@21:1/5 to Rich on Thu May 29 16:24:33 2025
    Rich wrote:

    Can you configure the camera to write to separate jpeg files

    Or is it just constantly streaming to an MJPEG file? How long a
    sequence does it write before moving on to a new one?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)