• FASTCIRC--High-speed circles for HGR graphics

    From Michael J. Mahon@21:1/5 to All on Fri Oct 21 00:01:44 2022
    If you've ever wanted to create HGR graphics containing circles of
    various sizes, you’ve probably been disappointed with the slow drawing
    speed. The straightforward method using sines and cosines can take
    several seconds per circle.

    I’ve just added a short (374 bytes) subroutine, FASTCIRC, to my website
    that draws circles in from 22 to 75 milliseconds, depending on radius.
    It’s self-relocating, so you can BLOAD it anywhere convenient.

    To use FASTCIRC, the center (Xlo, Xhi, Y) and the radius are POKEd into locations 6, 7, 8, and 9 and FASTCIRC is CALLed at its load address.

    My website has more complete documentation and a FASTCIRC.DSK image
    containing FASTCIRC, its Merlin source, and several demo programs to
    illustrate its use.

    Time to have fun with circles!
    --

    -michael

    NadaNet 3.1 for Apple II parallel computing!
    Home page: http://michaeljmahon.com

    "The wastebasket is our most important design
    tool--and it's seriously underused."

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From fadden@21:1/5 to Michael J. Mahon on Fri Oct 21 08:05:26 2022
    On Friday, October 21, 2022 at 12:01:49 AM UTC-7, Michael J. Mahon wrote:
    If you've ever wanted to create HGR graphics containing circles of
    various sizes, you’ve probably been disappointed with the slow drawing speed.

    It's nice to know I'm not the only one. :-)

    The self-relocation routine has that fun "living on the edge" feel to it. Very nice.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From TRS-90@21:1/5 to All on Fri Oct 21 23:44:11 2022
    Time to have fun with circles!
    -michael
    Home page: http://michaeljmahon.com

    Hi Michael, can I just say a big "thank you" for keeping your website http so folks like me (on a IIgs) can read your post and then go check out the website without switching to modern hardware? It adds so much atmosphere and fun to be able to do that. I
    even downloaded the .dsk via the IIgs, however I haven't yet figured out how to access .dsk, .po, etc... once downloaded without switching to moden hardware and making an image with CiderPress. But that's besides the point. Great website and it works on "
    vintage" (haha) hardware!

    Matt

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael J. Mahon@21:1/5 to fadden on Sat Oct 22 01:31:45 2022
    fadden <[email protected]> wrote:
    On Friday, October 21, 2022 at 12:01:49 AM UTC-7, Michael J. Mahon wrote:
    If you've ever wanted to create HGR graphics containing circles of
    various sizes, you’ve probably been disappointed with the slow drawing
    speed.

    It's nice to know I'm not the only one. :-)

    The self-relocation routine has that fun "living on the edge" feel to it. Very nice.


    Thanks! I’m pretty happy with my “key byte(s)” relocation scheme. I use it on most programs of few pages, since in a short chunk of code it’s extremely unlikely that all 256 byte values appear (and impossible in less
    than one page of code!).

    I actually use a short Applesoft program to histogram the byte values in
    the target program and print a list of all the unused values. It’s typical that there are several runs of consecutive “missing” values, so it’s not difficult to choose unique values to be used as “keys” to flag addresses requiring relocation. Setting the assembly ORG then completes the job. ;-)

    I haven’t seen this approach used by others, but I’d be surprised if it hasn’t been.

    Of course, if code can be written so that it’s location-independent, that’s great, but if you need internal subroutines or data, relocation is
    necessary.

    --
    -michael - NadaNet 3.1 and AppleCrate II: http://michaeljmahon.com

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Oliver Schmidt@21:1/5 to All on Sat Oct 22 09:28:19 2022
    Hi Matt,

    [...] however I haven't yet figured out how to access .dsk, .po, etc...
    once downloaded without switching to moden hardware and making an image with CiderPress.

    There are (at least) these options:

    - http://www.sheppyware.net/apple-ii-software/diskmaker.html

    - https://archive.org/details/diskmaker8

    - https://mirrors.apple2.org.za/ground.icaen.uiowa.edu/apple8/Utils/dsk2file.info

    Or bypass the image file step altogether with https://github.com/cc65/ip65/wiki/Wget65

    Regards,
    Oliver

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andrew Roughan@21:1/5 to Oliver Schmidt on Sat Oct 22 11:53:53 2022
    Oliver Schmidt <[email protected]> wrote:
    Hi Matt,

    [...] however I haven't yet figured out how to access .dsk, .po, etc...
    once downloaded without switching to moden hardware and making an image with CiderPress.

    There are (at least) these options:

    Have you tried MountIt ? https://www.brutaldeluxe.fr/products/apple2gs/mountit.html

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From TRS-90@21:1/5 to Oliver Schmidt on Sun Oct 23 02:52:20 2022
    "Oliver Schmidt" wrote:

    Hi Matt,

    There are (at least) these options:
    - http://www.sheppyware.net/apple-ii-software/diskmaker.html
    - https://archive.org/details/diskmaker8 https://mirrors.apple2.org.za/ground.icaen.uiowa.edu/apple8/Utils/dsk2fil e.info

    Or bypass the image file step altogether with https://github.com/cc65/ip65/wiki/Wget65

    Thank you, man, Will do. I hope comp.sys.apple2 sticks around for another 30 years. I learn so much here. Hahaha any hope for a favoites page in the 8-bit Contiki browser?

    ** Proudly posted from an Apple IIgs **

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Oliver Schmidt@21:1/5 to All on Sun Oct 23 08:30:20 2022
    Hi Matt,

    Thank you, man, Will do.

    You're welcome :-)

    I hope comp.sys.apple2 sticks around for another 30 years. I learn so much here.

    Me too!

    Hahaha any hope for a favoites page in the 8-bit Contiki browser?

    Not much memory left for any feature whatsoever. Maybe a prefilled start
    URL (aka home page) set to http://frogfind.com

    Regards,
    Oliver

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kent Dickey@21:1/5 to Michael J. Mahon on Tue Oct 25 04:56:25 2022
    In article <[email protected]>,
    Michael J. Mahon <[email protected]> wrote:
    If you've ever wanted to create HGR graphics containing circles of
    various sizes, you’ve probably been disappointed with the slow drawing >speed. The straightforward method using sines and cosines can take
    several seconds per circle.

    I’ve just added a short (374 bytes) subroutine, FASTCIRC, to my website >that draws circles in from 22 to 75 milliseconds, depending on radius.
    It’s self-relocating, so you can BLOAD it anywhere convenient.

    To use FASTCIRC, the center (Xlo, Xhi, Y) and the radius are POKEd into >locations 6, 7, 8, and 9 and FASTCIRC is CALLed at its load address.

    My website has more complete documentation and a FASTCIRC.DSK image >containing FASTCIRC, its Merlin source, and several demo programs to >illustrate its use.

    Time to have fun with circles!
    --

    -michael

    Very cool little program, it draws fast circles.

    The FASTCIRC.dsk image is actually in ProDOS order. Could you rename
    the image to FASTCIRC.po?

    Kent

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael J. Mahon@21:1/5 to Kent Dickey on Wed Oct 26 06:59:36 2022
    Kent Dickey <[email protected]> wrote:
    In article <[email protected]>,
    Michael J. Mahon <[email protected]> wrote:
    If you've ever wanted to create HGR graphics containing circles of
    various sizes, you’ve probably been disappointed with the slow drawing
    speed. The straightforward method using sines and cosines can take
    several seconds per circle.

    I’ve just added a short (374 bytes) subroutine, FASTCIRC, to my website
    that draws circles in from 22 to 75 milliseconds, depending on radius.
    It’s self-relocating, so you can BLOAD it anywhere convenient.

    To use FASTCIRC, the center (Xlo, Xhi, Y) and the radius are POKEd into
    locations 6, 7, 8, and 9 and FASTCIRC is CALLed at its load address.

    My website has more complete documentation and a FASTCIRC.DSK image
    containing FASTCIRC, its Merlin source, and several demo programs to
    illustrate its use.

    Time to have fun with circles!
    --

    -michael

    Very cool little program, it draws fast circles.

    The FASTCIRC.dsk image is actually in ProDOS order. Could you rename
    the image to FASTCIRC.po?

    Kent


    Hi, Kent!

    I’m glad you like it—it was fun to write!

    Interesting that you should ask about the disk image. There are so many
    disk images “out there”, many made before .po existed, that almost all emulators disregard the image name and just examine the first sectors to determine how to treat it. As a result, I’ve adopted the convention of
    naming all 5.25” images .dsk!

    I’m fact, I think all the disk images on my site are ProDOS images, but all are named .dsk—and (so far) you’re the only one to mention it! ;-)

    I’m willing to rename it, but then it would be the “inconsistent” one, or I
    could change them all (which would be a pain since I refer to them by name
    in several places in the text). I figure that if I name one image .po
    people will assume that I consider the suffix meaningful, and all the other images are *not* ProDOS! ;-)

    Frankly, I don’t think the Apple II world will ever get the naming of all
    the images consistent with their content. (And we haven’t even gotten to disks that are neither DOS nor ProDOS.) I don’t think the rampant inconsistency is a source of much confusion, either. Certainly you weren’t confused. ;-)

    Put another way, any of us who deal with disk images or write programs that deal with them, have already adapted to “misnamed” images and learned to cope with them, automatically or manually. That’s a useful skill in the
    Apple II world. And, frankly, having a single suffix about which no assumptions should be made is more consistent than trying to retroactively impose a new standard which will never apply universally.

    I originally tried posting .sdk ShrinkIt images, which are universal for recreating physical disks, but received several inquiries about what they
    were and why didn’t they work with emulators! So I switched several years ago to simply posting disk (.dsk) images. Maybe .woz images will finally become universal.

    “The great thing about standards is that there are so many to choose from.”
    And our “standard” solution is to create yet another “to rule them all”.

    If my reasoning is faulty, please straighten me out. ;-)

    --
    -michael - NadaNet 3.1 and AppleCrate II: http://michaeljmahon.com

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Oliver Schmidt@21:1/5 to All on Wed Oct 26 07:28:57 2022
    Hi,

    Interesting that you should ask about the disk image. [...]

    If there is any "standard" at all, then it's this:

    .do: DOS3.3 order
    .po: ProDOS8 order
    .dsk: Go figure

    Regards,
    Oliver

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael J. Mahon@21:1/5 to Oliver Schmidt on Fri Oct 28 06:02:00 2022
    Oliver Schmidt <[email protected]> wrote:
    Hi,

    Interesting that you should ask about the disk image. [...]

    If there is any "standard" at all, then it's this:

    .do: DOS3.3 order
    .po: ProDOS8 order
    .dsk: Go figure

    Regards,
    Oliver


    Fair enough, though the Apple II world seems to have lots of other image
    types, many of which aim(ed) to be more “universal”.

    But I stand corrected, and will review my site and adopt the “standard” as I have the time.

    --
    -michael - NadaNet 3.1 and AppleCrate II: http://michaeljmahon.com

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Oliver Schmidt@21:1/5 to All on Fri Oct 28 11:13:43 2022
    Hi Michael,

    If there is any "standard" at all, then it's this:

    .do: DOS3.3 order
    .po: ProDOS8 order
    .dsk: Go figure

    But I stand corrected, and will review my site and adopt the “standard” as
    I have the time.

    There seems to be a misunderstanding. I do NOT see an issue with .dsk files having ProDOS ordering!

    .do and .po both imply a statement regarding their ordering but .dsk does
    NOT imply any ordering - and therefore a .dsk can per definition never have
    a wrong ordering.

    At least that is my POV which I wanted to express with my last posting.

    Regards,
    Oliver

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kent Dickey@21:1/5 to [email protected] on Fri Oct 28 17:42:30 2022
    In article <tjgdh7$4qa7$[email protected]>, Oliver Schmidt <[email protected]> wrote: >Hi Michael,

    If there is any "standard" at all, then it's this:

    .do: DOS3.3 order
    .po: ProDOS8 order
    .dsk: Go figure

    But I stand corrected, and will review my site and adopt the “standard” as
    I have the time.

    There seems to be a misunderstanding. I do NOT see an issue with .dsk files >having ProDOS ordering!

    .do and .po both imply a statement regarding their ordering but .dsk does
    NOT imply any ordering - and therefore a .dsk can per definition never have
    a wrong ordering.

    At least that is my POV which I wanted to express with my last posting.

    Regards,
    Oliver

    The two main problems with .dsk being "Go figure" is:

    1) It makes emulators harder to write. An Apple II emulator is pretty easy
    to get started with, and my guess is there are at least 20 freely
    available. But: needing to auto-detect .dsk can be quite complex to do well, and it's one of many features that emulators seem to need that raises the bar to get new emulators.

    2) It's actually not possible to detect PO or DO for an arbitrary .dsk
    image. Let me see your code you're using to auto-detect, and I can
    create an image which it will fail on. And many games have no OS, so there's no great way to detect PO or DO of these images.

    I think AppleWin's detection algorithm for .dsk is:

    - Detect .DO order first. See if the link pointers on Track $11 look
    like a DOS 3.3 catalog track for all sectors 0-14, looking at byte
    +$02 in each sector. If it is, it's .DO. Then look to see if
    blocks 2,3,4,5 on track $00 look like they have the ProDOS links
    in the first 4 bytes of each block look valid for DOS 3.3 order.
    If so, it's .DO. Otherwise, it's MAYBE .DO, which will be the
    choice if nothing else matches.

    - Detect .PO order next. Checks DOS 3.3 catalog track first, but only checks
    some of the sectors (I'm not sure why, probably a trick of sector
    numbers). If it looks valid, it's .PO. Then check blocks
    2,3,4,5 for the ProDOS links in the first 4 bytes. If it
    matches, it's PO order.

    - Anything not detected defaults to .DO order

    Note, this can be confused by a file on a ProDOS image in ProDOS order
    that is on track $11 (blocks 136-151) and matches the DOS 3.3 pattern it's looking for. It would be better if AppleWin checked ProDOS directory patterns first when considering ProDOS order, to help avoid this confusion. And I'm
    not picking on AppleWin, it just is the easiest source for me to look at.

    Kent

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From D Finnigan@21:1/5 to D Finnigan on Fri Oct 28 19:23:33 2022
    D Finnigan wrote:
    Kent Dickey wrote:

    2) It's actually not possible to detect PO or DO for an arbitrary .dsk
    image. Let me see your code you're using to auto-detect, and I can
    create an image which it will fail on.

    But that's not the criteria is it? We don't need to write code that can detect any arbitrary image which you create solely for the purpose of
    failing said code.

    The set of all Apple II disks in existence is finite.


    When I wrote and tested my Apple II disk format detection code for the Mac
    GUI Vault web site, I tested it against however many thousand disk images I
    had in the Apple II collection.

    A finite set.

    --
    ]DF$
    The New Apple II User's Guide:
    https://macgui.com/newa2guide/

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From D Finnigan@21:1/5 to Kent Dickey on Fri Oct 28 19:21:38 2022
    Kent Dickey wrote:

    2) It's actually not possible to detect PO or DO for an arbitrary .dsk
    image. Let me see your code you're using to auto-detect, and I can
    create an image which it will fail on.

    But that's not the criteria is it? We don't need to write code that can
    detect any arbitrary image which you create solely for the purpose of
    failing said code.

    The set of all Apple II disks in existence is finite.

    --
    ]DF$
    The New Apple II User's Guide:
    https://macgui.com/newa2guide/

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From fadden@21:1/5 to D Finnigan on Sat Oct 29 07:47:00 2022
    On Friday, October 28, 2022 at 12:21:40 PM UTC-7, D Finnigan wrote:
    But that's not the criteria is it? We don't need to write code that can detect any arbitrary image which you create solely for the purpose of failing said code.

    The set of all Apple II disks in existence is finite.

    Even if your auto-detection succeeds on 100% of all known DOS and ProDOS (and Pascal and CP/M) disks, you will be unable to auto-detect the layout of disks with a customized operating system, of which there are many. I believe most ".dsk" images are DOS
    order, which means the odds of success are > 50% on a blind guess... but why guess at all, when giving the image the correct filename extension removes the ambiguity?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael 'AppleWin Debugger Dev'@21:1/5 to Kent Dickey on Sat Oct 29 10:17:36 2022
    On Friday, October 28, 2022 at 10:42:32 AM UTC-7, Kent Dickey wrote:

    I think AppleWin's detection algorithm for .dsk is:
    - Detect .DO order first.
    - Detect .PO order next.
    - Anything not detected defaults to .DO order

    Yes, that's correct. CDiskImageHelper::Detect() runs through the supported disk types (initialized in the CDiskImageHelper::CDiskImageHelper() ctor) calling each Disk Type's Detect() function.
    i.e.
    CDoImage::Detect()
    CPoImage::Detect()
    etc.

    Note, this can be confused by a file on a ProDOS image in ProDOS order
    that is on track $11 (blocks 136-151) and matches the DOS 3.3 pattern it's looking for. It would be better if AppleWin checked ProDOS directory patterns
    first when considering ProDOS order, to help avoid this confusion. And I'm not picking on AppleWin, it just is the easiest source for me to look at.

    Tom (or 4am or maybe qkumba) would know better then me but I don't believe defaulting to ProDOS sector order is a good idea for the (historical) reasons Andy mentioned.

    Ideally, someone would do an in-depth analysis of the thousands of images and verify this assumption. In the 80's the number of DOS formatted disks dwarfed the ProDOS format. My gut feeling is that ProDOS didn't really take off in popularity until
    well into the 2000's when emulators made it trivial to switch. I would also say qkumba converting games to ProDOS also had an impact. And probably John Brooks excellent work that allowed newer version of ProDOS to run on 6502s.

    Given the fact that it is not possible to write a 100% fool proof system there will always be edge cases that fail. Whoever is making the initial disk image "should" be properly tagging the disk format. But alas not everyone takes the same pride so we
    have some (many?) disks that aren't properly tagged and as such need to guess.

    Regardless, please open an AppleWin issue so the rest of the team can mull this over and see if it is time to auto-detect a ProDOS volume directory first. Thanks.

    Michael

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kent Dickey@21:1/5 to [email protected] on Sat Oct 29 18:50:16 2022
    In article <[email protected]>,
    D Finnigan <[email protected]> wrote:
    D Finnigan wrote:
    Kent Dickey wrote:

    2) It's actually not possible to detect PO or DO for an arbitrary .dsk
    image. Let me see your code you're using to auto-detect, and I can
    create an image which it will fail on.

    But that's not the criteria is it? We don't need to write code that can
    detect any arbitrary image which you create solely for the purpose of
    failing said code.

    The set of all Apple II disks in existence is finite.


    When I wrote and tested my Apple II disk format detection code for the Mac >GUI Vault web site, I tested it against however many thousand disk images I >had in the Apple II collection.

    A finite set.

    --
    ]DF$
    The New Apple II User's Guide:
    https://macgui.com/newa2guide/

    Can you share some of this information? Do you know roughly what is the percentage of .dsk images that are in ProDOS order? I do know the biggest irony is DOS 3.3 Master.dsk on Asimov was in ProDOS order.

    I think what you're saying is, if someone wants to throw together an Apple II emulator (say, written in whatever new language is hot this year), before releasing it they need to:

    - Write .dsk auto-detect code. As far as I know, my description of AppleWin's
    algorithm is the first public documentation of .dsk sector order
    detection. Otherwise, they would need to do a bunch of research to
    decide what to do. Note: other disk formats other than DOS 3.3 and
    ProDOS are not covered, such as Pascal.

    - Download every Apple II .dsk image available on the web.
    How should I go about doing this? I am unable to see how to download
    all Apple II files from macgui.com, can you provide some help?

    - Test their auto-detect code and make sure they don't make any mistakes.

    This seems like a pretty high bar, and I'm not sure any existing emulator
    meets this requirement.

    Kent

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From fadden@21:1/5 to Michael 'AppleWin Debugger Dev' on Sat Oct 29 15:55:41 2022
    On Saturday, October 29, 2022 at 10:17:38 AM UTC-7, Michael 'AppleWin Debugger Dev' wrote:
    My gut feeling is that ProDOS didn't really take off in popularity until well into the 2000's when emulators made it trivial to switch.

    Early emulators assumed all non-nibble disk images were in DOS order, because that's what the disk transfer utilities available back then used.

    I suspect that what propelled ProDOS-ordered images was the use of ShrinkIt to compress disks, and utilities like NuLib that would unpack them into a ProDOS-ordered file. When using that approach, you'd need to use a second utility to reorder the
    sectors to DOS order so the image would be accepted by the emulator. Support for both DOS and ProDOS ordering in emulators came later, and I don't think any emulators used auto-detection until CiderPress started doing it (2003).

    The ".2img" format was designed to resolve the ambiguity once and for all, but it never really took off.

    One of my tests for CiderPress is to run MDC (Multi-Disk Catalog) on a partial mirror of asimov. (I occasionally update a copy here: https://faddensoftware.com/ .) One could instrument MDC to report the number of ambiguous / mis-labeled /
    unrecognizable images there are, were one sufficiently interested in doing that, but I'm not sure this discussion is really hampered by a lack of data. (FWIW, a simple grep reveals there are 10,411 disk image files with recognizable filesystems named "*.
    dsk" in my data set, though some of them are larger than 140KB and hence unambiguously ProDOS-ordered.)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Steve Nickolas@21:1/5 to fadden on Sat Oct 29 23:40:58 2022
    On Sat, 29 Oct 2022, fadden wrote:

    On Friday, October 28, 2022 at 12:21:40 PM UTC-7, D Finnigan wrote:
    But that's not the criteria is it? We don't need to write code that can
    detect any arbitrary image which you create solely for the purpose of
    failing said code.

    The set of all Apple II disks in existence is finite.

    Even if your auto-detection succeeds on 100% of all known DOS and ProDOS
    (and Pascal and CP/M) disks, you will be unable to auto-detect the
    layout of disks with a customized operating system, of which there are
    many. I believe most ".dsk" images are DOS order, which means the odds
    of success are > 50% on a blind guess... but why guess at all, when
    giving the image the correct filename extension removes the ambiguity?


    I consider .DSK to mean DOS order, and if a ProDOS order file is given the
    .DSK extension I consider it a mistake.

    But having the option to override on open is nice.

    -uso.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Oliver Schmidt@21:1/5 to All on Sun Oct 30 11:48:50 2022
    Hi Kent,

    It seems to me that your topic is by now rather far away from where we
    (or at least I) started.

    But let me first try to avoid potential misunderstandings:

    1. Is it great that .DSK images are often DOS3.3 and often ProODS
    ordered? No.

    Personal note: If I could decide from scratch, I'd have .DSK images
    simply being always in the pyhsical sector order. My first
    pre-internet emulator worked that way.

    2. Is it great that the most popular Apple II disk image type has to
    go without any meta data at all. No.

    Personal note: If I could decide from scratch, I'd have all disk
    images come with extensible meta data.

    Okay, now to what used to be the topic at hand:

    1. Is the issues with .DSK images noticably reduced by Michael
    renaming his .DSK file to .PO? No.

    2. Is Michael's .DSK file hard to auto-detect? No.

    Just my two cents,
    Oliver

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kent Dickey@21:1/5 to [email protected] on Sun Oct 30 18:07:51 2022
    In article <[email protected]>,
    fadden <[email protected]> wrote:
    On Saturday, October 29, 2022 at 10:17:38 AM UTC-7, Michael 'AppleWin >Debugger Dev' wrote:
    My gut feeling is that ProDOS didn't really take off in popularity
    until well into the 2000's when emulators made it trivial to switch.

    Early emulators assumed all non-nibble disk images were in DOS order,
    because that's what the disk transfer utilities available back then
    used.

    I suspect that what propelled ProDOS-ordered images was the use of
    ShrinkIt to compress disks, and utilities like NuLib that would unpack
    them into a ProDOS-ordered file. When using that approach, you'd need
    to use a second utility to reorder the sectors to DOS order so the image >would be accepted by the emulator. Support for both DOS and ProDOS
    ordering in emulators came later, and I don't think any emulators used >auto-detection until CiderPress started doing it (2003).

    The ".2img" format was designed to resolve the ambiguity once and for
    all, but it never really took off.

    One of my tests for CiderPress is to run MDC (Multi-Disk Catalog) on a >partial mirror of asimov. (I occasionally update a copy here: >https://faddensoftware.com/ .) One could instrument MDC to report the
    number of ambiguous / mis-labeled / unrecognizable images there are,
    were one sufficiently interested in doing that, but I'm not sure this >discussion is really hampered by a lack of data. (FWIW, a simple grep >reveals there are 10,411 disk image files with recognizable filesystems
    named "*.dsk" in my data set, though some of them are larger than 140KB
    and hence unambiguously ProDOS-ordered.)

    Can you describe CiderPress's algorithm for determining sector order of
    a .dsk file?

    Kent

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kent Dickey@21:1/5 to Oliver Schmidt on Sun Oct 30 18:06:08 2022
    In article <tjlob2$6u0v$[email protected]>, Oliver Schmidt <[email protected]> wrote:
    Hi Kent,

    It seems to me that your topic is by now rather far away from where we
    (or at least I) started.

    But let me first try to avoid potential misunderstandings:

    1. Is it great that .DSK images are often DOS3.3 and often ProODS
    ordered? No.

    Personal note: If I could decide from scratch, I'd have .DSK images
    simply being always in the pyhsical sector order. My first
    pre-internet emulator worked that way.

    2. Is it great that the most popular Apple II disk image type has to
    go without any meta data at all. No.

    Personal note: If I could decide from scratch, I'd have all disk
    images come with extensible meta data.

    Okay, now to what used to be the topic at hand:

    1. Is the issues with .DSK images noticably reduced by Michael
    renaming his .DSK file to .PO? No.

    Actually, my thesis is most of these .DSK images in ProDOS sector order are relatively recent creations (within the last 20 years), and are not historical. And so asking people who are creating these images now to use .PO for ProDOS order would achieve my goal.

    2. Is Michael's .DSK file hard to auto-detect? No.

    Just my two cents,
    Oliver

    What is your suggested algorithm for auto-detecting the sector order of
    .dsk images?

    Kent

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From fadden@21:1/5 to Kent Dickey on Sun Oct 30 14:43:35 2022
    On Sunday, October 30, 2022 at 11:07:53 AM UTC-7, Kent Dickey wrote:
    Can you describe CiderPress's algorithm for determining sector order of
    a .dsk file?

    "Complicated".

    It tries DOS, ProDOS/Pascal, CP/M, and physical sector orders. (CP/M ordering is not really a thing, since nobody is creating disk images with CP/M software... but we might as well collect the whole set.) It tests the filesystem for DOS, ProDOS, Pascal,
    CP/M, RDOS, Gutenberg, and is probably checking for HFS even though I doubt anybody put that on a 5.25" floppy.

    The tests for DOS / ProDOS filesystems have to take into account non-5.25" situations, like UNIDOS disks and DOS Master floppies, so there's a bunch of auto-detect code that doesn't really apply to 5.25" disks.

    The tricky part is finding the right thresholds for the filesystem check. If you find 15 catalog sectors in the right order on track 17, you're done. If you find 2, you've got the wrong order. If you find 4... could be the wrong order plus some lucky
    bytes, could be a DOS disk with a custom short catalog track. So you try each order, and come up with a "yes definitely", "no definitely", or "maybe go with this one if you can't find anything better".

    The filename extension is used as a hint, but enough files were named the wrong way that CiderPress doesn't pay much attention to it. It does allow the code to try the most likely order first, so if we get a "yes definitely" from a filesystem we can
    just stop looking. Also, if we can't identify the filesystem, we can open the sector editor with order indicated by the extension.

    It Would Be Nice if CiderPress complained about incorrect or ambiguous filename extensions to encourage people to correct them.

    cf. https://retrocomputing.stackexchange.com/q/15054/56

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Oliver Schmidt@21:1/5 to All on Mon Oct 31 10:09:40 2022
    Hi Kent,

    What is your suggested algorithm for auto-detecting the sector order of
    .dsk images?

    Maybe there's yet another misunderstanding because I wasn't precise
    enough: In my POV, the only ProDOS ordered .DSK files acceptable are
    ones actually containing ProDOS.

    For a ProDOS disk in ProDOS order, the complete 512 bytes ProDOS boot
    block is located at offset $000 - in contrast to a ProDOS disk in
    DOS33 order having the second 256 bytes of the ProDOS boot block at
    offset $E00.

    At least up to ProDOS 2.4.x, the ProDOS boot block of all ProDOS
    versions has the string "PRODOS " in the second 256 bytes. So
    if that string is found at $100-$1FF, the .DSK file contains ProDOS in
    ProDOS order.

    Every .DSK file that doesn't match this criteria, is to be considered
    in DOS33 order.

    Regards,
    Oliver

    PS: If the ProDOS 2.5 boot block doesn't contain that string anymore,
    than ProDOS 2.5 disk images MUST be named .PO.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From David Schmidt@21:1/5 to fadden on Mon Oct 31 08:29:57 2022
    On 10/30/22 5:43 PM, fadden wrote:
    On Sunday, October 30, 2022 at 11:07:53 AM UTC-7, Kent Dickey wrote:
    Can you describe CiderPress's algorithm for determining sector order of
    a .dsk file?

    "Complicated".

    I've mirrored the same algorithm in AppleCommander. It works in all
    cases where a person has a reasonable expectation of there being
    reasonable data laid down by a reasonable operating system.

    Of note - ADTPro (and ADT before it) always has written 140k Disk II
    images in DOS order no matter what. Every other disk type writes images
    in ProDOS order no matter what.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From D Finnigan@21:1/5 to Kent Dickey on Mon Oct 31 13:03:55 2022
    Kent Dickey wrote:

    Can you share some of this information? Do you know roughly what is the percentage of .dsk images that are in ProDOS order?

    Not off hand, but I can get that if you are really curious.


    I think what you're saying is, if someone wants to throw together an Apple
    II
    emulator (say, written in whatever new language is hot this year), before releasing it they need to:

    - Write .dsk auto-detect code. As far as I know, my description of AppleWin's
    algorithm is the first public documentation of .dsk sector order
    detection. Otherwise, they would need to do a bunch of research to
    decide what to do. Note: other disk formats other than DOS 3.3 and
    ProDOS are not covered, such as Pascal.

    Or CP/M. I wrote detectors for these formats too, and I can produce a
    catalog of their contents programmatically.


    - Download every Apple II .dsk image available on the web.
    How should I go about doing this? I am unable to see how to download
    all Apple II files from macgui.com, can you provide some help?

    I started at Asimov. I would recommend that as a source for an Apple II emulator author who wants a big stack of disk images to test against.


    - Test their auto-detect code and make sure they don't make any mistakes.

    This seems like a pretty high bar, and I'm not sure any existing emulator meets this requirement.

    Indeed, which is why I've often found using the real thing to be much more satisfying.


    --
    ]DF$
    The New Apple II User's Guide:
    https://macgui.com/newa2guide/

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From I am Rob@21:1/5 to All on Wed Nov 2 13:39:28 2022
    What is your suggested algorithm for auto-detecting the sector order of
    .dsk images?


    I imagined this as an auto-detector.

    I like the way Sweet16 loads disk images. It makes a best guess as to what order the disk image is and if unable to decide it loads it as a RAW file. And due to this RAW file format, one can load Commodore 64 disk images which are 174,848 bytes
    compared to Apple II disk image of 143,360 bytes. Which was then easy to convert C64 T/S's to DOS T/S's.

    I would assume the best way to auto-detect an image is to load all disk images as RAW data which can be used by an OS that reads/writes blocks. It is just simple math that converts to Track/Sectors.

    A Prodos image is identified as having these standard unchanged bytes that would be extremely rare to have duplicated on an image that is just data.

    Block #2 - has byte #2 = $03 and byte #4 = $Fx
    Block #3 - has byte #2 = $04
    Block #4 - has byte #2 = $05


    A DOS image has this:

    Block #88 (which calculates to Track 17 Sector 0) - bytes #1 & #2 show the Track/Sector start of the first Sector of the directory. Normally shows $11 0F, but modified disks can be anything. I have on occasion changed this to $11 01 and then have Track
    $11 Sector $01 - byte #1 changed to a $00 to indicate the end of the directory. This allows only 7 file names but frees up the other 14 sectors of the directory for data on Track 17.

    But technically, byte #1 of Track $11 Sector 0 should match byte #1 of the directory sector it points to. for example, if byte #1 and #2 of Track $11 Sector 0 is $11 0F, then byte #1 of Track $11 Sector $0F should also be $11, but can also be a zero for
    end of directory. This is very easy to find if the first directory sector is in the same track as the VTOC track number and since Sector $0F is in the second half of Block #88. From an emulators perspective should be very easy to get and compare byte #
    1 of the VTOC and byte #1 of the first sector of the directory to determine the disk image is a DOS disk image. But byte #1 of the next T/S pointer would still have to be read in to determine if this disk is DOS order or Prodos order.

    PCDOS and HFS are also very easy to determine using this RAW block method, as well as I imagine CP/M would be. I think an emulator should read in the entire track of $0-F and $11-1F into memory, then it should be able to easily identify any disk image
    that gets loaded.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Antoine Vignau@21:1/5 to All on Wed Nov 2 14:46:05 2022
    With values in hex...

    If I may, block 88 can be changed to getting the value of T1/SB/01. It contains the track of the VTOC. Using it allows to handle VTOC that are not on T11.

    Antoine

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From I am Rob@21:1/5 to All on Wed Nov 2 15:16:18 2022
    With values in hex...

    If I may, block 88 can be changed to getting the value of T1/SB/01. It contains the track of the VTOC. Using it allows to handle VTOC that are not on T11.

    Antoine

    Is this part of DOS and is it the same for all versions of DOS? Including DOS3.2 and Third Party DOS's?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From fadden@21:1/5 to [email protected] on Wed Nov 2 18:05:11 2022
    On Wednesday, November 2, 2022 at 3:16:19 PM UTC-7, [email protected] wrote:
    If I may, block 88 can be changed to getting the value of T1/SB/01. It contains the track of the VTOC. Using it allows to handle VTOC that are not on T11.

    Antoine
    Is this part of DOS and is it the same for all versions of DOS? Including DOS3.2 and Third Party DOS's?

    DOS: track 17 sector 0 holds the VTOC. Everything else is potentially variable.

    ProDOS: block 2 holds the start of the volume directory. Everything else is potentially variable.

    Most disks follow the standard pattern: track 17 is a full catalog track, ProDOS has 4 volume directory blocks followed by the bitmap. Detecting "most disks" isn't the interesting part of the problem though. :-)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From I am Rob@21:1/5 to All on Wed Nov 2 19:09:29 2022
    If I may, block 88 can be changed to getting the value of T1/SB/01. It contains the track of the VTOC. Using it allows to handle VTOC that are not on T11.

    Antoine
    Is this part of DOS and is it the same for all versions of DOS? Including DOS3.2 and Third Party DOS's?
    DOS: track 17 sector 0 holds the VTOC. Everything else is potentially variable.

    ProDOS: block 2 holds the start of the volume directory. Everything else is potentially variable.

    Most disks follow the standard pattern: track 17 is a full catalog track, ProDOS has 4 volume directory blocks followed by the bitmap. Detecting "most disks" isn't the interesting part of the problem though. :-)


    I was taking Antoine's comment of T1/SB/01 as the location of the VTOC within DOS. Unless that was a typing error and was supposed to be T11/SB/01, in which case it has a different meaning.

    The value of Track 17 Sector 0 for the VTOC is stored somewhere in Tracks 0-2 which is usually DOS, unless the disk is DOSless. The question I was asking Antoine was, is the VTOC Track/Sector number in the same location for all versions of DOS including
    third party DOSes that one can use a standard routine to find the VTOC without assuming it is always Track 17.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From fadden@21:1/5 to [email protected] on Wed Nov 2 21:29:30 2022
    On Wednesday, November 2, 2022 at 7:09:31 PM UTC-7, [email protected] wrote:
    I was taking Antoine's comment of T1/SB/01 as the location of the VTOC within DOS. Unless that was a typing error and was supposed to be T11/SB/01, in which case it has a different meaning.

    T1/S11 holds the code loaded into $ac00-acff. It starts with the tail end of a function that initializes some file manager stuff, including the VTOC location (the byte at +$01 in holds the track number). The value would not be useful if the disk weren'
    t bootable. There are likely some disks with a relocated VTOC that would benefit from looking there, though you'd want to test the code around that location to see if it looked like DOS before putting too much faith in what you find.

    For DOS 3.2, it would be T2/S1. Custom DOSs often just made a few in-place edits, so standard routines tended to stay in the standard places.

    If you wanted to include this in the strategy, it might be best applied as a fallback if T17/S0 doesn't look like a VTOC, and you were about to write the disk off as "unrecognized".

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From D Finnigan@21:1/5 to D Finnigan on Mon Nov 7 13:36:29 2022
    D Finnigan wrote:
    Kent Dickey wrote:

    Can you share some of this information? Do you know roughly what is the
    percentage of .dsk images that are in ProDOS order?

    Not off hand, but I can get that if you are really curious.


    Counting ProDOS format disks:

    mysql> SELECT COUNT(*) FROM files WHERE file_format = 11;
    +----------+
    | COUNT(*) |
    +----------+
    | 2700 |
    +----------+
    1 row in set (0.10 sec)



    Counting ProDOS format disks with file extension .dsk:


    SELECT COUNT(*) FROM files WHERE file_format = 11
    -> AND file_url LIKE '%.dsk.%';
    +----------+
    | COUNT(*) |
    +----------+
    | 2341 |
    +----------+
    1 row in set (0.10 sec)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Oliver Schmidt@21:1/5 to All on Mon Nov 7 23:38:05 2022
    Hi David,

    SELECT COUNT(*) FROM files WHERE file_format = 11;
    +----------+
    | 2700 |
    +----------+

    SELECT COUNT(*) FROM files WHERE file_format = 11
    -> AND file_url LIKE '%.dsk.%';
    +----------+
    | 2341 |
    +----------+

    Thanks for sharing :-)

    The result seems even (much) more clear than I expected: .DSK files with
    ProDOS order are simply the de facto standard. Period.

    Regards,
    Oliver

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From fadden@21:1/5 to Oliver Schmidt on Mon Nov 7 18:18:45 2022
    On Monday, November 7, 2022 at 3:38:07 PM UTC-8, Oliver Schmidt wrote:
    The result seems even (much) more clear than I expected: .DSK files with ProDOS order are simply the de facto standard. Period.

    The results indicate that 86% of ProDOS disk images are called ".dsk". It does not indicate what percentage of ".dsk" are ProDOS. I expect that's just as easy to pull out... but it doesn't matter. The goal should be to eliminate ambiguity, not clarify
    its bounds so we can cope with it using best-effort heuristics.

    Is this discussion moving in circles at high speed? :-)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kent Dickey@21:1/5 to [email protected] on Tue Nov 8 03:15:48 2022
    In article <[email protected]>,
    D Finnigan <[email protected]> wrote:
    D Finnigan wrote:
    Kent Dickey wrote:

    Can you share some of this information? Do you know roughly what is the >>> percentage of .dsk images that are in ProDOS order?

    Not off hand, but I can get that if you are really curious.


    Counting ProDOS format disks:

    mysql> SELECT COUNT(*) FROM files WHERE file_format = 11;
    +----------+
    | COUNT(*) |
    +----------+
    | 2700 |
    +----------+
    1 row in set (0.10 sec)



    Counting ProDOS format disks with file extension .dsk:


    SELECT COUNT(*) FROM files WHERE file_format = 11
    -> AND file_url LIKE '%.dsk.%';
    +----------+
    | COUNT(*) |
    +----------+
    | 2341 |
    +----------+
    1 row in set (0.10 sec)

    Can you provide some background on what your total images are, and
    what it means to be ProDOS format? Using the output from Andy's file list
    can only show which disks have a ProDOS filesystem on it, not which have sectors in ProDOS order.

    Kent

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Oliver Schmidt@21:1/5 to All on Tue Nov 8 08:03:43 2022
    Hi Kent,

    Using the output from Andy's file list
    can only show which disks have a ProDOS filesystem on it, not which have sectors in ProDOS order.

    That's of course the relevant question. Sorry for jumping to conclusions in
    my last post.

    Regards,
    Oliver

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From D Finnigan@21:1/5 to D Finnigan on Tue Nov 8 17:25:56 2022
    D Finnigan wrote:
    D Finnigan wrote:
    Kent Dickey wrote:

    Can you share some of this information? Do you know roughly what is the >>> percentage of .dsk images that are in ProDOS order?

    Not off hand, but I can get that if you are really curious.


    Counting ProDOS format disks:

    mysql> SELECT COUNT(*) FROM files WHERE file_format = 11;
    +----------+
    | COUNT(*) |
    +----------+
    | 2700 |
    +----------+
    1 row in set (0.10 sec)



    Counting ProDOS format disks with file extension .dsk:


    SELECT COUNT(*) FROM files WHERE file_format = 11
    -> AND file_url LIKE '%.dsk.%';
    +----------+
    | COUNT(*) |
    +----------+
    | 2341 |
    +----------+
    1 row in set (0.10 sec)


    I made a mistake. :-( These queries don't answer the question of how many
    disk images are in ProDOS order. Instead, these queries count disk images
    which use the ProDOS file system.

    I can still answer the desired question, but I'll need to write a little bit
    of code to record whether a disk image was in DOS or ProDOS order. Right
    now, my code discovers what order is used, but after the disk catalog is generated, the order is not recorded. (I'm talking about my code that
    generates disk catalogs for the Downloads section of the Mac GUI web site.)

    --
    ]DF$
    The New Apple II User's Guide:
    https://macgui.com/newa2guide/

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From I am Rob@21:1/5 to All on Thu Nov 10 12:36:57 2022
    Can you describe CiderPress's algorithm for determining sector order of
    a .dsk file?
    "Complicated".

    It tries DOS, ProDOS/Pascal, CP/M, and physical sector orders. (CP/M ordering is not really a thing, since nobody is creating disk images with CP/M software... but we might as well collect the whole set.) It tests the filesystem for DOS, ProDOS, Pascal,
    CP/M, RDOS, Gutenberg, and is probably checking for HFS even though I doubt anybody put that on a 5.25" floppy.

    The tests for DOS / ProDOS filesystems have to take into account non-5.25" situations, like UNIDOS disks and DOS Master floppies, so there's a bunch of auto-detect code that doesn't really apply to 5.25" disks.


    Just realized you were testing for file systems and not sector ordering.

    There should only be 2 sector orderings, is there not? Sequential like Prodos does it, and DOS ordering? Or does formatting with interleave change the sector orders as well?

    Is a pascal disk on a floppy using sector order, whereas a Pascal volume on a hard drive uses Prodos order blocks?
    Same with CP/M, does it use both, depending on which disk it is on, floppy or hard drive?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From fadden@21:1/5 to [email protected] on Thu Nov 10 16:05:42 2022
    On Thursday, November 10, 2022 at 12:36:58 PM UTC-8, [email protected] wrote:
    Just realized you were testing for file systems and not sector ordering.

    I'm doing both. If you can't recognize the filesystem, you can't figure out if you have the correct sector ordering. And if you have the wrong sector ordering, the filesystem won't look right.

    There should only be 2 sector orderings, is there not? Sequential like Prodos does it, and DOS ordering? Or does formatting with interleave change the sector orders as well?

    The formatters that alter the interleave are moving the physical sectors around. This has no effect on disk images, because those are created by software that requests specific sectors rather than just copying whatever passes under the drive head.

    DOS 3.3, ProDOS, UCSD Pascal, and CP/M remap the sector numbers in software. DOS and ProDOS are different from each other because the software sector skew tables are different, so when DOS and ProDOS request sector 1 they get different physical sectors.
    (Sector 0 is always mapped to 0, so checking its contents doesn't tell you anything about the ordering.)

    "DOS order" images come from anything that runs under DOS and reads sectors in sequential order. "ProDOS order" images come from anything that runs under ProDOS and reads blocks in sequential order. If we had CP/M disk image utilities, we'd need to
    worry about CP/M order too.

    Is a pascal disk on a floppy using sector order, whereas a Pascal volume on a hard drive uses Prodos order blocks?
    Same with CP/M, does it use both, depending on which disk it is on, floppy or hard drive?

    ProDOS/SOS and Pascal use the same sector skew. CP/M uses a different skew. Copy ][+ images ignore the software mapping and just copy the sectors in physical order. See _Beneath Apple DOS_ pages 3-22 and 3-23 for an explanation and a table of values.

    All of this is applicable only to 16-sector 5.25" floppy disks. DOS 3.2 doesn't do this, and pretty much everything else is treated like a sequential block device.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)