Here's a question for the techies :-)

In my sort out of what data I keep where, and how, I formatted 4 x 2TB
SSDs as exFAT so I could use them with both Windows Machines and Macs.

As a check that I am backing things up properly I am doing directory size checks and there is an enormous difference between the NTFS and exFAT
total directory size. One directory contains nearly 13,000 small files of Flight Simulator Scenery and TreeSize Free is measuring the NTFS directory
at 673 MB and the exFAT at 7.6 GB. The overall impact is that FS2002 is
3.6 GB on NTFS and 32.9 GB on exFAT.

It makes it look like I've screwed up my backup but the number of files is
the same so it seems to be just the different file systems. Does that make sense?

--
Jeff Gaines Wiltshire UK
Indecision is the key to flexibility

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 11 Aug 2021 at 10:37:20 BST, ""Jeff Gaines""
<[email protected]> wrote:

Here's a question for the techies :-)

In my sort out of what data I keep where, and how, I formatted 4 x 2TB
SSDs as exFAT so I could use them with both Windows Machines and Macs.

As a check that I am backing things up properly I am doing directory size checks and there is an enormous difference between the NTFS and exFAT
total directory size. One directory contains nearly 13,000 small files of Flight Simulator Scenery and TreeSize Free is measuring the NTFS directory
at 673 MB and the exFAT at 7.6 GB. The overall impact is that FS2002 is
3.6 GB on NTFS and 32.9 GB on exFAT.

It makes it look like I've screwed up my backup but the number of files is the same so it seems to be just the different file systems. Does that make sense?

It's just the difference in filesystems. They don't store files by the
byte, they store in blocks. Some filesystems will store the leftover bit
at the end of a file, and tiny files, cleverly - exFAT does not.

You can choose what size blocks your exFAT is formatted at formatting
time, but they've probably defauled to 64kb on a 2TB filesystem. Which
means on average 32kb of waste empty space per file, if the file sizes
are evenly distributed.

Even so, that does seem a little excessive.... are lots of the files
really really tiny?

(You may like to know that macOS can read NTFS disks, btw - so you could
have one read/write exFAT and the rest NTFS, if workloads suited that.
Or leave them plugged into one machine and access over the network,
where it doesn't matter what the underlying filesystem is as the host
computer handles it. Like a NAS. Weren't you going to put these in the
NAS anyway?)

Cheers - Jaimie
--
To every complex problem there is a solution which
is simple, neat and wrong.
-- HL Mencken

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 11/08/2021 in message <[email protected]> Jaimie
Vandenbergh wrote:

Even so, that does seem a little excessive.... are lots of the files
really really tiny?

(You may like to know that macOS can read NTFS disks, btw - so you could
have one read/write exFAT and the rest NTFS, if workloads suited that.
Or leave them plugged into one machine and access over the network,
where it doesn't matter what the underlying filesystem is as the host >computer handles it. Like a NAS. Weren't you going to put these in the
NAS anyway?)

Hi Jaimie.

I wrote a quick app in C# to do some measuring, it tells me that FS2002
has 66,032 files in 497 directories amounting to 3.571 MB. The scenery
texture files are 43 KB each and there are 44,155 of them in 12
directories amounting to 1.845 MB for all the scenery files (it's an add
on mapping program).

Interestingly when I run the program over the exFAT drive it barfs and I
get a Windows message saying th message pump has stalled, if I tell it to
carry on it does and comes up with exactly the same figure as the NTFS
drive which produced the result in seconds.

I've just run AS SSD benchmark, the read speed for the WD Blue OS drive is
492 MB/s (NTFS) the exFAT drive 377 MB/s and my brand new WD Blue NVMe (in
a PCIe card) 375 MB/s, oh deary me why is that so low? I think I need to
go off and see if WD do a driver for their NVMe.

This is fascinating and I keep getting distracted in polishing the app,
now where's the swamp pump...

--
Jeff Gaines Wiltshire UK
I've been through the desert on a horse with no name.
It was a right bugger to get him back when he ran off.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 11/08/2021 in message <[email protected]> Jeff
Gaines wrote:

I think I need to go off and see if WD do a driver for their NVMe.

WD dashboard says I am using an unsupported driver but doesn't tell me how
to get an updated one :-(

I have put a question in to WD support.

--
Jeff Gaines Wiltshire UK
If you ever find something you like buy a lifetime supply because they
will stop making it

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 11/08/2021 in message <[email protected]> Jeff
Gaines wrote:

On 11/08/2021 in message <[email protected]> Jeff
Gaines wrote:

I think I need to go off and see if WD do a driver for their NVMe.

WD dashboard says I am using an unsupported driver but doesn't tell me how
to get an updated one :-(

I have put a question in to WD support.

I have dusted off my Asus Z170-K machine. The WD NVMe is reading at just
under 2,000 MB/s but WD dashboard still complains of an unsupported
driver. The Samsung NVMe (boot device) reads at 2448 MB/s.

Definite gremlin problems though. Pinging the desktop machine from the
laptop frequently times out which may be the cause ofe the hesistancy and stuttering I often see. To cap it off my Gigabit landline 'phone started
to make crackiling noises last night and I can smell magic smoke!

Whoops, spell checker not installed, E&OE.

--
Jeff Gaines Wiltshire UK
George Washington was a British subject until well after his 40th birthday. (Margaret Thatcher, speech at the White House 17 December 1979)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 11/08/2021 15:58, Jaimie Vandenbergh wrote:

On 11 Aug 2021 at 10:37:20 BST, ""Jeff Gaines""
<[email protected]> wrote:

Here's a question for the techies :-)

In my sort out of what data I keep where, and how, I formatted 4 x 2TB
SSDs as exFAT so I could use them with both Windows Machines and Macs.

As a check that I am backing things up properly I am doing directory size
checks and there is an enormous difference between the NTFS and exFAT
total directory size. One directory contains nearly 13,000 small files of
Flight Simulator Scenery and TreeSize Free is measuring the NTFS directory >> at 673 MB and the exFAT at 7.6 GB. The overall impact is that FS2002 is
3.6 GB on NTFS and 32.9 GB on exFAT.

It makes it look like I've screwed up my backup but the number of files is >> the same so it seems to be just the different file systems. Does that make >> sense?

It's just the difference in filesystems. They don't store files by the
byte, they store in blocks. Some filesystems will store the leftover bit
at the end of a file, and tiny files, cleverly - exFAT does not.

You can choose what size blocks your exFAT is formatted at formatting
time, but they've probably defauled to 64kb on a 2TB filesystem. Which
means on average 32kb of waste empty space per file, if the file sizes
are evenly distributed.

Even so, that does seem a little excessive.... are lots of the files
really really tiny?

(You may like to know that macOS can read NTFS disks, btw - so you could
have one read/write exFAT and the rest NTFS, if workloads suited that.

If you decide on NTFS as the common filesystem between your 2 worlds,
Paragon sell a read/write NTFS driver for Macs: https://www.paragon-software.com/home/ntfs-mac/

I'd be more inclined to use ZFS instead, but don't know how stable it is
on Macs and Windows. At least it was designed this century ;-)

Option C would be to avoid the problem and move your SSDs into a NAS, as
Jaimie also says.

--
Chris

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)