On 5/18/2024 6:50 AM, R.Wieser wrote:

Wendelin,

I know if even such a rule to use internal OS copy functions for creating
temporarily copies (which would maintain ADS files) would exist this would >> not guarant that Office apps would follow the rule and would not create
and copy their own data only, but if such a rule would exist I would
expect that at least MS office apps would apply it.

Does your MS-supplied (with the OS) "file explorer" show if a file has an
ADS atached ? And allows you to access (create/delete/read/write) them ?

If not, why do you think any other MS product would ?

By the way, "nice" to see that ADS - which was available as far back as XP - still has next-to-no suport in Win10 ...

Regards,
Rudy Wieser

It's supported, because it is used to track the security zone
of a file (caused by where it was downloaded from). This feature
is "half-baked", but it's the usual thing of "dogfooding a feature"
and proving that ADS works. Initially, Kaspersky attempted to use
ADS and there were failures, and Kaspersky stopped using ADS as a
result. Microsoft learned a lesson from this, to not release
features without some sort of test plan.

ADS was likely originally created, to copy Apple Resource and
Data Fork capability. But I don't recollect ADS actually being
used that way (copy an Apple file, the two halves preserved).
You always had to be careful when working with Apple files on a PC.
FAT32 doesn't have ADS, so it's not like this feature is valuable
enough, for all the file systems to have it.

streams64.exe libcs50-10.1.1.zip

streams v1.60 - Reveal NTFS alternate streams.
Copyright (C) 2005-2016 Mark Russinovich
Sysinternals - www.sysinternals.com

D:\libcs50-10.1.1.zip:
:Zone.Identifier:$DATA 26 <=== presumably twenty six bytes

Using nfi.exe :

File 45
\libcs50-10.1.1.zip
$STANDARD_INFORMATION (resident)
$FILE_NAME (resident)
$DATA (nonresident)
logical sectors 492064-492103 (0x78220-0x78247)
$DATA Zone.Identifier (resident) <=================== The 26 bytes are inside a $MFT 1KB slot
That's what the word "resident" means.

notepad libcs50-10.1.1.zip:Zone.Identifier:$DATA <=== This is how you open a named ADS.

And that shows the 26 byte content as two lines of text:

[ZoneTransfer]
ZoneId=3 <=== Zone 3 of five possible zones

The ZoneID is a number between 1..5 and is what Internet Explorer
used to use for some security slider setting. 3 is from the Internet
and "not considered safe".

And that marker has to be preserved, and it seems to be managed
via $MFT entries.

It's simple enough to test. Open the main handle with Notepad,
edit the file and save, then do Properties and see if the warning
at the bottom is still there. It should be.

If I check that file right now, at the bottom of Properties, it
says: "This file came from another
computer and might be blocked to
help protect this computer"

That file was put on disk, two years ago.

This does not answer the OPs question in a satisfactory way,
because I doubt there is a web page "guaranteeing" a behavior.
It's just a feature, and then they don't say much more about it.
If it wasn't for Russinovich (streams.exe) and for the crusty old (year 2000) copy of nfi.exe , we might not know anything about these files.

When I check today with nfi.exe, the file has been moved, but I
did not move it. You don't defragment SSD drives, and (apparently)
the $MFT has no compaction routine. So how that slot changed, I
haven't a clue at the moment. One of the reasons I ran this check,
was to see if it moved. And it did. Which is... disappointing.

File 89138
\libcs50-10.1.1.zip <=== since there is only one entry for this file now
$STANDARD_INFORMATION (resident) the resident ~700 byte area is used for all the
$FILE_NAME (resident) things labeled as resident. If you added more
$FILE_NAME (resident) streams, they would eventually need a second $MFT slot.
$DATA (nonresident)
logical sectors 7978432-7978471 (0x79bdc0-0x79bde7)
$DATA Zone.Identifier (resident)

when a file is severely fragmented, there can be multiple "File xxxxx" entries. You could have fifty file slots, for the containment of info for a single file. If you do that enough (expand $MFT that way), you get "insufficient system resources" error.

Paul

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/18/2024 6:50 AM, R.Wieser wrote:

Wendelin,

I know if even such a rule to use internal OS copy functions for creating
temporarily copies (which would maintain ADS files) would exist this would >> not guarant that Office apps would follow the rule and would not create
and copy their own data only, but if such a rule would exist I would
expect that at least MS office apps would apply it.

Does your MS-supplied (with the OS) "file explorer" show if a file has an
ADS atached ? And allows you to access (create/delete/read/write) them ?

If not, why do you think any other MS product would ?

By the way, "nice" to see that ADS - which was available as far back as XP - still has next-to-no suport in Win10 ...

Regards,
Rudy Wieser

compare result of DIR to DIR /R on the same folder to see if an
alternate file exisits

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Paul,

It's supported, because it is used to track the security zone
of a file (caused by where it was downloaded from). This feature
is "half-baked",

It looks like we have a different perspective on "supported" and
"half-baked".

but it's the usual thing of "dogfooding a feature" and proving that
ADS works.

That I can easily believe, seeing the absense of MS support in regard to managing ADS.

As mentioned before, not even the "file explorer" shows any support for it.
It can show a shortcut icon overlay without a problem, but an ADS icon
overlay ? Thats to much effort. :-(

Regards,
Rudy Wieser

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Zaidy036,

compare result of DIR to DIR /R on the same folder to see if an alternate file exisits

I had to google for that "/R" switch, as it isn't present on XP(sp3). But
its good to see that win10 has at least *some* support for it - even if it needs a command-console to be able to use it.

Regards,
Rudy Wieser

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/18/2024 1:31 PM, R.Wieser wrote:

I had to google for that "/R" switch, as it isn't present on XP(sp3). But
its good to see that win10 has at least *some* support for it - even if it needs a command-console to be able to use it.

NT fully supports it. They're just not meant to be used
visibly. As I understand it, the original putrpose was to
match MS Office on Windows and Mac. Mac has the data
and resource forks. ADS files allowed MS to do something
similar on Windows.

The trouble is that ADS depends on NT. Moving a file to
an FAT32 partition or writing to DVD will remove and ADS
attachments.

It's mostly basic kernel32 functionality. I wrote a small
program some years ago to list ADS files and optionally
delete them, in the interest of avoiding malware. As Paul
said, the whole idea was half-baked to begin with. Then
they decided to use it to tag downloaded files, as though
downloaded files are inherently dangerous.

Instead, it's ADS files that are dangerous. There should
be a way to disable them, but it's very basic API level.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Newyana2,

NT fully supports it.

Yes, I can see that here.

They're just not meant to be used visibly.

Really ? I find those "contains a possible flaw" FUD blurbs whenever I try
to open a saved HTML document quite visible. :-) (and irritating too :-( )

The trouble is that ADS depends on NT. Moving a file to
an FAT32 partition or writing to DVD will remove and ADS
attachments.

Yep. Nice when you use FAT32 formatted thumbdrives (as I do).

It's mostly basic kernel32 functionality. I wrote a small
program some years ago to list ADS files and optionally
delete them, in the interest of avoiding malware.

I did the same, in the interrest of being able to inspect and delete them.

As Paul said, the whole idea was half-baked to begin with.

I would not say that. Being able to store (local) user-defined properties
in them (file hashes e.a.) isn't all that bad.

Then they decided to use it to tag downloaded files, as though
downloaded files are inherently dangerous.

And that even happens with images and textfiles. Go figure.

Instead, it's ADS files that are dangerous. There should
be a way to disable them, but it's very basic API level.

ADS in itself isn't be dangerous, but as long as they can be used to hide
stuff outof the users sight they act as a poor-mans rootkit.

Luckily I found documentation to create an MS file explorer "plugin" to add overlay icons, and have done so for ADS. Makes it easy to spot them. :-)

But yes, being able to disable a feature that doesn't have much of any
support would be nice.

Regards,
Rudy Wieser

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/18/2024 6:31 AM, Paul wrote:

On 5/18/2024 6:50 AM, R.Wieser wrote:

<SNIP> When I check today with nfi.exe, the file has been moved, but I

did not move it. You don't defragment SSD drives, and (apparently)
the $MFT has no compaction routine. So how that slot changed, I
haven't a clue at the moment. One of the reasons I ran this check,
was to see if it moved. And it did. Which is... disappointing<SNIP>

Agreed, defragmenting an SSD is rarely a good idea. On the other hand,
Samsung SSD firmware (if I understood their literature) will move files
from one place to another as part of the ware-leveling algorithm's behavior.

I think they measure the passage of time (metaphorically) using SSD
block read/writes as the second hand. Then if sufficient time passes and
a file has not been accessed, that file is moved so its blocks can be reallocated and more probably absorb some of the future activity. [I
think this was done at a file, not a block, basis since rethreading a
moved block would need an update in their directory structure and that
could trigger more updating, etc.] I'm not sure if the type of block
moves I'm talking about here could/would be observed at the OS level or not.
--
Jeff Barnett

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 18.05.2024 20:16, Newyana2 wrote:

NT fully supports it. They're just not meant to be used
visibly. As I understand it, the original putrpose was to
match MS Office on Windows and Mac.

And you can use it to create files which are (nearly)
undeletable and unreadable:

echo 123abc>passwords.::$DATA
echo xyz>>passwords.::$DATA

Try to read or delete the created file.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Herbert,

And you can use it to create files which are (nearly)
undeletable and unreadable:

echo 123abc>passwords.::$DATA
echo xyz>>passwords.::$DATA

Try to read or delete the created file.

-- delete
del passwords?

-- display contents
more < passwords.::$DATA

:-)

Regards,
Rudy Wieser

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/19/2024 3:34 AM, Herbert Kleebauer wrote:

And you can use it to create files which are (nearly)
undeletable and unreadable:

ADS files are fully readable, deletable, etc. via basic
kernel32 functions like CreateFile, ReadFile, DeleteFile, etc.
My own program only adds the more obscure NTQueryInformationFile
to those methods.

You can handle the files perfectly well. You just can't see
them in Explorer. So the issue is that you wouldn't normally know
they're there. Actually, Microsoft also does other similar
tricks. For instance, the IE cache used to be designed in such
a way that there seemed to be no content unless one deleted
a desktop.ini file and then looked quickly.

There are also hidden Registry values. It doesn't seem so
odd in context. Microsoft's job is to build a software
platform so that 3rd-party software can use the hardware.
The average person has no need or interest in knowing how
that works.

The real problem with ADS files is that it's essentially a hidden
file system that can be used by malware. You could have a 5KB
text file with a 100MB malware program attached and you
wouldn't know it. That's all the more true given all the funky
shenanigans that Microsoft have done with recent Windows
versions. Disk space use as reported by Explorer is not the
same as space actually used on disk. It's typically off by some
10%. Even the Softies don't seem to understand it. When they
first came out with the winsxs calamity, one MS official said that
there's actually nothing in that folder. It's just mirrors. Another
official said there's actually nothing anywhere else -- it's all
just mirrored from winsxs to system folders. Whatever it is, the
bloat is quite real. That makes it even harder to keep track of
what's actually on disk. Too much MS funny business.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/19/2024 4:45 AM, R.Wieser wrote:

Herbert,

And you can use it to create files which are (nearly)
undeletable and unreadable:

echo 123abc>passwords.::$DATA
echo xyz>>passwords.::$DATA

Try to read or delete the created file.

-- delete
del passwords?

-- display contents
more < passwords.::$DATA

:-)

Regards,
Rudy Wieser

On Win11, this is what happens.

D:\>echo 123abc > passwords.::$DATA

D:\>dir
Volume in drive D is Scratch
Volume Serial Number is 025E-9F1C

Directory of D:\

05/19/2024 09:14 AM 9 passwords.
1 File(s) 9 bytes
0 Dir(s) 94,268,583,936 bytes free

D:\>del passwords.::$DATA
The filename, directory name, or volume label syntax is incorrect.

D:\>del passwords.
Could Not Find D:\passwords.

Rename in Perl would not fix it.

But "mv" on Win11 bash shell, got it. No complaint in that case.

/mnt/d$ mv "passwords." "passwords"
/mnt/d$

Now, it's accessible.

Yes, that's a bit nasty.

Paul

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/19/2024 7:47 AM, Newyana2 wrote:

Disk space use as reported by Explorer is not the
same as space actually used on disk. It's typically off by some
10%. Even the Softies don't seem to understand it. When they
first came out with the winsxs calamity, one MS official said that
there's actually nothing in that folder. It's just mirrors. Another
official said there's actually nothing anywhere else -- it's all
just mirrored from winsxs to system folders. Whatever it is, the
bloat is quite real. That makes it even harder to keep track of
what's actually on disk. Too much MS funny business.

That's the hard link between WinSxS (the service area) and System32
(where the file is being used in a sense). The File Explorer design
does not have a way of displaying "what things in this folder are
sharing clusters with things elsewhere, and what things are ordinary
files with just ordinary clusters".

As File Explorer descends a tree, it can double count the clusters
of storage used. If you're using GetNextFile or whatever, and
descending the tree, you could count the WinSxS entry as 4096 bytes
and the System32 entry as 4096 bytes, for 8192 total, whereas the
reality is, only 4096 bytes of actual storage are involved. Counting
this way, your C: drive size determination (via tree descent) is
6GB too much.

If you do Properties on C: drive, and look at the "pie chart" graphic,
that relies on $BITMAP or so, for information. In Win10 we started
seeing:

"Volume Bitmap is incorrect"

By using the global space management metadata ($BITMAP), you can get
a better idea of the size of the entire C: . But I'm willing
to bet that even dual booting Windows 7 and Windows 10, that the pie
charts on Properties of those two OSes might not agree. As the
"management" of $BITMAP was loosened up. When the disk is data-at-rest,
the $BITMAP may not be correct. That's something Macrium had to patch
at one time, and we were using 6.3.1835 release or greater, to back
up a Win10 C: drive.

While "all your stuff might just work today", there could be
a few rough edges. It's not very efficient, if everything
ends up being a puzzle. Even using CHKDSK on various Windows OS
versions, they don't "agree well" on how things should be done.
A Win7 run might take forever, as it processes "Extended Attributes" info.

Paul

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 19.05.2024 13:47, Newyana2 wrote:

On 5/19/2024 3:34 AM, Herbert Kleebauer wrote:

And you can use it to create files which are (nearly)
undeletable and unreadable:

echo 123abc>passwords.::$DATA

ADS files are fully readable, deletable, etc. via basic
kernel32 functions like CreateFile, ReadFile, DeleteFile, etc.
My own program only adds the more obscure NTQueryInformationFile
to those methods.

The above echo doesn't generate an ADS. The syntax is
just used to create a file with a name ending with a "."
And such filenames are not supposed to exist and therefore
Windows gets in trouble when you want to view its content
or delete it.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/19/2024 10:28 AM, R.Wieser wrote:

Paul,

del passwords.

Could Not Find D:\passwords.

That is why I used

"del passwords?"

, using the "?" wildcard to catch the "."

Regards,
Rudy Wieser

The problem is, typically one part of
a command sees "passwords." and the
other part of the command sanitizes that
and makes "passwords" from it, then when
it attempts to act on "passwords",
"no such file exists". That is the typical
failure sequence.

In Linux, the mv command might not know that the
underlying file system is NTFS, and consequently
it does not sanitize "passwords." and turn it
into "passwords" before acting upon it. That's
the only reason "mv" as a solution works.

Paul

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Paul,

del passwords.

Could Not Find D:\passwords.

That is why I used

"del passwords?"

, using the "?" wildcard to catch the "."

Regards,
Rudy Wieser

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 5/19/2024 10:51 AM, Herbert Kleebauer wrote:

echo 123abc>passwords.::$DATA

The above echo doesn't generate an ADS. The syntax is
just used to create a file with a name ending with a "."
And such filenames are not supposed to exist and therefore
Windows gets in trouble when you want to view its content
or delete it.

I'm afraid I don't understand what you're talking about.
You're using DOS to see if you can make Windows choke
on a file path? ADS files can be read and deleted. That's
all I said.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Newyana2,

echo 123abc>passwords.::$DATA

The above echo doesn't generate an ADS.

I'm afraid I don't understand what you're talking about.

The above ::$DATA postfix is indicating the main file itself. It doesn't
create an ADS on that file. You don't /need/ to use it, but you can. Yeah, confusing, I know. :-\

The first indication of that is that the filesize (of "password") is
non-zero. ADS are not included in (the main files) file-size.

If you want to check than remove that the "." from the filename and try
again. You will see that the text that you echo into it is in the
("password") file (and not hidden in an ADS).

Regards,
Rudy Wieser

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Paul

That is why I used

"del passwords?"

, using the "?" wildcard to catch the "."

The problem is, typically one part of
a command sees "passwords." and the
other part of the command sanitizes that
and makes "passwords" from it

It looks like we are not talking about the same thing here.

I posted a "you can't delete" solution. What are you talking about ?

But if you want to talk about parsing, than that is exactly the problem :
The command-interpreter part dealing with redirected files isn't parsing the same way as its own internal commands. Why ? Beats me.

Regards,
Rudy Wieser

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Windows/NTFS supports alternate data streams, so if an application adds an
ADS to a file copying the file using system file copy functions will keep
all ADS still alive.

Office applications like Word or Excel handle large files by working on
copies of the original file. External users cannot see if on saving file content the old file is expanded (and ADS are kept living), or if the old
file is renamed/deleted and a new file is created under the old filename. Creating a new file may ignore ADS.

Is there a Windows rule established to maintain ADS when creating such a
copy, that means can I trust to find an ADS created by an external app still after an Office app has edited the data file and stored data in such a newly created copy?

I know if even such a rule to use internal OS copy functions for creating temporarily copies (which would maintain ADS files) would exist this would
not guarant that Office apps would follow the rule and would not create and copy their own data only, but if such a rule would exist I would expect that
at least MS office apps would apply it.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Wendelin,

I know if even such a rule to use internal OS copy functions for creating temporarily copies (which would maintain ADS files) would exist this would not guarant that Office apps would follow the rule and would not create
and copy their own data only, but if such a rule would exist I would
expect that at least MS office apps would apply it.

Does your MS-supplied (with the OS) "file explorer" show if a file has an
ADS atached ? And allows you to access (create/delete/read/write) them ?

If not, why do you think any other MS product would ?

By the way, "nice" to see that ADS - which was available as far back as XP - still has next-to-no suport in Win10 ...

Regards,
Rudy Wieser

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)