Lawrence D'Oliveiro <ldo@nz.invalid> writes:
> Johanne Fairchild wrote:
>> Lawrence D'Oliveiro <ldo@nz.invalid> writes:
>>> vallor wrote:
>>>
>>>> ... the Unix puritan in me abhors hard-linked directories.
>>>
>>> They are a very, very, very, very bad idea. And completely
>>> unnecessary.
>>
>> Why are they a bad idea?
>
> Consider these two questions:
> * How do you detect reference cycles?

Multiple cycle-detection algorithms have been well known for decades.

> * What happens if you don’t?

Something gets stuck in a loop.

In the hypothetical where hard-link directories was possible, filesystem
traversers would have to take into account the possibility of
cycles. It’d be an extra implementation detail, comparable to the way
real ones have to take into account the possibility of the filesystem
changing as they progress, or accept with the consequences if they
don’t.

You’re on stronger grounds with “unnecessary” but a three-very bad idea?
Insufficient evidence at best.

--
https://www.greenend.org.uk/rjk/

On Sun, 05 May 2024 09:23:55 +0100, Richard Kettlewell wrote:

> Lawrence D'Oliveiro <ldo@nz.invalid> writes:
>
>> * How do you detect reference cycles?
>
> Multiple cycle-detection algorithms have been well known for decades.

How many directories would you have to lock down while checking for cycles?

How deep can your directory tree go?

>> * What happens if you don’t?
>
> Something gets stuck in a loop.
>
> ...
>
> You’re on stronger grounds with “unnecessary” but a three-very bad idea?
> Insufficient evidence at best.

You think getting “stuck in a loop” is not a bad idea?

In article <v17g26$1onvc$1@dont-email.me>,
Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
....
>> You're on stronger grounds with unnecessary but a three-very bad idea?
>> Insufficient evidence at best.
>
>You think getting stuck in a loop is not a bad idea?

It *might* be a "bad idea", but it is hardly a damn, damn, damn bad idea.

--
The single most important statistic in the US today - the one that explains all the
others - is this: 63 million people thought it was a good idea to vote for this clown
(and will probably do so again). Everything else is secondary to that. Everything else
could be fixed if we can revert this one statistic. Nothing can be fixed until we do.

On 2024-05-04, Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
> On Sat, 4 May 2024 14:49:19 -0000 (UTC), Lew Pitcher wrote:
>
>> Note that unix filesystems support a limited, controlled subset of
>> hardlinks: the "." and ".." directories found in each directory.
>
> Even those are completely unnecessary. Their interpretation could easily
> be hard-coded into the path-interpretation routines in the kernel, instead
> of wasting space for these entries in every single directory on a
> filesystem.

But they're not: it goes to the core of the Unix "everything is a
file" philosophy. Applications can and do read directories as
regular files and extract the parent from them. No on DOS/Windows,
you can't do that, you have to use a specific syscall. It's not
"completely unnecessary", just a different way of acheiving the
same effect.

--
Andrew Smallshaw
andrews@sdf.org

On 2024-05-05, Andrew Smallshaw <andrews@sdf.org> wrote:
> On 2024-05-04, Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>> On Sat, 4 May 2024 14:49:19 -0000 (UTC), Lew Pitcher wrote:
>>
>>> Note that unix filesystems support a limited, controlled subset of
>>> hardlinks: the "." and ".." directories found in each directory.
>>
>> Even those are completely unnecessary. Their interpretation could easily
>> be hard-coded into the path-interpretation routines in the kernel, instead
>> of wasting space for these entries in every single directory on a
>> filesystem.
>
> But they're not: it goes to the core of the Unix "everything is a
> file" philosophy. Applications can and do read directories as
> regular files and extract the parent from them.

Can you name one such application? Just as a warm up; then two more.

> No on DOS/Windows,
> you can't do that, you have to use a specific syscall. It's not
> "completely unnecessary", just a different way of acheiving the
> same effect.

No, you can't just read a directory as a flat file, and expect your
program to work on any file system.

Modern (as in within the last 40 years) Unixes support multiple file
systems including DOS. The directory representation is different in
different file systems. Some use B-trees and hash tables and whatnot.

On the Linux kernel, reading from a file descriptor that refers to a
directory is not allowed; it fails with errno == EISDIR. (Not sure
whether that's at the VFS layer, or whether specific filesystems refuse
that individually.)

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

In article <20240505071604.263@kylheku.com>,
Kaz Kylheku <643-408-1753@kylheku.com> wrote:
....
>>> Even those are completely unnecessary. Their interpretation could easily
>>> be hard-coded into the path-interpretation routines in the kernel, instead
>>> of wasting space for these entries in every single directory on a
>>> filesystem.
>>
>> But they're not: it goes to the core of the Unix "everything is a
>> file" philosophy. Applications can and do read directories as
>> regular files and extract the parent from them.
>
>Can you name one such application? Just as a warm up; then two more.

I think you're taking him too literally. There are interfaces for reading
directories. Not specifically read(2), but you know better than to pick
nits that way.

>> No on DOS/Windows,
>> you can't do that, you have to use a specific syscall. It's not
>> "completely unnecessary", just a different way of acheiving the
>> same effect.
>
>No, you can't just read a directory as a flat file, and expect your
>program to work on any file system.

He never used the term "flat file". That is *your* construct.

--
I voted for Trump because I thought he'd make pussy grabbing legal.
I honestly don't see any other way America could be made great again.

Lawrence D'Oliveiro <ldo@nz.invalid> writes:
> Richard Kettlewell wrote:
>> Lawrence D'Oliveiro <ldo@nz.invalid> writes:
>>> * How do you detect reference cycles?
>>
>> Multiple cycle-detection algorithms have been well known for decades.
>
> How many directories would you have to lock down while checking for cycles?

Could be as little as 2. Depends on the algorithm used, what else you
can assume about the filesystem, etc.

> How deep can your directory tree go?
>
>>> * What happens if you don’t?
>>
>> Something gets stuck in a loop.
>>
>> ...
>>
>> You’re on stronger grounds with “unnecessary” but a three-very bad idea?
>> Insufficient evidence at best.
>
> You think getting “stuck in a loop” is not a bad idea?

I think my previous posting adequately covers this question.

--
https://www.greenend.org.uk/rjk/

On 2024-05-05, Kenny McCormack <gazelle@shell.xmission.com> wrote:
> In article <20240505071604.263@kylheku.com>,
> Kaz Kylheku <643-408-1753@kylheku.com> wrote:
> ...
>>>> Even those are completely unnecessary. Their interpretation could easily
>>>> be hard-coded into the path-interpretation routines in the kernel, instead
>>>> of wasting space for these entries in every single directory on a
>>>> filesystem.
>>>
>>> But they're not: it goes to the core of the Unix "everything is a
>>> file" philosophy. Applications can and do read directories as
>>> regular files and extract the parent from them.
>>
>>Can you name one such application? Just as a warm up; then two more.
>
> I think you're taking him too literally. There are interfaces for reading
> directories. Not specifically read(2), but you know better than to pick
> nits that way.
>
>>> No on DOS/Windows,
>>> you can't do that, you have to use a specific syscall. It's not
>>> "completely unnecessary", just a different way of acheiving the
>>> same effect.
>>
>>No, you can't just read a directory as a flat file, and expect your
>>program to work on any file system.
>
> He never used the term "flat file". That is *your* construct.

I don't believe you are correct; read the original points again.
They are all quoted above, nothing snipped:

- "Applications [on Unix] can and do read directories as
regular files."

(Regular files are flat: just bytes, so it is legitimate for me to use
that word. That detail is neither here nor there anyway. You can't
read a directory even if you know how to follow its tree structure
or whatever it happens to be.)

- "[O]n DOS/Windows [...] you can't do that, you have to use
a specific syscall."

(That lands under "interfacws for reading directories" that you
allude to. So no, we don't just have to use those on Windows.)

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On Sun, 5 May 2024 12:17:48 -0000 (UTC), Andrew Smallshaw wrote:

> On 2024-05-04, Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>
>> On Sat, 4 May 2024 14:49:19 -0000 (UTC), Lew Pitcher wrote:
>>
>>> Note that unix filesystems support a limited, controlled subset of
>>> hardlinks: the "." and ".." directories found in each directory.
>>
>> Even those are completely unnecessary. Their interpretation could
>> easily be hard-coded into the path-interpretation routines in the
>> kernel, instead of wasting space for these entries in every single
>> directory on a filesystem.
>
> But they're not: it goes to the core of the Unix "everything is a
> file" philosophy. Applications can and do read directories as
> regular files and extract the parent from them.

How do they find those directories? Via their pathname. How do you get a
parent from a pathname? Simple: just drop off the last component. No need
to even read anything from the directory.

In article <20240505071604.263@kylheku.com>,
Kaz Kylheku <643-408-1753@kylheku.com> wrote:
>On 2024-05-05, Andrew Smallshaw <andrews@sdf.org> wrote:
>> On 2024-05-04, Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
>>> On Sat, 4 May 2024 14:49:19 -0000 (UTC), Lew Pitcher wrote:
>>>
>>>> Note that unix filesystems support a limited, controlled subset of
>>>> hardlinks: the "." and ".." directories found in each directory.
>>>
>>> Even those are completely unnecessary. Their interpretation could easily
>>> be hard-coded into the path-interpretation routines in the kernel, instead
>>> of wasting space for these entries in every single directory on a
>>> filesystem.
>>
>> But they're not: it goes to the core of the Unix "everything is a
>> file" philosophy. Applications can and do read directories as
>> regular files and extract the parent from them.
>
>Can you name one such application? Just as a warm up; then two more.

Depends on the version of Unix. I'll pick 7th Ed. Here's three
programs that could: `cat`, `dd`, and `od`. Of course, that's
not what you mean, bt I think it's related.

>> No on DOS/Windows,
>> you can't do that, you have to use a specific syscall. It's not
>> "completely unnecessary", just a different way of acheiving the
>> same effect.
>
>No, you can't just read a directory as a flat file, and expect your
>program to work on any file system.
>
>Modern (as in within the last 40 years) Unixes support multiple file
>systems including DOS. The directory representation is different in
>different file systems. Some use B-trees and hash tables and whatnot.
>
>On the Linux kernel, reading from a file descriptor that refers to a
>directory is not allowed; it fails with errno == EISDIR. (Not sure
>whether that's at the VFS layer, or whether specific filesystems refuse
>that individually.)

Sure. But by the time that sort of support for multiple
filesystem types came around, the structure of many Unix file
systems (with hard links for "." and "..") had already existed,
and there was desire to retain compatibility with existing
on-disk filesystem images. So even though the landscape is
radically different now, as you describe, history got us here.

Btw, the bigger reason that hardlinks to directories are
prohibited isn't because file tree walkers would have to be
rewritten to take care when transversing such filesystems, but
rather because automated filesystem checkers like `fsck` would
have to be a _lot_ more sophisticated to cope, and they have
the potential to do real, significant damage if they get it
wrong.

- Dan C.

On Sun, 05 May 2024 15:52:18 +0100, Richard Kettlewell wrote:

> Lawrence D'Oliveiro <ldo@nz.invalid> writes:
>
>> You think getting “stuck in a loop” is not a bad idea?
>
> [Non-answer]

Consider three directories A, B and C which point to each other like this:

A → B ↔ C

Note this: the refcount of (number of pointers to) B is 2, and that of C
is 1.

Now consider what happens if B is removed from A. Its refcount goes to 1,
so it is not deleted.

B ↔ C

But there is no way to access it! There are no other pointers to either B
or C, so both of them are now effectively lost as far as users are
concerned--yet their space cannot be reclaimed (or that of any items that
might be in them).

Not, that is, unless you run some kind of “garbage collection” pass over
the filesystem, to reclaim orphaned storage. Filesystems are not typically
designed with this sort of requirement in mind. Wonder why?

Lawrence D'Oliveiro <ldo@nz.invalid> writes:
> Not, that is, unless you run some kind of “garbage collection” pass over
> the filesystem, to reclaim orphaned storage. Filesystems are not typically
> designed with this sort of requirement in mind. Wonder why?

Again, the techniques are well understood; GC has been used in other
contexts for decades. In general you can’t reliably infer from something
not being done that it would be a bad idea, merely that it hasn’t proved
necessary.

--
https://www.greenend.org.uk/rjk/

On Wed, 08 May 2024 08:00:47 +0100, Richard Kettlewell wrote:

> GC has been used in other contexts for decades. In general you can’t
> reliably infer from something not being done that it would be a bad
> idea ...

Hint: think about what’s different about filesystems versus management of
main memory.