Who invented file extensions in file names?

https://retrocomputing.stackexchange.com/questions/27926/who-invented-file-extensions-in-file-names

by azeemba • 3 years ago

243 197 3 years ago

retrocomputing.stackexchange.com

This is also sort of a flame war topic for some. Divided primarily by those who believe the file’s content should contain the type (or be sampled to infer it) and those who prefer that the file’s name contain the type. More or less.

netsharc• 3 years ago

I'm already tempted to throw in my arguments.

Is it a flame war or a religious debate? E.g. vi vs emacs

gustavus• 3 years ago

For those embroiled in this holy war there has come a mediator that has quelled the discord and brough harmony and his name is Doom, Doom Emacs

(Seriously if you've been a vimmer for a while and wanted to give emacs a go give Doom Emacs a try and drop hlissner a couple of bucks it's definitly worth it)

alexvitkov4• 3 years ago

Yeah, except one side is obviously correct. Not knowing what program will open up when they click a file is insane.

veidr9• 3 years ago

No, just an explicitly-defined metadata field. Inference would be worse than filename extensions (which are awful).

I think the original Mac OS fucked us all by implementing this pretty much perfectly in the 1980s, but using their weird HFS "resource forks" to do it so that it not only couldn't be implemented on other platforms, but also lent credence to the "ugh fuck it lets just mangle the file name... and then hide it~!!! wooo000!~!" argument..

¯\_(ಠ_ಠ)_/¯

UPDATE: My memory was faulty; the "what kind of file is this?" metadata was actually stored as per-file metadata, the "type" and "creator" fields in the HFS(+) filesystem.

pavlov4• 3 years ago

Frustratingly NTFS has had proper metadata support for 30 years but FAT32 was a bag of rotten hacks, so Microsoft couldn’t fix this either even though their forward-looking OS had everything needed.

Someone• 3 years ago

but using their weird HFS "resource forks" to do it

That’s incorrect. File type and creator code always were stored where they belong: in the file system.

Also, what’s weird about alternative file streams?

so that it not only couldn't be implemented on other platforms

The main reason it “couldn’t” be implemented elsewhere is because of the dominance of FAT, which didn’t think it worthwhile.

paholg3• 3 years ago

Why are filename extensions awful? I find it incredibly useful to tell what kind of file something is by just glancing at it. Or do you envision tools like `ls` and file browsers also showing metadata at all times?

gsich6• 3 years ago

No, just an explicitly-defined metadata field. Inference would be worse than filename extensions (which are awful).

Which are bad because those don't survive filesystem transmissions. A filename with extension will.

alana314• 3 years ago

I remember Mac OS 8 not having file extensions and it was very difficult to change a file's type. I believe hexedit was involved. Extensions IMO are way more reliable.

gus_massa• 3 years ago

For extra confussion, most files have the info in a header at the begining, but zip files have the info at the end, so someone made a file that is at the same time a valid .pdf and a valivalid .zip. (I can't find the link now.)

frabert• 3 years ago

https://www.sultanik.com/cv

capableweb2• 3 years ago

POC||GTFO does this with their releases, where the file released usually doubles as multiple file formats, and contains a ton of hidden messages all over the place. https://pocorgtfo.hacke.rs/

From the latest edition (#21):

Technical Note: The electronic edition of this magazine is valid as both PDF and ZIP. Thanks to Ange Albertini, it is also a PCAP-NG packet capture of an experiment by Yannay Livneh. See page 7.

Edit: Ah, and of course, Cosmopolitan (by @jart) that produces an amalgamation of formats bundled into one file (including ZIP) that runs across a bunch of OSes https://news.ycombinator.com/item?id=38101613

BobaFloutist• 3 years ago

That's actually a very fun idea.

saghm• 3 years ago

There also used to be an exploit where you could make a file both a valid .jar Java executable and a valid .gif, which back in the days of Java applets would potentially let you execute code when someone loaded your image (e.g. if you uploaded it as your profile picture): https://en.wikipedia.org/wiki/Polyglot_%28computing%29#GIFAR...

13of40• 3 years ago

My favorite is storing a batch file uncompressed as the first entry in the ZIP archive, because you can rename it with .BAT or .CMD and run it. Sprinkle in some PowerShell and you can make a self extracting archive/installer.

dylan6042• 3 years ago

Filenames are the worst place to store metadata. Even a file's specific metadata fields are not always trust worthy. In the realm of media assets, relying on the file's metadata is a giant footgun. I'm not even talking about anything malicious, as that's designed to attempt to fool people. I'm referring to simple mistakes which would cause incorrect processing down the line if solely depending on metadata. For example, marking the video field dominance as progressive when the content is truly interlaced or vice versa.

Even in the world of image uploading via browser, just checking extensions is discouraged. If the feature is that humans can be allowed to manipulate it, it must be sanitized/verified before accepting. This isn't just for text heading for a database.

pbhjpbhj3• 3 years ago

Filenames are the worst place to store metadata.

Where's the best place? It seems like being able to send a file handle to an app without having to open the file nor read a sidecar/db could be quite convenient?

babypuncher3• 3 years ago

The problem with declaring a file's type in the content itself is that you end up rendering all prior file formats obsolete because they lack this information.

The better solution I've seen proposed is to store file type information as metadata in the filesystem itself, but this would lead to compatibility problems with archiving tools and any other scenario where a file is moved between differing file systems.

File extensions are the best way to make determining a file type easy without parsing the whole thing while also maintaining cross and backwards compatibility.

morsch• 3 years ago

It's getting less relevant as regular users stop using the filesystem directly and instead rely on proprietary services for their data. Google Drive doesn't rely on file extensions. Though for all I know, Microsoft's web stuff does...

What's the web equivalent of using a floppy disk to move data between Mac, OS and Windows, I wonder. Downloading to the local computer and uploading, presumably. I guess that is pretty equivalent. Though you can't even download the native representation of a Google (text, spreadsheet, ...) document.

avgcorrection3• 3 years ago

Why is Java apparently one of the few languages that are brave enough to use four characters in its file extension? “Rust” becomes “rs”.

tbm57• 3 years ago

fewer letters saves valuable developer time

typical java, prioritizing verbosity at the expense of efficiency /s

dmoy• 3 years ago

/s

Not entirely sarcastic though, looking at e.g. DescriptiveFooThingFactoryFactory

duxup3• 3 years ago

I used to write all my html files with .htm… apparently that went out of fashion.

Edit: I got my htm and html backwards and now comment not sense. Fixed now but obviously the folks below were right / responding to an earlier version.

veidr9• 3 years ago

wh... what extension do you use now then

treyd• 3 years ago

I personally use .htm for anything that's an incomplete chunk of an HTML-ish document, like a Jinja2 template file, and .html for full rendered documents.

Aardwolf• 3 years ago

Possibly the following? It wouldn't 'sound' enough like a file name extension if it was the full name '.rust'

Where the inertia of the history of 3-letter extensions in DOS and the likes is what caused this convention, of course

xnx• 3 years ago

Some languages (e.g. Perl, Python) are old enough to have been concerned with the 3 character extension limit from MS-DOS: https://en.wikipedia.org/wiki/8.3_filename

throwaway167• 3 years ago

As a RapidSketch user on my organisations web ratings board working group living in a coastal area affected by soil erosion and with a penchant for DNA modelling, I find Rusts choice of file extension perturbing.

.rust.programming.language would make far more sense than pretending Rust will ever need to work in DOS.

capableweb2• 3 years ago

Rust DOS : Creating a DOS executable with Rust

https://github.com/o8vm/rust_dos

liampulles• 3 years ago

Likely it was not a brave decision - just a decision, with no reason to pivot.

icedchai• 3 years ago

Probably because the first release was in 1996, so Java never ran on a platform (like DOS) that needed only 3 characters. Originally it only ran on Solaris and Windows 95/NT.

js23• 3 years ago

There's no abbreviation of Java that isn't weird. These all work well:

perl -> pl

python -> py

rust -> rs

ruby -> rb

javascript > js

But java?

java -> ja? jv? jav?

Clearly Sun should've gone with a different name for the language entirely, I dunno, maybe they could've named it after an oak tree instead:

oak -> ok

Hmm, maybe I could get used to "jv" after all.

duped6• 3 years ago

".j"

adwn• 3 years ago

Why would "ruby -> rb" be okay but not "java -> jv"?

kps• 3 years ago

That's up with the oak leaves, over everyone's head. Anyway, `.ja` is German for `.ok`.

pbhjpbhj3• 3 years ago

Why not .jva ?

infradig2• 3 years ago

Prolog -> pl

Then Perl tried to steal it.

Izkata4• 3 years ago

The logo has always been a cup of coffee (one of the meanings of the word "java"), so maybe either ".cof" or ".caf" (for "caffeine")

halper• 3 years ago

Boiled down to its essence: ".137"

the_mitsuhiko• 3 years ago

Pretty sure the answer is that developers are lazy and want to write fewer characters.

hn927268192• 3 years ago

If you think .java is brave, wait until you hear about Mojo's extension: https://en.m.wikipedia.org/wiki/Mojo_(programming_language)

They allow the extension to be the fire emoji

CSMastermind• 3 years ago

Hot take: we should include emojis as reserved words in programming languages.

lp0_on_fire• 3 years ago

Maybe I'm just an old curmudgeon but things like "you can use emojis as variables" (or in this case "the extension can be an emoji") scream amateur hour and a lack of seriousness to me. There's zero benefit, IMO, other than "it's quirky".

Which is a shame because the people who created Mojo are definitely not amateurs.

qingcharles2• 3 years ago

I had to answer my own question the other day -- can you make a table or a column in MS SQL that is simply an emoji?

Yes. Yes you can.

And God help us all.

cesarb• 3 years ago

four characters in its file extension

You are not looking close enough. While Java source code has four characters in its file extension (.java), the output of its bytecode compiler has five characters in its file extension (.class).

otteromkram• 3 years ago

Markdown is .md and .markdown.

No one uses the latter, though.

gardenhedge• 3 years ago

Another brave language is Dart. example.dart

kibwen• 3 years ago

I can answer the Rust question. In the ancient era, Rust source code was split into two types of files: .rc files and .rs files. The latter were "Rust source" files, and the former were "Rust crate" files. The crate files were essentially header files; they defined the public interface of the crate by explicitly listing items to be exported from the crate. Eventually explicit export lists were removed (and replaced with visibility modifiers on items themselves), and so there was no reason for .rc files to stick around, and nobody really ever objected to .rs as the now-universal extension for Rust files, since it's still a valid abbreviation.

doubloon2• 3 years ago

"why use big word when small word do trick" - Kevin Malone

avgcorrection3• 3 years ago

Also Orwell.

280877267• 3 years ago

file content contains the type can be read by computer.

file name contains the type can be read by human.

pavlov4• 3 years ago

JPEG files should have the extension .jpeg. Fight me.

belter• 3 years ago

Legacy systems, web norms, and user habits dictate '.jpg' triumphs over '.jpeg' for compatibility, brevity, and conformity....Smack down...

pavlov4• 3 years ago

My only friend and ally is the macOS standard Image Capture app which writes scanned files with the .jpeg extension.

I’m guessing this could even be a NeXT leftover because Image Capture is ancient. The .jpeg extension was more popular on 1990s Unix.

1024core• 3 years ago

.. and not .JFIF?

mjcohen• 3 years ago

Originally extensions were only 3 characters long.

pavlov4• 3 years ago

Originally as in Unix? I don’t think so.

But later on microcomputers in CP/M and thence MS-DOS, yes.

daveslash• 3 years ago

I don't know who specifically invented file extensions, but Raymond Chen's The Old New Thing has some good insight into why they are important on Windows. Edit: In summary, it is to be able to support Tape based drives and avoid opening a file "unless a user tells you to".

https://news.ycombinator.com/item?id=32811707

tpmx• 3 years ago

Just to be clear: Bill Gates and Microsoft weren't anywhere near inventing this. They bought what was mostly a clone of CP/M and called it PC-DOS/MS-DOS. That's where they got the file extension scheme from.

Digital Research (the creators of CP/M) also didn't invent this. For more details, see the stackexchange link.

gilcot2• 3 years ago

The poorly written clone was QDOS (for quick and dirty)

PaulHoule• 3 years ago

I remember them on the DEC PDP-8 (I was taking classes at the high school part time in the 4th grade, the high school had one that had those big 8 inch floppy drives) and RSTS/E on PDP-11 early on (spent a lot of time with one at the high school in Milford one summer.) These were on CP/M, a microcomputer OS that was big on high end microcomputers from 1975 to 1985 or so that was the inspiration for MS-DOS which also had them.

rqtwteye• 3 years ago

It's always amazing to think that somebody at some point had to invent things we now take for granted and feel like they just have to be that way.

As a sidenote it's sad that WinFS failed.

thriftwy• 3 years ago

WinFS failed because users do not care that much about local files. Cloud is everything.

Local files are even easier to lose and even harder to search than cloud.

dleslie• 3 years ago

It was shelved in 2006. The product managers must have had incredible foresight if they expected cloud storage to supercede local storage.

Hell, try as they might, Microsoft still doesn't have ubiquitous OneDrive usage among Windows users.

pixelesque• 3 years ago

First part I'd agree with: the second part I'm not sure, if only because of my experiences and frustrations of using MS OneDrive as part of my work's O365 integration: it unbelievably seems impossible to actually list all files in the web interface - I can only see "Recent" documents (which are documents I've viewed or opened, not new ones I've added), and I end up having to search to find new documents I've added (i.e. saved from Outlook, or scanned docs from printer), open them, and THEN I can see them in the main doc view.

Maybe I'm doing something wrong, but other colleagues have complained about the same thing, so I'm not sure...

ravenstine• 3 years ago

That is until The Google inexplicably locks your entire account with no recourse.

HumblyTossed• 3 years ago

WinFS failed because users do not care that much about local files. Cloud is everything.

This is such a web weenie take. Not everyone lives in the "cloud".

So annoying that we have to call it "the cloud".

tamimio• 3 years ago

Local files are even easier to lose and even harder to search than cloud.

Not if you have your own NAS, and you can use programs like everything (1) for an easy quick search, and you are in control of everything not under someone else’s policy mercy.

(1) voidtools.com

stronglikedan• 3 years ago

That's funny. To me, "cloud" is just a backup that I've thankfully never had to restore from on my computer, and only ever when I get a new phone.

cafard• 3 years ago

Back 40 years ago, Data General's AOS/VS defined 255 file types. In many cases, maybe all, there was a corresponding extension for the file, e.g. .exe for executables, .txt for text and so on.

However, there was a trick that I encountered: copies made not by a CLI command or a properly written utility, say by sending a file over email, did not preserve the type in the filesystem. Then you would try to compile or assemble the code somebody sent you, and get a baffling message about your file being of the wrong type. If you had encountered this before, you would

  REN xyz.asm xyz.xyz
  CREATE/TYPE=ASM xyz.asm
  COPY/APPEND xyz.asm xyz.xyz
  DEL/V xyz.xyz

and go on with your work.

mixmastamyk• 3 years ago

Wow, begging for a script or direct utility.

user3939382• 3 years ago

We usually can use file extensions to differentiate files that otherwise share the same name. I once worked on a VAX mainframe OS that to my surprise supported completely duplicate filenames including extension!

didgetmaster2• 3 years ago

Using file extensions (or having to open the file and inspect contents) to determine file type is a slow way to find files. If you have one of the newest hard drives that is capable of storing 100M+ files on it (or a NAS made up of multiple, smaller drives); then it can take forever to find all files of a certain kind (e.g. photos, documents, etc.).

Just try to write a program that lists all your photos. You have to know all the various extensions that apply to still images (.jpg, .jpeg, .gif, .png, etc.) and do a string comparison against every file in the drive hierarchy. Might as well go to lunch waiting for it to finish if you have many millions of files to search.

This was the primary reason why I set out to build a better system that could scan through hundreds of millions of files and find all that matched a given subset in just a few seconds. https://www.Didgets.com

fooker2• 3 years ago

You seem to go out of your way to avoid mentioning 'index' when that's the usual way to solve this problem.

didgetmaster2• 3 years ago

Apparently, your experience with various indexing services has been different than mine. I found it is just too easy for the index to become 'out of sync' with the actual file system and has to be occasionally rebuilt. Even with just 1M files, that always took a long time; but with 100M+ files?? If the storage was removeable or if you booted occasionally with another OS (or even just a different version of the same OS) then I found it was near impossible for an indexing service to keep track of all the changes.

eviks• 3 years ago

But Everything does this kind of search instantly on NTFS

porridgeraisin2• 3 years ago

Yeah, it uses some kind of journal to keep updated on changes etc

crazygringo2• 3 years ago

Tangentially related: I find it weird that relating file extensions to GUI document icons still doesn't work intuitively.

On my Mac, the document icons for file types still correspond to whatever program I've chosen to open that extension type by default.

Which is so strange -- an .mkv file isn't a VLC document; an .mp4 file isn't a QuickTime document, an .mp3 file isn't an Apple Music document, a .jpg isn't a Preview document, and a .tiff isn't a Photoshop document. What the heck?

I wish the OS would just define reasonable, attractive, default document icons for every standard file type, that applications couldn't control/overwrite. Reserve application-defined document icons just for those apps who truly have their own proprietary file format (e.g. .psd or .docx).

ulfw• 3 years ago

To each their own. I prefer to know which app will open when I double click a file. Seems Apple agrees as that's how it's working on macOS.

radlad• 3 years ago

100%. Having the icon show what's going to actually open lets me know whether I need to right click and "Open with..."

Workaccount2• 3 years ago

I'm surprised people are adverse to this on here. Having both the extension and the graphical icon convey the same information is redundant and wasteful.

ajb92• 3 years ago

I share your pain. Luckily there are great apps out there (I use IconChamp[1]) that let you take full control of any given extension's associated icon, among other cool things. If your point is more purely about Apple's stance, then I extend a hearty "hear hear".

[1] https://www.macenhance.com/iconchamp.html

account42• 3 years ago

That's how it generally works under Linux desktop environments - there are generic icons based on file type no matter what application is used to open them.

leejoramo• 3 years ago

I always liked the Classic macOS way for using "File Type" and "File Creator" meta data. File extensions were not used at the operating system level, but were useful for cross-platform sharing or as away to organize file.

The file icon would be assigned by the "Creator" Application. Double-clicking the file would cause it to be opened by the "Creator". You could choose to open a file with anything that advertised that it could work with a "File Type". There were utilities to change the "File Creator".

dheera2• 3 years ago

It was also a royal pain in the ass to change the file type should you want to open the same file with another application, edit it, save it, then go back to the 1st application.

robotnikman• 3 years ago

Its the same thing on Windows as well, and many Linux desktop environments. Seems to be a universal thing.

kube-system• 3 years ago

I think it is intuitive, the icon indicates which application you are opening. If I want to know which file type it is, I turn on 'show file extensions'.

toast0• 3 years ago

Which is so strange -- an .mkv file isn't a VLC document; an .mp4 file isn't a QuickTime document, an .mp3 file isn't an Apple Music document, a .jpg isn't a Preview document, and a .tiff isn't a Photoshop document. What the heck?

Aren't they? Most of these tools can open many specific types of documents, so you might want to know if a document is a movie or a picture in addition to what program will open it.

And of course there's the issue that Preview is actually an editor.

solardev2• 3 years ago

I have the opposite preference... I want to know if my OS is going to open a PDF in Preview or it was hijacked by Adobe again, or if a .cfg file is going to open up in some useless Microsoft System Configuration app, or a JSON file is gonna open up my super bloated IDE instead of a text editor.

It's an additional piece of information, rather than just mirroring what the extension already tells you.

nashashmi• 3 years ago

Makes for an intriguing concept in a tv show set in an alternate universe (Fringe?), where file extensions are not in the name but in the attributes of the file.

You would not need to hide file extensions then.

CharlesW• 3 years ago

This is how it worked in (Classic) Mac OS. I’m sad we lost this in (today’s) macOS.

wazoox• 3 years ago

ProDOS on the mighty Apple /// (and also the Apple ][) had a file type metadata back in 1983 :)

In fact even Apple DOS 3.1 (1978) supported 8 different file types.

jdswain• 3 years ago

It was called SOS on the Apple ///, originally Sara's Operating System but officially Sophisticated Operating System. It was a very advanced OS for its time, most notably one of the first to include a useful driver model.

The SOS filesystem was reused for ProDOS (and later GS/OS as well), but a lot of the advanced features were removed for ProDOS due to wanting to fit into the smaller RAM of the Apple ][.

xkcd-sucks• 3 years ago

Somewhat related, the question of "what IS a file's 'type' really" is pretty messy and AFAIK doesn't have any satisfying solution or any satisfying single answer [0]. A satisfying solution might look like a (long) feature vector. Libmagic goes in the right direction at least although the binary-fingerprint-at-the-beginning approach doesn't work well for everything / past some level of detail

[0] where different use cases might be satisfied by e.g.

    "a text file" 
    "a file exclusively comprised of alphanumeric UTF16 characters" 
    "a CSV file"
    "a file with tabular data and key-value metadata that calls itself CSV but is not spec-compliant so internally we'll call it CSVprime"
    "A file with tabular data written as ASCII text that represents $this_kind_of_measurement"
    ... etc

If you have any resources on this topic off the top of your head I'd appreciate it if you shared them

duped6• 3 years ago

GNU file doesn't just check magic numbers against libmagic, it also defines a scripting language and a series of tests/printers written in it.

That's what allows it to do complex things, eg identify all the flavors of ELF objects even though they share a magic number or determine if something is JSON or CSV without one.

prmph2• 3 years ago

The point is that, fundamentally, the concept of a file type is undecidable or not well defined.

Think about it: A JSON file can also be considered a text file. It could also be some higher level type of file, depending on whether it conforms to some application-specific JSON schema. Thus the kind of file it is has more to do with what you want to do with it; it's not some intrinsic property of the file.

clort• 3 years ago

I'm pretty sure the file command is not GNU btw, and I can't find anything about a GNU version.. do you have a reference?

https://en.wikipedia.org/wiki/File_(command)

spullara• 3 years ago

Not sure why it would think a module javascript file is java. Maybe just not updated very often.

    % file crawl.mjs 
    crawl.mjs: Java source, ASCII text

amelius2• 3 years ago

Well, you can define a file's type in the same way as you define in-memory data structures, with a type system.

prmph2• 3 years ago

See my comment: https://news.ycombinator.com/item?id=38117891

nomel• 3 years ago

Are you suggesting putting the (I assumed versioned) type system into the file's metadata, so that others can make sense of it? Or, some type system UUID, in the metadata, that can be used? I think anything like this would require many new concepts, like the move away from files being a collection of arbitrary stored bytes.

Lammy• 3 years ago

If you have any resources on this topic off the top of your head I'd appreciate it if you shared them

I wrote a Ruby library that attempts to be good at this https://github.com/okeeblow/DistorteD/tree/NEW%E2%80%85SENSA...

fsckboy• 3 years ago

A file is "of a type" if there exists some agent (program X or human H) that creates/writes such files, and whereby there exists some agent (program Y or could be Y==X or human I or could be I==H) that reads/modifies such files, and whereby the entire contents of such files can be made use of or understood in a useful/meaningful way; and other files not of the type's contents either can't be made use of meaningfully nor updated. Some file types may be sub- or super-sets of other file types (ex: .C source files are .TXT files, but .TXT source files are not necessarily .C files, they could be .CSV files) And files could be streams.

might wish to discuss files containing type information, and data/value information both, thus numerous but different valued files can all be of the same type.

you can get fancier if you want and talk about valid C source files or C source files which contain bugs, etc.

you could consider a file "weakly or adhoc typed" if there is only one or a pair of agents that use that type, and "strongly or standard typed" if there are numerous "different/distinguished" agents that "rely" on that file type.

this was off the top of my head

LargoLasskhyfv• 3 years ago

tl;dr MAD FAP

Who invented file extensions in file names?

Related Stories