Schrödinger's 😻 and outside-the-box naming

http://lwn.net/Articles/545741/

by edwintorok • 13 years ago

123 84 13 years ago

lwn.net

biofox• 13 years ago

More interested by the title... I had no idea unicode emoticons existed.

Found a complete list here:

http://www.alanwood.net/unicode/emoticons.html

frou_dh• 13 years ago

Not sure how standard they are, but there are a several hundred you can browse in OS X by selecting Edit > Special Characters... > Emoji in Finder. 🐈

ygra7• 13 years ago

They have been standardised in Unicode 6.

INTPenis• 13 years ago

I don't know how but the Homebrew project print a pitcher of beer somehow in my iTerm2. Most likely unicode but it was not listed in the page you linked.

Zirro• 13 years ago

🍺 Beer Mug - Unicode: U+1F37A (U+D83C U+DF7A), UTF-8: F0 9F 8D BA

You can find all of them in the Emoji section of Special Characters through the Edit-menu, as described in another comment. They work in the classic Terminal application as well, along with most other applications using standard OS X API:s or ones that have specifically added support.

bergie• 13 years ago

There are more symbols in http://www.alanwood.net/unicode/miscellaneous-symbols-and-pi...

snowwrestler• 13 years ago

I have an emoji apple in the Terminal prompt on my Mac. If I don't see that splash of red I know I'm on one of my Linux servers.

creamyhorror• 13 years ago

I've seen a bunch of companies start to use icon/emoticon characters in their email subjects to get attention - Newegg, LinkedIn, etc. It certainly works on me. I'm afraid soon every mail will have an attention-grabbing icon and we'll just get inured to it.

I have to admit, if I launched an email campaign, I'd use them too...

Evbn4• 13 years ago

Very easy to spam detect on non-alphanumeric chars in subject.

xentronium• 13 years ago

Why is unicode support so bad in 2013?

ygra7• 13 years ago

Because every implementation of anything has certain assumptions and when you're not someone who regularly juggles different scripts at once it's easy to miss a lot of things (e.g. blindly assuming that everything is ASCII).

Also part of the problem was U+0027 and its use as a quote character in shell scripts. So that was not a Unicode issue but rather a shell script injection issue and programmers do that all the time (although usually with SQL, I guess).

roel_v• 13 years ago

Because it's so much work for so little return.

zokier• 13 years ago

You might ask why shell scripting and string escaping are so bad in 2013 as well. Those have couple of decades more history behind them after all.

edwintorok2• 13 years ago

Not all Unicode characters are indexed by search engines, for example the 😻 is not indexed apparently: http://lwn.net/Articles/545993/

Millennium• 13 years ago

It's basically Y2K without the sense of urgency. People assumed ASCII, or at least single-byte supersets of ASCII, would pretty much be the norm, so they never bothered with anything else. And since there are no apocalypse nutters breathing down people's necks to fix it, it often gets deprioritized.

Plan 9 actually tried to do something about this: it assumed Unicode for everything, and invented their own encoding for the process. The OS didn't work out so well (regrettably), but we still use their encoding: it's called UTF-8 now. Still a superset of ASCII, I guess, but at least we've gotten beyond the single-byte assumption.

fusiongyro• 13 years ago

It's kind of a dull problem, unfortunately. I think this is also why so many of the XML standards are poorly implemented.

h2s2• 13 years ago

While disappointing, it's perhaps not surprising that the umlaut in "Schrödinger's Cat" caused trouble. The fact that the apostrophe was ruled too risky as well, however, is an indictment on software engineering as a profession.

If people are scared to put basic punctuation marks in the names of things, out of fear that badly-written software might break as a result, then that is a sign of just how far we still have left to go.

heja2009• 13 years ago

I'd argue it is worse that almost all languages can not be used to name things. A pain that many computer users in non-English speaking countries feel regularly. As a German I set all my devices to US English for this reason -although this causes other problems. The apostrophe is really just related to shells and scripting and should be a bit easier to fix.

praptak• 13 years ago

I agree and disagree. If everything had unicode support, we'd still probably want to limit the codepoint set used for identifiers that are supposed to be unique. Lots of characters have visually indistinguishable glyphs ('x' could be cyrillic kha as well as plain old x) and this already causes problems - see homoglyph attacks.

estebank• 13 years ago

Notice that the fix was to use "Schrödinger’s Cat" instead of "Schrödinger's Cat". The fix was removing the single quote, as the script checked if there were characters that would run afoul of string escaping. The problem as it is has nothing to do with Unicode and was actually fixed by using a Unicode character.

That doesn't mean that there aren't any Unicode bugs in other tools, but that wasn't the issue in this particular case.

j-kidd• 13 years ago

Handling of special character is usually a bigger problem than unicode. Just today, I tried to name a file with both a single quote and an exclamation point from bash shell. Ended up doing that with a GUI file manager.

pavlov4• 13 years ago

Unfortunately your comment is a sign of just how far we have left to go to get rid of the notion that computers are for use by English speakers primarily, and the rest of the world is an afterthought at best.

On 99% of the keyboards I've seen in my life, the middle row reads like this: ASDFGHJKLÖÄ'

The apostrophe is considered significant enough to be on that row, but so are Ö and Ä. It's reasonable for users to expect that software can accept the letter that is right next to L on their keyboard, yet there remain software engineers who assume that users won't be surprised that things break if they dare to touch this key.

h2s2• 13 years ago

My point was that Unicode is newer than ASCII, and that we can't hope to deal with Unicode (Ö) properly if we can't even cope with ASCII (') yet. Nothing to do with anglocentrism at all. I agree that there are lots of annoying computing problems for non-English speakers though.

MichaelGG• 13 years ago

I'm curious, are the Ö Ä characters so common that they warrant dedicated physical space? I've seen "Spanish" keyboards that have a dedicated ñ, but it seems like the letter isn't that common and would be better served via a key-combo.

In fact, some IMEs already intercept " ' : etc and won't print them literally, but assume they're going into a combo with another letter.

I've talked to a few friends, and they said it was a holdover from typewriters and not really designed with modern computing in mind.

Jakob• 13 years ago

For me in Chrome on Mac OS 10.8 instead of an emoticon it just shows a sqare. Safari showed the cat.

The square looks like the qed symbol, I thought the article was about a proof or a box. Both worked fine in my mind :)

omaranto• 13 years ago

It sounds like something weird is going on, I would expect the following behaviour: if you have a font with the cat glyph installed on your system all browsers display the cat, if you don't have a font with the glyph then no browser displays the cat.

Why does Safari show the cat while Chrome doesn't? I can think of two explanations: (1) Mac OS doesn't actually have a designated place to put fonts for all applications to find (I don't use Mac OS so I have no idea how it works, but this sounds unlikely), (2) Mac OS does have a central location for fotns but Chrome doesn't use them and just uses some fonts it comes bundled with. Is either of those two explanations correct? If not, what is going on?

dragonwriter• 13 years ago

> Why does Safari show the cat while Chrome doesn't? I can think of two explanations: (1) Mac OS doesn't actually have a designated place to put fonts for all applications to find (I don't use Mac OS so I have no idea how it works, but this sounds unlikely), (2) Mac OS does have a central location for fotns but Chrome doesn't use them and just uses some fonts it comes bundled with. Is either of those two explanations correct?

I suspect its an OS-related issue, as the cat shows in Chrome on Kubuntu 12.10 for me.

jakub_g• 13 years ago

In fact, some participants in the mailing list discussion proposed adding non-alphanumeric characters to future release names just to see what happens. (...) Peter Robinson proposed the project go right for the goal and choose "DROP table *;".

We need a courage like this! ;)

johnchristopher• 13 years ago

That part of the article made my day :)

lmm• 13 years ago

Who's really going to use os-release in an SQL database? Real courage would be 'Fedora 20 ; rm -rf / #'

deafbybeheading• 13 years ago

Don't forget --no-preserve-root

pjscott• 13 years ago

I believe this release name would combine the best of both:

    Fedora 20 ; rm -rf / # '; DROP table *; --

moron4hire2• 13 years ago

:( the cat glyph worked in Chrome on my phone but not on my desktop.

pavlov4• 13 years ago

Oh, it's supposed to be a cat glyph?

Chrome on OS X 10.8 shows me a square, and so I assumed that the joke is that it represents Schrödinger's box -- you know, where he keeps the cat which may or may not be dead.

moron4hire2• 13 years ago

Heh, that's actually kind of a neat, serendipitous, not-really-neat-because-this-shit-should-fucking-work-by-now-it's-2013-fer-chrissakes fallback.

meatsock• 13 years ago

mine only showed up as a box until i observed it.

evanb• 13 years ago

My OS X 10.7.5 Chrome shows a cat glyph in the title in the tab bar, but a square in the actual text. Quite weird.

kunai• 13 years ago

Odd, considering it works fine on Firefox on OS X 10.8 for me. Perhaps your Chrome install is broken? Or does Chrome simply not support the cat symbol?

yaddayadda• 13 years ago

I have yet to see the cat glyph in the title (firefox, chrome, or safari) is it 1F63B in http://www.unicode.org/charts/PDF/U1F600.pdf ?

Edit: changed glyph based on someone mentioning it had heart-shaped eyes

edwintorok2• 13 years ago

Yes it is 1F63B, to confirm use iconv,xxd and copy+paste the character from the title:

  $ iconv --to-code UTF-32BE|xxd
  😻
  0000000: 0001 f63b 0000 000a                      ...;....

qb453• 13 years ago

Maybe it's some special "Schrödinger's cat" glyph. My Firefox displays it properly on HN, but when I follow the link the tab bar shows a square. And title bar a cat.

sneak• 13 years ago

That's the wavefunction collapsing.

scilro• 13 years ago

It's a known issue in Chrome: https://code.google.com/p/chromium/issues/detail?id=62435

jonnyscholes2• 13 years ago

Hmm, is there a reference of who actually designed all these unicode chars anywhere? I remember looking a while back to no avail - these more 'graphic' ones resurfaced my interest. Did one poor sod do the whole lot [of Emojis] or...?

ygra7• 13 years ago

The ones you are seeing come from a font you have installed. There should be a copyright somewhere in there. For those in the Unicode code charts they come from several people and prototype fonts, usually.

jonnyscholes2• 13 years ago

Ah, of course. Thanks :)

gbhn• 13 years ago

Next release name: "' && rm -rf /"

Chris_Newton4• 13 years ago

There are two big lessons from this discussion, IMHO.

Firstly, broad character sets are all very well, but they have limited value if you can’t rely on everyone using your text being able to see them. Something like the ö (o-umlaut) is a reasonable thing to expect in modern Western fonts, but what about the emoticons I’m increasingly seeing in e-mails, or more advanced mathematical operators, or the cat in the title of this very article that many people commenting here can’t see in some contexts?

We need much better standardisation and prioritisation of sets of related glyphs that are, for example, permitted by the coding standards for a software project or supported by a font file. ASCII is too small, all of Unicode is too big, and picking glyphs for inclusion in fonts one at a time is too fine-grained for this purpose.

Secondly, it is crazy that with literally a million code points available in Unicode world, we don’t seem to have new control characters for “begin literal” and “end literal” to mark a range of text that should be interpreted verbatim regardless of context. Instead, we’re still using horrible hacks like quoting and escaping in environments like command lines and source code, and in text file formats like comma- or tab-separated values. These kinds of techniques are, invariably, horribly error-prone and terrible for usability; after all, in the case we’re discussing, it seems to be the apostrophe that is causing more problems than the ö! I think the computing world would be a much easier place for many, many people if there were one universal standard way of saying, “This is plain text”.

Jabbles• 13 years ago

Is it too much to expect that every device should be capable of rendering every unicode point (latest version at release) in at least one font? With the option to fall back to that font if it tries to render something in a font that doesn't have that character?

Obviously there are issues with unicode phishing of domain names and other cases where you might want to signal that a character is "strange", but surely the memory and processing requirements for this are low enough now. It doesn't have to be a good font!

Chris_Newton4• 13 years ago

Is it too much to expect that every device should be capable of rendering every unicode point (latest version at release) in at least one font?

Yes, I think it probably is. Unicode is vast, and the 100,000+ characters specified in the latest standard include numerous obscure, specialised, or downright gimmicky ones.

The effort required to create just one font that supports even a crude version of each and every character is probably measured in human lifetimes. Imposing that kind of burden as a barrier to entry for any new platform seems unrealistic.

Edootjuh• 13 years ago

You might be interested in GNU Unifont: http://unifoundry.com/unifont.html

Evbn4• 13 years ago

Um, how could you ever showll someone the string-termination character without escaping it?

ygra7• 13 years ago

> we don’t seem to have new control characters for “begin literal” and “end literal” to mark a range of text that should be interpreted verbatim regardless of context

That is not of Unicode's concern. Markup is reserved for higher-level things and is not done within Unicode. There are historical oddities like language tags and variation selectors but the former are deprecated and the latter exist to serve a particular need. You will never see semantics like those you describe applied to code points in Unicode. That's not what Unicode is for.

Chris_Newton4• 13 years ago

That is not of Unicode's concern. Markup is reserved for higher-level things and is not done within Unicode.

Well, I’m not sure what authority you’d cite for your statements about what is or is not Unicode’s concern, but right on page 3 in chapter 1 we have:

“The Unicode Standard began with a simple goal: to unify the many hundreds of conflicting ways to encode characters, replacing them with a single, universal standard. [...] Whenever textual data was converted between different programs or platforms, there was a substantial risk of corruption.”

That sounds familiar, given the various contexts I mentioned before. I wonder whether the cost of all the software bugs ever caused by getting quoting/escaping wrong is greater than other repeat offenders like dereferencing nulls.

Then on page 8 in chapter 2 there are several explicit text processes mentioned, including user interaction and treating text as bulk data.

I do understand that a basic principle of Unicode is to represent plain text rather than mark-up, but it seems to me that avoiding problems with representing widely used text strings in different ways using different quoting and escaping conventions is consistent with that principle and exactly the kind of lack of standardisation that Unicode should help with.

There are already more than 60 control characters in Unicode. There are also many characters that aren’t described as control characters but which convey things like layout information or rendering hints, such as the non-breaking space, soft hyphen, and direction controls. I know that language tags are deprecated, but those are rather complicated and context-specific and, as the standard itself observes, supporting them is a significant burden on implementers, so I don’t think it is reasonable to equate that situation with what I suggested.

On balance, as a practical matter, if you're working in a text-only medium, your character set is all you have. If anything else must be done using some form of mark-up built with the same character set, then a simple, unambiguous, standardised way to switch from one to the other seems entirely consistent with the general goals of the Unicode project.

Links to relevant sources:

http://www.unicode.org/versions/Unicode6.2.0/ch01.pdf

http://www.unicode.org/versions/Unicode6.2.0/ch02.pdf

unabridged• 13 years ago

There is a reason the latin alphabet is the most used in the world, and why 7bit ascii is the standard for computing. Simplicity. They are highly distinguishable characters consisting of mostly straight lines and simple curves.

Diacritical marks (training wheels) and unicode break this simplicity, you cannot program in a language with many characters that are hard/impossible to distinguish. Or with crazy characters that can switch direction of text and other nonsense. Unicode is a luxury to pretty things up for end users, not something to do serious work in.

Dewie2• 13 years ago

>, and why 7bit ascii is the standard for computing.

The US.

> Simplicity.

Oh...

> Diacritical marks (training wheels) and unicode break this simplicity, you cannot program in a language with many characters that are hard/impossible to distinguish.

Many languages get by with their communication in spite of having things like "spots over the o's" or whatever. I have no problem distinguishing them. Do you have experience with reading such languages? Or are you simply blowing hot air?

Ever looked at typewriter font for l and 1? Yeah, these things are not historical accidents at all...

> Unicode is a luxury to pretty things up for end users, not something to do serious work in.

It's a luxury for end users... yes, non-English users should count their blessings when they are able to use their whole alphabet. Part of this issue was correctly spelling the name of an Austrian physicist, not someone trying to write "cat" with some esoteric Unicode that looks like a cat.

Schrödinger's 😻 and outside-the-box naming

Related Stories