Invisible Characters

This is another good reason to have a text editor you really trust, which can show you these things. Whether it's different line-endings or weird invisible space stuff, I know I can just open it in Vim and figure out what's really going on pretty quickly. Wasted a lot of time earlier in my life on that nonsense (:

TacticalCoder2• 4 years ago

I agree with you.

I've got my Emacs set up to display in "bold, fluo foregound and a dark background underlined by a pink line" (yes, literally that obnoxious) any character which is not part of a list of characters I consider to be acceptable. And it's configured to show any "zero width" character as if it had a width. So any "invisible character" as well as any "invisible zero width character" does appear as a black square, underlined with a pink line.

And that for any buffer/file.

enchiridion• 4 years ago

Can you share that config? Sounds useful!

userbinator• 4 years ago

I use a DOS text editor for this, where no Unicode support is an advantage. The majority of the time I'm dealing with plain ASCII anyway.

nervuri3• 4 years ago

Vim does not display them all. The only program I checked which displays all such characters is `less -U`. You can test using this file:

https://gitlab.com/nervuri/nervuri.net/-/raw/master/gopher/z...

interroboink2• 4 years ago

Thanks for this! Good to add to the ol' repertoire (:

Looks like the only one Vim misses is U+17B5? Though it there could be more not listed there. Unicode is a deep dark forest.

----

For others readers, here's a non-gopher version of the article linked inside: https://nervuri.net/stega

saliagato• 4 years ago

    ⁭⁭⁭

30minAdayHN• 4 years ago

back in 90s on windows, our secret directory used to be alt+255 (it looks like a space but not space i think)

lifthrasiir• 4 years ago

Which is a non-breaking space (U+00A0), which is mapped to 0xFF in the code page 437. (You need to put a leading zero to access to Unicode code points, like alt+0255 for U+00FF ÿ.)

brunorsini• 4 years ago

I did the same thing. One more of those habits that didn't survive the transition to Mac OS X.

hamiltonians• 4 years ago

usefull for impersonation scammers , like on twitter

abrudz• 4 years ago

Great for doing tacit programming[1] in JavaScript:

  avg=ㅤ=>ㅤ.reduce((ㅤㅤ,ㅤㅤㅤ)=>ㅤㅤ+ㅤㅤㅤ)/ㅤ.length
  avg([3,1,4,1,5])
  2.8

[1] https://en.wikipedia.org/wiki/Tacit_programming

shepherdjerred• 4 years ago

I love point-free functions languages that support them, but please tell me you're not actually do this

orbital-decay• 4 years ago

My favorite is U+202E Right-to-Left Override, which doesn't appear to be listed there. A surprising amount of UIs (apps, sites) can be broken with it as they were never tested with right-to-left writing direction in mind. Even a Unicode reference website that I just used to recall the code is broken by it. [0] Entering RLO into arbitrary input forms for fun can bend spacetime, I swear.

[0] https://unicode-table.com/en/202E/

andreareina• 4 years ago

https://xkcd.com/1137/

emsixteen• 4 years ago

I've blissfully ignored RTL so far, but I know that I shouldn't - I just can't imagine the pain of actually figuring out how to deal with it. :')

My favourite character is the Greek question mark;

iamevn• 4 years ago

Even a Unicode reference website that I just used to recall the code is broken by it.

I like to imagine that sites like this have noticed the bug but have the sense of humor to choose not to fix it.

Mockapapella• 4 years ago

A while back I used these kinds of characters to encode programs into invisible text: https://www.thelisowe.com/sleeper-cell-a-method-of-embedding...

It doesn't do much on its own. I feel like it could, but the most effective use case I've come up with it you can invisibly plant a piece of code in some piece of text, then later on run another script that looks for that piece of code and runs it. I'm guessing that splitting the code up like this would make it harder to detect (not to mention that this code could even reside in other programs' comments undetected).

ttyprintk• 4 years ago

I’ve seen it as a Bacon code in capture-the-flag.

thirtyseven• 4 years ago

So I guess the only future-proof solution to check for this is to render user input off screen and count the number of solid pixels, at least until "falsehoods programmers believe about names" gets updated to include "Names must consist of at least one readable glyph".

voiper1• 4 years ago

"Names must consist of at least one visible glyph".

ludovicianul• 4 years ago

I've built a tool specifically to test if these kind of characters will reach API backends: https://github.com/Endava/cats. My idea was that APIs should explicitly reject or sanitise input containing such characters.

nervuri3• 4 years ago

Zero-width characters can be used to covertly watermark text and to figure out who copied text from a page and pasted it somewhere else. Server software can encode a hidden number between every few words, which corresponds to a server log entry with your username (if logged in), IP address, browser fingerprint, etc. I wrote more about this here:

https://nervuri.net/stega

I think the best solution to this type of problem would be a clipboard utility that warns you when you copy text which contains hidden characters, homoglyps, rarely used whitespace characters, etc.

Invisible Characters

Related Stories