Terminal escape sequences in Git commit email field

https://twitter.com/ryancdotorg/status/1375484757916672000

by whack • 5 years ago

322 155 5 years ago

Off topic, but the person posting this mentioned the artist who did the avatar and of course I love their art style, and of course they aren’t taking commissions right now. How does one find good pixel art artists who might do a small commission or three?

scrollaway• 5 years ago

Deviant Art used to be a good start a few years back. For pixel art you could also look at r/gamedev, lots of artists looking for work roam it.

ChrisMarshallNY• 5 years ago

They have some awesome artists, there, but I have had bad luck getting through to them on DeviantArt. There's a couple of artists that definitely could have made some money from me, but couldn't, because I couldn't contact them (or they ignored my contacts).

krasin• 5 years ago

Fiverr? I've had a generally decent experience with it, but I also follow the rule of three: always hire 3 different freelancers for any task, unless when I have established relationships with an artist who did very similar things in the past.

IgorPartola6• 5 years ago

Oh that’s a good point. I’ve used them before. It was neat.

aviditas• 5 years ago

This user does small commissions on occasion. http://imgur.com/gallery/qszHeHe

IgorPartola6• 5 years ago

Love the Slavic turtle tax, thanks!

opencl• 5 years ago

Try searching on skeb, it's a site specifically for art commissions and there are at least a few people doing pixel art on there.

IgorPartola6• 5 years ago

Oh this is a super cool concept! Thanks!

qbasic_forever• 5 years ago

I'd go spelunking in indie game dev forums and spaces. There are almost certainly artists looking for a little bit of side work.

wlesieutre• 5 years ago

You might find someone for hire at r/gamedevclassifieds

https://www.reddit.com/r/gamedevclassifieds

IgorPartola6• 5 years ago

Oh cool. I’ll look there.

DownToSky• 5 years ago

It's usually not hard to follow a few pixel artist and get a bunch suggested to you on Twitter. I know Kiana Mosser should have commissions open within a few weeks. https://twitter.com/kianamosser

tubularhells• 5 years ago

Try pixelation.org

hoten• 5 years ago

You can find forums where pixel artists look for work. Here's one such forum: https://pixelation.org/index.php?board=8.0

IgorPartola6• 5 years ago

Oh wow exactly what I was looking for. Thanks!

strogonoff• 5 years ago

Follow good artists you like on various platforms, get in touch as appropriate (use your best judgement).

junon3• 5 years ago

Go to the Aseprite discord server. They have a channel exactly for what you want.

asadlionpk• 5 years ago

I recently found one for a side project via instagram. Look under #pixelart tag.

ryan-c10• 5 years ago

I found Brian when someone I follow retweeted (or perhaps liked?) this post:

https://twitter.com/cmmrc2/status/1329991505524690944

I'd been looking for an artist for months at that point, and looked through his work and saw some nice portraiture and dropped him a DM.

I can't figure out who shared it, but I think it was another pixel artist who I liked and was hoping would open up commissions again.

I wish it were more common for artists to auction off at least some commission slots.

Blikkentrekker• 5 years ago

Well, one can put characters from the Canadian Aboriginal Syllabic Block into Go identifiers, so this would be the next step, I suppose.

bombcar2• 5 years ago

The next step is to add animation.

viraptor• 5 years ago

It should be possible: https://github.com/saitoha/libsixel

(Unless sixel required nul bytes in the encoding)

ryan-c10• 5 years ago

For animation, the real issue is frame timing. It looks like there's an animation API in kitty that allows this sort of control, so it'd work there, and iTerm2 supports inline animated gifs which I've already demo'd.

JohnCurran• 5 years ago

This same person tweeted a terminal animation out yesterday: https://twitter.com/ryancdotorg/status/1375880616789573633

Though I suppose it's not technically a git commit email, it's still a git command

bombcar2• 5 years ago

That looks very graphical - not the ASCII characters of the portrait.

ryan-c10• 5 years ago

It was still stuffed in the author email field.

Hendrikto• 5 years ago

Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should.

elchief• 5 years ago

You can name PostgreSQL tables with poop emoji

easton• 5 years ago

You can also use emoji to name any object in Active Directory, which is a heck of a lot of fun if you want to confuse the IT guy that needs to remote into your machine for maintenance.

rootsudo• 5 years ago

Okay, I'm going to try this now. This is perfect. I wonder if it'd break anything else.

This is gold.

pixl97• 5 years ago

Dont forget wifi also allows emojis in the SSID.

dharmab• 5 years ago

I have an Outlook folder named (trash can) (fire) and a rule set up so anything older than 30 days in there is deleted.

ExtraE• 5 years ago

Interesting. HN will let me submit a comment that contains emoji just fine (no error in dev tools network log or UI) but it doesn't show up in the thread (at least for me, if there's a stream of emoji from me testing that I can't see I apologize).

Izkata• 5 years ago

HTML/CSS also allows emoji in class names

wyldfire• 5 years ago

Oh wow, can someone repost that ANSI art documentary that was shared recently? Great inspiration for my next commit.

hedora• 5 years ago

Related: The Unix Haters Handbook, chapter 6:

https://web.mit.edu/~simsong/www/ugh.pdf

RealStickman_• 5 years ago

I will have a lot of fun reading this. Thank you!

ryan-c10• 5 years ago

Oh, wow, I'd heard of the unix haters handbook, but I never realized it was an actual published book.

genera1• 5 years ago

It's still a good read, not only as a historical artifact, but also because many of the criticisms still apply to the spiritual descendant of Unix, Linux. It's funny to see that 20 years ago people were already complaining about X window system or Unix security philosophy.

u801e• 5 years ago

I'm not sure if there are any format specifications for the author and committer lines in a git commit message object. I guess it really depends on whether commands that are used to display the information (git show, git log, etc.) can handle arbitrary data. I wonder how commands like git format-patch and git am would deal with a commit like this.

DarknessFalls• 5 years ago

I've not had much success showing block characters in the standard Mac Terminal.app. Is there a particular font I need to use?

dapids• 5 years ago

AFAIK terminal.app doesn't have any control code filtering before it's rendered. Although iTerm2.app does have this as a feature, if that helps.

tambourine_man2• 5 years ago

It can probably be done in grayscale, with even some dithering with shading characters from Unicode Block Elements to support Terminal.app and other emulators.

userbinator• 5 years ago

Many years of experience have taught me that fields holding data being able to hold any sequence of bytes is normal (and in the case of string data, anything except 00), so this didn't surprise me too much; in fact, it's usually when something is unusually restrictive that it becomes notable.

ryan-c10• 5 years ago

I was honestly more surprised at how much data I could fit in that field. Git itself doesn't appear to have any limit at all, though GitHub doesn't allow more than a few dozen megabytes.

invsblduck• 5 years ago

why is this so astounding to those commenting on twitter? because git doesn't artificially limit the line length?

i've used ansi sequences in my zsh prompt for 20 years to make colors and move the cursor; it's just in-band ascii that is interpreted by the terminal emulator, no?

danielheath• 5 years ago

By default, I'd expect data that likely came from third parties to be sanitized before output.

enriquto5• 5 years ago

Sure, sure, but why would such a sanitization remove useful things like terminal drawing codes? That would be untoward.

theamk5• 5 years ago

it was, note the twitter had you pass "--no-pager" to get cursor movement to work.

With default options, you only get colors, and this is pretty safe.

etaioinshrdlu3• 5 years ago

Its things like this that make me think terminals are irredeemably broken. One does not simply print a string to a terminal. I'm not aware of a standard way to handle user strings safely like you can on say, the web, or any other UI platform.

The Unix way seems to be to not solve the problem at all, unless you're writing a text editor, and then you reinvent the wheel and solve it yourself.

jasonjayr• 5 years ago

This is the SAME type of injection attack we've been fighting on the web for years.

Apache 1.3 (way back when) would escape escape sequences before printing them to the log, to prevent this exact same thing.

It is the responsibility of the program dumping data to the terminal to escape things prior to dumping to the tty.

etaioinshrdlu3• 5 years ago

Except, could you imagine a software ecosystem where 95% of the software is vulnerable to injection, and nothing is ever done about it, and there is no clear easy solution for developers either, and you have a legion of users who turn technology into a religion and think there's nothing wrong at all with Unix terminals.

It also probably doesn't help that the problem is unfixable in general without rewriting so much software... and that things mostly work okay, most of the time.

jojobas• 5 years ago

What's unsafe about it? It doesn't break out of the terminal as a system that shows data to the user. The fact that you can sneak data that looks weird doesn't make it unsafe.

yuubi• 5 years ago

Some terminals support a programmable answerback string, and of course the ENQ control character to send it to the host. Some have programmable function keys and a command to read them, which often (always?) sends the programmed string as consecutive unencoded characters (instead of hex or something).

lunixbochs4• 5 years ago

My vtclean project [1] cleans terminal escape sequences. It handles the full ECMA-48 spec, and every behavior described by all of the vt100 documentation I could find, and observed behavior of many command line programs and terminals.

It has good test coverage, fails safe (if it doesn't understand a sequence, it still strips the escape character), and can preserve colors in cleaned up text. It is directly and indirectly used by quite a few projects. It receives extremely few bug reports, because I took the time to do it right.

If you want a good way to do this, probably start here. It's three short regular expressions and a simple state machine (to clean up single-line edits).

My only complaint about the code at this point is it operates on string instead of []byte, but I don't want to break the API and nobody has complained.

[1] https://github.com/lunixbochs/vtclean

GSGBen• 5 years ago

Thanks for writing this! I used it a few years ago to remove the ANSI output from HP/Aruba network devices that clear your screen or line which was a nightmare for scripting. It works like a dream.

exikyut3• 5 years ago

It could be interesting to extract this to a context-free grammar that could be implemented in different languages.

There are many reasons users might not want/be able to use Go, or any other language this might be published in. Given the terminal's provenance I can't help but be absolutely certain (despite having no substantive evidence) a lot of people working with C in Interesting™ situations would love something like this, for example.

theamk5• 5 years ago

If you want to make string safe, filter out or escape all bytes / characters in 0-31 range. Maybe keep newlines if you expect multiple lines.

Done -- as simple as it gets.

caf3• 5 years ago

Some terminals support the C1 control set, and in particular 0x9b as the CSI is dangerous (it's a single character version of the ESC-[ CSI). So you have to watch out for that too.

tempodox3• 5 years ago

I'd posit that in this case it's git's failing to pipe content from some random internet connection unfiltered to the terminal. The terminal doesn't know where those control sequences come from. But anytime you interface with something external, you have to be prepared for hostile behavior or content that's out of spec. In this case it's git that interfaces with something external. Remember that any software can be hacked or abused.

teddyh• 5 years ago

I'm not aware of a standard way to handle user strings safely

I thought that this was what isprint() was for?

junon3• 5 years ago

I maintain chalk and a slew of other TTY related things.

Terminal emulators are, far and away, the most archaic computer technology we still use daily.

Believe me when I say, it's a very hard problem to solve since everyone has subscribed to Unix's philosophy of "everything is text".

As much as I dislike Microsoft's philosophies on software, they had the right idea with Powershell. Its execution is just abysmal.

Expect this stuff to change drastically in the next 5 years.

gugagore• 5 years ago

Why are the next 5 years going to be different than the last 50?

st_goliath• 5 years ago

Expect this stuff to change drastically in the next 5 years.

Bwahaha... is this your first encounter with backwards compatibility?

Changing this would be very hard, but this has nothing to do with somebody embracing some grand philosophy, but with tons of existing software relying on Unix like systems to, well ... be Unix-like, because that's what the software was written for.

Ericson2314• 5 years ago

I've given it a lot of thought, and below is my one "reformist" idea to significantly improve things without breaking back compat so bad

Invert the shell and terminal: every shell command (with unredirected streams) gets it's own pty.

- Like the those repl "notebooks" you can scroll each output separately, without hacks.

- Backgrounded processes won't spew garbage when you are trying to type. They just continue running in their terminal. You can even background something like vim or a game that takes the whole terminal, and it's just fine!

- Rather than hacking scroll back onto processes that might use tty stuff in expected ways, just straight-jacket them by giving them plain pipes instead! The OP exploits are then opt-in impossible.

- Having powershell-esque "post-text" commands that dispense with the bullshit are far easier to manage: also give them regular file descriptors and negotiate protocol or whatever, just like the above, rather than a pty.

- stdin can be a nice form with submit button rather than fixed (usually line) buffering policy.

- shell might even be simpler as there is less careful juggling of shared resources except where the programmer asks for it.

I also use ssh multiplexing instead of tmux for extra shells. When I do need to use tmux, I hate that read only mode prevents scrollback. The above builds nicely upon this:

- The persistent remote shell has as many ptys as needed before for chained commands which might still be running, or separate "sessions"

- The local client can use native UI elements for everything, no TUI jank.

- Scrolling over many (live or saved) terminals, the normal scrollback case, is possible and very side-effect free.

Anyone see any issues? I don't for anything I do daily (git, coretuils, etc.)! I don't think I have time to make this, so please someone else do.

junon3• 5 years ago

No, I'm saying this because there exists mailing lists and chat rooms where all of the terminal emulator developers are discussing these problems to collectively solve this issue.

tambourine_man2• 5 years ago

We've tried many times and in many ways, but text is still the main interface for programers (and even for a lot of end user tasks as well).

As long as this interface remains undisputed, "everything is text" is a feature, not a bug. Arguably, one of Unix's greatest.

rurban2• 5 years ago

The feature is "everything is text". The bug is the misunderstanding that "every text is identifiable". That names are garbage in, garbage out. Unix names are far worse than non-unix names in this regard.

Terminal sequences, unicode all violate the identifier rules. There is no u8ident library to sanitize this input or output.

barrkel• 5 years ago

And yet the web is text too - and has lots of fun vulnerabilities relating to failure to escape content.

I do not expect text to be significantly replaced any time in the next 5 years. Accessibility - screenreaders, increased text size with text reflow - is one massive reason.

na852• 5 years ago

I like the idea of Powershell and how it's object driven as well, but it doesn't get you away from text in the sense of you're still typing commands into a console.

Isn't the real solution to do away with the idea that we need to remain compatible with VT100 video terminals?

naikrovek3• 5 years ago

That's not what they are referring to. Powershell passes typed objects between cmdlets when pipes are used. This DEFINITELY better than plain text.

We've learned a lot since the unix philosophy was created. One of those things is that, if you want software that lasts, software that performs well, you need to be strongly typed.

Plain text is as far from strongly typed as you can get. Passing plain text between programs in 2020 goes against everything we've learned in the last 50 years.

tsujp• 5 years ago

I'm unfamiliar, what's the idea of Powershell you mention here?

serial_dev• 5 years ago

By default, it uses objects (structured data) instead of plain strings.

I'm also not extremely familiar with the concept, so at the moment, I'm reading this document to understand it better https://www.varonis.com/blog/how-to-use-powershell-objects-a...

ChrisSD2• 5 years ago

Nushell has a similar philosophy to powershell. They sum it up as "everything is data". Their book gives more details.

https://www.nushell.sh/

enriquto5• 5 years ago

it's a very hard problem to solve since everyone has subscribed to Unix's philosophy of "everything is text".

Look at it the other way round. Embrace text. Text is not the problem, non-text is. You'll take text from our cold, dead hands!

Scene_Cast22• 5 years ago

When I type "ls", I'm getting a text-serialized list of objects. Why can't I just get the (serializable) list directly? So that I don't have to mess with (implicitly) converting text back to objects, often involving regex and hacks.

ChrisSD2• 5 years ago

Text is fine for humans. The trouble comes when you want the machine to do anything other then simply print out the literal text (and even that's tricky, what with unicode and fixed width columns).

The moment you want to do something else with the text, you have to parse it in some way. At that point it's no longer text, it's a poorly specified serialization format.

exikyut3• 5 years ago

+++ATHokay.

silon422• 5 years ago

Actually text is now a problem... The old way of ASCII only stream of bytes was much superior.

proc0• 5 years ago

I think a huge factor is the dominance of the keyboard as the most used physical interface. Even in the emerging age of VR, keyboards are everywhere, even in software form in phones. I guess once we can solve language recognition, so it maps 1:1 with the input somehow, then we could have a voice-only terminal/shell equivalent. The building blocks for that would look different and not just saying commands out load.

mhh__• 5 years ago

I see the appeal, however outside of areas where you completely own the stack (like inside a company) it feels idealistic to imagine any standard being close to text in terms of pervasiveness.

I also worry about a bad standard becoming popular a la most of the web

stinos• 5 years ago

Its execution is just abysmal

It's not really good, but calling it abysmal is just too negative in my opinion :) It does get a lot of things right after all, especially in the "not everything is text" department. Also a lot of things wrong (my main gripes would be a lot of the syntax in general, case insensitive, usually more than a couple of ways to achieve the same thing). So in the end the learning experience/curve for me was the same as for bash (or languages like C++) and just as long/steep probably: feels like suffering until you've learned all the caveats by heart and in the process of doing that also learned how to actually use it, then it becomes ok to use. After that process I'm leaning towards favoring PS over bash though. Now it's possible there are already other shells which get more things right but I really doubt that in my lifetime I'm still going to spend (waste?) time on learning yet another shell. Or it must be convincingly good and provably fast to learn :P

themadsens2• 5 years ago

I'd say, what we need is a --json flag on all pipe producers like like ls, exa, find, grep, ripgrep. Then you'll have the best of both worlds: something that is equally(?) parseable by humans and computers. (jq to the rescue)

arthur2e5• 5 years ago

With NUL-sepearated output instead of line output people are doing fine. It's unstructured for sure, but how much structure can you put into something as general as grep?

burntsushi• 5 years ago

ripgrep does that already. But I just came up with my own format from whole cloth because there is no standard: https://docs.rs/grep-printer/0.1.5/grep_printer/struct.JSON....

Adding a --json flag is just the start. Ideally, other grep-like programs would need would want to emit the same format. But what happens when some grep-like programs have additional features?

Suggesting a structured representation as an output format isn't a panacea. It's just the start and it's not clear at all to me that it would be better than what we have now.

spookthesunset4• 5 years ago

I think one of the biggest peeves of mine with powershell is they dumped you into a god awful console. I dunno if this has changed much but good lord was the console horrible. No resize, nasty font, nasty color scheme, scroll back sucked, command history sucked, pain in the arse copy & paste.

Also the commands were super long. Sure there was the ability to alias them to something shorter but still....

That being said... powershell has a lot of potential.

(Also I’m woefully out of date... I haven’t touched powershell in a decade. I hope somebody tells me it’s all better now)

majkinetor• 5 years ago

Its better.

Windows Terminal is gr8 and removes all those problems. You have alternatives too like ConEmu.

Long commands are OK, use aliases. Bash has long command arguments too and nobody complains.

Saying Powershell has a lot of potential is kinda lame - its best of them all.

creamytaco• 5 years ago

Recent, 1999:

https://packetstormsecurity.com/files/18376/ADMesc.html

daniel5151• 5 years ago

Reading that just made me want to re-watch 1995's Hackers

gruez2• 5 years ago

The problem with this attack is that you have to drop the 6c file in your PATH for it to work, because otherwise you need to use ./ for it to execute. This makes the attack pointless because if the attacker can drop something to your PATH, you're already pwned since the attacker can just name his payload "ls" and wait for you to execute it.

theamk5• 5 years ago

Good thing I normally reach to "less" instead of "cat" nowadays. You get scrolling, and your terminal is safe from malicious injections or (much more likey) binary garbage.

And "git" for example applies "less" to pretty much all output by default, which makes most of the git-based attacks, including this one, irrelevant.

tempodox3• 5 years ago

From the thread:

Our continuing mission: To explore strange new platforms. To seek out new bugs and new software. To boldly shitpost where no one has shitposted before.

Mission accomplished, LMAO.

ryan-c10• 5 years ago

Yeah, it's kind of my thing :-)

One of the previous times I hit the front page of HN it was for inserting snark into an X509 certificate and then using it on my site.

https://rya.nc/cert-tricks.html

tempodox3• 5 years ago

I'd say the hat of a security researcher suits you well :)

kenrose• 5 years ago

Previous submissions over the last few days:

https://news.ycombinator.com/from?site=twitter.com/ryancdoto...

Terminal escape sequences in Git commit email field

Related Stories