How `curl -L cueup.com/jobs` works

http://tech.cueup.com/blog/2013/03/06/how-our-terminal-friendly-jobs-page-works/

by danicgross • 13 years ago

84 35 13 years ago

tech.cueup.com

danicgross2• 13 years ago

OP here. I'm curious to learn of any other creative uses of ANSI and curl. Does anyone know of websites that use it as an easter egg?

ysangkok2• 13 years ago

since you're only dependent on dumping stuff to the terminal, I think Telnet is comparable. Did you see the Blinkenlights Star Wars?

pjscott3• 13 years ago

There are some pretty cool Telnet BBSes still running:

http://www.telnetbbsguide.com/brieflist.asp

It's a good way to be suddenly overpowered by nostalgia. I feel this strange desire to PEEK and POKE, and perhaps to call interrupt 13h.

dice• 13 years ago

Not curl, but I did something similar for a particularly dramatic death scene in a MUD 15 years or so ago. The hardest part was shutting up all of the normal MUD chatter that the players get (OOC chat and the like).

songgao• 13 years ago

One thing that I often use is

  curl ifconfig.me

But this is so damn cool!

minikomi• 13 years ago

I just added one to http://poyo.co .. thanks for the idea!

pfranz• 13 years ago

Slashdot used to stick random Futurama quotes in their http header. I'm not seeing it anymore. curl -sI http://slashdot.org

patrickod• 13 years ago

Is anyone else unable to see this due to tech.cueup.com not resolving?

pjscott3• 13 years ago

You know the worst kind of DNS problem? The kind that only affects some people. Yeah, I can't resolve the domain name either. Ugh. For now, the Coral Cache hack works:

http://tech.cueup.com.nyud.net/blog/2013/03/06/how-our-termi...

EDIT: Looks fixed now.

re_todd• 13 years ago

It might help to add Google's DNS to the list of DNS servers, e.g. on my Ubuntu, I sometimes add "nameserver 8.8.8.8" to the end of my "/etc/resolv.conf" file.

ysangkok2• 13 years ago

Works with Wget too:

wget -q --user-agent="curl" -O - cueup.com/jobs

recroad• 13 years ago

Honestly, fuck anyone who does this for a job posting.

arjie• 13 years ago

Why? There's a normal (nice looking) page for other user agents. You haven't lost anything.

akavi• 13 years ago

I dumped it into a file out of habit, which sort of ruined the effect. I did wonder why a ~300 line textfile took so long to DL, though.

danicgross2• 13 years ago

Any hacks you can think of to detect this from the server?

tedchs• 13 years ago

Curl is not going to send different request headers to the server based on whether its stdout is redirected to a file.

bdarnell• 13 years ago

Start out by sending dummy data as fast as you can. Once you've sent a few TB you can be relatively confident that it's going to a terminal instead of a file (which would have filled up the disk by now). Then you can start slowly sending the real data.

spitfire• 13 years ago

Reminds me of an old dialup BBS. Can I post a comment on the wall?

cheeze• 13 years ago

Maybe it's just me (using PuTTY), but it is next to impossible to read while it scrolls up my screen. Once it's done scrolling I can't see most of the listings because by default my terminal doesn't save that many lines of history.

Awesome idea though

pjscott3• 13 years ago

I'm pretty sure there's a setting in PuTTY to increase the size of the scrollback buffer. Memory is cheap these days.

philfreo• 13 years ago

Where are these colorMarkdown and htmlToMarkdown functions?

hamburglar• 13 years ago

This is a cool idea, but anyone who needs "how it works" explained is really not the target audience.

raverbashing4• 13 years ago

I'm afraid my system does not have yet support to http protocol

This server does not apparently has any service listening on port 23 so unfortunately I can't telnet to it (I'm not sure what's this /jobs as well, is this a Gopher page? oh well...)

laumars4• 13 years ago

Port 22 is SSH. Telneting to it wouldn't do much good aside telling you which sshd was listening.

I'm also a little confused by what you mean when you said your system doesn't support "HTTP protocol", curl has been a staple for Unix and Linux for over a decade. Or are you trying to run curl from cmd.exe in windows?

raverbashing4• 13 years ago

I meant port 23, sorry

And maybe you're too young to understand the joke ;)

lloeki2• 13 years ago

I hoped this would work:

    $ telnet cueup.com 80
    GET /jobs HTTP/1.1
    User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
    Host: cueup.com
    Accept: */*
    <blank line>

but it redirects to Location: https://www.cueup.com/jobs, so this is necessary:

    $ openssl s_client -connect www.cueup.com:443
    GET /jobs HTTP/1.1
    User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
    Host: www.cueup.com
    Accept: */*
    <blank line>

And there you go (^D to stop).

Unfortunately things get munged up since it's using Transfer-Encoding: chunked, so there's the chunk size and blank line in between each content line.

Bonus points for cueup.com if they update the thing to not chunk it if User-Agent: telnet (I did test this and it doesn't work)

iso-8859-1• 13 years ago

Sadly there seems to be no MIME type for ANSI art. Of course this should depend on the "Accept" header and not the User Agent. But it seems infeasible.

redmattred• 13 years ago

Your /jobs link in your sites footer is 404ing

jkbr• 13 years ago

With HTTPie (master):

   http --pretty=none https://www.cueup.com/jobs User-Agent:curl

(`--pretty=none` is needed because the server incorrectly sends `Content-Type: text/html`)

How `curl -L cueup.com/jobs` works

Related Stories