Requestdiff – Send two HTTP requests and visualize any differences

by victordg • 11 years ago

142 33 11 years ago

smarx• 11 years ago

Seems like a useful tool; thanks for sharing it!

Two questions:

1) What's the diff logic? At first glance, it looks like JSON is reformatted (maybe canonicalized in some way) and then a line-by-line diff is applied. Is there more to it? Since the tool seems JSON-aware, I was surprised to see an added trailing comma up as a difference.

2) Do you have plans to expand the kind of HTTP requests users can make? It would be nice to use different verbs, headers, and request bodies. Runscope has a similar tool[0] built in that I believe (haven't tried it yet) allows a bit more flexibility, but it would be nice to have a standalone tool available.

[0] http://blog.runscope.com/posts/comparing-http-requests-using...

victordg5• 11 years ago

Great questions:

1) the diff is really simple, line by line and text based. I'm just using this really awesome library to do it: https://github.com/kpdecker/jsdiff The JSON is just being pretty printed to be able to do a text diff in a meaningful way.

2) Yes, I do, and thanks for the suggestions! But I just wanted to go with the minimal version first and see if people found it useful.

I didn't know about Runscope's diffing feature. Btw, I'm using a HTTPBin for the sample requests, which is a Runscope project as far as I know :)

zwischenzug2• 11 years ago

This would be useful behind a firewall. Is the source available?

victordg5• 11 years ago

Not yet, but I'm planning to open source it later today. I will reply here with the Github link. Glad that you found it useful!

weruiojkl24• 11 years ago

Awesome, waiting for the Github repo!

zwischenzug2• 11 years ago

Great, thank you.

kozhevnikov• 11 years ago

Reminds me of Diffy, a tool Twitter is using to algorithmically diff test their HTTP-based services.

https://blog.twitter.com/2015/diffy-testing-services-without...

jalfresi2• 11 years ago

For command-line junkies:

diff <(curl -sS -L https://httpbin.org/get) <(curl -sS -L https://httpbin.org/get?show_env=1)

jalfresi2• 11 years ago

Actually this is closer still to the output. Requires colordiff:

diff -u <(curl -sS -D - -L https://httpbin.org/get) <(curl -sS -D - -L https://httpbin.org/get?show_env=1) | colordiff

Aissen• 11 years ago

You can replace colordiff with vim's less tool (/usr/share/vim/vim74/macros/less.sh ) or vimpager (https://github.com/rkitover/vimpager).

icdiff is also nice to have a side-by-side colored diff (https://github.com/jeffkaufman/icdiff). And of course, you could use meld, vimdiff, kompare etc. with the same bash anonymous pipes indirection.

dgulino• 11 years ago

you can leave off the pipe: colordiff -u <(curl -sS -D - -L https://httpbin.org/get) <(curl -sS -D - -L https://httpbin.org/get?show_env=1)

tacone• 11 years ago

The website does show differences in the HTTP headers though.

Tepix• 11 years ago

As does this solution.

therein• 11 years ago

Good work but frankly this would be useful if it was:

(1) Terminal based

(2) Supported other types of HTTP requests

(3) Supported request body

(4) Allowed editing request headers

(5) Wasn't so easily exploitable to be used as a proxy or a DDoS relay (server-side bummer):

http://requestdiff.com/proxy?url1=https://www.amazon.com&url...

odabaxok• 11 years ago

(6) Support encodings other than utf-8.

JackuB2• 11 years ago

You might want to take a look at Gavel https://github.com/apiaryio/gavel (and its JS implementation Gavel.js https://github.com/apiaryio/gavel.js)

(disclosure: I work for Apiary.io)

diggan• 11 years ago

This is very cool! Looking forward to it being open source (also, things from Barcelona rocks ;) )

For people wanting to have a CLI tool instead, John Graham-Cumming's httpdiff might be worth looking at. https://github.com/jgrahamc/httpdiff

victordg5• 11 years ago

Thanks a lot :)

Httpdiff is a really good tool and way more powerful than mine.

I've used it multiple times and wanted to try making something that focused more on the request body (which in Httpdiff are saved to temporary files) and that was also a bit more visual than a CLI.

Btw, here is the code! https://github.com/vdel26/requestdiff

amelius• 11 years ago

I'd like to have a bookmarklet that lets me compare the states of the DOM trees of a website taken at two different times.

blemasle• 11 years ago

Requests are made server side, which make this unusable for local/dev environnement purpose. Too bad !

victordg5• 11 years ago

Right. I was aiming to do it client-side only but I found out that web browsers restrict the headers you have access to through cross-origin XHR requests [1][2].

So it was a trade-off and I preferred showing all the headers at the expense of having to proxy the calls through a server.

[1] http://www.w3.org/TR/XMLHttpRequest/#the-getallresponseheade...

[2] http://www.w3.org/TR/cors/#cross-origin-request

I will make it available on Github later, so that you at least have the option of running it yourself.

JackuB2• 11 years ago

For dev or CI you might want to take a look at Gavel https://github.com/apiaryio/gavel (or even Dredd for that matter https://github.com/apiaryio/dredd)

(disclosure: I work for Apiary.io)

vortico• 11 years ago

It's pretty easy to write a recursive diff function that compares JSON strings, in order to avoid the JSON -> diff by line hack that you're doing. But it's a clever hack that easily translates to the command-line.

andybak• 11 years ago

Wonderful.

Just the other day I needed something similar and was disappointed that I could find it.

I wanted to discuss something with a remote colleague and to illustrate it I wanted a visual diff of two files. I was hoping there was a nice little web app offering this but I was forced to screenshare (I could have terminal-shared but it was more hassle).

I was hoping for something like Etherpad but with a live visual diff.

no_gravity• 11 years ago

Comparing URLs comes up from time to time here on Hacker News and at one point inspired me to write a tool that batch compares pages visually:

http://www.productchart.com/blog/2015-07-19-urldiff

It renders a set of pages in a headless browser, compares them visually and alerts you if something changed.

Just a few lines of bash as you can see. But it turned out to be pretty useful. UrlDiff is a regular part of our regression testing at Product Chart now.

dannybtran• 11 years ago

Not working on Safari Version 8.0.6 (10600.6.3) Mac OS X 10.10.3

From JS Console:

[Error] TypeError: undefined is not a function (evaluating 'Array.from(e)') _toConsumableArray2 (app.min.js.pagespeed.ce.ozGaCBt6Kj.js, line 1) s (app.min.js.pagespeed.ce.ozGaCBt6Kj.js, line 1) f (app.min.js.pagespeed.ce.ozGaCBt6Kj.js, line 1) onload (app.min.js.pagespeed.ce.ozGaCBt6Kj.js, line 1)

skastel• 11 years ago

If you're looking for something similar, but with more features that natively understand HTTP, check out www.runscope.com. Full disclosure, I'm an employee and proud of what we've built!

krakensden• 11 years ago

Another solution: https://github.com/xthexder/httptee

A daemon that sends to two backends, and diffs the results.

pixelbeat• 11 years ago

Personally I use these from the command line:

http://www.pixelbeat.org/scripts/urldiff http://www.pixelbeat.org/scripts/idiff

See also mergely which supports diffing URLs: http://pixelbeat/programming/diffs/#mergely

SimeVidas• 11 years ago

Use cases?

gregingelmo• 11 years ago

Use Vim, it's the best diff tool by a long shot. Syntax highlighting, side-by-side diffs, and code folding (when large contiguous sections are equal)

vimdiff <(curl -s "https://httpbin.org/get") <(curl -s "https://httpbin.org/get?show_env=1")

It takes some getting used to but once you go vimdiff you'll never go back.

gdc• 11 years ago

Doesn't seem to work at all in IE11 / RTM Win10.

Requestdiff – Send two HTTP requests and visualize any differences

Related Stories