Proxy.app Lands In App Store

http://blog.websecurify.com/2014/05/proxy-app-lands-in-app-store.html

by passfree • 12 years ago

64 63 12 years ago

blog.websecurify.com

yincrash2• 12 years ago

I use Charles very often, but I use it to see requests coming from a mobile device. Can I proxy SSL connections from mobile with this? Will it let me use a custom self signed cert to do the SSL proxy?

toyg• 12 years ago

"There is SSL interception out of the box. Everything that has to do with SSL is controlled from the Keychain."

So I guess the answer is yes (although I haven't tried yet).

yincrash2• 12 years ago

But my phone doesn't use OSX's keychain to handle SSL certs. :(

passfree22• 12 years ago

You can intercept any http connection from the iOS simulator quite easily. For intercepting stuff on an actual device you require more extensive setup. We will publish docs how to do that soon.

peterkelly• 12 years ago

We will publish docs how to do that soon

Looking forward to this. I've just bought the app and being able to monitor e.g. what my iPad is doing would be great.

fideloper• 12 years ago

Is this mostly for monitoring, or does it have other uses? I'm unclear based on the app store and linked pages.

passfree22• 12 years ago

you can monitor and also intercept requests and responses and modify their contents.

joshcrowder• 12 years ago

It would be good to get a demo of this. I'm unsure of the benefits over Charles at the moment is it just native? I'm happy to pay for software that looks better but I'd like to try it out first.

passfree22• 12 years ago

We have no demo for this at the moment. We are not prepared to issue these kind of binaries at this stage. We will update as soon as we can.

hopeless• 12 years ago

I was only introduced to Burp at Scot Ruby conf last week as a tool for web security testing. If Proxy.app is targeting the same domain, it seems to provide none of the tools that Burp does.

Burp was a pain to setup (OS X makes installing and using java from the commmand-line a ridiculously complicated process) but it looks 1000x more useful than Proxy.app is.

On a marketing note, you might want to think about who your market is and what job they use a proxy for. It doesn't seem to offer anything for a security researcher but maybe there's enough there for a web developer.

passfree22• 12 years ago

This tool follows slightly different philosophy. We could have bundled all Websecurify security testing frameworks inside this app and make it more penetration testing friendly. Instead we thought that a different approach would be to create just a good proxy and add enough integration capabilities to be used side-by-side with other tools including the security tools we distribute elsewhere. In effect, it is a bit like the unix design which also closely matches the iOS application philosophy.

That being said, dumping more tools inside is not an issue. We already have the frameworks to support it but we are not sure if this will be a good thing to do from usability point of view.

calebm• 12 years ago

I like the Unix philosophy: Do one thing, and do it well.

sdevlin• 12 years ago

Burp Suite has a pretty rich feature set. How does this compare? For example, is something scriptable like Burp Intruder included (or planned)?

passfree22• 12 years ago

Burp Suite is one-size-fits-all type of solution. Indeed, it comes with a lot of other tools which are primarily used for penetration testing.

On the other hand, Proxy.app is just a good proxy and already comes with many of useful features. It is faster, feels native and available at the fraction of the cost of Burp. It is more general purpose proxy tool than a general purpose security testing framework. The tool can be nicely complemented by some of the other tools from Websecurify.

tptacek6• 12 years ago

Burp has a lot of features, but at it's core it's four things:

* Proxy records, categorizes, and makes searchable and navigable all the requests you make through Burp, and includes an internal CA to generate on-the-fly SSL certificates.

* Spider crawls websites and discovers new pages.

* Repeater takes requests from any other part of the tool (or, if you're masochistic, requests you write from scratch), delivers them to the target application, and renders their results. Repeater includes Burp's UI for breaking requests into keys and values for quick editing, and a small battery of fixups to make sure that edited requests are (if you want them to be) valid HTTP requests.

* Intruder takes a template request and a series of rules to transform that request, delivers the permutations rapidly to the target, and records and classifies the responses they generate.

Of these four tools, only the "Spider" is really particular to penetration testing. The other three tools are incredibly valuable for day-to-day debugging and testing. Intruder, in particular, is criminally undersold to web developers.

There are a bunch of other things in Burp (the Burp Scanner, for instance, is a second-tier web application security scanner that nobody I know relies on; I justify pulling it out of the core four features because it's also a recent arrival to Burp). Some of them are given whole tabs in the UI, but are really simple features --- the "Decoder", for instance, simply does character decoding. In any case, even penetration testers don't use them often.

iNeal• 12 years ago

Any significant advantages of using this over mitmproxy?

danpalmer3• 12 years ago

As far as I can tell, mitmproxy is very capable, but requires a lot of configuration to use effectively, and the interface isn't exactly brilliant. I've been using Charles for a while instead of mitmproxy just because it's quicker to set up, I'm much more productive with it. Looks like this app will be similar.

ahmadeus2• 12 years ago

I just downloaded it, and it is not intercepting anything? what setups should I do for it? Charles works out of the box!!

passfree22• 12 years ago

Hi ahmadeus,

Don't give up on us. :) It is probably something very simple. Just contact us via our contact form and we will get it sorted. http://www.websecurify.com/desktop/proxy.html

onethumb3• 12 years ago

I can't seem to find a contact form on that page?

ahmadeus2• 12 years ago

Ok so I got in touch with support, and I had to reconfigure the behavior of my mac's, which in turn affected my other iOS apps that I run on this machine. I think I would like a refund for this app, I truly need something like this but charles would just drop in and listen and not interfere or force me to change the behavior of my mac.

monkey_slap3• 12 years ago

Would you be able to give any examples, aside from being "native" (which I'm assuming means its built on AppKit+Cocoa) as to what your app does differently or better than Charles? I use Charles pretty frequently but my co workers haven't gotten on board yet.

As an aside, IMO claiming that Charles isn't native is a little disingenuous. Yes it's built with Java, but I wouldn't dismiss it as non-native (unless you're using native in the purest of ways, meaning that its "natural" - e.g. Cocoa - to the system). I consider tools like Eclipse and IntelliJ "native" even though their UI may be poor compared to Xcode.

passfree22• 12 years ago

Good questions. While Charles and other proxies are doing a fine job our dev team is also frustrated by the way they integrate with OS X. This is what we mean by native experience.

The application is in effect a single binary and makes use of various Apple frameworks which are optimised to run flawlessly on apple hardware. Only one platform is supported which means that there is less chance for bugs and platform-specific oddities. Proxy.app works as a sandbox application on OS X, distributed by Mac App Store and makes use of other native components like Keychain, etc.

Under the hood the tool is only making use of Apple frameworks. We believe that in the long term this will provide more stability and performance enhancements. Right now it takes just a moment to launch the application and there is zero friction when switching between different project files. We believe that it just works in the same way OS X applications are supposed to work and this in effect makes Proxy.app more pleasant to use.

danpalmer3• 12 years ago

I'll vouch for this. I've used Charles extensively, and while I haven't used Proxy yet, the most annoying things about Charles are always related to it not using proper UI paradigms, components, etc. I'd love something that was a proper Mac app for this.

As for the comparison with Burp, I have much less experience with Burp, but the pro version has a huge amount of functionality. Things like plotting the distribution of cookies so you can see if they are backed by a good RNG. It doesn't look like this is quite that advanced yet, but those are the sorts of features that only real pen-testers would be using. For most developers who just need to test stuff, this looks excellent, I really look forward to trying it out.

monkey_slap3• 12 years ago

Right on, thanks for the thoughtful response.

Did you guys run into any issues getting the app approved by Apple?

mikeash• 12 years ago

I think it's perfectly reasonable to describe Charles as "non-native".

For example, it doesn't integrate with Keychain, for SSL certificates, so you have to do a bunch of jiggery pokery with the openssl command if you want to generate your own CA for use with SSL interception.

For a more trivial but still annoying example, control-A and control-E don't work in Charles's text fields. In normal Mac apps, this goes to the beginning/end of the line, but in Charles it does nothing.

"Native" isn't about "poor" UI per se, but rather about whether or not it has the basic UI features provided by the system. In a native app, you have to go out of your way to disable stuff like ^A/^E because the system gives that to you by default.

I'm not a purist and will happily use non-native apps like Charles if they get the job done, but it's still a legitimate point against it.

mihaela2• 12 years ago

Java is not native. Period. I don't have a single Java app on my Mac and never will.

tptacek6• 12 years ago

To replace Burp, you need at a minimum a proxy that reliably works at high speed on a huge variety of sites, a UI for capturing, altering, and resending requests, and a "fuzzer" that will transform a template request according to a ruleset.

This turns out to be a pretty difficult thing to build well. Every pentester who can code that owns a Mac wants to rewrite Burp; I myself have died on that hill several times. But rewriting Burp is a bit like rewriting Microsoft Word; so much of the value is in the details, and there are so. many. details.

passfree22• 12 years ago

Hi tptacek,

Proxy.app is not meant to compete with Burp. Burp is a different beast and not necessarily, in our humble opinion, the best approach to penetration testing web applications. We know how difficult it is to write security tools because this is what we specialise doing. So you comment resonates with us very well.

Although we could have bundled-in all our security testing frameworks, it was decided to take a different approach for the time being. Later on we may decide to add security testing capabilities or these features may be provided by some of the other tools that we are currently working on.

zellyn• 12 years ago

If you're not trying to compete with Burp, you may want to remove it from the sentence, "While there were plenty of proxies such as Burp, ZAP and Charles…"

billyhoffman• 12 years ago

I think its important to say why it's difficult. The individual components of a web proxy/fuzzer range from easy to more challenging. At the extreme edge, they can get pretty hard:

HTTP proxy - Fairly trivial, annoyingly harder if you want to generate certs to MitM SSL.

Scaling it out - not too bad. fuzzing is highly parallelizable. How your parallel tasks work together is more challenging, (think rotating cookies) but do able, and you need to really do this right to maximize requests/seconds.

The audits themselves - If you actually want pre-built attacks, basic attack+response regexs (ala Nikto) are very easy, though building the DB is time consuming. Multi-request tests (looking for stored XSS, etc) are harder because an audit thread need to share state, or become somewhat sequential.

Dealing with the modern web - Harder. You need a JavaScript interpreter. I used Spidermonkey years ago, PhantomJS and the like more recently. Also means you need to support different packaging formats like JSON, MVC-style path parameters, etc and the associated escaping.

Keeping state - Harder. What does logged out look like? How do you detect this? (and no, don't make the user supply a Regex...) Has you audit screwed up your account so that "logged in" is all jacked up? Was there Stored XSS, but another audit clobbered the payload before your sweeper found the reflected value? (think updating your address on one page, displaying it on another, all while multiple things are auditing that update address form...)

All that said, I have found the main challenge people overlook when writing web pen test tools is the UI/UX. A shitty UI/UX ruins a tool because it ruins your flow. How do I specify what to fuzz with what rules? Can I resuse and reapply settings easily, how are results reported? Can I easily mark something as a false positive? But can I do that in a general way so that all issues matching on this host, or this file, or this URL frag are ignored? etc etc etc.

Please, if you are going to go down this path, think about how people are going to use the tool, and focus on maximizing their throughput.

passfree22• 12 years ago

Hi Billy. I wonder if you are the same Billy I met years ago. :) If yes, long time no hear.

These are all valid points. Writing security tools is technically very challenging even for the most basic things. Even a spider, regardless how basic this looks like in principle, is really hard once you get down to implementation. "In theory there is no difference between theory and reality but in reality there is."

This is why I am a big believer at simplifying the workflows in order to be more manageable. One of these workflows is recording requests and responses generated to/from the application. This is what Proxy.app does well. This feature can be extended further by other tools which we have already developed.

tptacek6• 12 years ago

I don't use bleeding-edge Burp, but whatever the version I run is, it doesn't have a Javascript interpreter, or for that matter a modern HTML engine. I'm also considering only the parts of Burp that are valuable for developers in general; things like stored attack vectors, like Nikto uses, are generally verboten on our team anyways.

So, if you asked me what the hard parts of getting Burp right when I took a stab at replacing it in Cocoa, and more recently in ko.js and Riak (more successfully):

* There is just a whole lot of UI involved in making a navigable proxy with the breakdowns you want, and, in particular, with editing and annotating raw requests and having those views do the "right thing" as the user edits the request and flips back and forth between views.

* Storage of requests is hard, because it's not intuitive how they need to be stored. The proxy wants long-term storage with indexes; the repeater wants rapid writes; the fuzzer wants to generate a shitload of requests (multiplying one request by 20k isn't rare), and a request from one part of the system can bounce through all the other parts.

* The proxy wants realtime communication/control with the UI, because some of your requests need to be intercepted and modified before delivery; in Burp, that UI shares most of the controls of the Repeater, which is surprisingly complicated.

Network performance is the easiest part of this to get right. Just don't use Java threads, and use a real timer library. But getting the network performance to cooperate with persistent storage is harder, unless you just let storage and memory explode.

We wanted fine-grained control over all of the tools so that we could automate retesting and regression. Like I said, we had a lot of luck hoisting the components of Burp into a series of web applications and backend systems backed by Riak (and Postgres, for search, index, and UI). We replaced Intruder with a command-line tool written in Golang that has a small specification language for the rules, instead of Burp's clunky interface, but there's still a whole lot of UI involved in parsing a request, automatically finding the places to template-ize, and then allowing the user to alter those selections.

When I took my stab at doing this in Cocoa a few years ago, I made a beeline for NSWebView; having a real webkit render is an enormous win. We do something similar to you with headless webkit now. This is a place Burp is genuinely constrained, due to Java.

Unfortunately, replacing Burp is a bitch.

hartator4• 12 years ago

That's a great news. We have been using fiddler inside a windows virtual machine just to do that.

I know everyone is asking the same kins of question but how does it compare to fiddler?

Our use case is kind of simple. We are building APIs in top of websites that don't have APIs. So a lot of "spying" is required. We have to use fiddler because it was the only one to correclty handle flash forms, file upload forms and to be able to globaly search for a particular token in every requests has been so much of a life saver.

passfree22• 12 years ago

Hi hartator,

Thanks for the support. Both fiddler and Proxy.app are proxies. Forms, JSON and XML are handled without a problem. At the moment there is no support for AMF if this is what you are referring to. We did not wanted to add this initially because it is not a standard and it requires creating a parser and serialiser from scratch - it is on our todo list though.

e28eta• 12 years ago

I don't know how feature complete it is, but I don't think you need to build one from scratch: https://github.com/nairteashop/CocoaAMF

hartator4• 12 years ago

Ha! Another question, do you plan to be able to filter by applications?

When you have a lot of tabs open, there is always pollution that get added to what actually matters.

passfree22• 12 years ago

Hi hartator,

That is a feature we are thinking about from day one but unfortunately to do this we either have to run multiple proxy servers and allow you to configure them individually or do process monitoring to understand who is connecting to the server. Both are on our todo list but we think that the second one have no chance to get approved by Apple due to the sandboxing limitations. We are currently researching this.

hartator4• 12 years ago

Okay, thanks for your response! Sandboxing seems to give ypu guys a hard time. I think the noise issue can be very problematic if you are using it on a real machine with plenty of web requests and not a virtual one.

Do you know if there is a way to setup the proxy manually? This way we can setup an instance of firefox doing the requests that matters but also monitor our ruby scripts that replicating these requests. It will make comparison so easy for our use case.

hartator4• 12 years ago

Okay passfree! thanks for your response! Sandboxing seems to give ypu guys a hard time. I think the noise issue can be very problematic if you are using it on a real machine with plenty of web requests and not a virtual one.

mihaela2• 12 years ago

When it comes to me, nothing beats Wireshark, but this app looks neat.

BradRuderman• 12 years ago

Has anyone done a comparison to find the best mac equivalent for Fiddler?

onethumb3• 12 years ago

Bought it, installed it, launched it. Then nothing happened. The iconography isn't clear (I have no idea what the first mode, the broken triangle icon, does) and has no tooltips on hover. It never prompted me for an Admin password, which probably explains why it's not actually capturing anything, since it probably can't automatically change the Mac OS X proxy settings on my network connections, unlike Charles.

onethumb3• 12 years ago

Ok, sure enough, if I manually go into my Network settings and add the HTTP proxy, stuff starts to collect. Nice. Wish it were automatic. Wonder how anyone will find this to do it when there's no docs?

But still can't get it to proxy HTTPS. The MitM certificate provided seems to be invalid: https://dl.dropboxusercontent.com/s/0z8khpq5p4guf9f/2014-05-... && https://dl.dropboxusercontent.com/s/6efwllajcnbpjvs/2014-05-...

I really want this tool to work, since I agree there's a big need, but frustrating so far...

mw6621• 12 years ago

I just bought it as well and was also trying to get HTTPS proxying to work. I am not a security guy, so I could be incorrect, but I think there are two things going on here:

1) the certificate generated by Proxy is 512 bits which OS X doesn't support; and

2) i'm not sure that Proxy will work with sites that use HSTS -- you might be able to clear the HSTS database in Chrome, but still for sites like Google, I think the certificate information is hard-coded.

passfree22• 12 years ago

Hi everyone. I just wanted to let you know that we started pushing the documentation here: http://learn.websecurify.com/help/proxy/ - soon to be bundled with the app itself.

Proxy.app Lands In App Store

Related Stories