OS X Auditor

by evandrix • 12 years ago

348 43 12 years ago

rrggrr• 12 years ago

Great idea. Love to run it, but... crash and burn:

~/Library/Safari/LastSession.plist Traceback (most recent call last): File "osxauditor.py", line 1702, in <module> Main() File "osxauditor.py", line 1663, in Main ParseBrowsers() File "osxauditor.py", line 808, in ParseBrowsers ParseSafari() File "osxauditor.py", line 745, in ParseSafari ParseSafariProfile(User, UserSafariProfilePath) File "osxauditor.py", line 717, in ParseSafariProfile LastSession = LastSessionPlist["SessionWindows"][0]["TabStates"][0] File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC/objc/_convenience.py", line 451, in __getitem__objectAtIndex_ return container_unwrap(self.objectAtIndex_(idx), RuntimeError) IndexError: NSRangeException - -[__NSCFArray objectAtIndex:]: index (0) beyond bounds (0)

bsmartt• 12 years ago

I can't even get pyobjc running on 10.9..

rmetzler• 12 years ago

I also have this problem. I'm not sure, but it might be caused by using brew python instead of the systems original python

yohanatan2• 12 years ago

Looks like you have an empty session list. Have you ever used Safari? Quick workaround might be to pop open Safari once (and browse a little) and then try again.

mwfunk• 12 years ago

Why on earth would anyone downvote this comment? The lack of a Safari user profile is the most obvious potential issue here, based on the backtrace.

mdaniel• 12 years ago

And that, folks, is why we don't do chained expressions. Variables increase readability and also offer one the opportunity to check a condition that should be true.

This, KeyError, NoneType exceptions and its Java friend NullPointerException drive me batty. They are so simple to defend against, if the author just took the time.

nsxwolf• 12 years ago

I love Optionals for this.

chc2• 12 years ago

Chaining itself isn't really the problem here IMO. Creating a variable for each level of the nested data structure and manually checking for the validity of each subsequent one would really, really not produce more readable code. It would turn a "get this value out of a data structure" expression into about 15 lines of code that each need to be individually read and understood. The problem is that Cocoa doesn't provide a great way to say "Access this stuff in this data structure unless it isn't there."

An option type is really the best way to kill this breed of problem, and thankfully Apple is including one in Swift, though it's still somewhat green.

arianvanp• 12 years ago

Maybe monads make this kind of chaining safe.

yohanatan2• 12 years ago

The accessor variant of the existential operator in CoffeeScript (which Swift apparently has taken influence from) handles this nicely.

mcescalante• 12 years ago

Very interested to try this out on my own machines to see the results. I know somebody who does computer security at a University and the staff there has been frustrated with the lack of available forensics tools for OSX, so this may nicely fill an empty niche for some industry people as well.

c0wl3• 12 years ago

OSXAuditor is pretty dope and we used it a bunch at Yelp. Over time, we created what we think is an inspired next version - https://github.com/Yelp/osxcollector

traveller_57681• 12 years ago

The passive scan approach has the single weakness that it appears to be effective, but mostly is security theatre.

Take for example, when we travelling into Australia, and they asked if we had any criminal convictions - I didn't know that a conviction was still an entry requirement for the island!

crag2• 12 years ago

I run OSXCollector and I get missing modules error. I'm not a python guy, but looking at the code, it requires:

import Foundation, import calendar, import os, import sys, import shutil

I assuming os, sys, and shutil are part of the system (are they?). What about the other two? Also, which version of Python do I run this under?

ddp• 12 years ago

...not a python user either. The README mentions 'pip' but I don't seem to have one of those either.

jschneier00• 12 years ago

calendar is a builtin as well.

It appears the OSX specific modules are located (on latest Mavericks) in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python

So you need to add that directory (for xattr) as well as that directory + PyObjC (for Foundation) to your PYTHONPATH

c0wl3• 12 years ago

On a reasonably modern Mac, all the dependencies should be setup by default - including modules and the correct version of Python.

fastball• 12 years ago

Foundation is part of the pyobjc package. You'll need to install that.

atmosx2• 12 years ago

If you're on a mac using brew or macports usually the paths are usually different. Macports/Brew python doesn't usually have 'pyobjc'. Try running osxcollector as:

sudo /usr/bin/python osxcollector.py

chrissnell• 12 years ago

What's up with the blurb on your GH page that says, "Exactly how Yelp uses the output from OSXCollector is a bit of our secret sauce"? How is workstation auditing a part of Yelp's secret sauce? You're a review site. Strong security on Internet-connected machines benefits everybody. There's no competitive advantage here. If you have something awesome, why not share it?

c0wl3• 12 years ago

Yeah, that could have been stated better. I guess I meant to say we're not handing out on GH the indicators of compromise for stuff that's currently giving us problems. But we do, in the README and when we talk to folks about OSXCollector, try to share how we do analysis. I'm hoping to release some related tools over the next few weeks that do some amount of parsing and analysis of the output.

There's some neat stuff we've played with (though nothing short of manual analysis is giving really high confidence at this point). Some of the ideas we played with but haven't written about yet: * Feed the output through a parser that finds domains, URLs, and IPs. Feed those into threat feeds and passive DNS APIs. Occasionally this surfaces interesting stuff. * Feed all the hashes to VirusTotal, cymru, or known lists of nasty stuff. Hits are generally nasty. * We've got some known indicators of commodity malware persistence in launch agents. We grep for those cause we know they've hit us before.

One of the recent things we did add to OSXCollector was pulling xattr's from downloads. This allows us to find the source URL - sometimes even the redirect chain - for a download by reading extended attributes of the file. This has been helpful.

atmosx2• 12 years ago

How do you read in human-friendly format the 'json' file created? Do you have any extra tool to create HTML, etc. or do I need to roll up my own json parser?

korzun2• 12 years ago

Cool idea.

But using raw md5 hashes to verify against a blacklist is kind of useless. Especially now.

You should be using smarter file signatures:

http://hooked-on-mnemonics.blogspot.com/2011/01/intro-to-cre...

jnazario• 12 years ago

so, if i understand it correctly you're saying create AV signatures (the link you point to is how to create ClamAV signatures). to do so in a worthwhile, comprehensive way would required the maintainer to basically become an AV company. possible, but requires a lot of effort to scale. by calling out to various services, the creator of this tool is essentially outsourcing maintaining a file reputation blacklist.

MD5s are, despite their limitations, the lingua franca of the security industry. nearly everyone who provides a file reputation query service supports them (as opposed to SHA1s or other hashes like ssdeep).

so, i think i get what you're saying, but i don't think it's a relevant suggestion here.

korzun2• 12 years ago

I'm pointing out that this tool is essentially useless from security perspective. Most of the payloads will have different hash results due to extensive use of packers.

If you are going to get hit with variant #11929 before the online databases obtains a hash of it, this tool is not going to pick it up but it will tell you that you are secure.

hueving• 12 years ago

so, i think i get what you're saying, but i don't think it's a relevant suggestion here.

It's pretty relevant. Without sending the whole file to the third party, the file reputation service isn't 'outsourced' as you suggest. Sending the MD5 will not do any good if the program makes non-deterministic modifications to its binary.

SmileyKeith• 12 years ago

This looks nice but there are a ton of issues around pyobjc. I've never been able to successfully install it on 10.9.

pudquick2• 12 years ago

This is because you don't need to. Especially for this particular script.

If you're using the natively included python, pyobjc comes free.

As soon as I saw that mentioned in the documentation, I knew this project was going to have issues.

johndoze• 12 years ago

I'm familiar with python bridges at all - how would someone transition something with pyobjc as a dependency to the native alternative you mention?

spiralganglion• 12 years ago

This looks like an interesting project. But, there's one bit of information, which is missing from the description and this discussion, that would help a lot with my understanding.

Is this a tool for a user who wants to learn more about their own machine, or a non-user who wants to know how a given machine has been used?

More information about the use cases of such a tool would be most helpful.

thinkling• 12 years ago

Ditto, in particular would be good to know whether this is only for post--intrusion forensics or also useful ahead of time.

entelechy02• 12 years ago

I'm getting an error after running it:

[INFO] Users' LoginItems [INFO] 's LoginItems [INFO] /Users//Library/Preferences/com.apple.loginitems.plist [INFO] Cannot parse /Users//Library/Preferences/com.apple.loginitems.plist (Binary or JSON plist may FAIL)

Traceback (most recent call last): File "osxauditor.py", line 1702, in <module> Main() File "osxauditor.py", line 1651, in Main ParseStartup() File "osxauditor.py", line 550, in ParseStartup if "SessionItems" in LoginItemsPlist: TypeError: argument of type 'bool' is not iterable

arm• 12 years ago

I’m guessing that’s probably happening because of the double slash after /Users.

entelechy02• 12 years ago

nono, that was originally asterisks but apparently hacker news replaces those with an empty string. that is weird, but unrelated to my problem

w0rd-driven2• 12 years ago

I'd say this isn't quite ready for prime time. The errors I received were annoying and you're not going to get an end user or most IT admins to parse the python just to find the errors.

I've only had 3 major errors but they were significant: 1) I'm on Yosemite so GetAuditedSystemVersion() looks for a PatchVersion variable that simply isn't there. The header reads 10.10 not 10.10.patch as expected. 2) The Safari parsing snafu listed in my previous comment. Opening Safari isn't enough, you have to use the browser quite a bit. The same could likely be said for all browser tests and it would be a good idea to outline precisely what this needs to be. Hint: A new system or install of Yosemite for instance will produce the errors I saw. 3) There's a parsing bug in ParseMailAppAccount() and I just commented out the call completely.

Any number of these could just be Yosemite related but I don't think so. All of the bugs I ran into are variations on index out of bounds due to some hardcoded assumption that mostly works, except in this instance apparently. I'm not the only one with these nagging bugs based on the issues list but mine do seem very specific to Yosemite or how I do(n't) use my system.

makeset• 12 years ago

Application path scanning follows symlinks recursively. One upwards link, and it never terminates.

OS X Auditor

Related Stories