GitHub Codescanning

https://github.com/features/security/advanced-security/signup

by jedisct1 • 6 years ago

390 73 6 years ago

Signez• 6 years ago

It seems that it will be free for open-source, public repositories, and quite pricy for private repositories.

"A member of our sales team will reach out to discuss details" is a great euphemism for "be ready to pay quite a few thousand bucks per year for this feature".

theincredulousk• 6 years ago

It is typical enterprise sales. If the prices aren't listed, it usually means you're charged based on what they estimate you can afford for the value-add. It may be $12k/yr for a big company and $1.2k/yr for a small one.

alexott• 6 years ago

or vice versa - I remember that O'Reilly Safari was almost 10 times cheaper per person when I worked in 100k+ people company, compared to only couple of thousands people...

marceloabsousa5• 6 years ago

No way it'll be $12k/year for a big company - I would say $50k minimum.

sbarre• 6 years ago

If you're a reasonably large org with lots of developers and tons of repos, this will be a cost well worth paying.

user59944614• 6 years ago

It's usually the opposite. The vendor tries to charge a crazy amount per user per month or charge for every employee in the company, which makes the solution acutely cost prohibitive.

Nobody wants to spend hundreds of thousands of dollars a year -if not millions- on something that's barely used. Better spend on anything else that is tangible.

marceloabsousa5• 6 years ago

I don't think this will be free for all public repositories. Having designed and implemented these kind of static analysers, it's quite costly to scale them - you do want to avoid useless CPU time on the millions of public repositories.

lonelappde4• 6 years ago

That's a problem that's simple to solve by putting a quota on # analyses per project per month, perhaps weighted by how popular the project is. Like everything else at GitHub, private project users pay extra to cover the public project users. It's proportionall regardless of a feature's cost.

Vosporos• 6 years ago

They said during the Keynote that they were willing to spend the millions of dollars necessary to run this on public repos that would activate the option because it's the right thing to do.

derefr• 6 years ago

Or it's in alpha and they haven't found out yet what the feature is worth to corporate users; so they want to do that before anchoring them to a price.

Also, it might vary per account in an unpredictable way, since it's a heavyweight operation that'd cost more compute-hours when run against larger repos, but also involves static-analysis tasks that don't necessarily scale linearly with LOC. So they might just not yet have a predictive model yet for the baseline cost to them, but instead are doing a trial run for each interested user, observing the cost of the workload on their repo, and then extrapolating that out as the cost to run that workload persistently.

Either way, GitHub has been strongly on the side of "clear pricing" so far, so I doubt they would plan to leave things this way. But it's hard to get enough data for a feature like this, when you know each run you do "just to model the curve" is costing you real money.

lonelappde4• 6 years ago

That means the price is negotiable. Rich users pay more, poor users pay less, both users get value, producer makes a profit; everyone wins.

adtac2• 6 years ago

What's the best way to put this politely from the seller's perspective lol. Do sales teams do background homework on clients' revenue and then come up with different numbers for the exact same offering?

canada_dry• 6 years ago

If you're an inside salesperson, cold calling ought to be pretty straight forward:

Customer: Sorry, I just don't think it's worth the price...

Sales: Oh really? Because we've identified 42 critical vulnerabilities in your code base

Customer: Do you have an installment plan?

duxup• 6 years ago

The usual theory on HN is that having to call means they're gong to milk you for money.

But I work with a SaaS product and the up front pricing is sort of irrelevant if only because the customers want a highly customized product and frankly you really need to know their business / work with them to find what they want to figure out what the price for implementation and customization is.

In our case it isn't a scam or a ploy for more money, it's just the nature of the beast / industry / and every single time it is what customers end up asking for (lots of customization and etc).

Now maybe that is the case for Github Code scanning, but also maybe it is very much a product that depends on how exactly you want it to work for you and that changes the pricing ...

kerng• 6 years ago

Well, Semmle was extremly expensive by itself, so I'd doubt it will be free

frizkie• 6 years ago

I wonder how much the difficulty of writing queries varies between languages. I was disappointed to not see Ruby on the beta sign-up list, and GitHub being a pretty heavy user, I'm sure they have their reasons for excluding it.

daeken2• 6 years ago

This is based on Semmle, which they acquired last year. According to their docs (https://help.semmle.com/lgtm-enterprise/admin/help/sys-requi...) it supports C, C++, C#, Go, Java, JS, TypeScript, and Python; no Ruby. (Edit to add: Woops, missed that this list is also literally on the page. I had even looked first.)

It's really hard to do any kind of static analysis on something like Ruby or Perl where 1) you need a ton of context just to parse it properly, and 2) tracing calls is a nightmare. Given that, I'm completely unsurprised they haven't supported it yet.

user59944614• 6 years ago

How hard can ruby be compared to python?

Python is very dynamic. I've used and worked on linting tools for python, tried commercial static analyzers, and they do a pretty good job in my opinion in spite of the language being dynamic. Not perfect but miles above anything I would have expected.

antpls• 6 years ago

Interestingly there is no support for Rust.

Is it because Rust is bullet proof, or too young to be considered for now?

greysteil6• 6 years ago

We (GitHub) absolutely plan to expand the list of languages CodeQL supports, and Ruby is a language we'd love to add (we're heavy users of it internally). In the meantime, because code scanning is extensible you can plug in third party analysis engines to scan the languages that CodeQL doesn't support.

the_duke3• 6 years ago

One thing that really annoys me about Github/Gitlab et al is that they don't provide a nice UI to show CI results in a structured way.

There are semi-standard XML formats that can be used to provide file position, severity and message, which could easily be produced from CI actions and would give devs and reviewers a great view of failing tests, compiler and linter warnings, ... With links to the files etc.

Instead we are still stuck with scanning text logs to figure out what failed.

I always assumed this is not implemented to eventually upsell something automated.

This is still a great feature though, which will probably prevent a large amount of bugs/vulnerabilities, assuming they can minimize false positives.

To give credit where it is due, I'd also note that most of Githubs new features since the acquisition were already present in Gitlab [1]. Github will be able to commit way more resources to make it polished, though.

[1] https://about.gitlab.com/stages-devops-lifecycle/secure/

Edit: apparently both Gitlab and Github have at least a limited version of this now, although Gitlabs implementation seems much nicer. See below.

Rockslide• 6 years ago

But GitHub actions do exactly support that in the form of code annotations, see e.g. here https://github.com/ember-template-lint/ember-template-lint/p... - no need to scan logs if you do that, you get the comments right on the code.

the_duke3• 6 years ago

Do you have a docs link for this functionality? I can't find anything.

The way of reporting seems somewhat odd, and the UI seems a bit limited, but it's a start!

Edit: https://help.github.com/en/actions/reference/workflow-comman...

kuschku• 6 years ago

Of course gitlab supports this: https://git.kuschku.de/justJanne/QuasselDroid-ng/pipelines/5...

It can automatically parse most XML formats, JUnit for example.

sytse• 6 years ago

Thanks! Docs for the GitLab feature to show CI results in a structured way are on https://docs.gitlab.com/ee/ci/junit_test_reports.html

GitHub Codescanning functionality is best compared to what GitLab has in GitLab SAST https://docs.gitlab.com/ee/user/application_security/sast/ and Secret Detection https://docs.gitlab.com/ee/user/application_security/sast/#s...

the_duke3• 6 years ago

I stand corrected, and gladly so!

This looks perfect.

shoyer• 6 years ago

Surprisingly Azure pipelines, which I understand is used underneath GitHub actions, does support report CI results in this nice way on GitHub, e.g., see these results from a recent PR on one of my projects: https://github.com/pydata/xarray/pull/4017/checks?check_run_...

For now, at least, it seems like this is one reason not to switch to GitHub actions yet.

616c• 6 years ago

Heard this suggested a few times but what is the source?

pabs33• 6 years ago

There is a standard for this: Static Analysis Results Interchange Format (SARIF):

https://docs.oasis-open.org/sarif/sarif/v2.0/csprd01/sarif-v...

tandr• 6 years ago

Do you know who supports it?

marceloabsousa5• 6 years ago

It will be interesting to see the false positive rate...

greysteil6• 6 years ago

At GitHub we're pretty proud of the scan results from CodeQL. Currently, 70% of alerts flagged in PRs are fixed (rather than marked as a false positive or won't fix). We think we can get that number up to 85%+ as we gather more data and iterative the queries (which are all open source).

marceloabsousa5• 6 years ago

Hmm, can you please share more details about this data: what kind of vulnerabilities you're finding, what does fix mean, what is the sensitivity of the analyser (flow, procedure), what are the underlying abstractions regarding memory, concurrency, etc? From the demos so far it's hard to see past a standard taint analyser. 70% precision on a static analyser is very high for a general purpose analyser unless you have a lot of missing vulnerabilities. The static analysis/formal verification community would be definitely interested in getting more details about your experiments.

daeken2• 6 years ago

Nice, it seems they finally integrated Semmle, which they acquired last year! https://github.blog/2019-09-18-github-welcomes-semmle/

This is the only static analysis tool I've really been interested in over the past few years; it's crazy effective from everything I've seen, and the queries are easy to write. Can't wait to play around with this beta on my own code.

p0rkbelly• 6 years ago

Hopefully this helps with people committing up their AWS access keys...

greysteil6• 6 years ago

PM for GitHub Advanced Security here.

We handle that with secret scanning - code scanning focusses on static analysis of your code to find vulnerabilities in your code, rather than committed secrets.

We have a partnership with AWS (and many other token issuers) that handles this really nicely. If anything that looks like an AWS credential is committed to a public repo we send it over to AWS - if it's a real token they notify the token's owner (and in some cases automatically revoke the key).

There's full details at https://help.github.com/en/github/administering-a-repository....

lonelappde4• 6 years ago

Please clarify who "we" is in your comments.

techntoke4• 6 years ago

I too would like to know.

We have a partnership with AWS (and many other token issuers) that handles this really nicely. If anything that looks like an AWS credential is committed to a public repo we send it over to AWS - if it's a real token they notify the token's owner (and in some cases automatically revoke the key).

So if something looks like a token from AWS or another token issuer, you automatically send the token to providers to check to see if it is "legit"? Is this something that is opt-in, or done automatically?

glram• 6 years ago

This is really nice. Even with code review secrets still slip through the cracks. I imagine this will be quite pricey tho.

pipework• 6 years ago

I had just stumbled on https://gitpod.io/ which comes with an extension to add a button next to the clone/download button-down (portmanteau of button and dropdown).

I also use codeanywhere for my personal use and whenever applicable I like to use codesandbox.io when it's JS-ish.

micael_dias• 6 years ago

The link is behind a login, I'm on my phone logged out but I believe it refers to this[1]?

[1] https://github.com/features/security

Someone1234• 6 years ago

Thanks. I'm surprised a mod hasn't changed it by now, but I guess we're the only two people not logged into Github.

delcaran• 6 years ago

I always delete my cookies, so that makes three of us.

zzo38computer• 6 years ago

Me too, because I don't have a GitHub account (I use Fossil).

pabs33• 6 years ago

Anyone know if there are any open source projects for doing secret scanning? Or if this uses any open source code scanning projects?

pabs33• 6 years ago

I wonder if this uses the Static Analysis Results Interchange Format (SARIF) standard internally.

https://docs.oasis-open.org/sarif/sarif/v2.0/csprd01/sarif-v...

greysteil6• 6 years ago

PM for GitHub Advanced Security here.

We use SARIF as the input format so third party code analysis engines can easily integrate with code scanning. Their results can then be shown in the same way that scans using our own CodeQL analysis engine are displayed.

Docs on how we translate each SARIF property into the code scanning display are below:

https://help.github.com/en/github/finding-security-vulnerabi...

(The beta notice on that page is very relevant here - we wanted to build extensibility options into code scanning from its inception, but whilst it is in beta the API won't be 100% stable. We'll do our best to avoid any unnecessary churn.)

0x49d1• 6 years ago

Seems like a competitor to https://snyk.io/ . Let's see when it comes out.

Hawxy• 6 years ago

Snyk is focused on dependency scanning and license management. This is static code analysis, more akin to sonarsource.

978e4721a• 6 years ago

Another electron app, nothing to look at here.

GitHub Codescanning

Related Stories