Most open-source software is libraries or frameworks

https://medium.com/@aserg.ufmg/most-popular-application-domains-on-github-4af418b77a5e

by bhjs2 • 9 years ago

123 57 9 years ago

asragab• 9 years ago

I am not necessarily surprised by the results.

It is worth noting that the second most popular "software tool" tucked in between oh-my-zsh and homebrew (both command-line tools/packages) is Tensorflow.

That has to say something about the current state of the industry, though admittedly, I am a little confused as to why it was classified as a "software tool" and not say, "a non-web library or framework."

droidist2• 9 years ago

I think a lot of people star Tensorflow and then never look at it again though.

matt4077• 9 years ago

In the last week, Tensorflow hat 59 new issues files. That seems to be #1 among those on these lists by a large margin. Second place is electron with 29 (only a bit more than half as much). After that it's react and Atom with 10 each.

That seems to indicate that tensorflow is getting plenty of use.

CiPHPerCoder• 9 years ago

Interestingly, I looked through their entire CSV and was surprised to find one of my projects in their 5000 most popular set. Unfortunately, it was a reading list (awesome-appsec), not an actual software project. But still, kind of neat.

acemarke• 9 years ago

Huh. After seeing you mention this, I took a look at the spreadsheet and was actually surprised to _not_ see my "React/Redux Links" repo ( https://github.com/markerikson/react-redux-links ) in the list. The dataset info page says they captured it as of sometime in January 2017. My repo should have had somewhere around 6000-ish stars at that point, but it's not included. Wonder why not.

In the meantime, I'm very pleased to note that my list will hit 10K stars within the next few days :) Based on that spreadsheet, I guess that would put it somewhere in the top 350 or 400 repos on GH.

Meaningless "Imaginary Internet Points", of course, but amusing to look at nonetheless :)

ivanbakel• 9 years ago

Very surprised by the ubiquity of JS in non-web libraries/frameworks. I know it's a well-played fiddle, but it's saying something damning about our ability to put together quick and easy applications with other languages - or maybe just the number of people who start with webdev - when JS becomes the first choice. Is it just a consequence of how much UI work has been put into HTML rendering engines?

mattgreenrocks• 9 years ago

Also consider how beneficial it is from an employer's perspective to just force all UI to be HTML/JS, impedance mismatches be damned.

Edit: another consideration is if HTML/JS is the standard UI, then it biases that generation of developers to favor that particular abstraction. Innovation is framed as, "look where we managed to cram HTML/JS!" Actual innovation, which would be more along the lines of, "here's a new paradigm/abstraction that replaces HTML/JS" is seen as eccentric and largely ignored. In effect, developers' over-attachment to the way things are done greatly slows forward progress. They're only able to recognize small, incremental improvements.

voltagex_• 9 years ago

Big bang improvements are how you get systemd. Isn't incremental improvement the core of Agile and hence all-the-rage?

TAForObvReasons• 9 years ago

Many of the top items in the subset of JS Language and Non-web libraries/frameworks (moment, async, request, underscore/lodash, bluebird/q) might be included in the standard libraries of other languages. Also, a bunch of reasonably popular tools like rollup and systemjs may be miscategorized

eeZah7Ux• 9 years ago

JS is vastly overrepresented due to its popularity amongst some "hip" crowds and some small companies.

johannes1234321• 9 years ago

What else should be top? - Applications and tools use libraries or are split up into library components ... if there were many more applications than libraries there would be a lot of NIH ...

jepler• 9 years ago

Yes, but at the same time you'd expect each library to be used by many applications. Say you have 100 libraries, you can make 10,000 applications just by picking 1 or 2 of those 100 libraries, applications should far outnumber libraries.

Maybe applications do far outnumber libraries, but it's a fact of how people use github that everyone whose application depends on a library stars the library; as every application uses many libraries, there are many library-stars. This is a total guess, but it aligns with how I think of using stars. (though stars are separate from notifications as github documentation takes care to point out)

dankohn1• 9 years ago

A decade ago many of us thought that open source applications like LibreOffice and Thunderbird would dominate. Instead, open source seems to be winning everywhere in infrastructure and libraries, as companies don't want to lock into proprietary middleware anymore like Weblogic or VMware. But many (most?) applications are now web-based and closed source (e.g., Google Docs, Salesforce, Gmail, etc.).

One critique: GitHub stars are not the best indication of interest and can be gamed. Instead, I prefer using commits, issues and unique authors as a better metric of project velocity: https://www.cncf.io/blog/2017/06/05/30-highest-velocity-open...

skewart• 9 years ago

Is it really accurate to call socket.io a "non-web" library? Even lodash is questionable in that category - while not strictly tied to the web, it is primarily used in web apps. I'd be curious to see lower level stuff and/or stuff further removed from the web in the non-web category.

garysieling• 9 years ago

I write a lot of ETL scripts in Node for https://www.findlectures.com use lodash in pretty much everything, for me it basically serves the role of collections libraries in other languages

aargh_aargh• 9 years ago

Neat site. Linking the related HN discussion here:

https://news.ycombinator.com/item?id=14484549

SwellJoe7• 9 years ago

This is a weird premise. It's github. Of course it is predominantly tools for coders. End user applications have wholly separate paths to the user. An application could have a handful of contributors/stars/downloads etc. on github and still have millions of downloads somewhere else (a project I work on fits that description). And, it might not be on github at all and still be used by millions daily.

I guarantee openssh, Firefox, LibreOffice, and probably a hundred other applications, are (orders of magnitude) more popular than the top applications on this list.

So, if this were titled, "Most open source software on github..." I wouldn't object. But, I have to completely reject the premise here, because I know that there's an entire iceberg of OSS software, including applications, that is completely excluded from the listing by virtue of either not being on github or being on github, but not using github as its primary method of distribution and promotion, and this data completely ignores everything below the surface.

Also, it's probably dangerous to begin to think of "Open Source Software" as only being "Software that has a public github repo".

Randgalt3• 9 years ago

You're right of course. I'm the main author of Apache Curator. While it does have a mirror on Github it's mostly consumed via its Maven Central repo. The Maven Central repo is most likely a better source of open source project usage.

SwellJoe7• 9 years ago

Here's how amazingly wide and diverse our OSS world is: I have never even heard of Maven Central.

jsgo2• 9 years ago

Honestly though, I don't think that's far off in that even those you mentioned are leveraged as a library (Firefox maybe not, not sure. Chrome can be used headless so guessing there's a possibility for Firefox. Some of the open source internals are used in other projects though). But OpenSSH for sure is used within other software. Okay, maybe libreoffice isn't a case (at least I don't think it is).

At this point, I just love seeing GUI software I leverage also coming with a CLI. I learn what it can do for me with the GUI and then can use the CLI to manage the heavy lifting for me.

SwellJoe7• 9 years ago

We could name hundreds, probably thousands, of OSS applications that are more popular than the laughably obscure #4 and #5 in the list in the article (not laughable in the sense that I mean to devalue their work, just that they are so clearly for an extremely niche user, and that user is a programmer).

I don't really care to argue semantics about what's a library vs. what's an app. Because many libraries have a UI and many apps have an API; that's another discussion, really (though, you've got tens of millions of people using Firefox through the UI daily and maybe a few thousand using it as a library daily...I know where I fall on the "is it an app or a library" continuum).

But, seriously, this article looked at one pile of data generated by a site that is entirely by and for programmers and declared that the entirety of OSS is represented by that data. The more I think about it, the more ridiculous, and maybe even offensive, it seems.

KamBha• 9 years ago

Regarding FF headless mode:- https://bugzilla.mozilla.org/show_bug.cgi?id=1338004

Basically, WIP. For now, if you have NodeJS:-

https://slimerjs.org/

It is kinda headless.

jcelerier4• 9 years ago

At this point, I just love seeing GUI software I leverage also coming with a CLI.

I really wonder what is the point of a CLI for, say, Krita or LibreOffice.

skgoa• 9 years ago

Yeah, this reminds me of the annual claims regarding what programming languages are the most popular based solely on github repos or stackoverflow questions. The methodology is completely inadequate for producing an answer that can be trusted.

dragonwriter• 9 years ago

It's github. Of course it is predominantly tools for coders. End user applications have wholly separate paths to the user.

To the user sure, but if they are open source, bazaar-model projects, they still need a route to contributors, and GitHub is as suited for that for end user apps as it is for libraries. So while, yes, it's possible that GitHub is unrepresentative for some other reasons, I don't think that “GitHub is for developers” is a reason to dismiss this kind of statistic.

OTOH, it may well be that apps, even open source, are more likely to be cathedral model (instead of bazaar) than libraries, which might reduce the need for using GitHub or some other public repository.

stcredzero• 9 years ago

Of course it is predominantly tools for coders. End user applications have wholly separate paths to the user

Libraries and tools for coders -- what's particular about these? These are the areas where programmers are also the domain experts! So is it any wonder that these are the projects that succeed?

I guarantee openssh, Firefox, LibreOffice, and probably a hundred other applications, are (orders of magnitude) more popular than the top applications on this list.

Let's look at the success of end-user open source applications. Is there a pattern of success being correlated with prevalence of the domain knowledge in the programmer population? There will be other influences, which will bring in corporate resources and skew the degree of success, but I think this pattern holds there as well.

SwellJoe7• 9 years ago

"So is it any wonder that these are the projects that succeed?"

...on github, a community of programmers.

cobbzilla• 9 years ago

Remove the words "open source" and the title makes more sense to me.

Most software is libraries and frameworks -- you just don't get to see most/all of the proprietary stuff, since it's not on github or anywhere else.

nickpsecurity• 9 years ago

Counterpoint: most proprietary apps are probably CRUD apps or spreadsheet stuff. There's still a lot of good, proprietary apps. Likewise for FOSS. Turns out the quick and easy solutions to common problems that please project owners happen more often than solutions to hard problems or things without immediate ROI. It says more about people's priorities than proprietary or FOSS software.

profpandit• 9 years ago

Open source is associated with philanthropy and with being really smart. If you're a really smart coder, out to change thw world, you'll obviously build a library or framework to help the other, not so fortunate coders around you as a way to bring relief from the tyranny of the big monopolistic corporates (read Microsoft). It's your David changing the evil ways of their Goliath.

slaymaker1907• 9 years ago

Why is lodash considered non-web?

rickycook• 9 years ago

i would assume because you use it in nodejs... which is a bit of a stretch i think

watwut3• 9 years ago

Why would it be considered Web? It is use pretty much anywhere where js is used.

Scarblac• 9 years ago

And that's overwhelmingly on the Web.

sigjuice• 9 years ago

The original title is misleading. It should say Most GitHub projects are libraries or frameworks. It is quite a leap to go from 60% GitHub projects to most open source software.

conceptme• 9 years ago

But every piece of software can use hundreds or thousands of libraries.

fermigier2• 9 years ago

1) Title is misleading, should have been "Most popular projects on GitHub are ...".

2) Judging from the top-5 list in the post, between 1/4 and 1/3 of the projects have been miscategorized.

Ex:

- https://github.com/chrislgarry/Apollo-11 -> should have been categorized as "Application", not "Documentation".

- https://github.com/tensorflow/tensorflow -> should have been "Library", not "Tool".

- Electron, Socket.io, Moment, lodash... are "Web libraries", not "Non-web libraries"

and probably more.

I hope the reviewers catch these errors before they publish this in a research journal.

fermigier2• 9 years ago

One could even argue that Jekyll is a web framework, not an application.

Cthulhu_• 9 years ago

Moment / Lodash are JS utility libraries for both browser and server-side runtimes.

majewsky2• 9 years ago

But even then, it just goes to show that a classification into "web" and "non-web" libs is fundamentally flawed. Maybe they want "JS" vs. "non-JS".

hanbura2• 9 years ago

Both server and browser are "web" in the typical use case, in my reading of the term.

jcelerier4• 9 years ago

How many of the software in Debian or any other big distro repo is actually being developed on github though ?

bigbugbag• 9 years ago

Most open source software is libraries or frameworks

Out of the 5000 most popular repositories of github, most are libraries or frameworks.

The author is not bad at clickbait.

edit I wonder if they actually checked if those 5000 licences are actually compatible with the open source definition[1].

[1] https://opensource.org/osd

Most open-source software is libraries or frameworks

Related Stories