Time for developers to address the chaos of Node.js and NPM

https://www.techrepublic.com/article/why-its-finally-time-for-developers-to-address-the-chaos-of-node-js-and-npm/

by intvocoder • 8 years ago

74 46 8 years ago

gringoDan• 8 years ago

An additional element adding to this chaos is the security vulnerability of npm packages. How often do we actually examine the package's code in detail before including it in a project?

Relevant article: https://hackernoon.com/im-harvesting-credit-card-numbers-and...

calibas3• 8 years ago

I don't know if it's still the case, but one of the first things I learned about JavaScript was how it was inherently secure. Then I actually started using the language, and learned what utter bullshit that is.

gramstrong• 8 years ago

inherently secure

Who told you that? I don't think "inherently secure" is a thing in the tech industry.

TheAceOfHearts2• 8 years ago

The ECMAScript spec doesn't include details about its environment. You can run it in varying environments, each of which has different security criteria. For example, you can run a fully isolated JS VM that has no interactions with the real world, which is certainly quite safe. Node has full access to pretty much all the user's system resources, while the browser is a bit more sandboxed and restricted in what it can access.

deusex_• 8 years ago

How is it different from maven deps?

kkapelon4• 8 years ago

Usually you don't use maven deps directly. You have a registry in your company that caches maven packages for you and runs vulnerability scans on the binary. So the binaries you end up using are "tested" internally.

Is that how people use NPM?

Also maven central does NOT support unpublishing of binaries. So if you have version 1.2 of something and you have scanned it once for vulnerabilities, it should always be the correct one. Nobody can publish again 1.2 in the central repo

arenaninja2• 8 years ago

It's troubling that PHP gets so much shit because it has an extensive if clumsy standard library, where node gets a free pass because you can get whatever doesn't exist from NPM, welcoming any number of security vulnerabilities into applications.

I recently cleaned my NPM cache at work after 9 months, it was 6GB. ES6 is a massive improvement, but Node's standard library is still lacking and the sheer amount of tooling needed to be effective and up to date is too much.

applecrazy• 8 years ago

I agree. Just beginning work on a react project installs 300MB or so of tooling, and when you throw Electron into the mix, expect a huge node_modules folder.

Analemma_• 8 years ago

If it makes you feel better, there are plenty of us out there who hate PHP and Node :)

slowmovintarget• 8 years ago

PHP isn't criticized because it has a big and clumsy library. The criticisms for PHP are very clear: https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/

It is badly designed.

davvolun• 8 years ago

I mean....you aren't arguing JS is a paradigm of perfect design, are you?

arenaninja2• 8 years ago

The fractal blog post will be 6 years old in a couple of weeks. I'm going through it again, and __a lot of it is about the big clumsy library__. Some choice excerpts:

There’s no clear design philosophy. Early PHP was inspired by Perl; the huge stdlib with “out” params is from C; the OO parts are designed like C++ and Java

There's even complaints about PHP 5.2, but PHP 5.4 had just been released a month earlier:

If PHP was compiled with --disable-url-fopen-wrapper, it won’t work. [...] Note that this flag was removed in PHP 5.2.5.
foo()[0] is a syntax error. (Fixed in PHP 5.4.)
PHP errors and PHP exceptions are completely different beasts. They don’t seem to interact at all. (Fixed in PHP 7)
There is no finally construct (Fixed in PHP 5.5)
Methods cannot be named e.g. “list”, because list() is special syntax (Fixed in PHP 7)

I'm going to stop there. Not saying any of it wasn't valid criticism or that some of it doesn't apply to this day. Just that too many people, yourself included, are happy to throw a 6 year old blog post at a language that has continued to evolve since. It's also strange that you say it isn't criticized for a big and clumsy library, because in the link you provided about half of the page is complaints about the library.

wvenable2• 8 years ago

Please stop quoting this article. One third of it was wrong at the time it was written. One third is just opinion. The last third is somewhat valid but a lot of it has been fixed since then.

The author never actually used PHP and just collected a bunch of "facts" from the Internet without even testing some of these things himself.

krapp4• 8 years ago

It might be a bit controversial here, but language faults aside, I think PHP's model of package management through Composer is better in many ways than NPM, at least in terms of namespacing, and not prefering to define every function as a separate package within its own universe.

nisten• 8 years ago

The problem relies in the obsession with monolithic design even if it ends up being a complete house of cards. This leads many new developers to sacrifice basic logic and first principles when judging what good software is; often resulting in products that no one uses and that just crash anyway. But hey, you get this great toy-like model to show on your next interview. Security and access rights are sadly not taught as being essential to having a proper monolith because they'd greatly slow down development time. Whereas an engineer or builder has no choice but to confine their creativity within those principles because a product crash for them has much more dire consequences.

st3fan• 8 years ago

Is this article truncated? The conclusion or end part seems to be missing. Or is this just oddly written?

yilugurlu• 8 years ago

It's 2018 and still leftpad issue, really?

Hamuko• 8 years ago

The article says that you can still unpublish packages from NPM at will. If that's still the case, then yes, leftpad should still be brought up in 2018 before someone unpublishes the "is-string-empty" package or whatever the dumb packages that people put on NPM are.

kabes2• 8 years ago

you can only within 24 hours of publishing. After that you need to send them an email to request it and they determine if it can be unpublished or not.

TheAceOfHearts2• 8 years ago

Clickbait. At least propose what it is that you think needs to be done to address this so-called chaos.

Worth noting that ES2017 added String.prototype.padStart and String.prototype.padEnd. The whole left-pad thing is blown out of proportion.

Second, you shouldn't be relying on npm for deployments. You dig your own grave. If package deletion is a concern, then host a mirror with something like Nexus [0], or pay for a service [1] [2] to do it for you. Over the years there have been a few local npm mirror tools [3], but I don't think any of em have really caught much traction. There's also the option of bundling each release. For that you can use a tool such as Webpack [4], or take it one step further and generate a single binary using something like pkg [5].

Developers also have the option of forking their dependency graph and updating references in package.json. Sooner or later you'll start to find bugs in your dependencies and have to fix them. It's your responsibility, you own all that code. Don't be afraid of maintaining your own fork. If you find a critical issue you'll probably need to be able to deploy your update without depending on a third-party anyway. This is fairly common in other ecosystems. Heck, you can also just check in your node_modules folder into version control.

Don't get me wrong, npm certainly has its fair share of problems, and historically they haven't had the best defaults... But there's usually ways to work around the issues, and they're gradually working towards improving things. It's the developer's responsibility to handle their ecosystem's problems, which is also nothing new in the world of software development.

[0] https://www.sonatype.com/download-oss-sonatype

[1] https://jfrog.com/artifactory/

[2] https://inedo.com/proget

[3] https://github.com/rlidwka/sinopia

[4] https://webpack.js.org/

[5] https://github.com/zeit/pkg

acemarke• 8 years ago

I personally recommend caching package tarballs locally inside your project [0], using either Yarn's "offline-mirror" feature, or if you're using NPM, a tool called Shrinkpack [1]. Might not be the best fit for every project, it but it guarantees reproduceable installs without having to go out to NPM's servers.

[0] http://blog.isquaredsoftware.com/2017/07/practical-redux-par...

[1] https://github.com/JamieMason/shrinkpack

FLUX-YOU• 8 years ago

It's the developer's responsibility to handle their ecosystem's problems, which is also nothing new in the world of software development.

This is where you have to start questioning the benefits of using node/npm/js on the backend. We're all stuck with it on the front end for the foreseeable future.

I've never had any of the problems in node/npm using NuGet/C# for example.

At some point, I raised my hands and emailed my boss about a new project using node, essentially saying "this is dumb. left-pad is dumb. is-number is dumb. is-odd is dumb. the amount of packages depending on these is dumb. And I won't be arsed to fix it because I value my time, so I'd rather just jump back to an ecosystem that has its shit together because the result is the same simple API pushing json either way we go."

node/npm sucks IMO. Sorry, but the problems I have to fix daily are not interesting, and conquering the mountain and being able to say I fixed all those every-day problems with it is not worth much on the job market.

If the amount of web backends suddenly needs node because they suddenly stop pushing json all day long, because node/npm is the only ecosystem that offers that solution, then you have a compelling reason. But other languages do json APIs just fine right now and have less ecosystem paper cuts.

jbboehr• 8 years ago

I'm kind of surprised no one has tried to make a standard library for node - although I guess underscore/lodash come close.

The package situation is a serious problem for distro maintainers, which AFAICT have basically given up packaging any npm packages the traditional way. Luckily Nix[1] and Guix[2] can most likely support automatic importing from npm.

Incidentally, one of the many things I love about typescript is that it has zero non-dev dependencies [3].

[1] https://github.com/svanderburg/node2nix [2] https://lists.gnu.org/archive/html/guix-devel/2017-03/msg008... [3] https://www.npmjs.com/package/typescript?activeTab=dependenc...

stephenr• 8 years ago

Related: 3m installs in 7 days, of a “module” to check if a number is odd.

https://www.reddit.com/r/programming/comments/886zji/why_has...

Zelphyr• 8 years ago

When I was checking out Serverless Framework I was floored that a single Lambda function (that barely did anything) resulted in 15+Mb of data being uploaded to AWS. I had only installed two NPM modules yet when I looked at the "node_modules" directory there were nearly 300 entries in there. Things like Redux and GraphQL, neither of which I ever intended to use in a simple test of Serverless Framework.

When Node and NPM first came out and I thought, while not perfect, they were a really promising set of technologies. Now, NPM specifically is something I will go out of my way to avoid. And by virtue of the fact that NPM is so closely tied with Node, I tend to avoid Node as well. Which sucks because I have always enjoyed programming in Javascript.

If anyone has come out with a curated alternative to NPM, please let me know.

pedalpete• 8 years ago

Though I agree with the article, and can appreciate that somebody just "checking out Serverless" (I'm assuming you were also new to Node), it just takes a little bit of awareness when installing packages.

It took a while to learn this lesson, but it pm;u tales a few seconds in the process of deciding if you want to use a package, to check out it's dependencies, I also check out the git page and license. Is it active, is it open-source, does it have far too many dependencies, is it in regular use.

https://www.npmjs.com/package/serverless

That's serverless. I was surprised to see it uses graphql as a default, and looking at graphql, how many dependencies does it have? Just one, which has no more. So that chain ends quite quickly.

It doesn't take long looking at the dependencies to see that the same packages get used again and again throughout the community. So you'll find you won't keep going down this path once you are comfortable with the packages.

Having said all of that, it is also good practice to look at the source of a package, and decide if you could do it better and easier yourself. It is amazing how many packages are only a few lines of code, and fairly unnecessary.

pan69• 8 years ago

One of my least favourite things about the NPM eco-system is that it doesn't have support for name spacing, i.e. package names are global. I think this is one of the things that Composer (PHP's equivalent of NPM) does very well [1].

Personally I'm not to worried about the size of my node_modules since it doesn't reflect what gets transpiled into the target anyway, but yes, if you have a bunch of projects sitting on your hard drive, it can get a bit busy.

[1] https://packagist.org/

simplify• 8 years ago

npm has namespacing https://docs.npmjs.com/getting-started/scoped-packages

amrx101• 8 years ago

There is a npm package to check if the number is even or not, Enough said.

boobsbr• 8 years ago

Yes, because a developer wrote a joke project, and nowthe whole ecosystem is garbage...

gr__or3• 8 years ago

There's two distinct problems I see here, which probably plague most package managers:

1. Package naming: All the good names will be gone soon, which'll probably lead to big players trying to get package names of little players. Which will then result in broken dependencies a la left-pad. Afaik the latter has been solved in NPM [1]. The former is probably more of a usability problem, we want installation to be simple and able to install deps just by remembering their name.

2. The power of deps and our reliance on good will: When installing dependencies, we basically grant them the power to wreak any kind of havoc they want. The ways we mitigate the worst: Belief in the magical power of open source, mostly using libs that are popular (hence might've gotten a good read by an IT-sec person) and at enterprise scale some static analysis tools to get an idea of how it interacts with your system. I think we're in need of a technical solution at the package manager level, where a dependency has to explicitly ask for certain capabilities (similar to how Android apps work).

[1] http://blog.npmjs.org/post/141905368000/changes-to-npms-unpu...

krapp4• 8 years ago

1) seems to plague NPM a lot more than others.

The former is probably more of a usability problem, we want installation to be simple and able to install deps just by remembering their name.

There's a point beyond which the desire for simplicity creates dangerous abstractions. A developer should expect to put more effort into installing deps than just remembering a package name.

gr__or3• 8 years ago

That it plagues NPM more than others might be due to size, NPM has almost 3x as many packages as the next large module repository (Maven) [1].

[1] http://www.modulecounts.com/

mabynogy• 8 years ago

Only NPM causes troubles. Node.js is stable and is improved quickly since the io.js merge.

You don't need to use NPM to run node programs. The binaries provided by node.js website are standalone (no dependencies): http://www.nodejs.org/dist/

Time for developers to address the chaos of Node.js and NPM

Related Stories