HN user
Please consider Scalyr: disclosure I work there, but objectively I will be surprised if someone beats us on price, focused log analytics - try and judge for yourself.
Does the speed matter? How exactly? I am genuinely curious: at Scalyr we _can_ be very fast but it is a balance with the cost that we want to pass on as price savings. Same with self-hosted Elastic: one can fine-tune it to be fast but minding the cost constraints gets it slower. WDYT?
A story of Netflix adopting StackStorm for autoremediation is a nice illustration of this approach. They began to build a tool, learned enough to be dangerous, discovered StackStorm and used it, experimented with it like with a breadboard, until finally figured out exactly what works - and reimplemented it. All these time learning the system was working and delivering vale.
Part of this story is here https://netflixtechblog.com/introducing-winston-event-driven...
Disclamer: StackStorm founder here (but not pitching).
The dichotomy is real and a reflection of Dev vs Ops dichotomy. DevOps made Dev and Ops collaborate but didn't blend the roles & skills. Ops appreciate logs but require consistent metrics to identify and root-cause the problem. Dev appreciate metrics but require logs to debug and fix the problem. Opinions on what is more important are informed by role and experience; the author makes it clear that as a team, we need both.
For many of the services I run, I haven't looked at logs in months, because the metrics tell the story. If service is degraded, I can usually correlate it to another downed service, a network failure, or a recent change. No logs needed.
Good point, echoing Brendan Gregg, the author of "USE" method commented:
The USE Method is based on three metric types and a strategy for approaching a complex system. I find it solves about 80% of server issues with 5% of the effort,” (http://www.brendangregg.com/usemethod.html)
Solving 80% of issues with 5% of effort is commendable; the rest 20% goes to developers where the other 95% of effort is spent debugging and fixing the problem, primarily by reasoning about the logs.
So: - "which of metrics or logs is more important?" is a relative and moot - "can metrics be extracted from logs?" - yes; "is it practical?" - it depends: likely NO for DIY. The fact that ELK is not making it particularly well doesn't mean that other products can't / don't do it. -
On top of "this sucks": such "fully automated" response is a violation of GDPR compliance that requires "right to obtain human intervention on the part of the controller".
Stunning that Google cloud can get away with this.
Rundeck is a good system, around for quite some time. I met many people who are using it in prod, some migrating to StackStorm when they need IFTTT, or those who prefer not to operate Java, some staying as Rundeck has more UI for one-off commands.
Nurtch DOES look like an awesome idea: jupyter notebooks proved to be a greatest interactive tool as well as for sharing the work. Good job with the site, too, best of luck with it.
If you’re serious about automating operations and wiring with slack/Chataops, check out StackStorm (disclosure I am one of the creators). Open source Apache 2.0,?installable, no mines. It’s much heavier than Nurtch or FireHydrant, as it comes with IfThisThanThat logic and workflows to string individual actions in runbooks, chat ops out-of box, but different strokes for different folks, we all runnops differently, pick your pill. https://GitHub.com/StackStorm/st2
If the comp, stocks, big $ is your primary motivation, look somewhere else.
We go to the startup for a thrill. For doing something you strongly believe. For creating something that didn't exist before. If you think you can do it in a big org, too, you are right, now go and try, get a job at Cisco or AT&T. Innovating there is like driving Kia Rio vs Porsche: it's a car, too. It goes thought the motion. But the thrill is not there.
Many people are happy in big companies. They won't be happy at startups. They're no worse because of this. Different stokes for different folks. There is no objective pro's and con's for startup vs big corp. There are properties, that match personal traits. The key is to know yourself and what are yours. Trouble is, we don't always know what we want, worse yet it changes over time with experience. And once we finally learn it, it's time to retire. Life is fun.
Now, you don't want to be taken advantage, right, so all written here about transparency, cap table, etc are the right things. Do check. Don't take below-market salary for the mirages of future. If you don't do these basics - you failed intelligence test and shouldn't be hired.
To be fair with other solid "on-top-of-K8S" frameworks: - Fission by Platform 5 https://platform9.com/fission - Kubeless http://kubeless.io/ - Nuclio by https://github.com/nuclio/nuclio
Not to mention few newer ones (there are over 30 serverless frameworks for any taste and size).
Similar but if you try the two side-by-side, you likely like Zappa much more... unless you work for AWS :)
It's what OpenStack has been - "opensource cloud platform". But yes, I agree, overstatement for OpenWhisk.
Define "Short". Condensed does not equate to easy to read. For a math book, time to read is in reverse proportion of the size. The shortest path is not the fastest.
Here's "Writing, Briefly" by Paul Graham http://www.paulgraham.com/writing44.html
Puppet vs Salt/Ansible is old story, they're compared as applied to config management.
Anthony is bringing a different angle here: how Salt / Ansible / StackStorm compare as event driven frameworks, applied to all-purpose automation, aka GLUE. Puppet/Chef aren't even playing.
Do you see this relevant? Or do you think "Salt beacons, who cares?" Why?
let's scrutinize. And please do challenge and point out what still feels wrong. * first, a library of scripts (actions), a shared one. each action is atomic, linux style, doing one thing well. A common pattern in ops. now with CLI, API and UI. Feels right so far? * second, combine these actions, building blocks,into workflows (workflow is action comprising actions). why not script? a) transparency of state (it ran 3 steps and failed on 4th) b) reliability, like 'restart workflow from a point of failure' c) carrying data - scripts pipe strings, workflows pipe JSON. * Add chatops. Any of these actions or workflows exposed in any chat with couple lines of meta. And any events sent to chat with rules
Good things begin to happen here, even before wiring events with actions. Shared context, integrations, quickly building more actions from existing actions, full audit...
* now, add IFTTT - firing these actions on events. Quite a lot of cases fall into this.
It's a challenge to single-out on one use case. A trivialized example, as log-file delete, is dismissed as "baidaid". Complex examples are domain specific and harder to grasp. We think we are on something here. We think it's not a bandaid, it's a glue. Needed in many cases.
Again, I work for StackStorm.
may be, when it comes to CI/CD. not familiar with concourse.
my comment above got cut out, i meant to say "generic platform with no focus on CI/CD". It's for auto-remediation.
it's not natural to use bosh or concourse to automate cleaning up log files or replacing a failed node in cassandra ring, or respond to identified DDOS attack.
terraform/juju/bosh are purpose-build for app and infra deployments.
StackStorm is a generic automation platform with no . One can use it to run arbitrary chain of actions on events. As such, it is used in auto-remediation & automating runbooks. We internally use it for variety of things like irc-to-slack relay, zombie ec2 vm periodic clean-up, ChatOps-ing JIRA, etc.
Some folks do run complex continuos deployment pipelines and blue/green deployments on StackStorm. Would be good if they can comment here.
Disclaimer: I work with StackStorm. In the past, I built Opalis IIS aka Microsoft SC Orchestrator. Seen both sides.
StackStorm to legacy runbook automations is what chef/puppet to legacy config management. It's open source, infra as code, and respect devops tools and mindset. Some folks on our team are devops with field experience putting their learnings in.
Our key design principles: 1) infrastructure as code, which means: workflows, rules, action metadata, and other artifacts are readable, source-controllable code (yaml)
2) integrations are "easy", which means: use python, ruby or shell, or turn any existing script into action by adding yaml meta-data. If you did an integration with something like HP OO or MS SystemCenter you appreciate the difference.
3) yes, opensource. I think it's a deal breaker, especially when it comes to integrations.
That's our perspective, how do you guys see it?