Show HN: Go Micro – A distributed systems development framework

by asim • 6 years ago

115 57 6 years ago

Not dissing your efforts, but you keep posting your Go Micro stuff, under different account names (chunk) in Show HN and normal threads. 90% of these posts add nothing and are just duplicates. Maybe add a blog post or something of value instead of the same old link every time.

asim12• 6 years ago

Thanks for your comment. I post a link to the github repo periodically because the project is moving incredibly fast and actively evolving to become something that's fundamentally valuable to those building distributed systems, cloud services and want to build applications with Go using a framework. People also come and go on HN so don't always get a chance to see it or dont' remember the project. I'm also always actively looking for feedback and so appreciate your comments and will look to publish a blog post next time with valuable content as I've done in times past on the blog https://micro.mu/blog/.

Otherwise as someone who started this project 5 years ago as his first open source endeavour and was really scared about the reaction from a developer community about the quality of my software or whether there would be any interest at all, I'd hope that positive encouragement is something we advocate for rather than negativity. And I hope others are not disuaded by your comment and actively post on HN to get positive feedback on their project!

Otherwise thanks for your comment.

Oh and also now being an old man I wanted to use an account with my real name rather than a pseudonym :)

lugged• 6 years ago

Don't apologize for selling yourself. If they're complaining, its working.

giancarlostoro• 6 years ago

the project is moving incredibly fast and actively evolving

Not that I'm hating, but one of the comments above made a valid point that this part of your comment suggests is correct: maybe you should be blogging about progress of the project and gain plenty of eyeballs?

One thing I saw that fascinated me is you guys are using mDNS for discovery, I was debating this discovery approach for a small microservices foundation for D but I couldnt even find a decent library so I havent really attempted much work in that approach. I did something that in a sense feels like a micro service architecture entirely with mDNS but in Python, then months later I worked with microservices in Java and it felt so bloated and overcomplicated compared to mDNS.

parhamn3• 6 years ago

If you can only confront someone using a throwaway account, you’re probably being unnecessarily mean. Glad they’re trying to get word out there on what they’re working on. Let the voting do the voting. Thank for your OS work OP, upvote from me. Looks like a cool project!

asim12• 6 years ago

Thank you!

jiofih2• 6 years ago

I understand the positive viewpoint, but don’t feel it’s in the spirit of HN to repost your own project every couple months. Back in the day even just posting your own stuff (instead of someone else who found it interesting) was frowned upon.

IMO a culture of curiosity vs blatant promotion is an asset for this board and should be preserved. I counted at least 26 reposts of go-micro. At what frequency should we just consider it a recurring (free) ad?

https://news.ycombinator.com/submitted?id=chuhnk

ABS• 6 years ago

genuinely curious to know why you created a throwaway account solely to post this?

city412• 6 years ago

That is very common practice on HN. Submitting a link here is a lottery that one rarely wins.

bsaul• 6 years ago

it looks interesting, but unfortunately the doc doesn't provide much beyond general diagrams. As an example, i wonder what are the properties of the pub/sub system in terms of delivery guarantees and tradeoffs. Or, if the goal is to have a third-party system implement the interface and just plug-it in, then at least give a list of available wrappers ( eg : "we currently have wrappers available for rabbitmq or kafka if you want to use them as your pub/sub system").

asim12• 6 years ago

Hi, author here. Thanks for your comment. You're right the readme doesn't go into detail and that's on purpose. I only wanted to provide a high level summary there. I need to do better and write some docs. There is a development repo where we're trying to slowly flesh out design docs https://github.com/micro/development. Please let me know if any of that helps and what we can add.

On the specific thing you're talking about. Yes exactly. We want to define high level abstractions for distributed systems which are pluggable. The idea being that we can't replicate the specific complexity and absolutely phenomal work of the teams building rabbitmq, kafka, etc but we can provide abstractions on top to simplify the developer experience and make it pluggable so those who want to use rabbitmq can but those who want to use kafka can swap that out instead.

The built in broker interface at the moment focuses on lightweight pubsub, notifications and events in a way that's fire and forget moreso than delivery guarantees, queuing, offsets, etc. You can use something like kafka or rabbitmq to get these guarantees but we want to write an Event interface that will be more inline with what people expect from that requirement also.

Hope that helps!

mrath2• 6 years ago

I do a little hobby work in Go. I looked at micro several times. For some reason I felt it is bit too heavy or unnecessary complexity. It does not feel fit in go ecosystem. But may be a lot of people are liking it. I could be wrong because I have not done much real work in Go. Does anyone use it for production apps?

asim12• 6 years ago

I think the Go community in its early years advocated for libraries over frameworks. 1. Because of the very powerful standard library which dictated the development model and culture of the community. 2. Because Go was still such a young project. I think its only natural over time that frameworks and tools emerge to simplify the development experience in every language. We realistically cannot ask people to piece together complex systems in that way.

If you imagine the thing you will end up writing in 200-300 lines of code in your main function, we're just encapsulating in a single function for you. The thing most companies end up building, a common shared library is by any other name a framework and so we're just piecing together that common experience for everyone else.

Now the key thing to note is you can use any of these packages independently. In fact you can think of Go Micro as a standard library for distributed systems development. Its just that at the top level we import the common features and create the construct of a "Service" and give you one way to initialise it with `micro.NewService`.

But the great thing is you have the freedom of choice to use things like go-kit, go-cloud, or any of the other thousands of tools in the ecosystem :)

atmosx4• 6 years ago

The “powerful standard library” to my experience is a myth that holds no water. Even trivial projects use external packages to handle trivial things like an HTTP server or logging.

No project I saw, however small, uses just the standard library.

bad_op3rator3• 6 years ago

Go is already an intentionally simple language with an excellent standard library, and dictates how to do things in the right way. Do we really need something on top of it? Not to mention, it's quite common that engineers like to reinvent the wheel, at least a little bit. It's just part of the fun, no one can blame them for that)

atmosx4• 6 years ago

Yes. Many open source application fairly complex & large are written in Golang: kubernetes, terraform, istio, etc.

All these run in countless production systems.

mrath2• 6 years ago

Sorry I intend to say, does anyone use micro in production?

wdb2• 6 years ago

I am wondering if a similar framework exists for Node.js or Deno?

I am really interested in Go but I have a lot of learning to do before I am confident to write solutions with it.

asim12• 6 years ago

I believe Molecular is a node.js project with similar ideals https://moleculer.services/ and also Seneca https://senecajs.org/

wdb2• 6 years ago

Thank you :)

pfalafel2• 6 years ago

You advertise "a powerful pluggable architecture." But Go binary plugins cannot be unloaded. At least not yet. How do you overcome this?

asim12• 6 years ago

Go Micro is a framework/library as opposed to a binary or server. You import the framework in code and it gives you the ability to add additional plugins via new imports. These can then be swapped out via environment variables or flags. We make use of Go interfaces to define the abstractions so all you have to do is create a plugin that implements the interface and that effectively makes it pluggable.

Here's the community led Go Micro plugin effort https://github.com/micro/go-plugins

pfalafel2• 6 years ago

What exactly does it mean to say that plugins "can then be swapped out"? I assume a binary Go plugin can not in fact be unloaded from memory. So modifying and reloading a Go plugin will result in multiple versions of it to then stay in memory. This is a shortcoming of the current Go implementation. Is my understanding correct?

npiit• 6 years ago

Dunno why the adversity in the comments. Great job, OP. I think there is a real need for something like this as the GitHub stars show, I considered using Kubernetes itself as a distributed infrastructure but it may be a bit heavy for many use cases. I think this can be a serious alternative for distributed systems that do not have intention to be built over Kubernetes.

eatonphil• 6 years ago

We used go micro in a production system for a while at a previous company. The nicest thing I found is that it abstracts a few common pubsub systems. But besides that I found it overly abstract. It's not really that hard to write glue for messaging systems as needed and that's the only part of go micro we were using.

chuhnk• 6 years ago

You're right. It's not very useful if you're just using it for pubsub. Go Micro was always intended as a full fledged framework for distributed systems development and you'll find if you build systems at scale you end up developing a similar framework in house. For those just using messaging or a database my advise would be to go directly to that system. Abstractions are only useful when they work together collectively for the foundation of something more. In our case when everyone is building Micro services we can actually run these absolutely anywhere regardless infrastructure and share them with anyone as a reusable building block.

wpietri2• 6 years ago

Are these things necessarily opposed? It doesn't seem impossible to me that a "full-fledged framework" also addresses the most common uses cases in ways that don't leave people befuddled.

aloknnikhil3• 6 years ago

Previous discussion: https://news.ycombinator.com/item?id=21878659

aloknnikhil3• 6 years ago

Service Discovery - Automatic service registration and name resolution. Service discovery is at the core of micro service development. When service A needs to speak to service B it needs the location of that service. The default discovery mechanism is multicast DNS (mdns), a zeroconf system.
Load Balancing - Client side load balancing built on service discovery. Once we have the addresses of any number of instances of a service we now need a way to decide which node to route to. We use random hashed load balancing to provide even distribution across the services and retry a different node if there’s a problem.

Trying to understand this a bit here. Is micro a replacement for k8s? Because I don't see any deployment capabilities here. So I'm assuming it's complementary to that. If that's the case, then I'm not sure how service discovery and load balancing would co-exist with the CNI in k8s.

asim12• 6 years ago

Not everything is k8s centric but also k8s also has very rudimentary models for discovery and load balancing, this is why things like service mesh have emerged to handle those challenges. As a framework Go Micro builds in client side discovery and load balancing which are completely separate to any third party system. You can imagine the next distributed database to be written with Go Micro which can operating independent of k8s.

In the case of k8s a lot of people swap out the load balancing for something like dns or integrate it with an envoy or linkerd considering our protocol usage is gRPC. But otherwise our service discovery acts as a central registry for all services. In a development centric environment this allows us to share feature rich metadata along with endpoints, description, and anything like team info, pager details, etc. All purely written in code rather than through yaml.

aloknnikhil3• 6 years ago

Agreed. I'm not saying that this needs to be integrated w/ k8s. I'm merely comparing it to k8s which promises the same abstraction albeit declaratively. The biggest plus point, IMO, for k8s is it makes managing deployments across physical hosts easy. So, I was asking if there was anything equivalent in micro that would help manage my deployments. And your comment on client side discovery and Load balancing partly addresses my original question. My main point here being, I don't see anything in the linked doc on why I'd choose micro over existing frameworks.

EDIT: https://github.com/micro/micro/blob/master/README.md

This page is more informative on why Micro over anything else. So I think this answers my second question.

latenightcoding• 6 years ago

Reminds me of https://github.com/atomix/atomix

solarkraft• 6 years ago

Why is "microservice" not mentioned anywhere on the page? Sure, it's not a great term, but it would make identification of its goal immediate instead of taking a few seconds.

Show HN: Go Micro – A distributed systems development framework

Related Stories