Clojure needs a Rails

https://blog.janetacarr.com/clojure-needs-a-rails/

by janetacarr • 4 years ago

252 228 4 years ago

blog.janetacarr.com

If we used interop for everything mundane, Clojure would really just be an S-expression shaped husk over Java code. Not a very good solution.

IMO some programmers pursue purity (no pun intended) over pragmatism, at their own expense. I understand the appeal, it's like hand planing a board with your vintage No. 8 instead of sending it through the Powermatic.

janetacarr15• 4 years ago

I understand this. Contrary to what I wrote, I do use interop to get work done when I have to, but it's certainly not an inviting solution.

seabird• 4 years ago

It's a lot more like buying a board planing robot cell, replacing the robot with a human operator because you couldn't get a teach pendant for it, replacing the automated part loading/unloading because you're now running parts that your fixtures can't handle, and ultimately replacing your low-cycle-time planer with a Powermatic because the vendor went under and you can't get any spare parts. Now you just have a Powermatic with a human operator and a distant memory of your initial machine.

You (at least should be) picking oddball stuff like Clojure because the unique advantages outweigh the problems. If you're just using it as a wrapper over another language that doesn't provide those advantages, you're taking the balls off the bull, and you probably should have just used Java, C#, Go, etc.

phtrivier2• 4 years ago

And clojure is actually in a better position than other niche languages because _at least_ you have the option of calling the boring and maintained java library to ship _something_ (it might be something non lispy, maybe a bit less efficient and maintainable but at least you solved the business issue.)

Try doing that in some other languages were your underlying platform has even less libraries than a JVM...

janetacarr15• 4 years ago

Interesting insight. I've never tried to call Erlang libraries from Elixir, but I'm not sure I'd want to either :)

tyre• 4 years ago

It’s very easy, as someone who went from Ruby to Elixir and had little understanding (to start) about Erlang.

It might be more natural than going down to Java because you’re mostly dealing with immutable data and Elixir is very consciously building on top of Erlang. In comparison, Clojure has always struck me as wanting to build on top of JVM but being an entirely different thing.

vendiddy• 4 years ago

It's surprisingly seamless to call Erlang libs from Elixir.

shakow• 4 years ago

I did it for some zip stuff that was already implemented in the Erlang stdlib, it was surprisingly easy. I assume the very small numbers of types in Elixir helps with that.

roguas3• 4 years ago

Yes, you get the eco system. But to what extend, the very thing mentioned in article rails-like framework is not really accessible right? I am ok with clojure not backing web framework, but also this one place where interop with springboot would be nice is not there (right?).

ttymck2• 4 years ago

At some point, do we have to consider that the problem might be Clojure itself? Does LISP lend itself to "Rails" -- a Rails that people want to work with? I ask, genuinely, as a near-total outsider.

The comparison to Elixir got me really thinking. I appreciate your "call to action", but community consensus gathering wasn't necessary for Phoenix to emerge as an option-of-choice, it gained traction because it is _good_.

Surely there are large organizations using full stack Clojure, similar to the Lawrence World-Journal. Is there something about LISP that makes it hard for them to abstract their "framework" bits into an open source package?

duncan-donuts• 4 years ago

The something is probably more philosophical within lisp communities. I’m relatively new to lisp (writing clojure/cl for about a year) and from what I’ve seen people prefer to bolt on small things instead of big frameworks. The full stack frameworks exist but they’re hardly frameworks. They’re closer to project templates than anything.

mark_l_watson6• 4 years ago

It might be that simply committing strongly to a language, any language, has benefits? Years ago, a long term consulting customer hired me on a short fire-drill job. He had also hired some Clojure celebrities like Stuart Sierra and Stuart Holloway. They were incredible productive and fast. It was a very long time ago, but as far a I remember they had a small set up libraries they used. BTW, I was hired for a few hours a week to help with dev ops, not code.

EDIT: I am currently working (part time) with an all in Common Lisp company. It is really nice to see a group of people who all love a programming language using it. As much as I like virtually ALL Lisp languages, I do sometimes feel guilty not just using Python when there are much better libraries available (e.g., machine learning, linked data, etc.). When a programming language has 1000x as many developers you simply get a richer ecosystem.

didibus15• 4 years ago

When a programming language has 1000x as many developers you simply get a richer ecosystem

Interesting comment given the article. That's the magic of Clojure being hosted, you get the rich Java ecosystem and get to use your Lisp that you love.

The downside is you still need to deal with the rich ecosystem that isn't as nice as if there was a rich one in Clojure.

jayceedenton2• 4 years ago

Is there something about LISP that makes it hard for them to abstract their "framework" bits into an open source package?

No. I don't know what else to tell you.

ttymck2• 4 years ago

That's fair, I didn't intend to sound entitled, but "let's all decide what Rails-like to use" doesn't seem to be working for Clojurists, so I was hoping that poking at other avenues might spark some productive conversation.

jmcgough3• 4 years ago

"community consensus gathering wasn't necessary for Phoenix to emerge as an option-of-choice, it gained traction because it is _good_"

I think it also because there's two camps of elixir devs - those who came from Ruby and those who came from Erlang. The Rubyists have a fairly easy time being productive quickly with Phoenix because it (at a glance) is very similar to Rails. From there you learn about supervisor trees, genserver, etc.

didibus15• 4 years ago

Surely there are large organizations using full stack Clojure, similar to the Lawrence World-Journal. Is there something about LISP that makes it hard for them to abstract their "framework" bits into an open source package?

There's a fundamental model shift that's not as compatible to frameworks.

Lisp optimizes for it to be really fast to implement perfectly tailored custom solutions, with your own DSL (domain specific language) that best fit your problem. Iterations are quick, development is fast, you can automate away repetitions and ceremony trivially.

The need for scaffolding and for complicated code-gen, which is half of what most web-frameworks provide, is just not there, because Lisps are already meta-programmable.

The other thing is Clojure's functional nature. Objects and Actors and Modules, these things naturally expand to become Singleton state or behavior containers that you call upon. They also naturally lend to a plugin architecture, where configuration setups a chain of implementing interfaces called into by standard modules from the framework, etc.

The functional, higher order, immutable, and data driven nature of Clojure doesn't fit well here either.

What happens instead is that libraries, not frameworks, tend to work better in Clojure. Things you can call and use as you want, instead of things that call you and ask of very specific extension and configuration from your part.

So the Clojure web stack is still full of reusable code, they're just exposed as libraries.

When you know what to do, you can easily put together a Clojure website together, in no less time then it takes using a framework in another language, and you similarly reuse almost as much code as in any other language.

The difference is that because you need to use the libraries, you also need to figure out how to use them and where and what to do to glue all of that stuff together.

In a framework, you can't choose how to glue them all, it's forced on you, that's both the issue with frameworks, and the benefit. So when you don't know how to build a website, a framework really helps, it guides you, and you just follow its strict structure and plug what needs to be plugged, and configure what needs to be configured.

With the library approach, that's left up to you, much harder to get started with, but not any less productive ounce you know.

Finally, a framework forces everything to be a monolith, that is built and designed within the rules of the framework itself. All behavior you'd want to reuse needs to be framework aware and compatible. This means that slowly people grow an ecosystem of compatible libraries all around the framework.

Again, with the library approach, the various libraries you choose to use were all independently designed and built, they weren't built to work together, so again, you have to find how to glue and connect them and map between them yourself.

You could still have a framework, but the mental model a framework follows is often contrary to the mental model Clojure enables, people often choose Clojure because you can quickly customize things to your liking and connect various independent building blocks together in the way you prefer and that works best for you.

That means whenever someone made their code structure available to others as a framework, everyone else went, I don't like the way you did it, so nevermind, I'll just use my way.

If the libraries all only worked with their structure though, it be too much effort to go your own way, but not so in Clojure, because all the building blocks are exposed as libraries, so you can easily structure it however you prefer.

lolinder4• 4 years ago

If we used interop for everything mundane, Clojure would really just be an S-expression shaped husk over Java code. Not a very good solution.

I don't have a lot of experience with Clojure, but speaking as someone who uses Kotlin on the server side, this attitude strikes me as odd. Most of the reason to use a JVM language is to take advantage of the mature and well-maintained Java libraries for basically everything.

If interop is considered a last resort (to the point where you reach for a 3-year-old unofficial Stripe library over the officially maintained Java one), what's the point of running on the JVM at all?

googlryas• 4 years ago

One problem with interop is you import the design/ergonomic considerations that may come from other languages, while neglecting what is uniquely powerful about your own language.

mark_l_watson6• 4 years ago

On Twitter, PureDanger had a nice comment, something like: it is better style to directly call into a Java library than to write or use a wrapper.

I am not sure if this really answers your comment, but I really liked his comment because I tend to write wrappers and re-evaluated my habits.

janetacarr15• 4 years ago

That's surprising. I listed the interop caveats here because sometimes it feels like a bandage intended to bridge some gap until Clojure matures.

But if Alex said that, then maybe it's not. ¯\_(ツ)_/¯

waffletower2• 4 years ago

You should take Alex's words contextually, and with a grain of salt. You can easily see his commits in this project, for example: https://github.com/cognitect-labs/aws-api

stingraycharles• 4 years ago

When I started out with Clojure, I was using wrappers everywhere, but after about a year or so realised that they added very little value, hurt performance, and made the code less simple. On top of that, wrapper libraries were often not complete in functionality.

The pivotal moment for me with this was when dealing with Kafka, just using the “raw” Java library ended up being so incredibly much simpler than any of the wrapper libraries.

hyperhopper• 4 years ago

Why does he say this? I would think the opposite and in a professional environment I've gone in the other direction of regretting not writing _more_ wrappers for external libs.

fithisux• 4 years ago

Very good piece of advice if there is no automated way.

For example you hand craft a wrapper for lib-version1. lib-version2 is released but you do not need it any more. So wrapper is outdated.

I prefer recipes to call other language code (or framework) instead of wrappers.

E.g. Call a C library from Dlang instead of waiting??? for some wrapper.

hibikir• 4 years ago

It Kotlin it makes a lot more sense to use Java libraries, because for most purposes, the interfaces you'd write in Kotlin and the ones in Java are not all that different, other than coroutines. From Scala or Clojure, the differences in interface design are far larger. You are going from languages where mutable state is an optimization you want to hide, to libraries full of setters that return void. A land of monads, and maybe macros vs imperative programming. This makes the ergonomics of many mature java libraries just too much of a bother to use directly. There are cases where the effort to rewrite is bonkers (say, ApachePOI), and others where wrapping is sensible (see a variety of json serialization libraries which support different parsing backends, some of which will be straight java), but there are many libraries that are sensible from Kotlin, but are worse than a rewrite if you are using an FP language.

Still, I think you are undervaluing the JVM itself, which is almost magical when it comes to production observability. It'd be a whole lot of work to come anywhere near that tooling support for ops if you were avoiding the JVM. Scala, for instance, can also transpile into Javascript if you feel like it, and there's a native project that compiles to LLVM, but the JVM has the vast majority of the use cases, and it's not due to the libraries.

lolinder4• 4 years ago

Yeah, I get that. Even in Kotlin there definitely a number of places where I prefer the Kotlin library to the Java, and I can see that it would happen more often in Clojure.

That said, the example in the article is the Stripe API, and I cannot see a good reason not to just use the official Java one. I've used it a ton, and it's as simple as can be—you pass a map in and get back an object. If that's a problem, either Clojure has a way worse interop story than necessary or the author is suffering from a variant of not-invented-here syndrome.

brightball• 4 years ago

One point to running on the JVM is that you can leverage all of the existing Java app server infrastructure in your company for deployment.

That stuff removes 90% of the arguments against using another language in a Java shop.

lolinder4• 4 years ago

To expand a little on this, here's Rich Hickey on the subject [0]:

I dislike writing Java/C# as much as anyone, and Clojure is my ticket to writing much less of it, while leveraging the efforts of multitudes. As I've said in my talks, most Clojure users go from "eww, Java libs" to "ooh, Java libs", leveraging the fact there there is already a lib for almost anything they need to do. And using the lib is completely transparent, idiomatic and wrapper free. OTOH, if they want to customize their interface to a lib, it is trivial to do so in Clojure.
If, however, the stigma of Java extends to libraries written in it, even when consumed from languages like JRuby and Clojure, and people think it's cooler to reinvent than reuse, they need a new definition of cool.

[0] https://groups.google.com/g/clojure/c/W_FTvyX6swY/m/UDhgIbkh...

bilalq2• 4 years ago

Interop with Java libs is decidedly not pleasant in many cases. I haven't used Clojure before, but I've worked on services built in Scala and JRuby over the years.

There are a few challenges you run into:

1. Many libs written in Java follow the worst of Java's design patterns and idioms. Rather than calling a function and passing it some data as an argument, you instead have to instantiate a WhateverClient that itself takes a WhateverThingyStrategy instance and a WhateverConfig class instance, then call a .executeCommand method with a new IWhateverCommand anonymous class instance that implements the execute, doCatch, and doFinally methods. And of course, you can't forget whatever subclass of AbstractOrigamiWhateverResponseMarshaller you need next. Oh, and because all these instances need to be defined as beans that will be DI-friendly later, there's even more boilerplate to look forward to.

2. In comparison to languages like Scala, Java's type system is often quite lacking. You can never trust that a non-Option type is not null.

3. If the rest of your code is free-floating with structural types in some other language, you inevitably need to write a ton of glue code for adapting these structural data type objects to whatever blessed POJO interface Java expects.

All that being said, I still would prefer to use the more mature Java lib if the more native choice isn't as well maintained. But there's no denying that it's painful. The ergonomic hit is just as bad (if not more than) execing a child process to run code in another well-suported target like Rust/Go/Node/whatever that maybe has more sensible idioms. There are of course a lot of reasons why that'd be a bad idea, but still...

kaba03• 4 years ago

The ergonomic hit is just as bad (if not more than) execing a child process to run code in another well-suported target like Rust/Go/Node/whatever that maybe has more sensible idioms. There are of course a lot of reasons why that'd be a bad idea, but still...

Come on, we both know this is false and it is only your feeling, nothing factual. How is IPC with all of its intricacies anywhere close to as ergonomic as calling one more method in the host language when all of your objects are also of said host language?

Frankly enough, creating a wrapper should not be hard at all, no matter the language. Especially not for libs that are mostly unidirectional in communication.

egypturnash2• 4 years ago

Suzi just needs to find an excuse to spend a couple years building a serious thing entirely in Clojure, no matter what. And then to disentangle all of the logic particular to her thing from the application framework she's built for this (or the existing one she used as a starting point), as well as disentangle anything that she thinks should be an external library, and release it all for free.

Suzi may need to wait to do this until after her current gig has won the Startup Lottery. Or perhaps she may need to quit her current gig and figure out how to get a bunch of other Clojure-lovers to pay her rent while she does this. Good luck.

janetacarr15• 4 years ago

One can dream ;)

egypturnash2• 4 years ago

Good luck! Getting this post on the front of Hacker News is probably a step in the direction of finding a way to pay your bills while spending a year or two building Canals For Clojure to the point where it attracts other people to build libraries and plugins for it.

avanai• 4 years ago

I think this article vastly undersells the value of Java(script) interop. The ability to call out to a well-maintained library in two of the most widely-used languages in the industry is one of the major selling points of Clojure as a pragmatic LISP, versus e.g. Racket or CL. The reason why there are so many half-baked wrappers around popular Java libraries is that it’s reasonably trivial to write one yourself on demand, so that’s what people do.

troutwine3• 4 years ago

It’s an interesting argument. However, I’m not sure about it. Consider Rust doesn’t have a Rails per se but the ecosystem it is thriving. It seems more like an indication that the Clojure community of developers is either focused on things other than the article is concerned with, or its period of popularity has waned and the libraries that do exist from that time are slowly rotting. I think about Erlang before Elixir. In its niche Erlang had a very good set of libraries, primarily centered around lower level network communication type things. But for integration with popular APIs etc. not so much. After Elixir and an influx of new people the off-the-shelf library situation got significantly better. I don’t think that’s because they were doing Rails type things necessarily but more because there was a more diverse pool of developer community, whereas the Erlang community was slowly shrinking through attrition.

If you want Clojure to thrive again you’ll have to stop the attrition of its developer community. Maybe that’s by building a Rails but even your example in-article where Go is more suitable isn’t really a web type problem. What problems can Clojure solve that other languages can’t, in a way that is worth the effort for someone new to the ecosystem?

janetacarr15• 4 years ago

I think you've answered you're own question here, in a sense.

Erlang was created by Ericson for use in telecom systems. (OTP stands for Open Telecom Platform) which gives credence to the idea that Erlang didn't need a Rails. Once Elixir came along, being a bit more general purpose, people started building those things.

Clojure, from the beginning, was intended to be a general purpose language, not a web language. So I think it would be fine to expect Clojure to be able to solve non-web type problems. Maybe the community has attrition, but I don't think so. The Clojurians slack seems bigger than it ever was, more companies are using Clojure than I've ever seen.

edit: general not generate

troutwine3• 4 years ago

Minor quibble, OTP _used_ to stand for Open Telecom Platform but that acronym was removed in the push to open-source. Ericson had hoped to sell the Open Telecom Platform to other telecoms but that didn't work out, hence the modern Erlang/OTP. Anyhow, Erlang's niche was and is fault-tolerant middleware, yeah, the kind of thing you see deep in the bowels of systems. The community built up a sizable bulk of code to support the kind of middleware you'd find in Enterprise-y type places in the early 2000s and then slowly started to fall off, leaving an ecosystem that slowly bit-rotted in much the same way you describe in your article. That, specifically, is what I'm referring to as the off-the-shelf rot problem. Perhaps I could have been more clear.

I agree with the argument that Clojure was intended to be a general purpose kind of language, but I think you article also suggests that -- unless you're tied to a JVM model _and_ you refuse to use Java -- that Clojure gets out-competed by languages like Go in that kind of work. It's not clear to me how or why building a Rails kind of thing for Clojure would reverse this situation. I think, also, the size of an online community and tally of companies can be misleading. Erlang's IRC and company use was at an all-time high even while the actual pool of developers had reached its peak and was declining, slowly but surely. It's argument from analogy, sure, but what you're describing in the article for Clojure looks damn familiar from my time with Erlang, pre-Elixir.

dustingetz2• 4 years ago

Clojure, from the beginning, was intended to be a general purpose language, not a web language

i don’t think that is strictly true, clojure is for “situated programs” which is basically a perfect description of cloud information systems and user interfaces.

immutability and macros is not what you want for games, for example. and data science is very SQL/struct oriented, it doesn’t hit clojure’s high notes. mobile is very platform interop heavy and best done natively without adding intermediate abstraction layers. databases are written in lower level langs for performance (Datomic is not known for lightning fast queries). what’s left?

Macha• 4 years ago

I think the lack of a Rails or other flagship framework has arguably hurt Rust if you compare it's trajectory to that of Go. Go had Docker and Kubernetes as early flag bearers to indicate it's a "real language", while Rust has only really had the critical mass to start getting treated as one in the last two years.

troutwine3• 4 years ago

Ah yeah, very fair if we’re talking “Rails” as a signifier of a big project that gives a language and ecosystem is “flavor”. For what it’s worth, I imagine Go has a broader remit than Rust even today. Go is a very good update on the pain points of Python for domains where integration with existing C libraries can be a little wonky, resource consumption isn’t a major concern. Rust puts resource consumption and fitting into existing systems projects as a first class design concern, which is great for me because that’s the kind of thing I work on, but I do admit it makes for a more challenging language to use.

int_19h3• 4 years ago

They're two drastically different languages optimized for different goals. Why would they have, or even want, the same trajectory?

slifin• 4 years ago

Clojure typically doesn't attract the kind of developers who want to work on "boring" things

So there is an imbalance there

On one hand I'm kind of happy there's no one true way in Clojure to do web dev, because as I think Rich said, it's not a solved problem

So if we couple too heavily to one opinion on web dev then it's going to fail us as a community on a big scale and be hard to move away from

Sets of libraries are typically easier to swap out

In reality I think most Clojure businesses end up on reagent or and reframe and they are the defacto ways of doing web dev

For "solved" problems like stripe calls it would be nice if we had more people working on maintaining libraries but as a Clojurian myself I know I wouldn't do that myself so I only have me and people like me to blame

capableweb• 4 years ago

Clojure typically doesn't attract the kind of developers who want to work on "boring" things

I'm quite the opposite. I love Clojure, just because it is "boring". Maybe not "boring" as in well-known, but "boring" as in the core language and it's runtime doesn't change and is stable, like really stable. I've upgraded projects using "ancient" Clojure versions (released +5 years ago) and not having to almost do any changes, as both the language, the runtime and most libraries are so stable over time. It simply just works. If people want to introduce breaking changes, they tend to prefer to create entirely new libraries instead.

int_19h3• 4 years ago

I'm kinda amazed that 5 years ago is considered "ancient" these days, and stability over such a short time interval is considered remarkable.

janetacarr15• 4 years ago

I agree. Boring is tough to work on.

I thought about touching on this in the article. Re-frame and reagent have become the default for Clojurescript development, but I have no idea how.

seancorfield6• 4 years ago

Could you elaborate on that? I don't do ClojureScript, but I've dabbled with it on and off since 2013 (I've been doing Clojure in production for over eleven years). When I first started looking at ClojureScript, Om was "the thing" and Reagent was the new kid. I built a prototype at work with Om and then rebuilt it with Reagent and found the latter much more pleasant and idiomatic to work with.

Last year, I dipped back into ClojureScript and built some simple projects with re-frame and I really liked it. I can definitely see why re-frame has become so popular.

keithasaurus2• 4 years ago

Clojure typically doesn't attract the kind of developers who want to work on "boring" things

I kind of have the opposite opinion. I worked on a large Clojure codebase for several years. There was so much effort put into building tooling and libraries in Clojure because the ecosystem isn't huge, (and some of its bigger pieces are just a misadventure, ie spec). It was really frustrating to me because I felt like we were spending 50% of our time writing inferior versions of code that Python, Ruby, or even JavaScript already had as libraries. I'd have preferred that time to be spent on interesting things like business concerns. Of course, you can use java interop, but it's much more natural in Scala and Kotlin.

huahaiy14• 4 years ago

Care to mention a few examples of missing libraries that you spent 50% of time writing on the job?

I can hardly recall a case where I could not find a Clojure libraries for. The only exception would be wrappers for proprietary APIs, such as Stripe.

thom2• 4 years ago

I don’t understand this at all. Clojure’s all over the place in banks and fintech startups and they’re all boring as hell. People don’t spend a lot of time on random proprietary wrappers because they don’t provide a lot of leverage, not because they’re ‘boring’.

eternalban• 4 years ago

This is the second appearance of "boring" in this thread. There is also "boring" libraries. Somewhere someone should-be/is writing a thesis on the psychology of programming and programming languages.

davedx• 4 years ago

I don’t think API wrapper libraries add that much value and I usually only use them if it’s absolutely necessary. Most APIs are just REST calls…

janetacarr15• 4 years ago

This is mostly true tbh.

But API wrappers handling things like retries, pagination, client-side business logic, and coercion from JSON. Having to account for all of that for two API calls is somewhat tedious and repetitive, and I'm sure we've all solved those problems at least a dozen times.

huahaiy14• 4 years ago

Retries, JSON coercions and so on do have Clojure libraries for them, so I am not sure why you want to solve these problems yourself, let alone a dozen times. Really not sure what your exact point is.

the_duke• 4 years ago

Writing high quality ergonomic API bindings is a surprising amount of work, especially in statically typed languages.

Every API has its quirks, sometimes returns different data, has edge cases for certain calls, has different error reporting patterns, rate limiting, authentication methods, sometimes has intended or unintended breaking changes, ....

Sure, you can usually just manually whip up the code for an endpoint or two in a few minutes , but then you need to continually add endpoins, fix the edge cases, adapt to upstream changes, ...

Good bindings make your life a whole lot easier.

joshlemer• 4 years ago

Correct. If we take their point to an extreme, HTTP calls are just bytes sent and received over TCP, so what's the use of an HTTP client library? Just open up a TCP connection and send/receive text to/from the server!

waffletower2• 4 years ago

Clojure libraries target microservices with a precision that no other language ecosystem has. In essence, Clojure web services developers rail against Rails and other bloated, unnecessarily complected (https://www.infoq.com/presentations/Simple-Made-Easy/) frameworks of the 90s. As a Clojurist I too rail against Rails. I don't think that expansive model fits the problem space. I have had painful experiences in the past maintaining Rails projects wondering why they didn't know of DRY. If there is a essence within Rails that you feel could be distilled into a lean Clojure model, build it out in a library and share it.

aantix• 4 years ago

Keeping things DRY is about the discipline of the developer.

It’s nothing specific to the language or framework.

janetacarr15• 4 years ago

My central thesis seems to be getting lost here.

I'm not advocating for a Clojure Rails because I love Rails. I've never used Rails to be honest. I'm arguing open-source efforts are repeatedly being spent on trying to build the next web framework/library/toolkit (Rails) for Clojure and not much else, so it would be great if there was one, so we can get on with filling the gaps in the Clojure ecosystem.

huahaiy14• 4 years ago

"open-source efforts are repeatedly being spent on trying to build the next web framework/library/toolkit (Rails) for Clojure and not much else", really?

That's contrary to what most of us know. There are some efforts in the Web front, but not much. At least, the community is not paying much attention to these efforts.

Let's look at the list of community funded projects, e.g. those in Clojure Together: the only Web related projects funded were clj-http in 2018, ring, re-frame and reagent in 2020. None of these are Web frameworks, and the rest of the funded projects are not Web related at all.

fmakunbound3• 4 years ago

Clojure libraries target microservices with a precision that ..

I’m not sure what that means exactly, but no one is using Rails only for building out micro services. They’re using Rails to go from nothing to a production ready web application, with all the incidental complexity taken care of, in a very short time.

It’s been a few years since I was involved with Clojure ecosystem - what is the Clojure experience equivalent to say, the original Rails demo from back in 2005? The last I tried, all the parts seemed to be there, but much painful assembly/“composing” was required and not all the parts fitted which ended up producing a lot of awkward complexity needing desperate decomplecting.

blain_the_train5• 4 years ago

MVC don't help with strip payment integrations.

vemv• 4 years ago

I also thought "Clojure needs a Rails" some 5 years ago. Nowadays (multiple, real-world Clojure jobs later) I don't.

I also learned that trying to second-guess why developers spend their time in certain libraries and not in other endeavors (like frameworks) comes across as pretty disrespectful and actually ignorant (considering that those people are very often more skilled/experienced than most of us).

(Good related read: Open Source Is Not About You)

So I'd recommend to anyone, open your mind, be ready to have some preconceptions changed, and contribute a significant deal to (Clojure) OSS to truly appreciate what we have. And write more interop ;)

justinhj• 4 years ago

I see exactly the same thing in the Scala community. Your first instinct when you need a library is to find a pure Scala one, when many times there are far superior (in terms of maturity, stability and documentation) in the Java word. Sometimes it makes sense to write an intermediate wrapper but not always.

antupis• 4 years ago

I think what Clojure needs is to be best at some very practical real world usecase so kinda what Rails was 2007 or Python and ML were 2013.

rubyist5eva• 4 years ago

Stop trying to make Rails in things that can’t be rails. Rails is Rails uniquely because it is RUBY on Rails. It plays uniquely into Rubys strengths, that’s what makes it what it is.

weavejester8• 4 years ago

What would a 'Clojure on Rails' framework look like? I think that's the primary reason why such a framework has yet to find traction; there's no single, obvious way to go about it.

The efforts made so far in this space have taken many different approaches, because they're all experiments. Explorations of what an idiomatic Clojure web framework might potentially look like. Clojure has been around for over a decade now, and it's fairly mature as a language; but it's ideology is young in comparison to more established paradigms like OOP.

To make matters more complex, Rails came to prominence in an era where web applications were mostly of one type: server-side HTML with a sprinkling of JavaScript. Nowadays there tends to be more options, e.g. a web application may consist of a single HTML shim, with a thick React client that talks to a GraphQL back end.

So on the one hand we have a language that's still exploring its identity; on the other we have a rapidly changing idea of what a web application should be.

My guess is that an idiomatic Clojure web service would look like a client-accessible relational database with a strong security model and datalog queries, along with some system for adding in side-effectful hooks to respond to data change. In other words, something rather different in design to Rails.

janetacarr15• 4 years ago

Thanks for the reply James.

I agree with everything you said.

To clarify, I'm not advocating to re-create Rails for Clojure, rather, I'm arguing open-source efforts are too focused on the search for Clojure's Big Web Framework TM. However, I do like your guess as to what a Clojure Web service might look like.

huahaiy14• 4 years ago

Your observation is simply not true. To the contrary, the consensus of the community seems to be that there's no need for a rails-like framework for Clojure. Most of Clojurians are satisfied with what we have in the Web front, and are not worrying about the lack of "big Web framework" at all. I don't know where you got your impression from.

suprfnk• 4 years ago

Something like Phoenix? Elixir is a newer language, with a lot of similarities to Clojure. Phoenix even made the most loved web framework in the Stack Overflow 2022 survey.

Why did they converge on something so, apparently, great, in such a low amount of time? And why is this so hard for Clojure?

Not taking sides or trying to be negative, just honestly curious what the driving factor is.

weavejester8• 4 years ago

I haven't used Elixir enough to be particularly confident in an opinion about it. However, Phoenix appears to have a similar design to many other MVC frameworks, and if a conventional design works for a language, there's less need to experiment. Initial development can converge around something that's tried and tested.

A conventional framework doesn't fit Clojure particularly well, so the community has been forced to tread new ground to figure out what works with the language.

refset• 4 years ago

Maybe it's because Clojure has typically attracted a demographic who are more shy about self-promoting and marketing their new ideas and tools. Photon is an exciting (and relevant) example defying that trend though: https://www.hytradboi.com/2022/uis-are-streaming-dags

Also relevant as a Phoenix-like alternative for Clojure: https://github.com/tatut/ripley

didibus15• 4 years ago

Because there's no alternative in Elixir.

In Clojure, you can make a website many other ways, use any existing Java server, implement your own server quickly, use any templating language you want, implement your own quickly, use whatever routing lib you want, implement your own quickly, etc.

Not only are there existing options from Java, but implementing your own is approachable to your average dev, and can be done in a reasonably short amount of time.

Maybe it's a bit of the Lisp curse afterall. A language too productive, that no one bothers settling for the one framework.

I also think Elixirs Ruby friendly syntax brought in a lot of Rubyists with Rails experience, and that attracted people who used Rails to Phoenix, giving it that inertia.

Chris McCord used to be a Rails developer, so he came directly from Rails, so it made sense to rebuild a similar framework for Elixir, and a lot of people followed from Rails to Elixir as a better ruby with better scaling.

newlisp2• 4 years ago

along with some system for adding in side-effectful hooks to respond to data change

Curious, can you elaborate on what you mean by this?

weavejester8• 4 years ago

So, for example, email verification. There could be a watch that triggers when a new user is inserted into the database, and this triggers a side-effectful function that sends an email.

ativzzz• 4 years ago

Nowadays there tends to be more options, e.g. a web application may consist of a single HTML shim, with a thick React client that talks to a GraphQL back end.

Best part about Rails is that you can do this as well. Rails has done a great job of being adaptable to new web technologies.

ben_jones• 4 years ago

Im sorry I couldn't get past a hypothetical Org allowing a single developer to yolo-yeet bespoke languages, frameworks, tooling, into internal-critical-business-project after internal-critical-business-project.

If your Org behaves like the above you could have the best Rails-Closure framework in the world but you're still destined for implosion.

hombre_fatal• 4 years ago

I think this is just a cultural thing where the ecosystem prefers to compose things together. The Node ecosystem also hasn't centralized around a Rails. People tend to just use Express.

Also, the syslog server example doesn't seem like something a Rails would solve.

can we please just get together at the next Conj conference decide what our "Rails" is going to be?

I think these days we, OP included, tend to use "Rails" to not just mean a monolithic framework, but also one that monopolizes the ecosystem which is one thing that made Rails good.

But since it's been tried before in the Clojure and Node ecosystems, maybe people just don't care for it.

nickjj4• 4 years ago

The blog post mentions Elixir and while Elixir has its own mini-Rails with a lot less opinions, you'll still run into the same problems as Clojure around things like wanting a Stripe supported library, or PayPal, or Braintree, or Paddle or a ton of other popular 3rd party SAAS SDKs that exist in Ruby, Python, Node, PHP, Go, etc..

It's one of the reasons I stopped using Elixir. When I reached out to Stripe (and other 3rd party tools) they said the demand wasn't there to justify the dev time to maintain an official library internally. I think this is the reality to expect when using a more niche language. That may be ok if you're ok with that but it's important to know that's what you're getting into when you choose languages like this. It's a personal preference.

bestinterest• 4 years ago

This brings up another point I hope someone tries to solve in programming. Every time a new language comes out we have to recreate millions of baseline libraries and it just sucks. As a dev I want to be able to make use of great libraries oblivious to what they are created with.

GraalVM is trying to solve this with its polygot VM idea https://www.graalvm.org/22.1/reference-manual/polyglot-progr...

nickjj4• 4 years ago

Every time a new language comes out we have to recreate millions of baseline libraries and it just sucks. As a dev I want to be able to make use of great libraries oblivious to what they are created with.

Technically this tool mostly does exist already with the OpenAPI specification if we're talking about REST APIs. If you as the API provider put in the leg work to create a very detailed specification which is a YAML file, you can generate programming language specific SDKs out of it as long as the language has an OpenAPI library to consume this spec in an accurate way.

Stripe has publicly mentioned[0] they mostly use this spec to generate their SDKs (even as of a few years ago), but I guess auto-generated code still requires some developer time and there's a quality assurance level of "hey we're dedicated to internally supporting this". It's a huge deal having a provider internally support your language's SDK.

[0] https://github.com/stripe/stripe-python/issues/694#issuecomm...

jrochkind12• 4 years ago

Honestly, as a rubyist, I more and more often run into new "SDKs" that don't have an official ruby version supported! Ruby is no longer always considered popular enough to get an SDK from a vendor.

I think the fundamental truth here is that so much of writing software these days is having open source to rely on. That applies to things like an SDK for a vendor API, but also applies to things like consolidating community effort on one approach to back-end web app -- instead of having a bunch of not really mature options, and also so effort can be consolidated on building additional things that work with that popular tool.

I don't think these two areas are actually related like the OP does -- if we can just reach consensus on a web framework, then somehow we'll get things like a Stripe SDK, I'm not sure about that.... but I think the general challenge of "how do we get a sustainable, supported, mature/polished open source ecosystem of software that we can use that works together" is kind something we don't really understand, how does that happen (or not) for a new language? I don't think it's just "well, if we only get popular enough it'll happen on it's own", and of course there is a chicken and egg issue there too.

nickjj4• 4 years ago

I think the general challenge of "how do we get a sustainable, supported, mature/polished open source ecosystem of software that we can use that works together" is kind something we don't really understand

I think one problem is people who tend to flock to niche languages like to write libraries so what happens is you get a random person on the internet who makes a library, then they eventually stop using it personally and the project dies because they were the sole maintainer championing the project. You have this happen in parallel and the next thing you know you have 3-4 libraries that are either missing big features, fall out of date, go into maintenance mode, etc.. I saw this a lot with Elixir's community libraries (Stripe, AWS SDK, pagination, etc.).

There is a chicken / egg aspect to it because I think the fix here is to eventually get popular enough where you have a large enough of a community around the language and ecosystem that the most popular projects for a specific feature get so popular that they can't die because too many people depend on it and then a bunch of folks start to contribute to it and the community informally blesses 1 solution as "the" solution. The chicken / egg problem here is that a lot of people want to use Stripe, not create a library to use Stripe, etc. so it never gets mass adopted.

I think the best way for this to happen is organically. Trying to create a committee with an org with specific contributors trying to invent solutions is going about it in the opposite way. The best libraries tend to be extracted out of real projects and grow to become popular because the person or people behind it have a skillset around not only creating high quality software but also a dedication to create great documentation, can handle support, have lots of tests and generally have created a maintainable code base that makes it easy for others to contribute.

Ruby is no longer always considered popular enough to get an SDK from a vendor.

Do you have a bunch of examples here? I haven't come across that with the usual libs I use (Stripe, Braintree, PayPal, Sentry, Datadog, Mux, AWS, etc.).

int_19h3• 4 years ago

The whole thing with niche languages remaining niche because they don't get access to popular libraries has been a thing for a very long time. It was actually worse before OSS became dominant, because proprietary libraries generally shipped as binaries, and so you'd be limited not only to specific languages, but to specific implementations and versions thereof that'd support the ABI.

thom2• 4 years ago

I think the thing people most struggle with in Clojure is that it looks like it should be an incredibly pure, deeply opinionated language but it’s actually extremely pragmatic. There is never going to be a long lived, vertically integrated, do everything framework for Clojure. If it was going to happen it would have already. Rails took off because burnt out enterprise Java developers wanted to quit and have fun again. Clojure happened because enterprise Java developers wanted to do enterprise Java better.

roguas3• 4 years ago

This is true, clojure is not super opinionated as people tend to think. Its nice, as you get to ramp up easily. There seems to be a steady path where you start with writing "java" in lisp and end up with pure functions and 1 reduce + something to trigger side-effects. I have worked with 4 codebases and they differed vastly.

cutler7• 4 years ago

Apples to oranges. You're comparing a framework to a language. Ruby itself was born out of the desire for developer happiness. It doesn't follow logically that momentum for a large framework could not have evolved out of Clojure's origins.

polote• 4 years ago

If you need a Rails, why not directly use Rails? For what kind of web application Clojure is better than python or Ruby or nodejs?

LesZedCB• 4 years ago

its not object oriented, for one.

i work in rails, and my main hobby lang is clojure. i hate the thinking patterns of OO more and more every day.

jayceedenton2• 4 years ago

In my experience Clojure teams use high quality Java and JavaScript libraries all the time. It's more often than not a piece of cake.

'Clojure needs a rails' has come up perennially for that last 10 years. I don't disagree, as I think convergence on one web solution rather than a constant churn would have great benefits. The idea that you can't use sysdig, or whatever, because there is no Clojure library, is nonsense. Interop is good and Clojure is nicer than Java for writing Java :)

Also, you can overload functions with further aritiss in protocols just fine. Nothing to do with variadics.

janetacarr15• 4 years ago

I never said it can't be done. I did write this in a hurry, so I may have muddled my central thesis.

I'm arguing two things:

First, Clojure has missing or rotting libraries, requiring re-implementing logic we've all done at least once because people are fixated building the Rails for Clojure.

Second, that the business case for using Clojure becomes weaker because the lack of open-source efforts on this front (due to the 'Rails Vacuum') causes either interal library building (extra time) or, more likely, interop. Interop, to _some_ management, sounds an awful lot like using Java, but with very expensive Clojure developers to do it.

huahaiy14• 4 years ago

What are the "missing or rotting libraries"? Maybe a few examples might be helpful. In my over 10 years of using Clojure in production, I am not sure I have "re-implemented" a lot of logic that are not already covered by some Clojure libraries. Such examples are rare.

Sure, I have used Java interop for working with Stripe, but that's because it is futile to maintain a 3rd party Clojure library if Stripe decided not to do it themselves. These proprietary APIs are constantly changing. It is perfectly OK to do Java interop in these cases. Any niche language will have the same problem, and Clojure interop with Java is quite pleasant to use, compared with using Java itself.

As far as I can tell, using ring and related things are fairly standard on the server side, and re-frame and related things are fairly standard on the client side, so I am honestly not sure what is missing for Web development in Clojure, as Web development is the most common use case of Clojure.

I am honestly confused.

jacobobryant• 4 years ago

It seems unlikely to me that efforts to build web frameworks are causing a decrease in time spent building wrapper libraries. Developer time (particularly, open-source developer time) isn't a homogeneous commodity; people tend to work on things that they personally need. Anecdotally I can say that if I were to cease development of my own Clojure web framework, I would not immediately start writing a Syslog library instead :).

wirrbel4• 4 years ago

I developed for fun in Clojure and ClojureScript probably for 3 years, it was tremendous fun and got me into Lisps. In many ways Clojure does still hit the sweet spots from a language design perspective for me.

I have revisited Clojure every now and then after and IMHO its in a weird state, too popular and successful to completely die, but also limited to reach wider adoption.

When revisiting old projects I have seen countless of Clojure dependencies having died in the meantime. I have replaced utility libraries with more recent utility libraries sometimes several times. Very annoying. Paradoxically the core API is really stable to a degree that bugs aren't fixed that are a behaviour userland code might rely on.

But its overall I think more of a symptom than a cause for frictions when developing Clojure. Community management is bad, even worse that RH nourishes this elitist aura around himself. Then the JVM integration is of course the blessing that led to Clojure's initial success but is at the same time limiting. The Java community by majority seems to gravitate towards Scala (weird scala version incompatibilities incoming, Scala DLL Hell) and Kotlin.

So overall Clojure is in a strange situation. For picking it up on-the-job its not the responsible choice probably for most situations I work in. For my private tinkering the JVM dependency feels just a little too heavyweight. I don't see that Clojure will hit an inflection point of adoption in any way in the future that would make it a viable candidate for me to use at work.

I really would hope that a Clojure-on-guile or Clojure-on-racket implementation becomes ready at some point that would make me want to switch from Scheme to Clojure for the toy projects at least.

lgessler• 4 years ago

Paradoxically the core API is really stable to a degree that bugs aren't fixed that are a behaviour userland code might rely on.

Can you give an example? As a medium user of Clojure I struggle to think of anything in the core that could rightly be considered a "bug" and fits this description.

wirrbel4• 4 years ago

Its been a while, I remember some strange behaviour of generic functions on sets* that differed from the behaviour in lists and bug reports about it.

casion2• 4 years ago

Community management is bad, even worse that RH nourishes this elitist aura around himself.

That's a weird take, considering it's consistently called out as one of the most positive aspects of Clojure on surveys.

wirrbel4• 4 years ago

I assume this is on community management. I am not taking part in Clojure surveys so its fairly obvious to me why my opinion is not reflected in the community surveys.

I got to admit that maybe community management might have improved in the last few years.

freshhawk• 4 years ago

It seems to come down to different views on what this means. Some people want leadership who will listen to user demands and apply pressure to the community to push them to work on things in those directions.

Clojure has more leadership along the lines of "hey, other devs, I am open sourcing this cool stuff that I think is good. Here is my philosophy on why I like these types of libraries and would love contributions if you agree with the direction. I'm happy there are other directions being taken as well, that's great. Oh, also, some non-contributors have been making demands that I spend time on their problems and my answer is a firm no."

I think it is a hugely positive aspect, but I also open source my own libraries and have the same feelings towards people who make demands of me without being willing to contribute so I'm certainly biased.

cr__• 4 years ago

Community management is bad, even worse that RH nourishes this elitist aura around himself.

I have never seen Rich say or do anything that matches this description.

roguas3• 4 years ago

I think RH emits fun grumpy vibes, the good kind. Seasoned dev who is tired that his way of dealing with programming is not more prevalent. Very down to earth, lets just do it better.

krn3• 4 years ago

When revisiting old projects I have seen countless of Clojure dependencies having died in the meantime. I have replaced utility libraries with more recent utility libraries sometimes several times. Very annoying.

Yes, the "best in class" libraries change every few years in Clojure's universe, but they always bring huge improvements that require completely new APIs.

For instance:

https://github.com/plumatic/schema -> https://github.com/metosin/malli

https://github.com/juxt/bidi -> https://github.com/metosin/reitit

https://github.com/bhauman/lein-figwheel -> https://github.com/thheller/shadow-cljs

https://github.com/stuartsierra/component -> https://github.com/weavejester/integrant

https://github.com/technomancy/leiningen -> https://github.com/practicalli/clojure-deps-edn

So overall Clojure is in a strange situation. For picking it up on-the-job its not the responsible choice probably for most situations I work in. For my private tinkering the JVM dependency feels just a little too heavyweight. I don't see that Clojure will hit an inflection point of adoption in any way in the future that would make it a viable candidate for me to use at work.

I think about programming languages as tools for different bottlenecks.

– Computational: Rust

– Networking: Go

– Business logic: Clojure / Python

And for many people Python has everything that Clojure is missing: a very low barrier to enter, and a very stable and well-documented ecosystem of frameworks and libraries.

huahaiy14• 4 years ago

"very stable" and python in the same sentence? I am not sure about that.

mdemare• 4 years ago

I think about programming languages as tools for different bottlenecks.

I’m going to steal this!

cutler7• 4 years ago

Don't forget Clojure's libpython.clj. There's also Hylang which was inspired by Clojure.

huahaiy14• 4 years ago

"countless of Clojure dependencies having died"?

What exact does it mean by "died"? It does not work any more? Very unlikely.

In the Clojure world, it is the norm that a 5 years old library works as well as a one-week old one.

If you mean "died" by "not updated any more", sure that's the norm in Clojure as well. Some libraries are done, and some authors moved on. But most likely, these libraries can still be used just fine.

My production code base contains plenty of libraries that are not updated for ages, but I consider that a benefit, not a fault.

hota_mazi2• 4 years ago

From everything I've seen these past years, Scala is on the same path to extinction as Clojure is.

Java keeps getting more popular, pushed by the very solid innovations that Oracle has been trickling into it every release, and Kotlin has certainly picked up a lot of momentum these past five years.

liveoneggs• 4 years ago

This comes up a lot and, I think, Laravel provides a great model for competing with rails. Stripe integration - https://laravel.com/docs/9.x/billing

fmakunbound3• 4 years ago

This might be what happens when the community seems to follow “composable libraries over frameworks”. You still have to do the composing, and it’s not obvious how to do that.

ejflick• 4 years ago

This is the one thing I found that prevented me from getting far with Clojure. I love the language and the concepts...but there's no stable well-documented go to tool that I can use to hit the ground running. Instead, I spent so much time looking at blog posts and random videos trying to cobble together some sort of system of various libraries to form the basic functionality of a web framework just to get to a point where I could actually start working on the meat of the application. Luminus is extremely helpful but I'd also end up having to learn about the individual pieces and found it hard to add libraries after I started my project.

The pervasive "compostable library" mindset also completely ignores the other benefits of having a standardized framework such as Rails that people can rally around: 1. Standardized documentation 2. Being able to create community that can provide support when you're having issues 3. Easily google-able solutions to common issues 4. A standard that can be iterated upon. I feel like Clojure libraries do this somewhat already so they can work with each other so what's the harm on standardizing these interfaces?

stazz12• 4 years ago

I reckon nobody opposes your fine idea, simply that when a template is transformed enough to be a useful web app starting point, it's often too opinionated to be generally applicable! But maybe the next time I start a few web apps I will think about how I can keep it a clean template as long as possible -- might go somewhere cool.

janetacarr15• 4 years ago

Since it keeps coming up, I'd like to clarify that I'm not advocating for a port of Rails or some kind of analog. I should have chosen a better post title.

I have never used Rails.

I'm arguing Clojure has missing or rotting libraries, requiring us to re-implement logic we've all done at least once because people are fixated building web frameworks, and I think it's motivated by people trying to build the Rails for Clojure. I really can't think of any other reason for people to build so many web frameworks at this point.

The Rails for Clojure is an idea, one the community rallies around, one where people won't reinvent the wheel over and over again, so that we can have a lot more non-web packages for Clojure.

I'm quite literally saying, the effort could be better spent elsewhere.

clusterhacks4• 4 years ago

I believe what you see with multiple web frameworks may more often be the misguided attempt of Clojure newbies to reimplement what they already know instead of kind of stepping back and understanding what they actually already have available via Clojure.

I can't find the reference anymore but I think Stu Holloway once said something along the lines of there is a tendency of new devs picking up Clojure to try and rewrite Rails. And that it was a mistake.

The real concern you express about Clojure having "missing or rotting" libraries is harder to quantify. I will say that has this has not been my experience at all. I actually find that I write one-off programs very frequently with Clojure - to call web services (REST or SOAP), serve a http-friendly API via liberator, and other similar work. Grabbing an older library that works seems to be quite easy most of the time.

I do think in the early years of Clojure some groups were looking at Clojure as if it could become what Ruby was to Rails. Meaning, could you use it almost as a marketing tool, e.g. "our devs use Clojure and thus are more powerful." It isn't/wasn't a terrible idea from that perspective but I do think that is one that has somewhat fallen by the wayside. The motivation behind some of the multiple "Rails-like" Clojure libraries may more have been an effort to build a new 38 Signals and not so much fix a real need for a web framework in the Clojure community.

Rich Hickey was looking at Clojure as a better tool for thoughtful software development and maybe even planning for what he might build (like Datomic) with it. I remember at the first Clojure/conj conference he gave a talk and expressed some amazement that other devs were already using Clojure to do production-like things. Everything since that time may have been an interesting side effect of Clojure being a lovely language for devs who were already strong with Java or Javascript so the interop was not a big deal. And its in the interop where maybe most devs should look for these "missing or rotting" libraries?

seancorfield6• 4 years ago

Most of the Clojure web frameworks I've seen have been designed and built by fairly experienced Clojurians -- but I'd have to go read each of those frameworks' rationale docs to see _why_ they actually built them.

newlisp2• 4 years ago

Can you share how you approached calling SOAP services from Clojure?

clusterhacks4• 4 years ago

For posterity, I found the quote I was thinking of from Stu:

"what we would quite often see is that people would write a Java app in Clojure, or people would write a Ruby on Rails app in Clojure. And not surprisingly, it would have weaknesses that you associate with idiomatic Java apps or idiomatic Ruby on Rails apps because that's, in fact, what they are."

https://www.thoughtworks.com/insights/podcasts/technology-po...

dustingetz2• 4 years ago

clojure is about simple composition not magic, so any solution starts by solving frontend/backend data sync in a composable way. Absent that you are just in the tarpit so most commercial shops might as well be in the typescript tarpit with everyone else and benefit from the larger ecosystem.

didibus15• 4 years ago

Next on her docket was the Stripe payment integration. Should be the same thing as the website. Suzi sees there are a number of Stripe Clojure libraries to choose from

Stripe is just a bunch of rest calls, why are you looking for a useless wrapper that only builds up HTTP requests?

I'll sound like a true Clojurian, but this is just the kind of complexity Clojure eskews. You don't need an SDK for Stripe.

And if you want one, just use the Java SDK directly that's well maintained probably by Stripe itself.

That's why each time you looked at the libraries they barely did anything, because there's barely anything needed to do.

That's also why if you look at the cognitect aws-api you'll see they've basically un-frameworked all the complexity from the official Java SDK, they've turned it back into plain http requests where all you have to do is put the JSON payload together and send it.

only to find there is no syslog server library for Clojure that she can find

To be honest, seems weird to not just use a off the shelf syslog server, but still this seems a kind of niche use case, if you look, most languages don't seem to have great existing syslog server libs. It's kind of surprising that Go does to be honest, this might be one of those where you bite the bullet.

Now, it does look like you can use Spring, and I don't think spring is very hard to use with Clojure, but I recognize there's no guides or tutorials to go by.

If you look at Java, Syslog4j is 10 years old without commits, so I mean, if Java even doesn't have an actively maintained syslog server, I'd say this really isn't a good example, why Go has an active one I have no clue haha, that's the more curious part.

Now, it also does seem like it's just some UDP protocol, and while that's more complicated then an http call, I still think there's a lot of these where if people learned how to just do them, again you wouldn't need a library, the idea behind syslog is how simple the protocol is to implement.

Suzi estimates the project with Go instead. The business case for Go is very strong now. Sh*t.

What was the estimate with Go as compared?

but it's really hard to sell management on building an entire library when another language has so many batteries built in

I didn't resonate unfortunately, I have the opposite feeling, I always find a Java or JavaScript lib for anything they want (well syslog maybe not haha), if there isn't a Clojure one. And that's the only reason why I can get away using Clojure, where I could never use Elixir, Haskell, CommonLisp, or any other language that haven't blown up yet.

When it comes to management, I don't say I'm building a library, I just do the work. Implement the behavior or feature and if I made it into a reusable library then that's just benefit on the next project where I might reuse it.

In the end, I'm always faster with Clojure, because it just tends to require so much less code, ceremony and the REPL is a super power for quickly exploring and iterating, at least for me.

If we used interop for everything mundane, Clojure would really just be an S-expression shaped husk over Java code. Not a very good solution.

It's an amazing solution. That's exactly why Clojure is so awesome. Doing it all in Java or Go or JavaScript would be 10 times worse. Use Clojure for your business logic, the core of your app, what gives your business true value, and delegate the mundane to old boring and chunky like Java and JavaScript, but now you have a much nicer language, syntax and a REPL on top. What's not nice about this?

A Clojure API might be slightly nicer, but often wouldn't, because the mundane generally is intrinsically messy, it's often IO and is all about side-effects. It's also so easy to make your own Clojure API, like just wrap things in a function done. Now you use that function instead everywhere.

People act like interop is this complicated, difficult, and effortful thing, that's going to drastically slow you down and render you code unreadable and unextandable. That's not true one bit. The interop is pretty damn good, and at least for me, in terms of the time to delivery, it takes me no longer to use a Clojure API that the interop directly with the Java API. But just like everything else, you have to learn how it works and then you're good.

piggybox• 4 years ago

Java has Play

Is Play still (or ever was?) the go-to web framework for Java?

didibus15• 4 years ago

It's awesome, but can we please just get together at the next Conj conference decide what our "Rails" is going to be? That way we can get down to business

I like the rallying call here. I do think it be nice for the community to do whatever to help Clojure's viability and popularity.

That said, my issue is that I disagree with the approach. I've already moved on to business and I use Clojure.

I think this is the clash the community faces. On one side you've got the people who seem discontent with the libraries approach, and refuse to embrace the hosted nature, they want a native Clojure, no reliance on interop. They might not know Java or JavaScript and might be unfamiliar with their runtimes, or simply discontent with them and don't want to touch them ever.

Then you've got the other side of the community, I think where I fall in, I want to do things my way, don't want to learn any framework, instead I'll just do it like I feel and it'll work better for me, as I'll know it ins/out. I dislike framework, I'd rather just straight up use things directly, I want as little indirection, and I have no issue using interop and having a few wrapper shims when needed.

That side of the community is just delivering business as usual with what's already there, and it doesn't seem much more is needed.

Unfortunately, I also think there's another dimension between this dichotomy, the latter group, where I fall for example, I think is leaning more experienced, senior, team lead, Clojure is their third to fourth professional language, they've settled in it. The Stackoverflow survey did show Clojure had the most senior leaning user base, and probably because of that is also highest paid.

I think this is an issue, because take me, in Java, C# or some other language I've worked in professionally, I'm all up on advocating for use of the frameworks, the DI, the design patterns, and I'll contribute. These languages are too heavy, you can never get anything done quickly or without the man-power, and you don't want too, it's not fun. So you need to rally yourself behind one thing, even if it's not perfect, there's no room for experiments.

The reason I'm using Clojure is to get away from that though, it's for the freedom to experiment, the ability to within the business budget, be able to deliver the same amount yet custom built half of it exactly as I want.

I couldn't get away with that in Java, I'd miss deadlines.

So if I assume half the community is like me, I think that's a lot of people who don't want to focus on one framework, or one way to do things.

fny• 4 years ago

I often wonder whether we're about to move from the golden age of software into the golden age of bitrot. Software ate the world. There's been enough money sloshing around that programmers have had the luxury to develop umpteen tools in umpteen languages that solve the EXACT SAME PROBLEM. At some point the party will stop and we're bound to run out of people willing to prep for FizzBuzz interviews just so they can support a ball of Clojure/JavaScript/PHP/Python/Ruby.

xwdv• 4 years ago

One of the hurdles to having a true Clojure on Rails is finding a suitably synonym for “Rails” except starting with a C.

Perhaps Clojure on Causeways or something?

quickthrower2• 4 years ago

What, JVM language with no ffi?

d_t_w• 4 years ago

If we used interop for everything mundane, Clojure would really just be an S-expression shaped husk over Java code. Not a very good solution.

Interop into the host system in Clojure is idiomatic. I think often there is a misunderstanding of the difference between information systems and artificial ones. Rich Hickey gives a plain english explanation in section 2.1 of his fantastic talk "A History of Clojure". [1]

I have been using interop into Java for the mundane for ten years. Our general approach is to simply use p/datafy to convert the result of interop calls immediately back into Clojure datastructures.

I don't find that approach onerous and it doesn't impact our overall system design any more than using a third party DSL. The interop ensures we take ownership of our Java dependencies rather than using an outdated thin DSL wrapper which provides no real value anyway (as I assume the Stripe example would be). In the past where I've worked on projects with those wrapper DSL I tend to throw them out, they're the wrong idea.

In my case I use stable, mature, artificial systems provided by Java to underpin my application. My application is an information system, where I use the lean power of Clojure to implement complex functionality without the drag of Java.

The result is an echo of the Destroy All Software talk "Funcational Core, Imperative Shell" [2], but rather more a "Data-Oriented Core, Type Oriented Shell". The end goal being to enable my small team to ship better and faster than our competitors

The most important artificial system in our application is Jetty, the networking framework. The next is the Kafka Client / Streams API. That's our Type Oriented Shell. The magic of the application is Clojure, everything in between. The glue is interop, p/datafy, and transit.

In some cases the magic, value-adding, bespoke software part a developer is attempting to add between the Type Oriented Shell parts is tiny, and the result feels like an S-Expression shaped husk. In my case that magic part is enormous, and the benefit of using Clojure obvious.

If you are shipping an information system, the problem domain is non-trivial, and you are delivering on the JVM, I believe you should use Clojure to attack that problem. For me it has been game-changing in terms of pace and quality of delivery. Right tool for the right task.

Clojure is also enormously more fun and liberating to work in than Java, and you shouldn't underestimate the power of joy as a morale maintainer when working on one problem for a long time. I first started working on our product in May, 2018 and I enjoy working on it today.

For context, I worked in Java for 15 years before adopting Clojure a decade ago so I understand the relative benefits of shipping production information systems in both languages. I'm also probably more comfortable just using Java interop than some Clojure developers.

We have plans to open-source the useful parts of our type-oriented shell and the glue that binds them in the next few months, along with our solution for managing dynamic system state, but then again I've been faintly promising that for three years so take it with a grain of salt.

The application I work on is enterprise-grade tooling for Apache Kafka: https://kpow.io/

[1] https://download.clojure.org/papers/clojure-hopl-iv-final.pd...

[2] https://www.destroyallsoftware.com/screencasts/catalog/funct...

b123400• 4 years ago

Not exactly Clojure-related, but I wonder how acceptable is it for a programmer to suggest a technical decision by one’s own passion over company’s interest.

The business case for Go is very strong now. Sh*t.

It sounded like a negative news for Suzi as she wanted to promote Clojure, but it’s positive for the company, as Go is a more suitable choice in this case. If a programmer is asked to make a technical decision, comes up if a solution based mostly on passion, isn’t it misleading? I’ve seen a Haskeller claiming “this thing can be better expressed in a custom DSL” and started to write a parser, which in retrospect absolutely unnecessary, the passion was so strong that decisions got irrational. Shouldn’t a programmer be honest and say “this is not the best tool, but it’s my favorite, using it makes me happy”?

jmcgough3• 4 years ago

I think it's a sign of maturity as an engineer to realize that your favorite language might not be appropriate for a particular business problem.

hota_mazi2• 4 years ago

This is bigger than a single individual.

Maybe this technical choice makes that person happy, but what if it makes ten other developers on the team unhappy, or unable to do their work because they don't know this new technology?

What if that technical choice makes the whole code base unmaintainable in the following months because it gets abandoned, or is just so niche that it's impossible to hire developers familiar with it?

Jeaye• 4 years ago

I disagree that Clojure needs a Rails, in the sense that there's a single framework to use for all web server needs. However, I do really think Clojure needs a better documentation story for web development across the board.

Getting a ring + reitit + malli + integrant + core.async + vertx-mongo server running requires putting together a _lot_ of different pieces. A framework is one way to solve this, but so too is documentation. Perhaps it's just my Clojure mindset, but I'd be hesitant to reach for a one-stop-shop framework for all of the above, but I would definitely be interested in some docs which helped guide me hook all of them together. Is there such a thing as a documentation framework? :)

-- Thinking about how Clojure devs feel about libraries makes me realize how ironic it is that so many of them use Emacs.

teodorlu• 4 years ago

but I would definitely be interested in some docs which helped guide me hook all of them together. Is there such a thing as a documentation framework? :)

What's the first step to making those docs?

There are resources floating around, but they can be hard to find. I really like the docs for http-kit and luminous - they are cohesive.

uxcolumbo• 4 years ago

I would definitely be interested in some docs which helped guide me hook all of them together

Yogthos put together this guide - in book format:

https://pragprog.com/titles/dswdcloj3/web-development-with-c...

Additional links:

https://yogthos.net/posts/2022-01-08-IntroducingKit.html

https://kit-clj.github.io/

https://luminusweb.com/

valyagolev• 4 years ago

Lein templates are a great way to get started with a lot of pieces already in. Perhaps there's already a template for your needs

tus666• 4 years ago

"It'll probably take two months to build this relay because there's no syslog server library for Clojure." she says.
"Oh wow, I didn't think it would take that long." Sam says.

The correct answer should have been:

"Who asked you to write it in Clojure?"

stazz12• 4 years ago

More like we want to use it for everything because it's awesome

otikik• 4 years ago

Lua never got a Rails, but it got a Grim Fandango (the best "mainstream" video game that shipped it [1]). That game marked an inflection point in adoption of Lua as a scripting language for video games, and that expanded to other areas afterwards.

Perhaps it worth exploring what other niches like that would make sense for Clojure. For example: It seems to me that its lipsy nature would meld well with artificial reasoning and perhaps AI. Some sort of PyTorch competitor, perhaps? How is the Java ecosystem with regards to AI libraries?

[1] https://es.wikipedia.org/wiki/Grim_Fandango

danjac• 4 years ago

There's no "Rails for Go" either, so not sure why that is brought up as a comparison.

I suspect (from dabbling in Clojure some years ago) that the Clojure community has more enthusiasts than practical users, who enjoy building stuff that meets a Platonic ideal of language purity instead of the messy work of supporting long-term libraries that are far from Lispy perfection but meet the needs of real world projects.

jwr• 4 years ago

From a point of view of someone who wrote and maintains a big SaaS written in Clojure and ClojureScript:

* The libraries could definitely be improved and we do need better approaches to web apps.

* But we do not necessarily need all-revolutionary-totally-different-reinvent-everything approaches that require you to throw away everything you have ever seen.

* "Last commit 3 years ago" is not necessarily a problem.

* I've spent an incredible amount of time on things which I expected would have been solved a million times. Stuff like session management, authentication, webhook handling, all of which become way more complicated if you use things like websockets.

* While there, I learned that it's very difficult to come up with abstractions that handle all the edge cases. Sooner or later you need that webhook that is authenticated differently, some API calls do not need authentication, and some of your upper layer code needs that request source IP address, because the EU said so in the laws for handling VAT.

Also, even with all the reservations, I still can't imagine writing and maintaining my app in anything other than Clojure and ClojureScript. It would no longer be a one-person project.

So, while I would like to see more stuff being developed, I am not sure if a "Rails for Clojure" makes sense, and I have no idea how to write one. I have a suspicion I am not the only one thinking along these lines, and it could be one of the reasons why we don't have anything of the kind.

Instead, what I do is contribute a little bit to various Clojure funding projects, either directly to developers or to organizations like Clojurists Together.

carl_sandland• 4 years ago

Apologies if this is a bit daft, I just experimented with Java/Clojure interop and I think most people here are talking about Clojure/Java interop: note the different order. I built a Java 'application' that uses Java's rich libraries to handle all the "I/O stuff and interactions", and called out via clojure.api to a clojure functional core. This seemed to work very well.. But I can see this means learning two languages (not 1.5). Just wondering if many people do this?

Get the Java code to collect a bunch of abstract data values and feed it into the functional core, which is pure clojure with very few libraries at all.

blain_the_train5• 4 years ago

yes, everyone in these communities does this to some degree:

typescript, coffeescript, scala, kotlin, etc...

blain_the_train5• 4 years ago

err. i think i misunderstood your question.

z9znz• 4 years ago

There is the Kit framework https://kit-clj.github.io/ which looks promising. It is rather new, and the documentation is quite limited, but I think it has a good chance at growing into something new people would find an accessible starting point for Clojure web development.

There is also a project called Coast which appears to be a direct attempt to make a Rails-like framework. However, the repo activity seems ... quiet. Granted, Clojure is special for having a lot of libraries which are quite old and appear dead but are instead apparently complete or satisfactory, and as such they don't get new commits.

I'm still waiting for a configurable platform (in any language) which allows you to specify the things you need, and it provides the working foundation. It then has a plugin structure so you can build your business logic in any of several languages, and importantly it doesn't suffer from the usual code generation problem where it only works one time (the first time). Wordpress is sort of this, but it's slow and architecturally awful.

justnotworthit• 4 years ago

I've used lots of little utilities and scripts in many languages, but I can't recall running across a clojure project. Occassionally java of course, but nothing with source code in clojure. Is clojure more serverside/business?

Clojure needs a Rails

Related Stories