JEP 457: Class-File API for Parsing, generating, transforming Java classfiles

https://openjdk.org/jeps/457

by carimura • 3 years ago

102 34 3 years ago

openjdk.org

spreiti• 3 years ago

I highly recommend you to watch this video by Brian Goetz if you want to learn more about this JEP.

https://youtu.be/pcg-E_qyMOI

shellac2• 3 years ago

And as a companion piece I really enjoyed Paul Sandoz's 'Code Reflection' talk https://www.youtube.com/watch?v=xbk9_6XA_IY, which considers how java code might understand java code. The sort of thing that enable pushing functions to SQL or GPUs for example.

Edit to add: slides here https://cr.openjdk.org/~psandoz/conferences/2023-JVMLS/Code-...

bafe3• 3 years ago

Interesting, but I don't see how operating on the code model is going to be any easier than working on the AST. It seems much more hard to reason about many transformations as compared to manipulating the AST. I do hope they take inspiration from scala and rust macros

waynesonfire• 3 years ago

At round 8:50, Goetz talks about the design, in essence the quote is,

"We designed it as a functional library because functionally inspired libraries are successful at meeting these goals."

Ironic.

za3faran• 3 years ago

Why ironic? Java is a multi paradigm language (though obviously not at the same level as say C++). It has been getting "functional" features for a while now, including streaming libraries, pattern matching, immutable records, etc.

wpollock• 3 years ago

Will this enable lombok officially?

cogman102• 3 years ago

Lombok's headaches are because they are touching Java compiler internals to accomplish their magic. This won't fix that.

Lombok wouldn't be nearly as troubled if it was just doing simple bytecode manipulation.

If lombok wants to stop the pain, then they'll need to stop reaching into internal APIs. They'll possibly need to remove a few features (like some of the `private final` work they are doing).

In other words, you can expect lombok to be a headache for years to come. (Maybe consider not using it? That'd be swell. Speaking as someone that curses lombok everytime jdk updates roll around.)

the-smug-one• 3 years ago

Why isn't Lombok just written as a pre-javac pass? javac(lombok(srccode)). Just gotta keep that parser alive and up to date, which open source IDEs also must so they're probably available.

exabrial3• 3 years ago

Lombok is a compiler extension, where this is an API for the JVM. So two very different things. Think of Lombok/compiler extensions as #include directives in C... things that happen at compile time. This API is for writing programs at runtime, past the compile phase.

pron2• 3 years ago

Lombok is not a compiler extension. Compiler extensions, aka annotation processors, are offered only specific capabilities that ensure that they preserve the Java language specification. Particularly, code that compiles successfully with an extension also compiles without it (perhaps requiring other classes to be available) and it compiles down to the same bytecode. Annotation processors are used to implement pluggable type systems (e.g. https://checkerframework.org) or to generate other classes (e.g. https://immutables.github.io/).

Unlike compiler extensions, Lombok compiles source files that do not conform to the Java language specification. Lombok is an alternative Java Platform language, like Clojure or Kotlin or Scala, except that it's a superset of the Java language. However, rather than forking `javac` source code and modifying it to compile Lombok source files, the Lombok compiler modifies `javac`'s operation by hacking into its internals and modifying them as it runs to compile Lombok sources rather than Java sources.

Having alternative Java Platform languages is perfectly fine. The problem with Lombok is that it doesn't present itself as such but as a library or a compiler extension even though it violates the Java language specification in ways that compiler extensions are forbidden from doing.

xxs2• 3 years ago

Lombok should not be touched, esp. since records have been available. Morealso, lombok is just a sugar coating during compilation time.

Even before records, just use public final fields, and be done with the getter/setter nonsense. (IDEs do a good job of offering options for toString(), and c-tors)

Personally, I consider lombok one of the better anti-patterns widely used.

javanonymous• 3 years ago

Lombok can be used for more than just getters/setters.

I personally use the @Builder annotation on records with more than 3-4 fields. I find it much more readable than a long list of arguments to the constructor.

It also makes it easy to return a copy of the record where only a few fields have changed:

   var r = book.toBuilder()         
           .lastUpdated(now)
           .title("...")
           .build()

I also use other annotations, but I could work without them if a future version of Java provides a builder-like pattern (or named arguments)

ImprobableTruth• 3 years ago

Records are the way to go, but there are many situations where the ecosystem doesn't work with them e.g. if you're stuck using something like Hibernate.

cogman102• 3 years ago

This is the sort of thing that will have a LARGE impact on the ecosystem for updating from one JDK to the next.

One of the big headaches we see with moving up JDK version is bytecode generation/manipulation libraries choking on newer versions of the JDK. It's generally a simple update, but sometimes it's not (for example, when someone is shading asm).

Having bytecode generation as part of the JDK will mean all libraries, including asm, can migrate to that and benefit from a perpetually supported API that updates as the JDK does.

This is probably 80% of the headaches I've experienced going from the likes of Java 11 to 17 and 17 to 21.

exabrial3• 3 years ago

This is pretty exciting... I've used them all libraries at this point in my career: CGLib, ASM, BCEL, ByteBuddy, Javassist, etc... each has its pluses and minuses. I've designed everything from profiling agents, to systems that pack decimals into EBCDIC and invoke COBOL programs on big IBM iron, to lightweight JIT compilers, all using these libraries.

In 2002, the visitor approach used by ASM seemed clever

I couldn't agree more. The visitor pattern was very hard to explain/justify back then, and still difficult to explain to newbie programmers just entering the profession.

Looking at the examples, I think this is going to be an official replacement for ASM, meaning it's going to be pretty low level. The use of streams pretty straightforward.

If anyone from the JEP is reading this: I have two pieces of feedback!

First, take some inspiration from the way CDI Portable Extensions work. This is probably the most delightful extension API I've ever used. The @Observe callbacks are super simple to explain to people and it's really easy to write extensions for the framework.

Next, I wouldn't ignore the need for a higher-level API akin to ByteBuddy or Javassist. Sometimes I just want to write an interpreter or intercept a method call and thats it.

For example in my Junit/Mockito extension https://github.com/exabrial/mockito-object-injection I need to intercept a call to the class under test in order to lazily inject dependencies at the last possible moment. While I certainly could do this with ASM, Javassist makes this fairly simple with it's MethodHandler api.

Side note, it's a damn shame we don't have a mobile operating system that is JVM native :/ All this cool APIs simply never reach a huge number of devices.

vips7L• 3 years ago

I just want to share my support of CDI extensions. They just make sense!

xxs2• 3 years ago

First, take some inspiration from the way CDI Portable Extensions work

It's way too late, the API is effectively locked in preview mode.

While I am not a fan of ASM's visitor pattern, explaining it to anyone would be the least of my concerns, ASM requires pretty extensive knowledge on class structure, method signatures, and most of the byte code instructions. Whoever ventures in byte code editing mode should be able to read the byte code directly.

twic• 3 years ago

How would something like CDI portable extensions fit in here? Are you thinking it would another way to replace ASM's visitors? Doesn't it lack all the useful structure that the design in the JEP as, around streaming and building and transformation and so on?

exabrial3• 3 years ago

I meant the "style of" CDI portable extensions :)

When you write a CDI portable extension, you register a bunch of observation handlers. So as the CDI environment is discovering stuff, it calls your observers and you have the chance to make chances to the runtime. I was thinking something akin to that here... when reading a class, your observers get called and you have a chance to manipulate the class data.

I find this style intuitive.

nvm0n2• 3 years ago

Surely the problem you're objecting to is that Android is JVM native, so the Java APIs are a part of the OS itself.

lemming• 3 years ago

I wonder if this will be available as a standalone library for older JDKs, it looks very nice.

saagarjha• 3 years ago

I’ve always wondered why the APIs for reflection in Java were always so low-level. If you look at, say, Objective-C (a language with a very similar model of Java) almost nobody drops down to assembly to patch things, because the language is expressive enough that you don’t need to do this basically ever. Why not add the same to Java?

JEP 457: Class-File API for Parsing, generating, transforming Java classfiles

Related Stories