Fury – Fast multi-language serialization framework powered by JIT and Zero-copy

by chaokunyang • 3 years ago

142 68 3 years ago

chaokunyang33• 3 years ago

Fury is also the fastest jvm serialization framework in the https://github.com/eishay/jvm-serializers/wiki

Weird they aren't emphasizing competitiveness with Protobuf and Avro, which is what I'd look at long before java serialization if I wanted fast object serialization.

chaokunyang33• 3 years ago

Protobuf/Avro doesn't support polymorphism and circular references, maybe that's one of thge reason? But even compared with protobuf, fury is 3.2x faster. When comparing with avro, fury is 5.3x faster

chaokunyang33• 3 years ago

See https://medium.com/@shawn.ck.yang/fury-a-blazing-fast-multi-... for fury implementation details

usrusr• 3 years ago

Just imagine the project landscape you'd need to be in to consider starting this worthwhile, given all uncertainty of success and so on. Or a very large middle finger to YAGNI, but I doubt it's that.

chaokunyang33• 3 years ago

Good point! The success of a project is not only determined by what it can do, but more determined by what it chooses not to do. Only with clear and simple goals can we focus on what truly matters, while also attracting more developers to join our community.

chaokunyang33• 3 years ago

But I want to say that Fury will be continuously maintained. I created it 4 years ago, open sourced it in 2023.07. I've maintained it with little credits for 4 years. Now with it open-sourced, it will just make me put more efforts on it.

But on the other hand, the success of fury does not depend on whether I put in enough effort but on whether I can build a thriving community to involve more people to join us. I must admit that I am still learning in this area and have a long way to go.

hardwaresofton2• 3 years ago

Beware of burnout -- if companies take you at your word and expect continuous maintenance (for free), the load might be hard to bear after a while.

The best way to avoid it I know of is to delegate responsibility and find other people to maintain the library with you (and/or charge for development/encourage corporate sponsorships!)

egamirorrim• 3 years ago

OOI, in a 'modern' app, what would you be using this kind of serialisation and deserialisation for? Some kind of complex RPC?

chaokunyang33• 3 years ago

There are 3 kinds of scenarios are serialization bounded: 1. RPC for large microservices application, which are latency&throughput sensitive. The domain model can be huge, serialization will be the bottleneck. Some domain object graph may contains thousands of objects, serializing using jdk will take seconds, and the serialization result may be 50~100kb. Take Order as an example, imagine you need to process an order which has 500 items.

2. Data transfer for bigdata distributed systems: bigdata systems will handle much data, the data needs be transfered between workers. The serialization can be the bottleneck too. Spark RDD，Flink DataStream all have this bottleneck. The use bianry format such as arrow/tungsten format to reduce serialization overhead. But it's limited, sql oritiened, and can't express complex logic such as graph/event/domain.

3. Task scheduling: Image you have a mpp distributed systems, you need to schedule thounds of tasks in a process every sub-seconds. Serialization will be the bottleneck too.

specialist• 3 years ago

Yes and: Kyro was motivated by building multi player games.

npstr• 3 years ago

The comparison to JDK serialization is surprising me. Noone should be using it for anything serious in production. Even the Chief Architect of the Java platform has called it a "horrible mistake". https://www.infoworld.com/article/3275924/oracle-plans-to-du...

chaokunyang33• 3 years ago

There are too many java serialization libraries, I compared fury with jdk/kryo/fst/protostuff/protobuf/flatbuffers/thrift/msgpack/canproto/acro/jackson/json. Fury are fastest too. See https://github.com/eishay/jvm-serializers/wiki for detailed benchmark results.

It's a hard choice to select one for the title, so I use JDK for it, which may be not a good choice.

BTW, fury support jit serialization for jdk17 record, which is super fast compared to other serialization frameworks such as kryo

pron4• 3 years ago

Just to be clear, though, the mistake that Mark Reinhold refers to isn't the particular implementation of the JDK's core serialization, but the design that allows arbitrary objects to be deserialized while bypassing their constructors (and so their established invariants). Unfortunately, many other serialization libraries — faster or slower — are repeating the same mistake. I.e., any serialization library that can serialize any class that implements Serializable suffers from all the same flaws of core serialization.

Correct serialization can only be done for classes that are designed for it by having a well-known construction protocol, which are currently basic collections, enums, Strings, records, and classes that register specific serialization code.

chaokunyang33• 3 years ago

I see, totally agreed! If any objects can be serialized, the deserialization will introduce security issue too. For example, `constructor/equals/hashCode` may contains malicious code, which introduce the deserialization risks.

Fury put much work on this to avoid the open dynamic deserialization risks.

But although `basic collections, enums, Strings, records, and classes that register specific serialization code` are the only objects should be allowed for serialization, and they are serialized by the construction protocol in fury already. There are so many applications has used the existing serialization protocol assumption, we have to keep compatible, otherwise most of application can't use fury.

mkleczek4• 3 years ago

Why?

Zero copy state transfer is a viable and high performance alternative.

Security and integrity can (should?) be implemented at a different layer.

LispSporks22• 3 years ago

There's also a chapter on JDK serialized in Bloch's "Effective Java" worth a read:

    Item 85: Prefer alternatives to Java serialization
    Item 86: Implement Serializable with great caution
    Item 87: Consider using a custom serialized form
    Item 88: Write readObject methods defensively
    Item 89: For instance control, prefer enum types to readResolve
    Item 90: Consider serialization proxies instead of serialized instances

chaokunyang33• 3 years ago

Those pattern are all supported in fury. And it seems fury are the only framework which implement the jdk `writeObject/readObject/writeReplace/readResolve/readObjectNoData` methods except jdk iteself. Other java serialization will jsut ignore those methods, and get incorect results. But today I'll suggest to avoid to use JDK `riteObject/readObject/`. Since to be compatible with JDK serialization API behaviour, it will introduce performance and space overhead. We can register custom Serializer by fury.registerSerializer(xxx.class, XXXSerializer.class).

`writeReplace/readResolve` are an useful pattern, and can be used when needed.

jarym2• 3 years ago

I wonder how this compares with Microstream One which looked to have quite slick serialisation capabilities

chaokunyang33• 3 years ago

I benchmarked with Microstream using jmh and jvm-serializers data. Hare are the results: 1) fury is 43x faster for the speed. 2) Fury is 5x smaller for serialized binary size

Here is the code for reproduction: https://github.com/chaokunyang/fury-benchmarks#fury-vs-micro...

vlovich1232• 3 years ago

How does Fury compare to static schema serialization frameworks that aren’t Java focused? For example, flat buffers, alkahest, protobuf etc.

jarym2• 3 years ago

Oh nice!! I will certainly try this out. Much of my work these days is Java (Kotlin), C# and Python so a good serialization framework is always helpful.

arnold_palmur• 3 years ago

No comparisons to SBE in the benchmarks?

chaokunyang33• 3 years ago

SBE needs to define the xml for the java bean, and generated the code. It need lots of work to implement a multiple-layer nested objects serialization. So I don't added it for now.

belfthrow2• 3 years ago

Should still be added. Are are talking a lot about perfomance so this is where SBE shines. Worth comparing if you want to show of the numbers.

fakecrusade• 3 years ago

Interesting, I work in the same company but have never heard of it

chaokunyang33• 3 years ago

haha, maybe it's time to use it now. It's used wisely in distributed systems at our company. I posted several blogs in our internal ata bbs, you can search it

SpaghettiCthulu2• 3 years ago

I'm curious, why do you appear to generate and compile Java source code at runtime instead of generating bytecode directly or using `MethodHandle`s?

chaokunyang33• 3 years ago

Good question! Part of code are generated using `MethodHandle` such as some field getters, or constructor. But the overall code are generated as source code first at runtime instead of bytecode. This is because binary protocol are very complicated. Generated the bytecode directly will make the troubleshooting more difficult. but it's possible to generated the bytecode techniquely. We have an IR abstraction, it's possible to change it to generate the bytecode

aardvark179• 3 years ago

The danger with generating source code is that you now depend on source rules which may have changed, never applied to your source language, or are made more complicated by class loading refs. I’d definitely go for byte code generation, and possibly using constant dynamic to handle any really tricky constants, over source code any day.

gigatexal2• 3 years ago

can someone help a noob like me understand where i would use something like this? is this a way to get faster json parsing?

chaokunyang33• 3 years ago

Json doesn't support polymorphism and circular/shared references, those scenarios are more suitable for fury. And json is a text protocol, which is inefficient compared to fury binary protocol. Taking jackson json lib as an example: 1) Fury is 41.6x faster than jackson for Struct serialization 2) Fury is 65.6x faster than jackson for Struct deserialization 3) Fury is 9.4x faster than jackson for MediaContent serialization 4) Fury is 9.6x faster than jackson for MediaContent deserialization

For http rest api, json is more suitable. For RPC/bigdata/game/scheduling, fury may be better

gigatexal2• 3 years ago

Thank you chao!

chaokunyang33• 3 years ago

Another issue for json format is that it will bloat the payload. For example, for an int `1111122222` will take 10 bytes in json, but only 4 bytes in fury. `0.3333333333333333` will need 18 bytes in json, but only 4 bytes in fury.

Game_Ender• 3 years ago

Two questions:

- Does Fury support versioning?

- What is source of truth for the schema when dealing with multiple languages and the same data?

chaokunyang33• 3 years ago

Good questions. 1)Fury support versions. 2) The schema will be included in the data, and will be used for deserializing/validating the data. The schema will be sent to peer once for a tpc connection.

RantyDave• 3 years ago

Great that it's fast - but I'm not sure it would make all that much difference in Python code.

chaokunyang33• 3 years ago

For python, the speedup won't be so huge, we get a 50%~100% speed up for dataclasses. We haven't put much efforts on it. In future I believe the speed up will be larger.

knutwannheden• 3 years ago

Interesting. I will be curious to see how it stacks up against Jackson Smile, both in terms of features and performance. Since it appears to support both polymorphism and back-references, it looks like it already has some advanced features.

chaokunyang33• 3 years ago

Yeah, you are right. Fury supports polymorphism and circular/shared references, which are not supported by json/jackson. I did a benchmark with jackson before, here are the results: 1) Fury is 41.6x faster than jackson for Struct serialization 2) Fury is 65.6x faster than jackson for Struct deserialization 3) Fury is 9.4x faster than jackson for MediaContent serialization 4) Fury is 9.6x faster than jackson for MediaContent deserialization

see https://github.com/chaokunyang/fury-benchmarks for detailed benchmark code.

chaokunyang33• 3 years ago

Another issue about json is that json is not storage efficient. For a float 0.3333333333333335, json needs 18 bytes, but fury needs only 4 bytes.

Fury – Fast multi-language serialization framework powered by JIT and Zero-copy

Related Stories