Google and Binomial Open-Source Basis Universal Texture Format

https://opensource.googleblog.com/2019/05/google-and-binomial-partner-to-open.html

by jasondavies • 7 years ago

265 20 7 years ago

opensource.googleblog.com

edwintorok• 7 years ago

There is a paper describing the technique here: http://gamma.cs.unc.edu/GST/gst.pdf

Scaevolus• 7 years ago

That paper described the inspiration for Basis, but Basis itself is slightly different.

https://github.com/binomialLLC/basis_universal#quick-basis-f...

Internally, Basis files are composed of a non-uniform texture array of one or more 2D ETC1S texture "slices". ETC1S is a simple subset of the ETC1 texture format popular on Android.

yingw7872• 7 years ago

This is amazing to me from an outsider's point of view; I thought Binomial was a two-person shop (idk if they expanded recently). To be able to partner with Google and Chrome Media on an equal level just doesn't seem possible to me. Binomial must have some pretty nice tech. Congratulations on the partnership!

sanxiyn• 7 years ago

They are still two persons. https://twitter.com/sehurlburt/status/1131371251790835712

yingw7872• 7 years ago

Thanks for the link!

flohofwoe• 7 years ago

Well, they fixed a problem none of the GPU makers is interesting in solving directly: providing a common image compression format for 3D rendering. This is all a patent-encumbered mess, and the reason we are in this mess are the GPU vendors alone.

When Microsoft still had power over desktop computing, it pretty much dictated GPU vendors to use the DXT image compression format which is supported on all desktop GPUs, but nothing similar has happened on mobile.

In the current situation, this problem couldn't have been solved from the "inside". It isn't a trivial problem by far to solve (probably one of the few topics in programming that deserve the name "rocket science"), but it also doesn't require a huge workforce or standardization committee (and thank goodness this hasn't been "designed-by-committee"), "just" many years of focus and dedication.

nod• 7 years ago

Congratulations to https://twitter.com/sehurlburt ! ...who is also a font of interesting and insightful business advice and life/growth thoughts

emit_time• 7 years ago

Seconded. Very friendly too :)

modeless4• 7 years ago

Compressed texture formats are underused. I think few people outside of game development understand the benefits. The linked post explains it but I'd like to take a stab at it too.

Regular image compression formats like PNG or JPEG save network bandwidth and disk storage but not RAM. They must be completely decompressed and stored uncompressed in RAM as long as they are displayed. Compressed texture formats do save RAM. They remain compressed in RAM while they are being displayed, typically 1/6 the size of regular images or even smaller. That saves not only RAM but memory bandwidth and power too; especially important on mobile. Switching to compressed textures can make applications significantly faster.

Compressed texture formats have two main disadvantages that Basis solves. One is that their compression ratios are fixed and not as good as JPEG. Basis adds extra compression to make them more comparable to JPEG on disk.

The other, larger issue is that compressed texture formats require hardware support on the GPU, and GPUs don't agree on what formats to support. There is no single format with guaranteed support on every platform, which means that cross platform applications must encode each image multiple times and ship different formats to different users. Basis solves this by efficient runtime transcoding to most common GPU formats. This capability is unique and a really big deal.

Anyone with images in their application should take a look at Basis. Now that you can ship a single compressed texture file that's supported everywhere, I hope that this technology can expand beyond games.

throwaway2048• 7 years ago

Part of the issue is they have been patented to the hilt by parties like S3. It hasn't been an issue in video games because companies just agree to implement other S3 apis in return for a patent license, but in cases like this that obviously isn't an option.

https://en.wikipedia.org/wiki/S3_Texture_Compression

It has been an obstacle for open source drivers too, but luckily this specific patent expired.

markdog122• 7 years ago

beyond games

Accept: image/basis

CMCDragonkai• 7 years ago

Could this be used for convolutional neural networks?

modeless4• 7 years ago

I'm not sure where it would help. The encoding process is slow so it wouldn't make sense to encode anything that changes like the inputs or activations, or the weights during training. You could maybe encode the weights after training but you'd really have to massage the data to fit into these formats, and the lossy perceptual tricks used here are designed for the human visual system, not high-dimensional convolution filters.

ggg2• 7 years ago

is it lossless? what is the trade off?

modeless4• 7 years ago

No, compressed texture formats are lossy. Also the fixed bitrate that allows random access into the compressed pixels constrains the encoder a little more than the variable bitrate of JPEG or other image formats.

60654• 7 years ago

From this [1] it sounds like the texture is a modified variant of ETC1, which gets loaded up from disk into memory as such, and transcoded on the CPU into target GPU format (DXT, PVRTC, etc), and then uploaded to VRAM.

If so that sounds similar to their previous Crunch library [2], in that it does transcoding on the CPU before moving the texture to the GPU - but most likely much faster than Crunch because it starts with ETC as shared basis. (And artifacts are a common with transcoding between compressed formats, especially when transparency is present, but I have high hopes for this one considering the authors! :) )

Authors suggest that a version that transcodes on the GPU would be possible - that would be really interesting, I would imagine that would be even faster, although at the cost of higher texture memory usage.

[1] https://github.com/binomialLLC/basis_universal#quick-basis-f... [2] https://github.com/BinomialLLC/crunch

modeless4• 7 years ago

Crunch only supports one family of formats, DXT. (Unity has a fork that supports the ETC family, but it does not support decoding ETC and DXT from the same compressed file). Efficiently transcoding one compressed file to multiple compressed texture formats at load time is the key new feature that only Basis provides.

markdog122• 7 years ago

Three.js already supports it: https://github.com/mrdoob/three.js/pull/16522

manigandham• 7 years ago

Is this going to lead to a new general image format? What happens with webp? That seemed to have never taken off much because of legacy support.

ttoinou• 7 years ago

Is anyone thinking of adapting this create a new GPU-based video format ? Like HAP https://hap.video/

Google and Binomial Open-Source Basis Universal Texture Format

Related Stories