We Need A Standard Layered Image Format

http://shapeof.com/archives/2013/4/we_need_a_standard_layered_image_format.html

by ingve • 13 years ago

246 142 13 years ago

not a huge conspiracy theorist, but isn't this the kind of thing that would make us not rely on purchasing Adobe CS7 exclusively (for fear of incompatability) in the future?

Standards wise, open-source wise, and generally it's a good move (who wouldn't want to get a standard complex image format that would make sense to the common programmer?), but I don't think the finance department over at Adobe is going to like it very much?

rlpb2• 13 years ago

Won't this format leave behind traces of what was in the file before? I'm not sure that this is something that we want in a standard interchange format. Users will assume what they are sending is only what they see in an application. This is reasonable. I am concerned that using this format will lead to data leaks.

angersock3• 13 years ago

That ship has pretty much sailed with EXIF and embedded metadata anyways, right?

This would seem to be more targeted at digital content creation (and toolchain application interop) than a normal-use file format.

rlpb2• 13 years ago

It's up to your editor what metadata it chooses to save. It is not up to your editor what data was in the sqlite row that it just modified or deleted but may still be forensically available. There is a fundamental difference here.

It is possible to clean it up of course - perhaps by vacuuming, or definitely by copying to a new file at database level. But it seems dangerous to me to start with a native format with this behavior and attempt to clean it up later.

eurleif2• 13 years ago

It could be defined in the format spec that applications should VACUUM the database after every save.

seanalltogether3• 13 years ago

How is that different from a psd or editable fireworks png file? I can have several groups with various visibility toggles that any normal user wouldn't see unless they dig into the layers.

SQLite3• 13 years ago

Not if you set "PRAGMA secure_delete=ON;" See http://www.sqlite.org/pragma.html#pragma_secure_delete for additional information.

aceofspades192• 13 years ago

Whats wrong with SVG? Doesn't it accomplish what the OP wants to do already and is already a standard supported by a lot of applications? I must be missing something here

arielpts3• 13 years ago

XML is a really bad format for containing large chunks of binary data (bitmap)

aceofspades192• 13 years ago

And why are we storing binary data? The whole point of SVG is everything is in XML unless I'm very mistaken

WiseWeasel• 13 years ago

The XML can store references to the binary chunks in standard formats though, alongside the vector shapes and filters.

  <svg viewBox="## ## ## ##">
    <image xlink:href="layer1.png" x="##" y="##" width="##" height="##"/>
    <image xlink:href="layer2.jpg" x="##" y="##" width="##" height="##"/>
    <rect x="##" y="##" width="##" height="##" fill="red"/>
  </svg>

All that's left is to standardize a way to package it.

aidos2• 13 years ago

This whole idea seems totally mad from a distance but as a developer I'd love to be able to open complex files as DBs.

What would be the downfalls of passing around a DB as an image file? Does the compression suck? Does the performance suck? What makes this a terrible idea?

EDIT: to clarify, by compression I'm referring to the non-lossy type - I'm assuming that within the db you'd have already processed assets.

thristian• 13 years ago

One downside would be that tools like file(1) that try to guess a file's format by examining its contents would return "SQLite database" for every image file, which isn't the most helpful result.

Since SQLite aims to be a generic way for applications to store data, it'd be nice if its "open database" and "create database" functions allowed you to supply, say, a 4- or 8-byte magic value that would be stored at a fixed location in the file's header so tools could distinguish your SQLite-based file-format from everybody else's, without having to load the thing up and examine the schema.

eli2• 13 years ago

I'm not convinced this is actually a good idea overall, but...

There's no reason `file` couldn't be made to recognize special cases of SQLite databases. It certainly handles far uglier cases already.

And SQLite already provides a place to store meta-data about a table: http://www.sqlite.org/pragma.html#pragma_user_version Though the bummer is user_version is only 32 bits.

takluyver2• 13 years ago

There's already a similar situation for files like ODT, which are actually just a zip file with specific contents. But maybe the zip format has more flexibility to stick magic values in than sqlite does.

coldtea• 13 years ago

>One downside would be that tools like file(1) that try to guess a file's format by examining its contents would return "SQLite database" for every image file

If the format catches on, they wont. For one, it would have it's own fixed file extension, and second, file(1) will be updated to perform and extra check even if something initially looks like a common SQLite database file.

SQLite3• 13 years ago

There are currently 24 contiguous bytes of unused space in the SQLite header. If need be, and if SQLite catches on for use as a portable image format, I will be willing to allocate some or all of those 24 bytes to an identifier string for file(1).

bane2• 13 years ago

why not reserve a specially named table with the format identifier in there that file(1) could poke into and take a look at?

KaiserPro2• 13 years ago

There are already formats that allow you to keep n layers of images. openEXR for example allows you to keep multires, multi bitdeptph images all in the same file, with lots of metadata

It also has options for byte order to allow faster read/writes

jared314• 13 years ago

Whatever happened to the debate around how in/efficient the blob storage format in SQLite was? Did that get changed in v3?

EDIT:

Found the official examination[0] on the performance of blob storage.

[0] http://www.sqlite.org/intern-v-extern-blob.html

angersock3• 13 years ago

I would like to suggest this minor change:

  create table layers (id text, parent_id text, sequence integer, uti text, name text, data blob);

become (Vec2 used for notation's sake to make the declaration simpler--just two reals):

  create table layers (id text, parent_id text, sequence integer, uti text, name text, data blob, offset Vec2, bounds Vec2, rotation real);

This would make it really easy to ignore layers which have no component in the view space, or to compactly represent raster subimages.

"bounds" is the size of the axis-aligned bounding-box of the image. "offset" i a translation from the center of the axis-aligned bounding-box of the layer image data to the origin, and "rotation" is the rotation (in radians, CCW) of the layer. Order of application is rotation, then translation.

EDIT: Fixed missing def for "bounds".

eridius7• 13 years ago

I think you're missing something in your comment. There's no change between the two lines, and no reference to offset or rotation.

aeontech• 13 years ago

mouse over the declaration line and scroll left.

seanalltogether3• 13 years ago

I would argue you're not gaining much by complicating that specific table structure, since that information is going to be stored in the layer_attributes table where it naturally belongs. There may be several other attributes that can affect whether or not to load the resulting binary blob into memory.

darkstalker• 13 years ago

What about OpenRaster? It's used in open source apps like MyPaint and Gimp.

http://www.freedesktop.org/wiki/Specifications/OpenRaster

angersock3• 13 years ago

If I can just use sqlite, than I don't have to open an archive, parse xml, and otherwise get into a bad mood (see their file layout here: http://www.freedesktop.org/wiki/Specifications/OpenRaster/Dr...).

eridius7• 13 years ago

> OpenRaster is an open exchange format for layered raster based graphics.

How's that going to work for vector images?

franksalim• 13 years ago

You can have SVG layers in OpenRaster. See: http://freedesktop.org/wiki/Specifications/OpenRaster/Draft/...

mkr-hn3• 13 years ago

Why not XCF?

https://en.wikipedia.org/wiki/XCF_%28file_format%29

takluyver2• 13 years ago

That page says: "The use of XCF as a data interchange format is not recommended by the GIMP developers, since the format reflects the GIMP's internal data structures, and there may be minor format changes in future versions."

mkr-hn3• 13 years ago

It still seems like a better basis to fork for a format than something new and untested.

spartango• 13 years ago

Interestingly, in the GIS/Mapping space Mapbox has faced a similar kind of problem, albeit with image tiles rather than just layers. Their solution was to come up with an image format called mbtiles (http://mapbox.com/developers/mbtiles), which actually uses SQLite. It's not the same problem, but perhaps it throws some credibility to the use of SQLite as an on disk format for structured images.

saraid216• 13 years ago

ESRI Shapefile formats include a DBF file, which is a Foxpro database.

peatmoss2• 13 years ago

And I still dream of spatialite becoming the one interchange format to rule them all.

itafroma5• 13 years ago

> Last summer, Adobe killed their image exchange format "FXG".

Edit: previously I had balked at this claim, but I'm wrong. While I can't find any official notice of its death, and it's still in use in Scene7 and somewhat supported in the open-sourced version of Flex, trawling message boards does indicate Adobe shying away from it in favor of SVG[1][2] and it's not supported in CS6 without an extension.

Still, his problem with FXG (or a format like it) seems superficial. Yes, FXG defines links to other assets instead of combining them into one file, but to me, that's a feature, not a problem: I can them programmatically swap out assets easily by just manipulating the references within the file. It makes it really useful for generating customized assets on demand. In practice, it's like HTML for layered documents. We don't complain that HTML is broken because it links to images instead of including them, do we?

If you don't like that you have to send a folder: zip it up and give it a custom extension if need be, a la browser extensions and Microsoft Office formats.

[1] http://mail-archives.apache.org/mod_mbox/flex-dev/201303.mbo... [2]: http://mail-archives.apache.org/mod_mbox/flex-dev/201303.mbo...

eridius7• 13 years ago

We also don't typically use HTML as an interchange format between editors.

eli2• 13 years ago

But that's because HTML has complicated and ambiguous rendering rules, not because everyone hates structured, human-readable index documents.

itafroma5• 13 years ago

At its core, a layered document is a set of declarations about the constituent parts the document is made up. They link to other objects, like images and fonts. Whether you include everything the document links to into one container or let them hang out in a folder, it doesn't change how layered files are created or interacted with within editors or elsewhere.

So to transmit web pages? Sure, we use HTML as an interchange format. You receive everything about the page to manipulate it, including the locations to where the images and other related files are.

In other words, a layered document is not the same in kind as a JPEG or a GIF, which are flattened images. You need to be able to manipulate a layered document in the same way you could manipulate a Word document or an HTML file.

Whether you use a plain text format like XML or a SQLite database, the end result is going to be the same: in a SQLite database, you could place the component images/fonts into the database as blobs, just as you could technically place the component images/fonts in an XML file as data URIs.

But at least with XML, it's editable as plain text and doesn't require the component assets to be transmitted with the layout-defining file. I much prefer that over a format that dictates everything must be encapsulated within.

justincormack• 13 years ago

Well ePub is a zip file of HTML documents. So some people do.

tjoff3• 13 years ago

"We don't complain that HTML is broken because it links to images instead of including them, do we?"

Actually, we do.

Saving a webpage and getting tons of file is a nightmare while the IE approach of lumping them all together is just so sensible it's hard to imagine a universe where this isn't standard.

Replacing resources is the special case. Sharing, moving or just storing an image isn't. Just get or write a tool that can replace the resources for you and you have completely satisfied the 0.001% as well.

itafroma5• 13 years ago

Truth be told, you're the first person I've seen ever argue that HTML is a broken file format, particularly because it doesn't contain everything it links to. Imagine what it would mean if HTML was an encapsulated file format: you couldn't pre-fetch or cache static assets at all, because you'd have to get everything any time the structure of the document (the HTML part proper) changes. That's not a 0.001% problem.

As I and others have mentioned throughout the comments here, the single index file—be it XML like FXG, HTML, or otherwise—and linking to assets within it is pretty common and standard approach to complex documents. Combining them by zipping them up is a trivial addition and, again, common way to solve the "but I don't like having multiple files to transfer" problem.

Zikes• 13 years ago

How about an additional frames table?

malkia• 13 years ago

And why not TIFF?

splicer• 13 years ago

According to Wikipedia, Adobe Systems holds the copyright to and has control over the TIFF specification.

bsimpson• 13 years ago

Alias used TIFF as the native format for layered images in Sketchbook Pro 1.0 in the early 00s.

rogerbinns2• 13 years ago

FXG could have solved the single file problem by grouping all the files up and making a zip file, and picking a different extension. For example this is how Android (.apk) and iOS (.ipa) applications are distributed.

It also has the advantage that if you extract the zip you can now access whatever pieces are inside as separate files even by tools that don't understand the containing format. (eg a bunch of jpg/png inside the zip are perfectly viewable in any image viewer). Trying to do the same with SQLite is problematic as you have to write queries to grab the blobs and then dump them to disk.

Where SQLite as a format does shine is if you need to keep historical information such as an undo/redo log. That makes it good for an application format, but not for an interchange format.

derefr2• 13 years ago

> For example this is how Android (.apk) and iOS (.ipa) applications are distributed.

A more relevant example: this is how .docx files work, for very much the same reasons (one XML file at the root, and then a bunch of referenced resources.)

Someone• 13 years ago

Apple also uses zip for Keynote, Pages and Numbers documents, again for very much the same reasons, and Java's .jar, .war, .ear, .whatevar files also use zip.

.zip seems to be the universal 'combine some stuff in a single file' format, probably because it is old enough to be guaranteed patent free (OTOH: http://broadcast.oreilly.com/2010/06/is-zip-in-the-public-do... and http://www.pkware.com/support/zip-app-note. http://www.pkware.com/documents/casestudies/APPNOTE.TXT only mentions encryption and patching as patented)

Wingman4l72• 13 years ago

Also done with CBR / CBZ files (comic book archives) -- basically a renamed .zip of images with filenames numbered in ascending order.

vlaube2• 13 years ago

The file extension actually makes a difference here, CBRs are RAR compressed archives.

kalleboo• 13 years ago

Does the zip format let you replace an arbitrary file inside an archive without rewriting the whole file? It's not uncommon to see Photoshop files that are 400+ MB, and rewriting the whole file on save would have some pretty poor performance. The other examples of files that use the zip approach are either distribution-only (ipa/apk/jar) or generally don't get very big (word processing).

vlaube2• 13 years ago

This is an interesting problem, I wonder if Photoshop or The Gimp actually do something fancy here. There shouldn't be much of a problem replacing files when the files never get bigger and uncompressed ZIPs are used (compressed would make any sense for PNGs and JPGs anyway). Small variations in file size could be compensated with some kind of padding, but it seems to me that we are approaching a point of complexity here, where we should just use another kind of filesystem, one that supports fragmented data.

masklinn• 13 years ago

You can easily add new files at the end, then rewrite the central directory header (which is at the end of the file for exactly this kind of reason: not have to rewrite everything to add new files). And because the CDH specifies offsets to files, you can replace the old entry and put the new one there (replace old offset by new offset, done). This does however mean the old entry will still be in the file until you "garbage collect". That's if the new version of the file is bigger than the old one of course, if it's smaller or the same size you can just overwrite the old entry and not touch anything.

One could even imagine padding files in the archive so there's some leeway in growing the compressed files, it'd likely depend on the format expression.

rogerbinns2• 13 years ago

You can append just fine, and SQLite doesn't let you replace arbitrary content either. If the new content fits within the space of what it is overwriting then all is fine (as with zip), but if there is insufficient space then the new data is written at the end (as with zip). SQLite has a vacuum command which creates a new database only with used content.

Joeri• 13 years ago

Exactly. An interchange format should be editable with standard tools included with windows, and it should be easy to write all the code needed to extract part of the file yourself. Sqlite has many benefits, but realistically the only way to interact with it is through someone else's code. That rules it out as a suitable interchange format.

Any performance benefits sqlite might have are easily outweighed by the ease of interacting with the zip-based format.

seanalltogether3• 13 years ago

I'm interested in why Acorn ties the concept of a layer so tightly to a bmp or shape blob rather then segregating the two. It seems that having a library table that the layer table links to would allow for better reuse of images.

Library table <- pixel data

Layer table <- heirarchy/duplication of library items

Attributes table <- attributes of layers

Also, are the concept of Pages accounted for by the ordering of parentless layers, or does Acorn not support the notion of pages? Would that be its own table or a modification on the Layer table.

flutty• 13 years ago

Zip archive containing an SVG file embedding multiple png/jpg/jpeg2000 images.

groby_b• 13 years ago

"I want a filesystem in a single file. I'll pick the one I'm most comfortable with and pretend that it's much better than separate files".

Meanwhile, we'll ignore all the semantic issues around the data. What exactly _is_ a layer? Any given image file? What if some tools only handle e.g. alpha-layers? What if a layer is actually a filter kernel?

How are the layers composited? How are they ordered? Is it a linear order, or is there actually a tree of layers?

The reason PSD works so well is only partially that things are all in one file. It's also that the semantics of its contents are extremely well defined. (OK, if you're willing to consider PSD documentation "extremely well defined")

And Gus is almost completely punting on that part - define _that_ well enough, and it might make sense. Until then, it's just another VFS with a blob of assets that the receipient probably can't parse exactly as intended.

robbles• 13 years ago

Couldn't someone come up with a relatively sane schema for storing the same information that a PSD would contain in a SQLite database?

That way, you could standardize on the semantics that are already documented and familiar to other developers, but avoid the difficulties of the file format, which seem to be the issue here.

* disclaimer: I've never had to write a PSD-parser, I'm just going on hearsay.

jcampbell1• 13 years ago

Photoshop layers have all the nastiness of threaded comments plus the added complexity of "blending". I frankly can't comprehend the complexity of modeling Photoshop's layers as SQL tables.

Converting PSD's to JSON seems like a reasonable thing to do, but converting them to normalized SQL seems like a never ending shit show.

cachvico• 13 years ago

How about just strip PSD of all its crud, Adobe specifies it openly and gives it to the commons. I have a dream..

skue• 13 years ago

But a layered image SHOULD be a single file and not a filesystem. No non-technical user would expect otherwise.

And why all the spite? Gus isn't proposing that Acorn's database schema BE the standard. He says that clearly. He simply wrote a blog post giving an example of how he has used SQLite for storing layered images. (Also, he did happen to describe how he does layer sequencing - did you read the full article?) Any final spec would of course need to define additional semantics. That wasn't the point.

comex2• 13 years ago

> But a layered image SHOULD be a single file and not a filesystem.

But if all (or most of what) you want is a VFS, SQLite isn't necessarily a good way to make one. As has been mentioned elsewhere in the thread, it's fairly common for modern file formats to be zips of a structured filesystem; although this has upsides and downsides compared to SQLite, it seems much easier to run unzip than look up how to get SQLite to output the binary content of a table row to a file.

Millennium• 13 years ago

If what you want is a filesystem in a file, then it's probably best to just start with .iso files and work from there. You'd still have to standardize the directory structure and "internal" formats for actual data, but that would get you the filesystem-like structure you're looking for in a format that is already optimized for use as a filesystem.

zellyn• 13 years ago

Some of your blobs seem to contain a lot of structured data: I would further specify a format for them somehow: perhaps as further tables that join to the main ones, perhaps as JSON, perhaps as protobufs. Burying application- or format-specific blobs in SQL fields just pushes the problem down a level.

hoov• 13 years ago

> and Adobe, if you are listening you should really give SQLite a serious look

I'm not in the Photoshop group, but I do make liberal use of SQLite in the projects that I'm involved in at Adobe.

hiker• 13 years ago

OpenEXR solved that problem years ago and it's supported by virtually all content creation tools. An open source implementation with a well designed C++ api and bindings to several other languages.

For reasons that I don't understand, it's adoption is weak outside of the VFX industry.

jbarham• 13 years ago

Yeah, OpenEXR (http://www.openexr.com/) is a very nice format, well designed and documented, and with an open-source implementation.

Apart from VFX and post-production, OpenEXR has also become the standard file format in HDR applications.

IMHO it isn't mainstream yet because most people don't know what HDR imaging is and why current image formats are inadequate for HDR. For still image editing Photoshop is overwhelming dominant, whereas in VFX pipelines there's a lot of custom tools so you need an open format.

aaronblohowiak• 13 years ago

> When you want to send someone an image you want to pass them a single file, not an XML file with a folder of assets. While there are technical benefits to this, it's an incredible burden on the customer.

Or, you could just zip that whole thing up and everyone who doesn't have to care thinks it is "just one file", like .war's.

arielpts3• 13 years ago

Other common examples: .app .docx

notatoad• 13 years ago

this is like saying "i've invented a standard file format for all types of files. i call it binary. it's made up of zeroes and ones. i've solved interoperability forever".

benatkin• 13 years ago

The author didn't put the punchline in the title. Why did you put it in the title, HN submitter or editor?

comex2• 13 years ago

On the contrary, in my opinion, this isn't a joke, and since the "punchline" was likely to draw attention to the article from people that might be more interested in a debate over the wisdom of using SQLite "trivially" like this than a generic discussion of image formats, it was useful and shouldn't have been removed from the title.

Semiapies2• 13 years ago

This is an interesting idea, though I cringe a little at using a relational database as a key/value store (as in the case of layer_attributes).

I wonder whether it would be more useful to just represent the whole thing as a big JSON structure with base64 blobs. Each layer can be an object in the "layers" array with the attributes as keys.

gilgoomesh• 13 years ago

Layered images can easily grow to gigabytes in size. You do not want to store that much information in a format like JSON that must be opened into memory, parsed and stored in native objects. SQLite is RAM efficient and quick to open, even at large file sizes.

Semiapies2• 13 years ago

Fair enough - I rarely work much with images, so didn't even think of size.

nvk• 13 years ago

Amen!

bane2• 13 years ago

One nice thing about using SQLite is that various compressed versions of the flattened image could just be stored as blobs in another table in the same file. For example, my camera shoots with a RAW+JPG mode that produces two files, why not store the RAW data as described here and the JPG as an easy to grab/display item for quick image viewers etc. Hell store various common versions of thumbnails and icons in there as well.

fnordfnordfnord• 13 years ago

If someone does this right, it should go over pretty well for EDA (Electronics Design) file formats. EDA is a wasteland.

est• 13 years ago

IMHO SQLite is horrible format. using Matroska (.mkv) would be suitable for anything, vectors, bitmaps, audios and so on. It's essentially a container format like XML but for binary

WebM is based on mkv as well.

shuzchen2• 13 years ago

Could you elaborate on what specific benefits using Matroska would have? One downside I can think of is that there are no programming environments (that I can think of) that ship a Matroska parser, whereas sqlite readers/writers are found in many standard libraries.

jfb• 13 years ago

I thought about this (actually, an ISO based format like MPEG-4, but same idea). You could even represent edits as samples over time, and then play back the file as a movie!

xiesx• 13 years ago

xkcd.com/927 :)

mceachen• 13 years ago

TIFF, http://en.wikipedia.org/wiki/Tagged_Image_File_Format has been around since 1993. It supports layers. It supports compression. It supports paged(!) images. It has well-defined supported image encoding formats. It's already supported by all the image editing software you've ever used (from this millennia).

We have a Standard Layered Image Format. HURRAY. NEXT PROBLEM, PLEASE.

sneak3• 13 years ago

http://xkcd.com/927/

elsamuko2• 13 years ago

Fun fact: Even though Adobe sits on the TIFF standard for more than 20 years, Photoshop doesn't support layered TIFF images.

jwarren• 13 years ago

Actually, it does: http://cl.ly/image/0j383w2d0t2v

Open this test image if you don't believe me: https://dl.dropboxusercontent.com/u/824493/layertest.tif

jjcm• 13 years ago

Image formats have come a long way. I think the author wants an image format that can replace .psd in a sensible way. TIFF sadly cannot do that. While it can contain layered raster data, it lacks a lot of other capabilities. The author explicitly mentions vector layers, but if you want parity with .PSD you're looking at also having to support adjustment layers (hue/curves/levels/etc), blending modes, opacity/fill, raster masks, clipping masks, vector masks, locks, annotations, guides, grids, gradients, tags, text, comps, channels, and countless other things I can't even think to list (I wont even dive into smart objects). Keep in mind that's just the layer data, there's tons of other metadata in the format as well. It is a behemoth of a file format, and the years of backwards compatibility inside of it have multiplied that complexity to levels of insanity. I have .psd files that if I enable backwards compatibility when saving will increase the file size by half a gig. Granted those are enormous source files to begin with, but with higher def screens on the horizon file sizes are increasing dramatically. Having something more sensible than .PSD is something we're going to have to deal with sooner or later.

kruhft2• 13 years ago

.XCF

twelvechairs• 13 years ago

To provide a better answer than the GP:

.PDF .SVG .DXF/.DWG

All of these are open standards or basically open standards which are already widely implemented and allow the use cases mentioned (vector and raster combinations and layers) and also many of the ones you have thrown in.

Of course the real problem is that they are far too complex for most people's needs so why bother.

People really don't really often need to share the 'layers' of images combined with vector data (usually you use the layers yourself then just share the final product - a flattened JPG or similar), however if they do the above formats are there for them.

virtualritz• 13 years ago

Anything that TIFF lacks can be added using custom tags. All people need to do is to agree on names and types for these custom tags that store the new information. This means any old app will still be able to work with such TIFF files and just ignore the information it doesn't understand. So the problem isn't solved but the answer is not coming up with yet-another-file-format.

A good example of extending TIFF with tags is Pixar's PhotoRealistic RenderMan. It uses a TIFF flavor as a base for its mip-mapped textures. Such a TIFF contains a pyramid of layers of the image at different pre-downsampled resolutions and a bunch of custom tags, the renderer interprets when it reads such a file.

However, I can use 'Open As...' in Photoshop and open this as a TIFF. I will simply get the highest resolution layer from the file. That's a simple and battle-proven example of why TIFF is indeed a great answer to the problem.

npsimons• 13 years ago

Came here to say almost exactly this; of course, people being people, TIFF won't be "good enough", where "good enough" is either 1) invented in house (NIH syndrome), or 2) monetizable (TIFF is too open/not proprietary enough).

KaiserPro2• 13 years ago

or if thats not good enough: http://www.openexr.com/

has photoshop plugin aswell. No real support in the gimp though (cinepaint possibly has it though.)

Groxx• 13 years ago

This is kind of like saying "We don't need PNG/JPEG/ETC, we just need to put image data in an XML file! It can do everything! Standardization at last!"

Which is true. Except for that little "implementation" detail.

I like the idea of using SQLite as a complex-data-in-single-file format. Hadn't really thought of that for cases like this. But it does nothing to solve the problem of standardization.

VoxPelli• 13 years ago

Standardizing a format/API upon a tool like SQLite was something that was proposed in the web world as WebSQL but scrapped due to not wanting an entire standard to rely on a specific code base but rather wanting to have the standard be possible to reimplementable by others and thus creating the IndexedDB standard instead.

I'm not proposing that an image format like this one should use IndexedDB, but if relying on SQLite means being reliant on a specific code base that has to be used by anyone that wants to read the file - then it's probably not such a good idea.

When discussing this I think one should also have in mind eg the move to web apps as well - a reliance on SQLite will make it hard for them to read the format. A standardized simple base format that can easily be supported in new languages would be preferable as I see it - but I'm no expert in the image processing area.

greggman• 13 years ago

It's not going to happen for the same reason a standard 3D format will never happen. (and no neither FBX nor Collada fit).

The problem is, at least in the case of image editors (and 3d editors), standard formats arguably stifle innovation because you have to break the standard to add anything.

Maybe you decide it would be be better if all vector colors were stored as HSVA instead of RGBA except now no one can read your files. Maybe you'd prefer to store floating point colors with values greater than 1.0 to represent light emission. Maybe the standard only defines circles but you want eclipses. Maybe the standard says circles are defined based on rounded rect with the maximum roundness but you'd like them to be based on a center point and radius. Maybe the standard doesn't support text on a path but you want text on path. Maybe the standard doesn't support linking paths so that if the text doesn't fit on one path it bleeds into another. Maybe you'd like to justify text across multiple paths but that's not in the standard. Maybe you like columns but that's in the standard. Or you want to be able to define areas to be cut into pngs, each area with a specifiable filename. Or you want some of the settings to be per animation frame but the spec never thought about animation so your SOL. Or you want layer fx. Or you want to add a new layer effect. Or you want to be able to embed a PDF as a layer. etc.. etc.. etc..

Photoshop's basic chucky format is well known. It's not that hard to write some code to read all the chunks you care about. Putting those chunks in an SQLite format will not make it any easier to deal with new chunks your code does not understand.

itafroma5• 13 years ago

> The problem is, at least in the case of image editors (and 3d editors), standard formats arguably stifle innovation because you have to break the standard to add anything.

Indeed: that's exactly why Adobe created FXG instead of adopting SVG in the first place[1] (though they are now circling back to SVG).

[1] http://www.mikechambers.com/blog/2008/09/30/why-adobe-chose-...

lucian19002• 13 years ago

Even if all vendors deviate, it's still easier to interoperate, since at least some of the structure is transparent.

ChikkaChiChi• 13 years ago

Wasn't metadata supposed to solve this issue?

If the parsers are only looking for data relevent to them, I see no reason why a standard couldn't be developed.

I might be oversimplifying the problem...

7952• 13 years ago

A limited subset of HTML would be nice.

schrijver• 13 years ago

Don‘t think the discussion is complete without http://en.wikipedia.org/wiki/OpenRaster which is already supported in a number of programs.

pbhjpbhj• 13 years ago

What I'd like is the ability for two editors to work on a file at the same time, so layer locks might need to be built in. A db seems to me, a novice in everything relevant(!), to be a good way to allow multiple applications access whilst controlling for collisions.

nanoscopic• 13 years ago

Correct me if I am wrong, but FXG files are an archive with an XML file and assets, just as xlsx files are.

How is this a problem? How could is possibly be easier to open a file in sqlite than it is to just extract the file into it's folder structure with any standard decompression software?

I don't know about you, but last time checked there is no easy way to open a file contained in a sqlite database in my editor directly. I for one would much prefer the archive method than using a sqlite database.

Also, I would rather edit xml, than edit a sqlite database. XML can be edited by any text editor. I'd say text editors are a bit more universal and widespread than sqlite editors.

jbverschoor• 13 years ago

Good luck opening and modifying that 150MB psd file.

aoloe• 13 years ago

.ora? (OpenRaster)

http://en.wikipedia.org/wiki/OpenRaster

gimp (plugin), krita, and mypaint

We Need A Standard Layered Image Format

Related Stories