theefer

Not to nitpick, but XManager was built by the Platform team at DeepMind, not DeepMind for Google, though they helped roll it out beyond DeepMind.

(I was on the XManager team in Platform.)

The frontend repository above is only for the user-facing front-end website (aka the guardian.com). Other back-end components such as the content API and the content creation tool (Composer) are separate projects in their own private repos (not open source). That said the Guardian has many other open source projects on their GitHub org [1], including their image management tool for instance [2].

[1] https://github.com/guardian [2] https://github.com/guardian/grid

`await import('./foo.js')` could kick off a speculative fetch for foo.js

Interesting, and kind of makes sense.

However:

Of course that doesn't help for dynamic cases like `` await import(`./language-packs/${navigator.language}.js`) ``.

Isn't that the main, possibly only advantage of `await import(...)` though?

For static import names, why write `const x = await import('x');` if you can simply write `import x from 'x';`?

It seems you have chosen a bit of an edge case. Agreed the videos shouldn't pre-load, but to take a more standard example of a simple article:

AMP: http://tools.pingdom.com/fpt/#!/brWXpA/http://www.theguardia... Requests: 67 Load time: 485ms Page size: 627.3kB

Non-AMP: http://tools.pingdom.com/fpt/#!/eNuTPX/http://www.theguardia... Requests: 136 Load time: 2.88s Page size: 1.8MB

And to be fair, note that the non-AMP page has a lot more content, onward links, components, etc.

As you can probably see yourself from the screenshot and video, not much.

There may be some subconscious influence from the old system and other image systems we have used (Lightroom, Picasa, Google Photos, Flickr, etc), but I can't think of particular features inspired from Picdar.

are there some video features (like kaltura or youtube simple editing)?

Not at this stage, no. We're currently focusing on images.

how does "Indesign integration" exactly work?

InDesign integration works by drag and dropping an image into InDesign. Associated metadata, including the canonical ID, are then read by an InDesign plugin so we can store a reference to the original asset.

Other integrations with other web-based editorial tools in our suite also use drag and drop (+ drag metadata). It's also possible to copy paste URLs.

is it possible to use local storage, not Amazon services?

Not out of the box, but there is no reason why it couldn't be done with a few changes.

is there some ansible / salt / chef / whatever installation scripting available?

I'm afraid not. We mostly use CloudFormation to spin up our infrastructure and inject the relevant configuration on each service. There is a lightweight CF script for development purposes in the Grid repo, but our main CF script is currently in a private repository.

picture approval workflow

We had discussions with our picture desk about the level of permissions desired. Currently, we prefer the approach of keeping publishing open, so as not to introduce tedious barriers and slow down publishing (e.g. in case of breaking news, etc). There is a balance between the cost of publishing the wrong thing (e.g. not rights cleared) vs the missed value of publishing too late.

To balance the openness, we are focusing on providing high degrees of visibility to our picture editors, e.g. a feed of all images about to be published that aren't fully rights cleared. We also work really hard to get the information properly recorded (ideally automated) so the rights information is accurate about whether a picture can be used or not.

At the end of the day, we want to empower desks with the tools to come up with their own workflows and ways to limit risks associated with allowing all (or most) people to publish content, rather than imposing a rigid workflow.

- Séb, lead developer on Grid

As other replies have said, this is functionally a DAM.

It is particularly adapted to the requirements of publishers, in that it supports large number of images (we have over 3M currently), can scale to ingest many new images continuously quickly (publishers often receive lots of images from agencies and wires), indexes all the metadata to power a very fast search, allows collaboration of various roles involved in the use of assets and production of content, etc.

Unlike most commercial DAMs, which can be quite costly to run and acquire, Grid is also Open Source. We didn't find any existing DAM (incl Open Source) that fit our requirements, in particular in terms of ease and speed of use, powerful Web-based interface, rich APIs, etc.

You will have to review alternatives to know which one is the best fit for your use case.

Hope this helps clarify what Grid is wrt other DAMs.

Best,

- Séb, lead developer on Grid

It has some similarities with Thumbor (incl. cropping and resizing assets, etc), but the two are quite different. Thumbor goes a bit further than Grid with features like face recognition, etc, which we are looking at for the future. Thumbor also supports dynamic resizing, whereas we prefer to generate static assets and use external services (ImgIx in our case) to handle dynamic resizing and optimisations.

Grid is more of a media management service, allowing quick search of indexed metadata, organisation and collaboration using labels, quick upload into the system, rights management, etc. It also has extensive APIs to allow various degrees of integration.

Grid stores images in S3 (could be abstracted to any storage system with a small amount of effort). FTP is only used as a source for ingesting images into the system, and we're looking to scrape that and replace with S3-based ingestion in the near future.

- Séb, Lead developer on Grid

People interested in Duo may also want to have a look at jspm.io. It solves a similar problem, but with a few differences which to me are advantages:

- Transparently supports modules from CommonJS, AMD, ES6 or globals.

- Enforce a manifest (config.js) that let you pin dependencies (incl. transitive dependencies) to exact versions. Unlike RequireJS config, jspm automatically manages that file for you.

- Support multiple package providers, e.g. NPM, on top of Github.

- Based on SystemJS, a polyfill for the upcoming standard System loader. This hopefully makes it future-proof.

- Does not require a compilation step: dependencies can be pulled dynamically from a CDN over SPDY. Alternatively they can be cached locally as well. A compilation step (jspm bundle) is still available.

- Works both in the context of Node and the browser.

We've been successfully using jspm and SystemJS in production at the Guardian. It's still early days, but the devs are very active and responsive.

This isn't meant to distract people away from taking a look at Duo and making up their own mind, but I noticed nobody mentioned jspm in this thread and thought people may want to look at both and compare.

Markdown was indeed considered, but we ended up rejecting the idea for a variety of reasons. Given the interest around this decision in several comments here, maybe we should write up about it as well.

They're quite happy with it (except when it breaks, understandably). Mostly, they just want it to work as they expect (i.e. like Word or Google Docs), so the goal is for them to not notice it. Doing the Right Thing on paste from GMail or Google Docs (a very frequent use case) is therefore crucial. At the same time, we want to rely on Scribe to enforce correct, standard typography rules, valid markup (unlike if it were free-form HTML), etc.

We're only at the beginning, but the curly quotes plugin mentioned in the blog post is a good example of that. Other ideas in the pipeline include automatically enforcing and converting to UL/LI lists instead of paragraphs with bullet point characters, warning on punctuation issues, etc.

Scribe has also allowed us to integrate contextual options, such as buttons to add images when the caret is on an empty line, or a button to embed any URL pasted into the body.

So the biggest challenge is therefore to provide a reliable UI that responds as one would expect, while allowing extra features to be built on top of it without too much effort.