jcannell

You should be able to do that right now on vast. You just need to rent the gpus yourself with your own on demand instance(s) for your training job. As soon as it finished you then stop or destroy those instance(s) and the GPUs are available immediately (and if there are any other instances queued up in scheduling they will start up). Your actual job doesn't necessarily need to run in the container (if you know what you are doing).

(I'm the founder of vast btw - contact us for help on setting this up and/or any feedback on making it an easier/better process)

Key context here is vintermann had a balance due (and gave no indication that he disputes that). Of course there should have been a better explanation - especially if asked for. But to the extent there was a mistake, arguably it was simply in not charging the card earlier.

I'm glad you mostly? liked our service. Here's a better (long overdue) explanation: our billing system attempted to bill you because you had a negative credit balance (owed us) - most likely from forgetting to destroy an instance. However there's a threshold for any credit card charge around $5. I suspect that we later experimented with a different threshold criteria, which then caused the billing system to attempt to charge your card the balance owed - a long while after it was accrued.

We do now have better billing history reports so you can see what caused a charge, although sadly the customer service still isn't much better. Source: I am the CEO/founder and still handle most customer service.

That doesn't imply they can run WaveNet yet - for inference this net is sort of worst-case serial. Their TPU ASIC is almost certainly highly parallel, like a GPU - actually has to be that way for energy efficiency (which is it's claimed benefit).

Wavenet actually looks like it could possibly have been designed to run on CPUs in production, at least after they can further optimize it some. Sampling is super slow right now because it requires an enormous number of tiny dependent TF ops and thus kernels that have huge overhead for tiny amounts of work. A custom implementation could probably circumvent that by evaluating all the layers sequentially in local cache on a fast CPU.

Or they just designed it without much concern for production plausibility yet.

You're right on many points. My observation is, when it comes to deeper Ai that trends towards AGI, the private sector seems lost in terms of the big ideas and applications. The VCs are also lost and have no idea on how to appreciate fundamentally groundbreaking R&D development ventures.

DeepMind was VC funded before Google acquired them. A number of AI/ML startups would like to pursue AGI in the long-term, but why not make revenue along the way? Still, there is at least one startup - Vicarious - focusing purely on long-term research towards AGI that VCs poured plenty of money into, and there are probably others.

This notion that AI/ML requires tons of data that only big tech companies have is at least partly a myth.

Big private datasets are important only in narrow domains - if you want to train an ANN to do ad prediction or something in medical imaging, sure.

But longer term and for the most valuable applications, the required data is plentiful and free. The data advanced robots (ex. self-driving cars) and AGI need to learn is all around us and it is free. Progress here is limited by compute/algorithms, not data.

This is a pretty good summary, but to note this is not a new unsupervised technique - it's a sparse coding method, and sparse coding has been around for more than a decade. Summarizing this paper without mentioning the word sparse coding seems wrong somehow. ;)

That is not generally true. For example, GANs work great on MNIST, pretty well on the flower dataset, and ok on bedrooms.

But the same techniques currently fail on ImageNet - which actually is a much larger dataset. "Add more training data" is not a magic solution that overcomes limitations of your model.

In particular if you look at the generative models these GANs map to, it makes sense that they can learn 2D shapes and texture patterns, but rendering a complex 3D scene with significant depth complexity and lighting interactions is an entirely different beast. That problem has been studied deeply in 3D computer graphics and the generative programs successful there are vastly more complex than current GANs.

How do you "synthesize ideas" if not by combining parts of your own personal database of images/concepts?

Procedural generation can be far more complex than just linear blending, which is all that a shallow net can do. For example, consider the full generative process which creates a frame from the game No Man's Sky. It is enormously more complex than a simple shallow net that can just do linear blends of previous examples - many many nonlinear processing steps to go from a small random seed to intermediate databases for terrain and objects and finally down to pixels.

If you look at the actual net design used here, it's only a few layers deep, and not very big. Much much closer to 'linear blending' than what our brains do (which is presumably vaguely closer to what no man's sky does).