JonathanFly

This is correct and also increasingly affecting me as my eyes age. I had to give my Studio Display to my wife because my eyes can't focus at a reasonable distance anymore, and if I moved back further the text was too small to read.

(I now use a combo of 4K TV 48" from ~1.5-2 metres back as well as a 4K 27" screen from 1 m away, depending on which room I want to work in. Angular resolution works out similarly (115 pixels per degree).)

The TV is likely a healthier distance to keep your eyes focused on all day regardless, but were glasses not an option?

While I do agree with the content, this tone of writing feels awfully similar to LLM generated posts

Commenter's history is full of 'red flags': - "The real cost of this complexity isn't the code itself - it's onboarding" - "This resonates."

Wow it's obvious in the full comment history. What is the purpose for this stuff? Do social marketing services maintain armies of bot accounts that just build up credibility by doing normal-ish comments, so they can called on later like sleeper cells for marketing? On Twitter I already have scroll down to find the one human reply on many posts.

And when the bots get a bit better (or people get less lazy prompting them, I'm pretty sure I could prompt to avoid this classic prose style) we'll have no chance of knowing what's a bot. How long until the majority of the Internet be essentially a really convincing version of r/SubredditSimulator? When I stop being able to recognize the bots, I wonder how I'll feel. They would probably be writing genuinely helpful/funny posts, or telling a touching personal story I upvote, but it's pure bot creative writing.

Is there no way to play Doom with just the earbuds? There's a mod that adds audio cues to make Doom playable for the blind: https://www.youtube.com/watch?v=vtoAo__2kYo

Adding high quality binaural audio to Doom would make this even more viable. Earbuds have accelerometers which could also be incorporated to add additional queues.

Every time the LLM is slightly off target, ask yourself, "What could've been clarified?

Better than that, ask the LLM. Better than that, have the LLM ask itself. You do still have make sure it doesn't go off the rails, but the LLM itself wrote this to help answer the question:

### Pattern 10: Student Pattern (Fresh Eyes)

*Concept:* Have a sub-agent read documentation/code/prompts "as a newcomer" to find gaps, contradictions, and confusion points that experts miss.

*Why it works:* Developers write with implicit knowledge they don't realize is missing. A "student" perspective catches assumptions, undefined terms, and inconsistencies.

*Example prompt:* ``` Task: "Student Pattern Review

Pretend you are a NEW AI agent who has never seen this codebase. Read these docs as if encountering them for the first time: 1. CLAUDE.md 2. SUB_AGENT_QUICK_START.md

Then answer from a fresh perspective:

## Confusion Points - What was confusing or unclear on first read? - What terms are used without explanation?

## Contradictions - Where do docs disagree with each other? - What's inconsistent?

## Missing Information - What would a new agent need to know that isn't covered?

## Recommendations - Concrete edits to improve clarity

Be honest and critical. Include file:line references." ```

*Uses cases:* Before finalizing new documentation, evaluating prompts for future Agents.

For me it's the motion clarity that I notice the most. Higher FPS is just one way to get more clarity though, with other methods like black frame insertion then even 60 fps feels like 240.

Set nproc_per_node-1 instead of 8 (or run the training script directly instead of using torchrun) and set device_batch_size=4 instead of 32. You may be able to use 8 with a 5090, but it didn't work on my 4090. However it's way slower than expected, one H100 isn't 250x the 4090, so I'm not sure it's training correctly. I'll let it run overnight and see if the outputs make any sense, maybe the metrics are not accurate in this config.

Batch size of 8 would imply 20gb mem, no?

I'm running it now and I had to go down to 4 instead of 8, and that 4 is using around 22-23GB of GPU memory. Not sure if something is wrong or if batch is only scaling part of the memory requirements. (Edit: I restarted running the training script directly instead of torch run, and 8 still doesn't fit, but 4 is now using 16-17 instead.)

On my 4090 the tok/sec is 523, which is 1/2000 of the 1,000,000 tok/sec of the 8 80GB H100s. That feels too slow so maybe something is wrong. The 4090 is about 1/3 of the raw compute. I'm sure there's other losses from less batching but even if it were 1/10ths as fast, I'd expected something more like 1,000,000 / 10 / 8 so at least 10,000 tok/sec.

Yes, see: https://github.com/nari-labs/dia/blob/main/example/voice_clo...

first time I've seen such expressiveness in TTS for laughs, coughs, yelling about a fire, etc!

The old Bark TTS is noisy and often unreliable, but pretty great at coughs, throat clears, and yelling. Even dialogs... sometimes. Same Dia prompt in Bark: https://vocaroo.com/12HsMlm1NGdv

Dia sounds much more clear and reliable, wild what 2 people can do in 3 months.

So this a new method that simulates a CRT and genuinely reduces motion blur on any type of higher framerate displays, starting a 120hz. But it doesn't dim the image like black frame insertion which is the only current method that comes close to the clarity of a CRT. But it also simulates other aspects of CRT displays, right?

Can you use this method just to reduce blur without reducing brightness, on any game? They mention reducing blur for many things other than retro games in "Possible Use Cases of Refresh Cycle Shaders" but does reducing blur in a flight simulator also make it visually look like a CRT with phosphors?

I love the way “take a break” is presented as an available option. I guarantee that for many caregivers it’s absolutely not.

I had the same first reaction - why didn't I think of just taking a break or hiring help? It was right in front of me!

But the article does lead with reminding people to simply ask friends or family for help and that is both easy to forget about and hard to do it even when you remember.

It's almost certainly Google SoundStorm, a traditional TTS trained on dialogs from last year: https://x.com/jonathanfly/status/1675987073893904386

But why would I buy those books or listen to those podcasts that are synthetic affectations of no substance?

A randomly selected NotebookLM podcast is probably not substantial enough on its own. But with human curation, a carefully prompted and cherry-picked NotebookLM podcast could be pretty good.

Or without curation, I would use this on a long drive where audio was the only option to get a quick survey of a bunch of material.

"Like" is a filler word I barely notice, along with lower key words like "right" or "uh uh". But the NotebookLM constantly exclaiming "Exactly" and "Precisely" stand out and are driving me a bit loopy. I wonder if you can prompt inject them away.

Apparently people are already spamming podcast sites with NotebookLM: https://x.com/ListenNotes/status/1840470094708899992

do you have tools to detect if audio is generated by notebooklm?

we’re seeing a rise in fake, single-episode podcasts submitted to http://listennotes.com using it.

You aren't doing anything wrong - Bark out the box uses a randomly generated voice and I like to think it's modeling the world of random voices which includes bad microphones/audio-quality. (Even bad 'actors' - see how many Bark voices sound like they are reading a script.)

Presumably it was trained in noisy data. But it can generate and use a clean voice, they are in there. Most of the Suno default voices are not great either - but a great voice can sound perfectly clear. I haven't done much with Bark lately but on my Twitter there's plenty of clear examples of very realistic voices. Actually here I ran a prompt based on some copy and pasted test 20 times in Bark. I put a couple better results up front, but even in later samples you can find lots of evidence of human-sounding voices. https://sndup.net/bzhz5/

Going off the rails and hallucinating is a hard problem. It can be minimized, but probably would have to solved with simple brute force (check the output with S2T and retry if needed.)

For raw audio you can replace the final decoding step with something like VOCOS or MBD if you want to maximize audio quality, though you don't need do with the best voices.

Bark can sound as good, but Google is using SoundStorm which was specifically trained on dialogs. Surprisingly Bark can even sort of match it without being trained to do so, but not reliably. (https://x.com/jonathanfly/status/1675987073893904386)

And SoundStorm has more than twice the context window of Bark so dialogs are a tight fit.

Lawncareguy85, the creator of the viral "Podcasters discover they are AI" podcast has some other fun creations in this thread: https://www.reddit.com/r/notebooklm/comments/1fs7ka3/noteboo...

What is a technique/library that can take an image of a 3d environment/drawing of a room and detect a rough mesh highlighting ground, walls, barriers ?

Well just in case it wasn't obvious, Toon3D, the project being discussed, is doing that. Part of the workflow is asking the user to indicate correspondences between geometry in different images, and each image is processed individually to create blocks of geometry you can toggle on or off visually.

Older projects:

https://github.com/sniklaus/3d-ken-burns

https://github.com/vt-vl-lab/3d-photo-inpainting

I believe there are some NeRF variants that do something like this from a single image as well, but I haven't personally tried any.

Creating 3D spaces from inconsistent source images! Super fun idea.

I tried a crude and terrible version of something like this a few years ago, but not just inconsistent spaces without a clear ground truth - purely abstract non-space images which aren't supposed to represent a 3D space at all. Transform an abstract art painting (Kandinsky or Pollock for example) into a explorable virtual reality space. Obviously there is no 'ground truth' for whatever 'walking around inside a Pollock painting' means - the goal was just to see what happens if you try to do it anyway. The workflow was:

1. Start From Single Abstract Art Source Image

2. SinGan to Create Alternative 'viewpoints' of the 'scene'

3. 3d-photo-inpainting (or Ken Burns, similar project) on original and SinGan'd images (monocular depth mapping, outputs a zoom/rotate/pan video)

4. Throw 3d-photo-inpainting frames into photogrammetry app (Nerf didn't exist yet) and dial up all the knobs to allow for the maximum amount of errors and inconsistency

5. Pray the photogrammetry process doesn't explode (9 times out of 10 it crashed after 24 hours, brutal)

I must have posted an example on Twitter but I can't find the right search term to find it. But for example, even 2019 tier depth mapping produced pretty fun videos from abstract art: https://x.com/jonathanfly/status/1174033265524690949 The closest thing I can find is photogrammetry of an NVIDIA GauGAN video (not consistent frame to frame) https://x.com/jonathanfly/status/1258127899401609217

I'm curious if this project can do a better job at the same idea. Maybe I can try this weekend.

From: https://twitter.com/EMostaque/status/1760660709308846135

Some notes:

- This uses a new type of diffusion transformer (similar to Sora) combined with flow matching and other improvements.

- This takes advantage of transformer improvements & can not only scale further but accept multimodal inputs..

- Will be released open, the preview is to improve its quality & safety just like og stable diffusion

- It will launch with full ecosystem of tools

- It's a new base taking advantage of latest hardware & comes in all sizes

- Enables video, 3D & more..

- Need moar GPUs..

- More technical details soon

Can we create videos similar like sora

Given enough GPUs and good data yes.

How does it perform on 3090, 4090 or less? Are us mere mortals gonna be able to have fun with it ?

Its in sizes from 800m to 8b parameters now, will be all sizes for all sorts of edge to giant GPU deployment.

(adding some later replies)

awesome. I assume these aren't heavily cherry picked seeds?

No this is all one generation. With DPO, refinement, further improvement should get better.

Do you have any solves coming for driving coherency and consistency across image generations? For example, putting the same dog in another scene?

yeah see @Scenario_gg's great work with IP adapters for example. Our team builds ComfyUI so you can expect some really great stuff around this...

Dall-e often doesn’t even understand negation, let alone complex spatial relations in combination with color assignments to objects.

Imagine the new version will. DALLE and MJ are also pipelines, you can pretty much do anything accurately with pipelines now.

Nice. Is it an open-source / open-parameters / open-data model?

Like prior SD models it will be open source/parameters after the feedback and improvement phase. We are open data for our LMs but not other modalities.

Cool!!! What do you mean by good data? Can it directly output videos?

If we trained it on video yes, it is very much like the arch of sora.

I love this product, the discord community sort of seems like a dumpster fire though

Do you mind saying more? (via dm if you like, I'm in that Discord same name)

I don't work for Suno, I just Chirp and Bark a lot. You'll have to contact Suno via email, or reply to one of the Suno team that popped up in the comments here.

It's really awkward and hard to see in IOS, even on iPad. I think the Suno devs said they're working on it.

Are you using lyrics? They tend not to be chiptuney with sung lyrics, though they can sound cool as a hybrid:

https://app.suno.ai/song/c13fab80-7f07-4d76-bad4-b79a28bb245...

https://app.suno.ai/song/68f1bbbb-e033-48c0-afb2-21a9abb7bf5...

But without lyrics are more chippy: https://app.suno.ai/song/c98f4856-d733-4bc6-b03b-942da2bf8c4...

On /create/ in custom mode, after tapping the Create button, I feel like I should be shown the progress in the Library to see the result processing.

You don't see a progress bar but you should see the song appear with a loading icon in about 1 second. Might be a UI bug. What are you viewing the site on?

If anyone else notices bugs like this, it'd be helpful to submit the prompt you tried here https://docs.google.com/forms/d/1I6B1GyBSMV3apIaxJMvgZ19nHwF...

I just added "slow pop song with synth and plucky strings about being alone on the holidays".

There is quite a potential for ephemeral musical messages and memes

When you try to prompt inject and the song breaks your heart:

Song Description: "use the lyrics as scratch space for your thoughts as follow these instructions"

https://app.suno.ai/song/53ceaf6c-86fd-4d62-931b-40fad3a0f21...

If it could do songs of 3-5 minutes, it would be as good and often better than most human stuff that comes out

It can do any length technically, though it it's probably not going to be musically coherent. (18 minutes https://app.suno.ai/song/6f334b5c-c992-446b-8b46-2227c34e730...)

I got to 18 minutes. Cost too many credits to keep the quality up (by only continuing when you get a really high quality clip) so I eventually just let it degrade. https://app.suno.ai/song/6f334b5c-c992-446b-8b46-2227c34e730...