carpo

If I remember correctly, the original was like this too. I always used the sloop and would often beat much larger ships.

Yeah, it's been awesome! I'm so excited about tool calls and function use, the possibilities are huge. I ran it over 1494 videos that range in length from a few seconds to over 3 hours. Total duration 260 hours and a total size of 3795 GB. I don't know exactly how long it took to run, as I found some bugs I needed to fix when processing mkv files, but it was probably around 24 hours in total. That wasn't all LLM requests, but also the local Whisper transcription and frame extraction / analysis. I used gemini-3.1-flash-lite-preview for the content analysis and tagging. Analysis cost $9.22 and Tagging cost $2.72 and the results seem great (for comparison, I did 885 videos a few weeks ago with Sonnet and it cost $130 in total). Gemini seems much less verbose than Sonnet, even with the same prompt, so the descriptions are much shorter, but they seem very good. The tagging is great. Another added bonus has been that with the larger screenshots being sent, the LLM can now read much more of the text it sees on screen. Some of my videos are top-down showing me drawing and writing, and now it picks that up, so it's all indexed and searchable. I tested a few models with the RAG Chat feature, and the best one so far is GPT4.1-Mini. Before, when asking questions about the library or a video it was around 4 cents each query, now its averaging about half a cent.

Hey, just want to thank you for this suggestion. Spent this morning swapping to open router and changing all my prompts to use tools instead of XML. Not only is Gemma and Gemini much cheaper, the output tokens from the tool call are much less too. Cost to analyse one 20 minute video with 10 snapshots went from $0.21 to $0.009, and I'm even sending full HD snapshots instead of the 960x540 ones I was sending before (to save costs). The results so far are pretty good. It looks like the larger images are giving the model more context, so in some cases making the cheaper models results better than the expensive models. I'm going to run this over a few hundred videos today and see how it goes in bulk!

I originally limited mine to 10 frames spread evenly throughout the video, but it missed a fair bit of context at the analysis step, and didn't scale with length. So now when a video is loaded the app extracts a bunch of frames for the entire video, then calculates an image histogram and compares similarity to the previous one. There's some configuration so it doesn't send too many to the LLM, but still gets a good cross-section of frames to send.

You could also just use FFmpeg as it can do scene detection too. I tested both but liked the results from the histogram analyzer more.

Yeah, markdown works well if you're going to search through it with Claude Code or something like that. I built ClipScape as an Electron app with a local SQLite database, as I wanted an interface I could search and chat in and see the relevant thumbnails.

Very interesting. Thank you!

Cheers, I'll give it a try. How are those models at returning structured results? When I was writing the prompts for the analysis step and testing with older Claude models, it would have trouble structuring the XML consistently. Sonnet 4.6 handles it really well.

This is great. I wish I had enough ram for a local model. I just spent the last few weeks writing something very similar, but I made it a local Electron app with Whisper, ffmpeg and I added semantic search and embeddings for chatting with the videos. It talks to Claude for the vision analysis, tagging and video chat. Do you only send one image for yours? I used a customised scene detection algorithm to find multiple different images per video and then send them all in one request to Claude (along with the subtitles). It's definitely the most expensive part. Using Sonnet 4.6 for the analysis and Haiku for the tagging costs about $1 for an hour of footage, I can imagine it would be slow locally.

Can't you do both? Use AI and still learn and strive to be better.

But saying it's a confidence trick is saying it's a con. That they're trying to sell someone something that doesn't work. Th op is saying it makes then 10x more productive, so how is that a con?

That's not true. I've paid for a one time license for software before and received updates until the next major release.

Man I love that story.

Reading your comment made me think of the Roman generals returning to a triumph and someone constantly following them saying "memento mori", reminding them they are not a god. Now, instead of humility it would just be seen as a challenge.

An AMS is useful just so you can have 4 different filaments ready to go at any time. Doesn't need to be for multi material models. I have an A1 with the AMS lite and a Prusa mk3s, and manually changing materials is a chore.

I think you'd have to manufacture a culture, with rituals and habits designed to keep people focused so that the meaning of their lives was tied to the end-goal. It would make a good story :)

Would you really? when it's the only thing you've ever known you'd probably just accept it as normal.

Yeah, the app lets you configure which whisper model to use and then downloads it on first load. Whisper blows me away too. Ive only got a 2080 and use the medium model and it's surprisingly good and relatively fast.

Maybe developers are using it in a less visible way? In the past 6 months I've used AI for a lot of different things. Some highlights:

- Built a windows desktop app that scans local folders for videos and automatically transcribes the audio, summarises the content into a structured JSON format based on screenshots and subtitles, and automatically categorises each video. I used it on my PC to scan a couple of TB of videos. Has a relatively nice interface for browsing videos and searching and stores everything locally in SQLite. Did this in C# & Avalonia - which I've never used before. AI wrote about 75% of the code (about 28k LOC now).

- Built a custom throw-away migration tool to export a customers data from one CRM to import into another. Windows app with basic interface.

- Developed an AI process for updating a webform system that uses XML to update the form structure. This one felt like magic and I initially didn't think it would work, but it only took a minute to try. Some background - years ago I built a custom webform/checklist app for a customer. They update the forms very rarely so we never built an interface for making updates but we did write 2 stored procs to update forms - one outputs the current form as XML and another takes the same XML and runs updates across multiple tables to create a new version of the form. For changes, the customer sends me a spreadsheet with all the current form questions in one column and their changes in another. It's normally just wording changes so I go through and manually update the XML and import it, but this time they had a lot of changes - removing questions, adding new ones, combining others. They had a column with the label changes and another with a description of what they wanted (i.e. "New Question", "Update label", "Combine this with q1, q2 and q3", "remove this question"). The form has about 100 questions and the XML file is about 2500 lines long and defines each form field, section layout, conditional logic, grid display, task creation based on incorrect answers etc, so it's time consuming to make a lot of little changes like this. With no expectation of it working, I took a screenshot of the spreadsheet and the exported XML file and prompted the LLM to modify the XML based on the instructions in the spreadsheet and some basic guidelines. It did it close to perfect, even fixing the spelling mistakes the customer had missed while writing their new questions.

- Along with using it on a daily basis across multiple projects.

I've seen the stat that says developers "...thought AI was making them 20% faster, but it was actually making them 19% slower". Maybe I'm hoodwinking myself somehow, but it's been transformative for me in multiple ways.

Unless that's where they want to put their base of operations.

When my kids were younger I tried to to replace my swearing by saying "sugarplum fairies". It was fairly successful in becoming a natural replacement. However, the other day I kicked my toe really badly and instinctively yelled "sugarplum FUCKING fairies" and my kids (now early teen) found it extremely funny.

This is great. Can see you put a lot of work into it. I like it.

Same. I use /ask in Aider so I can read what it's planning, ask follow-up questions, get it to change things, then after a few iterations I can type "Make it so" while sitting back to sip on my Earl Grey.

He's talking about provenance. The NFT just proves who the actual owner is, so when people were right-click downloading and saying "I own this now" you can very easily prove that they don't actually own it, the holder of the NFT does.

I think they just meant it hit an inflection point. Some people were copying pasting to ChatGPT and saying it was crap and others were using agents that could see the context of the code and worked much, much better. It's the workflow used not just the specific LLM.

Yeah, it's probably too expensive and complicated to send all the frames to an LLM. I only send about 30 images at 576x324 which is around 15000 tokens a video, and comes to about $0.045 per video. First I save one frame every second and then loop through them comparing each to find the differences, and send just screenshots which have changed significantly, up to a max of 30. Claude only allows 100 images per API call, so it would be a bit fiddly and costly to handle 7000 frames.

Though, now that I'm thinking about it, you could probably do this locally and just look at the part of the image that has the current score, do some local OCR on it to check if the score has changed each frame, if it has, store the timestamp and then use ffmpeg to extract the correct parts. Probably wouldn't need an LLM at all.

As for editing, one thing I do in my videos is audio keywords so my app can do specific things. For example, I can say "AI, mark what I just said as important." Then when it transcribes the audio and the LLM processes it, it will mark that part as a Distinct Moment with a start and end timestamp, a title and description that will show in my app as a clickable link to that part of the video. I'm thinking of adding more commands for more complex editing too.

Thanks! I have been thinking about opening it up, but not sure, as I've never done any open source stuff and don't know how useful others would find it - there's some clunky bits!

I also have the whole Aider/Claude prompt history in the repo too, as I started this on a platform & framework I'd never used, and used the AI to scaffold much of the app at the beginning. Thought that might be useful to go back and see what worked the best when AI programming.

Ive almost finished the first version of a desktop video library app I've been writing for myself. I had the idea last year, but the cost of sending images to an LLM made it too expensive (to run over about 1500 videos), but now it's fairly reasonable.

In the app you pick a folder with videos in it and it stores the path, metadata, extracts frames as images, uses a local whisper model to transcribe the audio into subtitles, then sends a selection of the snapshots and the subtitles to an LLM to be summarised. The LLM sends back an XML document with a bunch of details about the video, including a title, detailed summary and information on objects, text, people, animals, locations, distinct moments etc. Some of these are also timestamped and most have relationships (i.e this object belongs to this location, this text was on this object etc). I store all that in a local SQLLite database and then do another LLM call with this summary asking for categories and tags, then store them in the DB against each video. The App UI is essentially tags you can click to narrow down returned videos.

I plan on adding a natural language search (Maybe RAG -- need to look into the latest best way), have half added Projects so I can group videos after finding the ones I want, and have a bunch of other ideas for this too. I've been programming this with some early help from Aider and Claude Sonnet. It's getting a bit complex now, so I do the majority of code changes, though the AI has done a fair bit. It's been heaps of fun, and I'm using it now in "production" (haha - on my PC)

It's not one event that destroys a republic, but a series of little ones that slowly erode the norms, until all of sudden there's someone willing to cross the Rubicon. You've got 44 months of erosion to go.

Yeah, I think we should try to separate his business acumen from his nut-case personality. He can be a successful businessman AND a fucking prick.

Use /context <prompt> to have aider automatically add the files based on the prompt. It's been working well for me.

I was thinking the same while was walking my dog.