selalipop

But their comment is about 2 years out of date, and AI image gen has got exponentially better at text than when the models and LoRAs they mentioned were SOTA.

I agree we probably won't magically scale current techniques to AGI, but I also think the local maxima for creative output is going to be high enough that it changes how we approach it the way computers changed how we approach knowledge work.

That's why I focus on it at least.

I've been working on post-training models for tasks that require EQ, so it's validating to see OpenAI working towards that too.

That being said, this is very expensive.

- Input: $75.00 / 1M tokens

- Cached input: $37.50 / 1M tokens

- Output: $150.00 / 1M tokens

One of the most interesting applications of models with higher EQ is personalized content generation, but the size and cost here are at odds with that.

I'm applying generative AI to interactive fiction with Spellbound

https://www.tryspellbound.com/app/scenario/65821/create

(You can click on the dice at the bottom to turn on D&D mode)

I've taken the approach of starting with the #1 problem with Gen AI for this application: that it writes bland prose with not much going on by default.

From there you can layer on systems that address things like object permanence, but even with a basic engine capable of generating fun to read pages of text I think you already get a pretty fun experience

This is fun! I've been working on Spellbound which is in a similar vein: https://www.tryspellbound.com/app/scenario/65838/create

It's a bit more open ended though, are the constraints on actions intentional (ie. they're predetermined), or is the model just adamant on picking from options provided

If you actually look into the "SEO slop", they're constantly fighting a battle with Google to keep their place.

It's all garbage so no one notices when some of it suddenly disappears off the face of the earth and gets replaced with other garbage: but for the ones making it, their revenue essentially goes to 0 overnight.

technically launched 6+ months ago, but yeah current form of it did "re-launch" 2 months ago

Making good revenue but mostly I'm just very bullish based on current revenue and willing to bootstrap

That's picking up pennies in front of a steam roller: Google is incentivized to punish you when the content is garbage, and people are disincentivized to share what you generate.

It's an entirely different game once you can generate useful content worth reading with AI. People will even pay you good money for it.

Spellbound is an instruct model to NovelAI's completion model: you enter commands which in turn dictate what happens to your character, then the AI models how others would react to you

I was going to disagree with the article because the content 4o-mini generates isn't there yet.

I run a content site that is fully AI generated, https://tryspellbound.com

It writes content that's worth reading, but it's extremely expensive to run. It requires chain of thought, a RAG pipeline, self-revision and more.

I spent most of yesterday testing it and pushed it to beta, but the writing feels stilted and clearly LLM generated. The inflection point will come for content people actually want to read, but it's not going to be GPT-4o mini.

I've experimented with 30 or models so far, my general finding is closed source models like Claude have GPT-isms, while open source models do have a little less of a default tone but their ability to understand existing worlds is directly tied to how many tokens they were trained on.

Since existing worlds are (currently) where most of the stories are set, it's worth it to use a closed source models and wrangle their issues with dialogue.

To it's credit though, Llama 3 is the first OSS model trained on enough tokens to not feel lost for most worlds, so I've started routing some traffic to it for free users

The output format the site uses is also really really hard for most models to follow without fine-tuning, but fine-tuning then causes them to pick up the vocabulary of whichever model they were fine tuned on, which is a bit unfortunate

Unfortunately Supabase charges extra for the luxury of setting that URL, and the site is wildly unprofitable right now so I'm sticking to their free offering for the time being

Privacy policy and terms of service should be visible on homepage and in the sign up dialog

The pricing page is also visible when signed in and in a story (no dedicated pricing page yet but it's in the backlog)

I'm working on a website [1] that's essentially "Choose your own adventure with AI NPCs" and I've found two things:

a) LLMs are excellent at keeping a "linear enough" storyline without being linear. They'll let you do outlandish things, but given the assignment of "tell a cohesive story" they manage to corral the story back to something sensible unless the player intentionally keeps pushing at the boundary (in which case they probably do want things to go off the rails)

b) LLMs can do delightfully colorful dialogue, they just need to be grounded in a character. Everyone thinks of factual grounding, but given enough description of speech patterns, character motivations, etc. they're capable of dialogue that's lively and completely rid of "GPT-isms", which are what tend to break immersion

I actually trained an open model [2] on the task of grounding LLMs in characters and actions as opposed to factual things like RAG, and eventually I want to build a game demo out of it

[1] https://www.tryspellbound.com [2] https://huggingface.co/hf-100/mistral-spellbound-research

I recently built this at a hackathon: similar to website-to-chatbot products it crawls a webpage to understand a business and provide a model

But instead of a chatbot, this generates a set of guardrails for a chatbot based on your webpage

-

For example, if your website has information about a hotel, an LLM using RAG would attempt to answer most questions about hotels.

But by default there's no real-time information on things like weather or traffic conditions.

Rather than risk the chatbot hallucinating an answer, the guardrail model would detect a query likely to result in a hallucination and preemptively block it from reaching the underlying model

My users don't find what these tools generate to be spam. They're enjoying a classic format with a novel level of flexibility and (understandably) find that very fun.

Not really useful to give up the fight in the infancy of something with as much surface area as generative AI.

Is being used to create spam is not the same as needs to be spam, and we mostly just need platforms that leverage generative AI natively to bridge the gap.

Sounds more like a typical caching issue than anything

I’ve used Plasmo without touching their framework (essentially using it for HMR and bundling) and it works pretty well

But it’s never a bad thing to have two projects solving a painful problem

I’m working on a platform for reading hyper niche e-books: https://tryspellbound.com

I don’t think this form of generative AI needs to become a source of spam, carefully designed platforms can let people enjoy their niche content without making them feel isolated

OnStar dates back to the 90s and could track vehicles en-mass by the mid 2000s

The hashing algorithms in question require solving all the same problems as the classifiers: they're perceptual hashes not cryptographic.

Look at how NeuralHash is implemented.

A hash doesn't work without solving all of the same problems their provenance classifier solves: you need a way to be able to detect the steganogram even after the image has been modified.

Say your service would cost an unsustainable amount with GPT 4 because of excessive prompting needed, or you need to retry X out of every Y generations.

You could eat the cost temporarily knowing when GPT 5 arrives you'll be able to cut down on that significantly.

Or you build a simpler product that gets a foot in the door in organizations knowing that you'll have an additional value proposition that's enabled by an incremental improvement in capabilities.

It's not just about web requests: Small things like the select that reflects various font choices correctly, consistently styling form elements as deeply as the site does, the live updating rich formatting inside the main input (shockingly annoying to deal with, even with React), the per-word fade-in effects for text with memoization and other state-dependent optimizations since Chrome chokes on too many spans, handling accessibility for all of these little pieces...

There'd be a lot of "just a little bit of vanilla Javascript and CSS" sprinkled all over that'd all need to have state managed in a manual way that wouldn't be a meaningful improvement over React.

Like I said, I'm working on something where seemingly minute UI/UX details matter to the target user. So I can't just say "CSS and Vanilla JS don't support that without pulling teeth, it's a small thing, so I'm going to ignore it". Sometimes that's going to be a valid strategy and sometimes it's not.

I recently launched an app in an extremely saturated space (AI characters, https://tryspellbound.com)

Most of my differentiation is in the UX: specifically things like animations that were carefully selected to make the UI feel snappy even though the AI isn't that quick.

It sounds silly, but there are literally 100 apps doing exactly what mine does. If my site felt like submitting a form, and the UI had native selects, and overall it felt like a site that leaned more on native web technology, the demographic that uses it would have shunned it for other options before giving it a chance.

Sometimes you're in a market where you get the luxury of ignoring looks and just delivering value in a way that doesn't allow people to complain, but other times you have to give the people what they want: and a lot of them want that "it feels like an app" experience.

What about chatbots that understand products? https://notionsmith.ai

You had a chance to prove my assumption wrong by writing this same exact comment without all the snark.

At the end of the day if you're just unmoved by the implications that an ML model went from a bag of tokens to a structured, explained chain of thought, and a final response on an unknown task with rewards defined in natural english (!) and intentional ambiguity most humans wouldn't even try to confront... there's not much conversation to be had.

I think the rest of us (including your colleagues) will continue to build on these models, and like most advancements there'll be a vocal crowd insisting the car isn't useful because it can't be fed with grass.

not in recommendations

You didn't have to say that after complaining ChatGPT's web interface didn't give both us the same reply (most people in ML understand how temperature relates to LLM output)

_

By the way, if making your own personal music recommender seems like "magical thinking", maybe you're a little lost on which parts of Spotify's recommender systems are complex due to scale: if Spotify only needed to make song selection work for one person at a time, they'd have a lot more leeway in architecture.

In fact, when the problem was flipped and they were scaling humans attaching their recommendations to songs, they built on OpenAI: https://newsroom.spotify.com/2023-02-22/spotify-debuts-a-new...

Not unexpected when you're a founding organizer of the "NLP 4 Music and Audio Forum".

Your reply is not indicative of someone capable of a good faith conversation on the topic, but I'll bite.

I think you don't understand what the hard and easy problems are that underly the solutions I'm talking about.

For example: you repeatedly reply to people talking about the length of the prompts, but end users don't need to write prompts. It's trivial to append instructions around what a user says.

On the other hand, you keep replying to people with "how is that not just Siri" when people describe the LLM demonstrating zero-shot classification for example, but you don't seem to understand how difficult of a problem that has been for ML. Those contrived chat logs you see are demonstrating multiple discrete classifications that would have each cost untold hundreds of thousands of dollars in development of recommender systems to replicate just a few years ago.

—

Most people couldn't even dream of building a Spotify song recommender from first principles that could capture nuance like that chat demonstrated with an army of engineers. The fact is today, right now, that's something someone could hack into a real usable personal recommender in a weekend.

At the end of the day LLMs don't make all problems easier, and they make some problems harder: but the problems they make easier are extremely hard problems. I think if you're not familiar with how hard some of the things they're doing are, then the things they're doing poorly glare out much brighter.

If you spend half that weekend is spent fighting the LLM to output JSON the right way, it sure sounds like LLMs are just dumb hype machines... but it doesn't reflect the sheer impossibility of the value they're providing within that same system.

That JSON isn't something you'd type, it's something that you can programmatically generate if you have a Home Assistant setup.

With super primitive wake word detection and transcription, the most you get is:

- What the user said

- How loudly each microphone in the house heard it.

If you take a look at the mock object in that transcript, that's what it maps to...

```json { "request": "I'm finding it hard to read" "observedRequestVolume": [ 3eQEg: 30, iA0TN: 60, h1T3y: 59, 5Qg1M: 10 ] } ```

The only part that would be human provided is: "I'm finding it hard to read"

The invented challenge was to see if using a suboptimal set of inputs (we didn't tell it where we are) it can figure out how to action.

It's zero-shot capability that makes LLMs suitable for assistants: traditional assistants can barely handle being told to do something they're capable of in the wrong word order, while this can go from hastily invented representation of a house and ambiguous commands to rational actions with no prior training on that specific task

I think you fundamentally don't understand the topic if you're talking about two pages of text?

The end user would never type in a word of that: they'd say "[Wake word] play me some music"

A piece of software running on a device would transcribe what it heard, and fire off a request to the LLM with all of that text wrapped around their statement.

For ease of sharing I used the web interface to provide the instruction, but you'd use the API with a prompt which also dramatically increases determinism.

No one is writing out the state of each light bulb: you trivially query that information programmatically and bundle it with the request.

—

In a real product there'd be explicit handling of detecting where the request came from, that's already a problem that's been worked on, but I wanted to demonstrate the main difference vs Siri: zero-shot learning

The LLM wasn't told what those volumes mean, but it was flexible enough to infer the intent was to provide a form of location, rather than ask.

It's a forced example so if you want to get caught up on the practicality of audio for locating people be my guest, but it's to show LLMs are great at "lateral applications" of capability:

You give them a few discrete blocks of functionality and limited information, and unlike Siri they can come up with novel arrangements of those blocks to complete a task they haven't yet seen.

—

Honestly the fact you keep going back to "look at all the text" feels a bit like if I showed you the source code for an email messaging app, and you told me: "No one will ever use email! Who would write all that instead of just writing a letter and mailing it?!"