HN user
Founder http://Interdimensional.ai, former OpenAI creative applications, prompt whisperer & Science Communicator and WSJ best selling author.
I may have missed something. What are saying specifically?
Okun's law shows a strong correlation between GDP and employment. Which runs counter to your previous claim – as I understand it (please forgive me if I'm not understanding it correctly.)
Did you make a comment somewhere else that I missed?
Fair question. In my anecdotal experience everyone I know who had persistent energy, focus and a mechanism for improving their craft achieved something interesting. Sometimes it was ended up in screenwriting or an adjacent field, but something came of it.
My friends who produced excuses became really good at that. My friends who produced real works and continued to improve became good at that instead.
Exactly. I'm very glad I ignored this kind of thinking.
Exceptions happen. When I decided I wanted to be a novelist I wrote ten books in one year and only released the ones I liked. I'd write a book then read a book on writing and try to improve. I'd give my books away to anyone who asked and turned them into free podcasts.
A year later I was one of the top ten best-selling indie writers on Amazon, I had my first movie deal and representation from an amazing agent who stuck by me through the rough parts.
Luck and timing were big factors. I also locked myself in my house for 12 months straight and did nothing but focus on being a better writer and create as much output as I could to get traction. I spent a tremendous amount of psychotic energy at building my writing business. Most people give up. I was 37 and desperate to make it work.
When I went with a traditional publisher it was frustrating at first. I was getting critical nods (I was a finalist for the Edgar and the Thriller Award) but the sales numbers weren't the same as when I was independent.
Eventually I found a great publisher (an imprint with Amazon Publishing) and it was a perfect match. My first novel with them (The Naturalist) spent six weeks in the number one spot (even outselling Harry Potter.) I now do two books a year with them.
I don't have the quite the sales or name recognition of a James Patterson or Stephen King, but I became a millionaire from publishing and investing on those earnings before I started working in AI.
I can't speak to the part of the article about advances. They're based very heavily on your sales history. I had to work my way up to six-figure advances. If you have John Scalzi sales numbers you can pick your terms. David Baldacci doesn't ask for an advance. He just splits the profits with the publisher. People like him are in a whole different league.
tldr: Luck and timing matter, but so does being an absolutely focused psychopath who gives up a social life, television, games and everything else to get what you want.
Okun’s Law has held up pretty well over the last 60 years. Are you aware of any research that disputes that other than the degree of the coefficient?
Eventually that will probably be the case.
The future isn’t a fixed point. But before then H&R Block (or a company like them) will probably shift to AI. Then eventually your AI will mean you don’t even need them.
The disruptors will get disrupted.
Original author here:
Yes. Rich people tend to re-invest their money and not throw it into a Scrooge McDuck vault to swim in. The re-investing is what helps create economic growth.
Original author here:
A. Some parts of the economy will grow. Some will shrink. My claim is that if you believe that AI will create new efficiencies and continue to do so, then overall economic growth will increase at a faster rate than ever.
B. Predicting what architecture could lead to this growth is beyond the scope of this article. The point was to say that if we keep getting increased efficiencies we'll get increased growth.
C. Job creation accelerates. Before I was hired at OpenAI in 2020, nobody on the planet had ever been hired as a prompt engineer. Now it's a career (at least for the time being.)
D. I'm assuming maximizing profit in the most near-sighted way possible. Same as it has ever been.
Is there a specific claim or argument that you disagree with?
Original author here (not ChatGPT):
My claim is that H&R Block would replace all their workers for more efficient AI if it was more profitable. Then they would re-invest that money in the most productive way they can – other companies, etc. This is the same strategy as Apple, Microsoft and other companies do with large amounts of cash.
I don't claim H&R Block would be directly creating new jobs. They would try to maximize profit by investing their profits and diversifying.
Some of these can be done with the right prompting. Wordle was solved a year ago. The problem is that what works and what doesn’t can be counterintuitive or at least require you to think about how something is perceived at the token level.
https://andrewmayne.com/2023/03/29/how-to-play-wordle-with-g...
Here's his YouTube channel. I don't know about the viability, but I loved the concept so much I started writing a sequel to my Station Breaker novels around the concept.
6 Days to Air: The Making of South Park
You’ll probably get better results by putting the examples only in the completion part of the training examples.
GPT-3.5 learns how to generalize better when it’s just in the completion.
This is the same problem that vexed the researchers who did the paper on the alleged reversal curse.
(https://andrewmayne.com/2023/11/14/is-the-reversal-curse-rea...)
If Pac-Man and the ghost positions are on the x axis, what is the y axis used for?
GPT-3 Davinci from 2020 could do this too:
Segment this text into proper sentences:
Input:
UNDERNEATHTHEGAZEOFORIONSBELTWHERETHESEAOFTRANQUILITYMEETSTHEEDGEOFTWILIGHTLIESAHIDDENTROVEOFWISDOMFORGOTTENBYMANYCOVETEDBYTHOSEINTHEKNOWITHOLDSTHEKEYSTOUNTOLDPOWER
Output:
1. Underneath the gaze of Orion's belt, where the sea of tranquility meets the edge of twilight, lies a hidden trove of wisdom forgotten by many, coveted by those in the know. 2. It holds the keys to untold power.
1. I don't think the original research paper demonstrated the reversal curse. They claimed that you'd only get random answers from their example prompt. I showed that wasn't the case. I also pointed out what I believe to be a flaw in how they trained their model that when corrected for gave results that were non-random.
2. That depends on what you mean by logic. What would be an example of logical reasoning that would settle this?
3. Alan Turing created the Imitation Game thought experiment to show the futility of this question. If intelligence is something that can be observed and tested, then when we should be able to describe what to test for.
4. I don't make any specific claims about LLMs logic or intelligence. I just wanted to put their claim that LLMs can't generalize from B to A to the test.
I've added your suggested test to the blog post: https://andrewmayne.com/2023/11/14/is-the-reversal-curse-rea...
Spoiler: It doesn't say "Tom Cruise."
Hello!
I’m the “shark-diving science journalist” in question.
First of all, you can run the experiments like I did and test this yourself. I’m not asking anyone to take my word. Just do what I did: Read the original paper. Test the claims for yourself.
And to clarify a couple things:
1. The shark-diving part is true.
2. I’ve never been a journalist of any kind that I’m aware of unless you count writing for Skeptic Magazine. I’ve had many, many jobs though.
3. I started at OpenAI as a software engineer and member of technical staff. When I started there was just over a hundred people there. The lines between engineering were and are blurry.
4. I was the original prompt engineer at OpenAI and discovered many of the examples for using GPT-3 and wrote a lot of the original documentation. Internally my title was “prompt whisperer.”
5. I’m in the GPT-4 research paper for my contributions to model capability. I helped find abilities for long-text, vision, etc.
6. I was given the title Science Communicator when I started doing background briefings for media, etc., but still worked on model capability and other things.
7. I left OpenAI two months ago to work on a startup.
Best,
Andrew Mayne
Those don't explain the relation between neuron cost and length.
Could you give us an example of what that means in practical terms?
The post says that 1000 neurons will give you 130 LLM responses - but of what length?
(LLMs are generally priced by input and output tokens. The longer the tokens the longer the compute time. Without an idea of what you mean by a response it's hard to understand.)
Likewise: 1,250 embeddings – How big is the text size in the example?
I'm VERY excited to see you doing this and understand it's early stages, but I wan't wrap my head around the pricing without context.
This is very cool. I'm still trying to understand the pricing. What is a "neuron" in this context? A token? A character?
"Neurons are a way to measure AI output that always scales down to zero (if you get no usage, you will be charged for 0 neurons). To give you a sense of what you can accomplish with a thousand neurons, you can: generate 130 LLM responses, 830 image classifications, or 1,250 embeddings."
130 LLM responses of that length? 1,250 embeddings what size of text?
I lived across the street from Warner Brothers Ranch. It was surreal to watch a show like the Sarah Connor Chronicles and see a town blow up and look out my window and see the same church steeple. My nightly walk took me past the backyard of the Wanda/Vision home. Apparently I was trapped in the Hex.
I look at their personal GitHub repos (when they're relevant.)
I started working at a prominent AI company in 2020 with no formal background in AI/ML. I was able to bring outside understanding of language and theory of mind into use with large LLMs and create a role as a prompt engineer. My résumé was basically doing interesting things and helping make their models more useful.
Even though I'm not a formal researcher, I've been able to contribute to research projects and be included in papers because the field is so new.
The most important criteria I look for when I interview applicants is what they have built. Github repos, papers even cool Product Hunt projects can have impact.
I always have trouble figuring out what to put in my bio. Every five years I'd shift into something new that caught my interest.
After a stint in entertainment I realized that AI was where everything was heading. I took up programming and started studying AI.
Original author here. I'm a programmer. I started on the Applied team at OpenAI back in 2020 as a prompt engineer (I helped create many of the examples in the GPT-3 docs.) I became the Science Communicator for OpenAI in 2021.
My blog audience is very non-technical so I write very broadly. We've been super busy with the launch of GPT-4 and Plugins (I produced the video content, found examples, briefed media on technical details, etc.) so I was only able to grab a few hours to put these demos together.
As far as the ChatGPT prompts go, I included a few, but they're just simple instructions. Unlike GPT 3.5 where I'd spend an hour or more getting the right instruction to do zero-shot app creation, GPT-4 just gets it.
Before I got into writing novels I had a small publishing business centered around books for magicians. I started this when James Randi off-hand mentioned to me that every magic book sold at least 300 copies because there were that many diehard collectors out there who bought everything.
I figured that if my break even was at or below 300 units I could put out two or so titles a year and not worry about going into debt. At the time a $40 book earned about $16 wholesale. Cost of printing was ~$3. I’d usually sell 200 copies directly ($7,400) and another 500 to wholesalers ($6,500.)
It averaged out to about $15,000 per book a year….but then the books kept selling year after year. Some made me over $250,000 in a ten year period. I hadn’t factored in for the long tail - which made it even more lucrative.
There are much better opportunities than publishing, but there’s some money to be made if you understand the actual addressable audience and have a passion for it.
The United States Public Health Service said TEL (leaded gasoline) was safe to use in fuel in 1925 as did the US Surgeon General 1926.
The $445M number doesn't count the total cost of the program, which put each Shuttle launch at ~ $1.5 billion.
Average cost of a SpaceX resupply is $152 million. That's the total amount the US government pays per mission.
You couldn't send the shuttle up with 10% of the cargo capacity and only pay 10% of the cost. The Shuttle was $1.5 billion whether it was 16,000 kg to the ISS or 2,000 kg.
Nor could you just use it to send two people to the ISS and only pay $170 million for two seats (Really $400 million - the actual cost for two seats.)
This is wonderful. There's so much potential for doing ML in the browser and this makes for a great demonstration.