forrest2

This is largely a side effect of mimicking the distribution on the internet via pretraining.

It's a good basis for setting up a model of the world since we have so much data and it's free.

Post-training techniques like DPO and RLHF are then about using minimal hand-curated data (expensive!) to shift that distribution closer to standard human / desired behavior.

It will continue to get better -- early versions of chat gpt were taught to say "I don't know" with something like 20 training examples and it got substantially better off of those. As that number of training examples increases with the amount of capital invested, there will be more patterns that get latched onto and expressed by attention in these models.

----

It will take time but they'll get pretty robust. Models will still be susceptible to Dunning-Kruger / ignorance. They aren't perfect AND it's in their training data thanks to us humans that they're copying.

A single synchronous request is not a good way to understand cost here unless your workload is truly singular tiny requests. Chatgpt handles many requests in parallel and this article's 4 GPU setup certainly can handle more too.

It is miraculous that the cost comparison isn't worse given how adversarial this test is.

Larger requests, concurrent requests, and request queueing will drastically reduce cost here.

Of course, we already need to be working in Spaghetti ints or prepping them will be as complex than a linear scan.

Can't wait for spaghetti arithmetic.

Do we have a better algo than log(n) for locating the min?

I'm thinking spaghetti max align them, lay them across an arm at the midway point, sweep the shorts that fell, and repeat until all remaining are same length.

A lot of the yolo stuff from ultralytics is AGPL3 fyi. Recommend caution depending on what code or models / model lineage are used

Then it's an arguably arbitrary classification right? The borders seem potentially ill-defined and/or social in nature.

In some places, it's "gay" if two guys hold hands. On the racist bit: it can be hard to disambiguate why someone treats you in an unfavorable way -- it could be because they're actually racist or they could just be a pissy person who hates their job.

Just because a person or group of people classifies you a certain way doesn't make it universally true.

You can get most products on prem at a certain price & size. A lot of companies will apply resistance though unless the contract size is right b/c on-prem contracts tend to be less unit-profitable, risky, + unique annoying terms or constraints.

If you * need * it, you find a human to talk to (sales or connect from your network).

Is there a design paper somewhere? Curious how this was accomplished (and what trade-offs / failure modes it has): "System uses a consistency sharding algorithm, lock-free design, task scheduling is accurate down to the second, supporting lightweight distributed computing and unlimited horizontal scaling"

I think I have this: https://www.nationwidechildrens.org/conditions/auditory-proc...

Might be the same thing depending on your full set of symptoms.

You might not see it available via self serve interfaces, but I think the h100 cards can do 256x to a single host.

I think people said the same thing about NNs in general before we hit a scale where they started performing magic.

There could be exponential or quadratic scaling laws with any of these black boxes that makes one approach suddenly extremely viable or even dominant.

If you have tight interfaces that fully encapsulate the use of any, you're fine. If you want to return `any` or pass it around outside of an insulated zone, you're gonna have a bad time.

`any` is a contagious/viral construct where anything it touches could become `any`. And then you spiral out of control and the whole codebase gets nothing from typescript but gets all the operational overhead.

If your browser doesn't support text fragments, can cmd/ctrl+f for "contagious" and/or "gradual" https://www.typescriptlang.org/docs/handbook/typescript-in-5......

Final note is that you likely shouldn't blindly cast unknown to stuff that you hope it is but instead use type guards to find out what the type is.

Zod is really good at making this ergonomic for most use cases. More niche types using nonprimitives or uncopyable attributes require hand rolled type guards :/

Type guards also update the type of your unknown var after it passes through them if you gate scopes with them if(isX(y)) {...}

Additional guards will merge attributes onto the type

You make that impossible with application logic or constraints. Just like any linked list, you need to update the other pointers

Example constraint would be a unique index for the Playlist member table on the combination of Playlist id and previous Playlist member ID.

or perhaps for some applications it would be sufficient to do

playlists {

  id

  song_ids []

-------------

and just store the ordering in an array. Some postgres drivers might start shitting the bed though at some gigantic array sizes, but a playlist probably has reasonable enough limits that you wouldn't have a big problem.

Or model it as a linked list and you can sidestep the limitations / complexity of some kind of numeric (or bytes / text based ordering field)

------------

playlists {

id

}

------------

playlist_members {

id

playlist_id

prev_playlist_member_id

(and/or next_playlist_member_id)

song_id

}

------------

you could then just select * from playlist_members where playlist_id = ... and sort on the client side. you'd probably add an application limit where playlists have a max length of some kind.

re-orders can be done in a fixed number of row updates and typical application queries are still possible / fast.

I don't wanna speak for them, but for some people, going in deep on something is the only way to stay healthy / happy / sane.

Different strokes for different folks

One could argue that what you're describing is one of those alternative business models. You don't care if someone comes to your page anymore for ads if the scraped page is the ad.

I'm sad about what I think the stable equilibrium looks like for internet vs AI, and I hope there are some platforms that are able to put in the good fight to keep the internet awesome.

The internet is such an amazing public good and has moved technology forward at such an insane pace. It'd be depressing to explain to my grandkids that you used to be able to learn anything on the internet without subscribing to the X platform or Y curated channel.

There is flywheel issue with generative search: content producers, information curators or outlets, etc need to get paid.

People are okay-ish with Google in part because it drives traffic and traffic can be profitable. Fancy features that snatch website content and show them in Google's result pages are already not appreciated by indexed websites.

If generative search becomes the dominant interface, we will eventually see severe public info stagnation until alternative business models can grow around it or avoid it altogether. I suspect we'll see more platforms like Spotify for X and a continued shift toward subscription platforms and youtube.

Of course, what's to stop the bots from watching all of Netflix, listening to every podcast, etc? It will be an interesting decade for law / regulation / licensing.

Anecdote:

I'm a midwesterner and half of my siblings, most of my wife's siblings, and some of my friends' siblings are in their late 20s, have no jobs (or <10 hrs a week), and live with their parents.

From the outside, they look nervous / afraid to try to get into the job market or to date people. Men & women, but it leans men. The ones in poorer families stay home all day and play video games, and the richer ones venture out to spend their parent's money at restaurants or on trips but otherwise do the same. Half of them were doing minimum-wage work and left at the start of the pandemic and the other half have never had jobs.

I could just be in a local pocket of people like this, but I'm worried about how many people must have fallen off of the wagon and will never get back on.

Trying to get a skilled entry-level job after having done literally nothing for 5 years is hard for a lot of reasons. One being the mental hurdle you have to get over: you know you'll face a lot of rejection, you're out of practice, and your work peers will be a lot younger.

--------------

Controversial opinion: The whole thing has burnt me on UBI. I am afraid that the average American doesn't have the discipline to be productive if we introduce too much free income / free state of subsistence.

A great counter-argument is "well why does someone have to be productive? why should anyone have to work?". I don't know anything, but I suspect that we get to ask that question because of how dominant/rich the US is, and that is bound to end if we aren't more than competitive against countries that work their asses off.

Another counter is something like: exceptional people are responsible for the 10x-1000x outcomes that carry the economy, but those individuals are only the catalyst and do it on the backs of the rest of us. Takes all parts to make the machine work.

--------------

Back on topic. Here's some graphs:

FRED Graphs:

Hours worked by full-time and part-time employees by year: https://fred.stlouisfed.org/series/B4701C0A222NBEA

US Pop: https://fred.stlouisfed.org/series/POPTOTUSA647NWDB

Median weekly real earnings: https://fred.stlouisfed.org/series/LES1252881600Q

Employment-Population Ratio - 25-54 Yrs.: https://fred.stlouisfed.org/series/LNS12300060

Real gross domestic product per capita https://fred.stlouisfed.org/series/A939RX0Q048SBEA --------

US Employment rate by age 2000-2021: https://www.statista.com/statistics/217899/us-employment-rat...

Haven't used yet but systemd flag added a month ago:

https://devblogs.microsoft.com/commandline/systemd-support-i...

Vespa looks pretty compelling; indexing looks like a dream.

I'd recommend basically anything else over a customized ES / Solr cluster. Some of the least fun clusters to manage. Great for simple use-cases / anything you see in a tutorial. The moment you walk off the beaten path with them, best of luck.

Just an anecdote

I am not talking about investing in companies. I am talking about how these companies invest their R&D capital in projects.

Some food for thought:

These companies do have economies of scale on their side for internal tech investments. Things like this are a cheaper bet for them than for anyone else. Investments that seem irrational in a vacuum (aka for anybody else) can be rational for these specialized parties.

They already have hardware specialists, contacts / relationships, software expertise, idle bodies, etc.

Yeah this is what reminded me. Figured I'd revive the "mega earthquake" while the news was buzzing about the "mega flood".

https://en.wikipedia.org/wiki/Megathrust_earthquake

An absolutely nuts quote from the article -- "the northwest edge of the continent [...] will drop by as much as six feet and rebound thirty to a hundred feet to the west"

I've used an emulator in the past for GPS spoofing too. Obviously useful for testing and what not but also for location based apps / games / etc.

Tangent: IIRC Pokemon GO (and probably other more professional usecases for GPS spoofing) mitigates spoofing via a variety of techniques

Investors usually have liquidation preference. If they do, they'll have a first dibs on that $50m up to their invested amount.

Arguably needs to be abstracted out to some kind of multiplayer middleware; multiplayer sounds like one-size-fits-all but it's very nuanced.

And that feeling that it needs to be abstracted/genericized might be the same reason why it never gets done.

TLDR: Usually you start with num_interesting_event / total_measurements

This biases the result toward 0 or 1. Instead, you can bias toward .5 if you know that both sides of a binary outcome are possible: ( num_interesting_event + 1 ) / ( total_measurements + 2)

At 0 measurements, your probability is .5 based on the rule of succession instead of 0, 1, or undefined. Further measurements effectively iterate on this initial fair guess.

Ah sorry, I misread your claim.

Sounds high but if it's true it's true.

find that ts normally brings a downside, actually. In one project I maintain, a solid 30% of the code is nothing but ts stuff.

That 30% number is almost certainly why. The experience with codebases with 100% type coverage / tight types have been glorious, and the ones with partial coverage / plain js files / lots of "as any" / lack of strict compiler flags https://www.typescriptlang.org/docs/handbook/compiler-option... have been miserable and a waste of time. Can't stress the strict flags enough.

One of the "pros" of typescript is its gradual typing, but I think it's a trap and forces poor impressions on people.

One of those reasons is how `any` works. In gradual adoption, lots of stuff turns into `any`. `any` is a contagious/viral construct where anything it touches could become `any`. And then you spiral out of control and the whole codebase gets nothing from typescript but gets all the operational overhead.

If your browser doesn't support text fragments, can cmd/ctrl+f for "contagious" and/or "gradual" https://www.typescriptlang.org/docs/handbook/typescript-in-5...