carterschonwald

this mirrors my approx experience.

i've had trouble finding any anecdotes or data about how to actually set/explore logit sampler settings

5.6 sol is pretty good, its definitely very very well tuned, i'm not quite sure which of the others i should use near term, but theyre doing great work

this so true. its really hard to make sure a model isnt going off the rails if i dont see full cot. the fact that oai and anthropic models hide it now has made them less reliable. which is a shame

its a wrapper around mlx, so thats gonna be the portability bottleneck

not sure about that, but im actively working on designing ultra sparse models that i want to have perform competitively with stuff 100-10_000 times larger. ehich does yield similar throughput. time will tell id it works out

one of the most hilarious examples of how the current wh admins false protectionism, in my mind, is the crucible steel bankruptcy in jan/feb 2025.

its a technological tragedy because it was the only facility im aware of globally that could actually manufacture steel based carbide alternatives at commercial volumes. idk if the relevant equipment is being operated by anyone post bankruptcy. powder steel equipment is a bit less destroyed when turned off, but i think the key blocker is that heat cycling a 3k centigrade furnace will age the material and cause cracking thatmakes it hard to resume the powederization flows

ive had doom loops on release day with opus 4.6. quantization aint the culprit ;)

even before the llm era sites would flag me as a bot for opening 15 links to read later. its fucking infuriating now

this is pretty cool. i think part of the root cause is current rlhf post training design around confidence and optics rather than cooperative transparent honesty. though its kinda an expensive hypothesis to dig into as a private individual

This paper points at an idea, but its really only legible if you have a more developed version of the idea already. I really should write more

i literally had a co2 sensor for my engineering team last fall cause the space was so poorly ventilated. just measuring it continuously radically changed how everyone approached using the space packing wise and ventilation. smelled better too :p

good. Of course the precise language of the ruling matters, but good.

so the most notoriously patent oriented tech firm is buying this up. lol ;)

good for the founders. also explains why my resume got dropped on the floor as a desk reject :p

.... i thought this was more widely known, granted i did write up a pretty wacky doc explaining way more fun experiments than these, and i have a fix that even prevents role collapse in my harness on github

https://github.com/cartazio/oh-punkin-pi/blob/main/scripts/b...

use that to install assuming you have whatevers needed to set up.

ive a much fancier next gen thing i hope to make available as a sass mid to late summer, but if you have any feedback or questions on mine do reach out

“sit with” is way over used lately

“worth sitting with ” is in the way to overused scale :(

whats your fave/best one shot code gen?

mine def has to be the initial impl of my python to my personal fave compiler ir. i got claude chat to write it in two sentences. 4 turns helping it rmeemver where it put shit becsuse of transcript issues

my heavily modified test bed for of oh my pi fixes this

gemini models are also fantastic at understanding non spoken sounds

this helps so much. i do it too. with some of the newer frontier models its unclear if you can even turn it off in the first party chat apps. havent compared api semantics yet.

its part of making sure the model actually engages in emotive communication, if i'm inventing insults i've never even thought about, i'm furious :)

saying i'm "furious" has lower entropy that incredibly implausible abuse. In some first party harnesses it just results in doom loops, but thats usually because the COT is hidden after the immediate turn in those setups. COT persistence helps with a lotta stuff

thats a harness issue not a model issue. eg i have my own reasoninf harness that forced persisted cot

context window size isnt quite the issue though, its that the attention mass kinda spreads out too much and everything kinda converges to a sortah global average region full of what we know to be slop! theres some really cool ways at the harness or model layer to mitigate this. just isnt really prioritized by the labs often.

i had the very strange experience last week of a recruiter listing your org as an example client, and when i looked stuff up i saw the current state.

i actually think this is too tame. it really has to be stuff youd mever say to a real person.

i want to view desktop on my mobile… and it wont lettt meeeee

you might like the stuff in my work of oh my pi, its a test bed for my ideas around making these tools more reliable. hoping to maybe have a native ui iter of the real thing that this is a test bed for this summer.

https://github.com/cartazio/oh-punkin-pi/blob/main/scripts/b...

the moment this clicked after reading it a few years ago its been my daily driver. that resource is a treasure