HN user
Contact me about anything - (ted suzman) firstname@lastname.net
Quanta Magazine consistently explains mathematics/physics for an advanced lay audience in ways that don't terribly oversimplify / still expose you to the true ideas. It's really nice! I don't know of any other sources like this.
My hope is that my own cadaver is ripped apart by somebody as crazy/appreciative as me =D
Nice!
Re: favorite systems for migrations — not really; I've always just kind of not used one, or rolled my own. Desiderata:
* fully atomic (all goes through or none goes through)
* low-boilerplate
* can include execution of arbitrary application code — data-query-only only migrations feel kind of limiting.
* painless to use with multiple developers multiple of which might be writing migrations
(1) This is awesome. Feels like this wraps enough complexity that it won't just be a toy / for prototyping.
(2) When a schema is provided, is it fully enforced? Is there a way to do migrations?
Migrations are the only remaining challenge I can think of that could screw up this tool long-term unless a good approach gets baked in early. (They're critically important + very often done poorly or not supported.) When you're dealing with a lot of data in a production app, definitely want some means of making schema changes in a safe way. Also important for devex when working on a project with multiple people — need a way to sync migrations across developers.
Stuff like scalability — not worried about that — this tool seems fundamentally possible to scale and your team is smart :) Migrations though... hope you focus on it early if you haven't yet!
I haven’t seen a better solution than remolacha’s #2 (create separate temporary state for the form).
Forms just inherently can have partially-finished/invalid states, and it feels wrong to try and kraal model objects into carrying intermediary/invalid data for them (and in some cases won’t work at all, eg if a single form field is parsed into structured data in the model)
slightly charred ends of the sticks had been cut specially to stick into the fire, and both were coated in human or animal fat.
Wait, what? Where did they get human fat from…?
In all the cases in the article it looks like shame plays a big role. I wonder if hikikomori is caused by a loop of [adverse circumstances that cause the person to feel shame] -> withdrawal to avoid shame -> being ashamed of having withdrawn [loop]
How’s PostgreSQL’s code quality? If projects have tons of technical debt or poor abstractions it can often be hard to make significant changes. Is that the case here, or no?
It was certainly an unresolved question before they did this work!
Naively, it seems reasonable to believe that if you adjust all the weights of a neural net towards the behavior you want via SFT and RLHF, that it would compete with/mute/obscure undesired behavior like a back door. But it seems not to be so… Indeed the cute mask does not cover the entire shoggoth— it may still have tentacles (https://images.app.goo.gl/YW9g3BvwGqGwYTgd6)
An LLM can be executed in an “OODA” loop like in AutoGPT and given a goal towards which it takes agentic actions, especially if the LLM is fine-tuned for function calling. So, it can be the main component of an agent that does have goals/de-facto motives! The wrapper code can just be a couple hundred lines.
AutoGPT itself is pretty weak, but it’s possible to write wrapper code that leads to stronger agency. Also, agents formed this way with GPT4 are way stronger than with GPT3.5… so expect this trend to continue.
Hmm wonder if that applies to https://www.hacksmith.tech/lightsaber :D
But the ways in which a LLM can “talk about” the universe (and everything it contains) are limited to the ways in which humans have previously talked about the universe.
This is often said, but it isn't so.
The task of predicting the next token in human speech really well requires immense intelligence — potentially far more intelligence than possessed by the original speaker! Imagine yourself engaging in the task of listening to someone who isn't that smart speak and then trying to figure out what they'll say next — in doing so, you might make all sorts of extrapolations about the person, their motivations, their manner, their dialect, etc — calling on all sorts of internal models that you've built up about people over time. This is what models are being trained to do when we train them on predicting tokens.
There are concrete examples of models inventing new ways of thinking that are not described in their training set. For example, when training a transformer from scratch to perform addition mod P (and having no training data other than examples of addition mod P), the transformer was able to discover the use of discrete fourier transforms and trigonometric identities [1]. As we can see, neural nets can build all sorts of internal mental models that no one explained to them beforehand. These internal mental models can then be elicited and used for other purposes by e.g. fine-tuning.
I think a good mental model for transformers/neural nets is that they're automatic scientists. They figure out ways of modeling things in order to predict the output from the input — which is what scientists do! As part of this, they can de-facto discover new theories, and come to rely on the theories that prove useful in their prediction task.
Also, not all tokens in the training set are from human speech, so models are being trained to model all manner of data-generating processes.
but LLMs can’t do anything, really
Do you think https://github.com/Significant-Gravitas/Auto-GPT et al will become more performant as models improve?
Happen to have one more? thank you either way!
Is it me, or have there been more earthquakes in the Bay Area lately?
LLaMA isn't built on RLHF, so it may be necessary to create a more extensive prompt. For example:
```
You are a super intelligent honest question-answering system.
Q: What's 2+2?
A: 4
Q: What color is the sky?
A:
```
It's true that many of the people on Jan 6 didn't view themselves as extreme and didn't realize they were being fascist — they probably mostly truly did believe they were protecting democracy — however this does not preclude them being de-facto fascists.
Caro understands his subjects so so well that you feel you are inside their heads.
Caro’s works have spoiled me — other biographies feel out-of-focus and surface-level and speculative.
(And all this despite the fact that Caro isn’t even really focusing directly on his subjects, because actually his books are about power and its dynamics.)
One strategy I’ve seen employee is to tear the book into three parts.
For me, more frisson is evoked by some of the passages in this rendition of Elgar's Cello Concerto than by anything else: https://www.youtube.com/watch?v=OPhkZW_jwc0
I wonder what happens if you ask it to draw a self-portrait
Computational proofs of integrity (STARKs, SNARKs) could detect silent data corruptions (at the cost of a ~1000x slowdown)
I wonder if we’ll see them used for large scale applications whose correctness is critical.
what all this effort is supposed to bring about
In the limit, all this effort brings about an artifact that could be thought of as a "magic" computer where:
- Everyone trusts that the computer operates to spec,
- Anyone (or any user interface) can type whatever they want into the terminal and press "enter",
- Anyone (or any other computer program) can read state out,
- The computer has immense amounts of storage and compute,
- Anyone who wants to change the computer's state in some way that cannot be achieved by using the terminal can freely make a copy of the computer with whatever changes they want, and encourage others to use their version instead.
What, one might wonder, would the use be? There are many, many potential uses — it almost boggles the mind — it's kind of like trying to imagine the uses of the internet when it was first invented.
To name one example: you could use this magic computer to operate a VR metaverse, instead of relying on a centralized Facebook one. Failing to do that would lock meaningful sections of our lives into a single vendor whose motivation is profit. Using the magic computer instead would mean that the metaverse would operate according to its predefined rules (which could leave plenty of space for human elections or other forms of collective decision-making), and that if things went awry anyone could try to set us back on track by making a modified copy of the machine and raising awareness about using it instead.
Just like regular computers, this magic computer could also have plenty of negative consequences. It depends on how people opt to use it. I think we'll probably have more luck adopting technology and trying to drive it in the right direction, though — the alternative is widespread Luddism, which might actually be a good idea, but I don't see a realistic path to achieving it.
A lot of folks who are very skeptical of web3 due to its dark sides are probably people who probably have great values and, if they adopted web3, could probably do a lot to improve its trajectory.
Early blockchains have also solved the problems they were designed to solve. For example:
- Bitcoin offers a transferrable store of value which cannot be inflated by governments,
- Stablecoins, thanks to being cross-border, are often used in e.g. Argentina where the local currency is unstable and it's not legal to buy dollars,
- Proof of Humanity + universal basic income has provided extra income to Argentinian people (e.g. heard of someone who was able to purchase a ticket to visit their family for Christmas thanks to crypto UBI),
- Crypto has been used to send remittances to economically unstable places (Lebanon, Turkey, Venezuela)
- Gitcoin has provided public goods funding and advanced our conception of mechanism design,
- Helium has created a new 5G network that people can actually roam onto,
- NFTs have provided a new funding model for artists (who create public goods),
- Zcash and Monero have allowed for fully private digital transfers,
- Dark Forest (https://zkga.me/) has been an amusing game,
- Snapshot has helped create a delegative voting system that governs a $3B treasury,
These might not be problems that you face or believe are important, but all these are examples of intended problems being solved.
P.S. the tone of your comment made me a little sad :(
When writing code for a blockchain today, even on an Ethereum L2, you're in a very resource-constrained environment where you end up using bitpacking tricks and the like. That's how it was for early computers, too, of course — they may serve as a better analogy. Programmable computers have taken decades to develop, beginning in ~1950, and I expect that decentralized computation will follow a similar path.
There simply are numerous hard problems to solve to make this all work at greater scale. In a healthy ecosystem like Ethereum's, there are frequent research discoveries (discovery of the concept of data availability, the application of BLS signature aggregation, proposer-builder separation, zkevm, data availability sampling, ...). The software engineering effort required to implement such research is colossal as well. Eventually we'll even see e.g. specialized hardware for efficiently producing or verifying zero-knowledge proofs.
It would be easy to look at the very early computers, which were perhaps not all that useful, and shrug — but that take wouldn't have extended well into the future as the technology scaled.
I wonder whether this could be a good early human application for Neurallink — with sufficient direct brain I/O we could interact with these patients in a metaverse setting!
Ethereum will scale thanks to validity rollups and data shards. The technical design is pretty much done; it's just engineering work from here!
The valuation is easy to justify if you assume e.g.:
(a) that they build robotaxis ~5 years ahead of competitors
(b) they continue to sell ~50% more cars each year (they have a stated goal of 20m cars/yr by 2030)
How much is a robotaxi worth? A regular taxi can drive ~100k miles/yr @ ~$2/mile. Let's say that Tesla robotaxis undercut on price by a factor of two, thus making ~$100k/yr in revenue. Let's say 50k goes to the buyer, and 50k goes to Tesla. Five years of that would be worth $250,000. Let's divide by two to be a tad more conservative. If you're selling 20m cars a year, that's $125,000*20m = 5T of value in cars in a single year. If Tesla were doing that in a single year, their market cap would be in excess of $10T]. If you think Tesla has a 1 in 10 chance of pulling off the above, then you'd expect a fair market valuation for them today of 1T — which is roughly what the market puts them at.
Plus add Tesla Energy, which could be as big as the car business.
Plus add Tesla Bots (labor), which is probably the largest market in the world.
Not quite: under typical Ethereum chain conditions, you can include a small constant-sized priority fee to encourage miners to include your transaction. (This is really simple, and can be a default which most users are never even aware of.)
This priority fee is free money for the miner; the only reason they would not include your tx is because it would make their block slightly larger, which would make it propagate over the network slightly more slowly, and slightly increases the chance that the race is won by a different miner who discovered a block nearly simultaneously. So, the priority fee just needs to be big enough to make up for this tiny extra risk.
This article checks ShadowStat’s inflation work using both theory and a variety of crosschecks, and ultimately even discovers a computational error: https://www.thestreet.com/economonitor/emerging-markets/deco...