convolvatron

if 5% of the work that I'm reading to keep up is slop, that's a shame. that's time I spent puzzling over connections that weren't real, trying to impose some kind of logic on the arguments, and wondering why the data doesn't really support the conclusion.

if 50% of the work is nonsense, then there's a serious concern that we can't move forward at all.

some guy left a few hundred pounds of steel in the loading area of my workshop for stupid reasons. maybe about $80 worth in scrap. he kept coming by and claiming someone was going to pick up it up, and getting really threatening about us stealing the value from him. the scrap yard is 200 ft away. he drives a big jacked up truck. after a couple weeks of this I'm like 'look, I'll cut it down, and we can throw it in that truck of yours and you can roll 200ft down the road and we'll be done with it'. he was incensed, his bed liner would get all scratched up.

after that I dragged it out onto the curb for the meth addicts to sell.

also no real mention of body construction or bed dimensions. nice to see a 2 door though I guess

I agree that something like *lisp would be a welcome relief from the hairball that is CUDA. while the lisp part was actually compiled, the parallel part reduced pretty 1:1 onto Paris, (the 'assembly language' designed by Steele). that was was basically a jump table into the microcode, which was just a big control word issued by (what I remember vaguely was being an amd 2900 bit slice) sequencer.

today I guess you'd translate those vector instruction into MLIR and hope for some layout and fusion.

but absolutely, you can just use tensors like normal variables. what a awful regression

yes. Calton Pu. is that really such a problem? JITs seem to work pretty well, I'm pretty sure synthesis was more about creating custom paths than being strictly self-modifying, but it's been a long time.

update: oh, I just remembered why it _is_ an issue for modern processors, Massalin leaned very heavily on the general purpose double compare-and-swap that was present on some generations of the 68k architecture, and x86-64 only has double contiguous compare and swap

it could also be a real advance, a valuable tool, lead to a bunch of great ideas and products, and still not be a good return on the trillions that have been dumped into it mindlessly.

I certainly see that the OS interface is limit to language innovation in that it assumes a lot the things I mentioned, and these tend to get shoehorned in and turn into a series layer violations to poke through all the needed flags and controls.

we definitely gain a lot of acceleration from having a common substrate, but it acts a constraint.

virtual machines are really a good out, whether that's a hardware isolation layer, or just a software abstraction. I think the view that the hypervisors present (simple virtio style devices) are maybe a better canonical OS interface than the sprawling syscall+library model.

but sure, unless I wanted to develop a language and runtime that _really_ looked different, I would try to paper over posix to the degree possible.

that's really unlikely. in the 80s the US military had several AI labs and was working with academia...I worked with some. the focus was nearly exlucsively on logic programming and 'expert systems', a path which failed miserably leading to an 'AI winter'.

somewhere around 1990, working at a Dod lab, I picked up on the 'neural network' craze and did a little work. it seemed interesting, but not that much different than using adaptive filters for signal processing. that seemed to be the general consensus, that there wasn't a lot of of interesting behavior or depth there. the number of weights and sizes of super computers at the time were a good 3-4 decimal orders of magnitude lower than what's being thrown today.

so unless the military ran a giant expensive program just to hide the fact that they actually had machines 10000x faster than the $50M (in 80s money) supercomputers of the day, and had trained them on a corpus they whipped up out of nowhere (the amount of information on the internet at the time could probably have fit on a few of todays phones), it probably didn't happen.

that seems backwards to me. one of the primary constraints in language development is the OS api. programs that don't interact with the world are increasingly less interesting, and you really have to work hard and be clever to change the file and its semantics, the socket, or the thread. these things have sharp edges and tend to be leaky.

just wait. the current gold rush has left any consideration of efficiency or price-performance at the side of the road. the entire enterprise is structured as a 'whoever can spend the most money wins the game for all time'. if we can get past that and invest more in theory and systems at a natural pace it'll just keep getting more affordable over time.

that's still comes up far short of the total bill. the US never talks about the destruction to civilian infrastructure, the number of innocents killed, the loss of any ability to claim to be on the 'right side'.

these are seeds that will ensure that the US is treated like Russia, or WWII Germany, or Iran. A rabid aggressor that will need to be isolated and eventually contained.

what would consider to be the essential characteristics of a third way?

I know a lot about kernel programming. and the last thing as I would ever suggest as being core to kernel programming is that is a specialized discipline that uses different rules and shouldn't be accessible to neophytes. its just code. sometimes the restrictions are unfamiliar, but there is nothing magic going on here.

sorry, I meant for a systems programmer the parts where there is a kind of dual correspondence developed between statements in a language and transistors on a board I think would probably open some mental doors for a systems programmer.

but I haven't gone through the video lectures or even all of SICP. but those that I did have had a lasting impact. particularly the erasure of the declarative/procedural dichotomy..thats been a very useful tool

addendum: actually I think the case for SICP in systems programming is stronger than that. There are several places in the material where the gap between 'high level programming' and 'construction of machines using gates' is thoroughly walked through and evaporated. maybe some of of the other similar treatments for logic programming and continuous analysis won't strike as deep, but that part should really be required reading.

these talks distill out the core questions of topics like mutability and state management and abstraction. almost uniquely so. so I consider them deeply relevant to systems programming in as much that its primarily concerned with..state management and abstraction.

unless you mean 'systems programming' as just 'the crap one does to try to glue together all the grotty pre-existing systems' and 'developing a good sense of taste about 3rd party libraries', in which case no, its not really very relevant.

although even here there is insight, I watched a video of Sussman describing why they were putting down SICP and demanding that MIT develop new introductory courses. he was so graceful and considered, putting his polished jewels away. the time when we could reasonably be expected to see across and through all the layers of abstraction was over.

consider the alternatives. it could have been written in PL-1 and rapidly become dated. or it could have been written in a slightly higher level custom language and that would also have to be taught and would be less clear about what was going on under the hood. or a kind of pseudo-code that would also admit ambiguity. or it could have been rewritten in pascal, and then java, and then javascript and then rust.

given the timespan and the focus on complete analysis of running times and not just asymptotics, in the end maybe it wasn't so terrible a choice.

we completely fail to run a marathon by turning the notion of distance covered into completely fictional metric, and anathematize the idea of setting targets beyond that two weeks since that requires collaboration and forethought. by attempting to run a marathon in a completely stateless way we end up walking in a brownian fashion but get thoroughly exhausted by the process even though we aren't getting anywhere really.

that sounds pretty related to Kolgomorov complexity, which is uncomputable in general. https://en.wikipedia.org/wiki/Kolmogorov_complexity

I too would be interested in approximations or heuristics if anyone has any

concurrent programs enable parallel evaluation. concurrency is necessary but not sufficient for parallelism.

or go up a diameter if that's feasible.

that seems very unlikely. margins are already quite thin, and the airlines are making most of their money with the higher fare tiers. if you remove some of those customers, then its entirely possible that the base fare will increase in response.

in CS we define a complexity class as a set of problems that have the same growth characteristic. that is for a problem size N, how long does it take in the worst case to find the solution for that problem.

one such class is the Polynomial class, or P, where the time to solution is some fixed exponent of N (like N^2, or 3).

the next big step is NP, which require a polynomial number of nondeterministic steps, whose solution can only be verified in polynomial time. usually solutions to NP problems are exponential in cost with respect to N (like 2^N), but thats not part of their definition.

problems in NP are generally identified by mapping them into a well known problem known to be in NP, where the mapping has to occur in polynomial time.

its an open question as to whether NP as a class can actually be solved in P time, but most people doubt that that is really the case.

eventual consistency as generally used doesn't guarantee that events are presented in the same order. I use 'monotonic consistency' for that, but idk how common that is.

or, we could just wait a hot second, get GPU and associated hardware over the 30% utilization mark, develop a fault tolerance strategy that recovers more useful work, and spend a bit more time researching how models actually converge. 50% savings on training time would mean even more energy savings because of the add-on effects of cooling.

this spending of billions just to get a 4 month lead, without even trying to invest in getting this stuff to run properly is wasteful to the point of insanity. I don't think it's at all productive to chide people for not wanting to dump their resources into a black hole.

it seems pretty clear that the investors and the AI companies _like_ to throw around big GW numbers. it gives them a moat, and it fuels the bubble.

no, they want a questionnaire, a coding sample, and an example of technical writing. there's a reasonable interpretation of that that doesn't involve writing a distributed unix.

I've gone through the same process, not so much that I don't think I would be worth considering, but serious code and documentation examples aren't something I can really give out given that they're proprietary. this last winter I started a whole guest-kernel based syscall intermediation and distribution framework in rust just for the application. with all kinds of design documents. I was about 30% finished by the time I landed a job somewhere else :)

but I still applaud the intent. I self-selected out by giving into scope creep

as someone who actually worked at the NSA pointed out earlier in this thread, they have plenty of resources, but also plenty of politics and some execution problems. so I wouldn't put money on them making a great model, but to say that they are completely incapable of doing anything is probably quite wrong.

the issue here that is a forgone conclusion, regardless of where the model comes from and which chips it runs on, is that now they can reasonably comb through all the stuff that they've been collecting. that's a pretty huge operational change.

NSA has had their own supercomputing program for decades. they design and produce their own large scale machines. chips, fabrics, arithmetic units, all of it. they also employ quite a number of hardcore mathematicians, computer scientists, and systems wranglers. if they decided it was of strategic importance there is absolutely no reason they couldn't train their own models.

idk how old you are, but there there was a style in the 70s where goto was the only control flow operator used in a lot of codebases, and it was absolutely horrible.