singingfish

as a rule the interviewee should be assessing for goodness of fit, as well as the interviewer. If my simplified nuanced answer to why I limit my usage of LLMs was poorly received, I wouldn't want to take a job somewhere likely to make me miserable.

aka LLMs can not learn from experience - this is a fundamental limitation. c.f - individuals with Korsakov's syndrome - who also confabulate in a similar manner.

Both Yamaha and Roland keep a long back catalogue of reasonably priced parts for at least their mid-range pianos and up as well. I should check Korg too I guess.

I've had some impressive customer service from Yamaha concerning decades old saxophones that they have zero prospect of generating revenue from in the future, for an unrelated (musical) data point.

Of course they do - it's the same as bookmakers or other gambling syndicates, it only makes sense to operate at a certain scale otherwise the rounding errors and stochasticity will kill you.

If you want to just stop yourself from shooting yourself in the foot in your current checkout and maintain privacy around your own stupidity, you can add patterns to `.git/info/exclude`

and long may he idle

dumb, sick, stupid, other adjectives - generally not negative although could be in the right context - both in the Australian and North West England (where Matt was from) context. Just watch out for being called a fucking cunt. In almost all cases that one is pretty unambiguous. I do remember at one point having a conversation with Matt about how handling East Coast and West Coast North Americans was quite a different proposition from a cultural perspective, also related to this thread.

Matt would have liked this discussion. And given that his and my mutual friend has some actual legitimate serious expertise on obscenity in the English language even more so.

and child prodigies are notorious "arseholes" QED.

I totally get it, some people found his mess very difficult and it could easily lead into a death spiral. Others had a very different experience. He was certainly someone needed to be managed by those who knew him well from time to time, when possible.

Matt taught me everything I know about how to make commercial programming creative, engaging, artistic/craftsman type activity, aligned with my desire to keep everything open source to the maximum extent practical.

Another former colleague who is way more talented than I am emailed me privately to express a similar sentiment.

You'll find Matt's indirect influence in things like SQLAlchemy, and chunks of the enduring parts of the javascript ecosystem as well. He was known in the perl community, but his unparalleled thinking skills have a much wider indirect influence

Matt was a child prodigy, and child prodigies have it notoriously tough. He and I worked closely for a while. There is someone else important in my life that has somewhat close to mst's intellectual gifts, and similarly to mst they also have difficulty controlling their reaction to other people. However, unlike this other person in my life, mst did know how to express accountability and had been on a learning process to deal with his limitations. Matt and I never had beef, perhaps because we recognised that our respective strengths and weaknesses were complementary.

Yeah, there's useful, but not that useful. It help to think of them as "language extrusion confabulation machines" to understand the actual limitations.

In other news, water is wet and the sun rises every morning.

What about extra feed - grain or silage or whatever? Or do they keep stocking density low?

Tyre is clearly the correct spelling. I quickly tire of anyone claiming otherwise. Gaol is an archaic spelling which I've found incredibly confusing ever since I was a small kid a long long time ago.

All that fertiliser that gets put into Cattle grazing lands increases the methane and other emissions, so your read on this is to a significant extent incorrect.

Was just talking to a Cambodian wildlife biologist about this very thing last night. In a South East Asian context rather than North American that is.

me? No. Ex trainee neruopsychologist and failed academic who was in the right place at the right time back in the mid 90s who didn't pick up computers for professional interest until the mid-late 2000s after getting excited by Neil Stephenson's Cryptonomicon when I was looking for a career change. These days I identify as an international computer hacker, but mainly to take the piss (due to the tiny element of truth sitting underneath)

Yeah the internal representation of organic neural networks are also weird - check out the signal processing that occurs between the retina and the various parts of the visual cortex before any decent information can emerge from the signal - David Marr's 1980s book Vision is a mathematically chewy treatise on this. This leads me to start thinking that human intuition may well caused by different neural network subsystems feeding processed data into other subsystems where consciousness and thus intuition and explanation emerges.

Organic neural networks are pretty energy efficient in comparison- although still decently inefficient compared to other body systems - so there is the capacity to build things out to the scale required, assuming my read on what's going on there is correct, that is. So it's not clear to me that the energy inefficiency of ANNs can be sufficiently resolved to enable these multiple quasi-independent subsystems to be built at the scale required. Not even if these interesting looking trinomial neural nets which are matrix addition based rather than multiplication come to dominate the ANN scene.

While I was thinking this comment through I realised there's a possible interpretation wherin human activity induced climate change is an emergent property of the relative energy inefficiency of neural architecture.

The reasoning is quite subtle, and because I'm not a very coherent guy I have problems expressing it. In the LLM space there are a whole bunch of pitfalls around overfit (largely solvable with pretty standard statistical methods) and inherent bias in training material which is a much harder to problem to solve. The fact that the internal representation gives you zero information on how to handle this bias means the tool can itself not be used to detect or resolve the problem.

I found this episode of the nature podcast - "How AI works is often a mystery — that's a problem": https://www.nature.com/articles/d41586-023-04154-4 - very useful in a 'thank goodness someone else has done the work of being coherent so I don't have to' way.

Yeah, give 'em small constrained jobs where the lack of coherent internal representation is not a problem.

I was involved in ANN and equivalent based face recognition (not on the computational side, on the psychophysics side) briefly. Face recognition is one of these bigger more difficult jobs, but still more constrained than the things ANNs are useful for.

As far as I understand none of the face recognition algorithms in use these days are ANN based, but are instead computationally efficient versions of the brute force the maths implementations instead.

I've been following the whole thing low key since the 2nd wave of neural networks in the mid 90s - and made a very very minor contribution to the field which has applications these days back then too.

My observation is that every wave of neural networks has resulted in a dead end. In my view, this is in large part caused by the (inevitable) brute force mathematical approach used and the fact that this can not map to any kind of mechanistic explanation of what the ANN is doing in a way that can facilitate intuition. Or as put in the article "Current AI systems have no internal structure that relates meaningfully to their functionality". This is the most important thing. Maybe layers of indirection can fix that, but I kind of doubt it.

I am however quite excited about what LLMs can do to make semantic search much easier, and impressed at how much better they've made the tooling around natural language processing. Nonetheless, I feel I can already see the dead end pretty close ahead.

The extra boilerplate noise with excessive repetition doesn't help one little bit.

Aligning the computer's and the humans' thinking processes. Cognitive load is exceptionally important - one of the few uncontravenable facts in human psychology is that healthy human short term memory has a capacity of 5 items plus or minus 2. So reliably 5. And thus the maximum number of thinking balls you should be juggling at one time.

Which then leads to thinking about designs that lead to the management of cognitive load - thus the nature of the balls changes due to things like chunking and context. Which are theoretical constructs that came out of that memory research.

So yes, this is pretty much principal zero - cognitive load and understanding the theory underneath it are the most important thing - and are closely related to the two hard problems in computer science (cache invalidation, naming things and off by one errors).

Thank you for attending my TED talk

I've had some C# code inflicted on me recently that follows the pile of garbage design pattern. Just some offshore guys fulfilling the poorly expressed spec with as little brain work as possible. The amount of almost-duplicate boilerplate kicking around is one of the problems. Yeah it looks like the language design encourages this lowest common denominator type approach, and has lead into the supplier providing code that needs substantial refactoring in order be able to create automated tests as the entry points ignore separation of concerns and abuse private v public members to give the pretense of best practices while in reality providing worst practice modify this code at your peril instead. It's very annoying because I could have used that budget to do something actually useful, but on the other hand improves my job security for now.

I thought it was more that python was basically perl with the fun bits removed.

Or to put it another way: python helps you think more like the computer does. Perl helps the computer think more like you do.

I believe it was NSA not NASA - in his hubristic style larry wall described perl as being born from "a secret project in a secret laboratory".

I'd say the biggest influence on my programming work is recognising how to use that style of humour can help deal with situations that can sometimes get to quite high pressure - e.g. "please rescue this failing 9 month long $x^e6 project before it goes live in the next fortnight"

I think the lack of discussion of perl is in part because from a bird's eye view, python and perl do exactly the same stuff in exactly the same way, but python has the mindshare. Because of perl's depth and flexibility, for the average developer team it can be really hard to manage.

There's a similar story around javascript - note how es6(?) introduced `"use strict"`. Old school javascript always struck me as a bit like crippled perl with really good vendor support.

nah, it was created for their day job in order to provide reliable quick to write code under difficult conditions