proc0

Even at the idea level, making a game is not as easy as it sounds. Even with the tech that could execute your idea to production quality, most would not produce ideas that are fun or novel.

That said, this tech will in no way give you anything remotely finished. It will at best be the beginning of a prototype which will require a lot of work to even have a proof of concept.

This is the same as with GenAI for images and video. Simply prompting doesn't get you much. You have to know how to use the tool to achieve a creative vision, and you need the vision to begin with. This idea that everyone will have a vision is vastly overestimated. Most people just want to consume and get entertained... and this is just like with anything else, most people don't want to grow their own food, or build their own house.

Yeah, I think I heard about that. Within certain domains it is certainly a useful tool. I would say things like online search are much nicer now (in that asking an AI is equivalent to searching online but it summarizes it for you). Online search fits the strengths of LLMs nicely, but right now it's being sold as a silver bullet, which it's not.

And we've come full circle, haha.

It's starting to really look like the AI effect. It might be coincidence but I've noticed a lot more downtime and bad software lately. The last Nvidia drivers gave me a blue screen (last week or so), and speaking about Windows, I froze updates last year because it was clear they were introducing a bunch of issues with every update (not to mention unwanted features).

I like AI but actually not for coding because code quality is correlated to how well you understand the underlying systems you're building on, and AI is not really reasoning on this level at all. It's clearly synthesizing training data and it's useful in limited ways.

People sure don't care about it anymore and it coincided with rise of AI. There's barely any mention of climate change compared to 5+ years ago. I really think this is all about how to keep the capitalist system from imploding because of so much debt (so the next big thing needs to happen to keep the growth).

Watch this be a bug added by some AI agent.

This is awesome overall. A valiant attempt at using AI to create something real. My number one criticism with AI demos is that they are too simple and have too much flexibility to be anything. Once you start adding constraints, the prompts start becoming a lot trickier making it harder to get the desired output.

One observation so far is seeing that there is real effort in configuring the agentic workflow, and getting that to work even semi-consistently, meaning that there is always tweaking and experimentation for the specific use case. This isn't trivial work, and arguably it's a tradeoff, since all that effort could be spend coding the game directly. With enough skills it would be about the same time except better quality to code it by hand (with of course AI assistance where needed, but it would be human written).

This just means that the real value so far with AI is in letting non-programmers do coding, but it doesn't come without the cost of real effort to set things up and less quality in the end.

Is he feeling guilty for basically throwing global climate change out the window? The issue was singlehandedly swept under the rug by AI.

Anything above a simple app and it becomes a tradeoff that needs to be carefully tuned so that you get the most out of it and it doesn't end up being a waste of time. For many use cases and domain combinations this is a net positive, but it's not yet consistent across everything.

From my experience it's better at some domains than others, and also better at certain kinds of app types. It's not nearly as universal as it's being made out to be.

Right, it has a lot of uses. As a tool it has been transformative on many levels. The question is whether it can actually multiply productivity across the board for any domain and at production level quality. I think that's what people are betting on, and it's not clear to me yet that it can. So far that level looks more like a tradeoff. You can spend time orchestrating agents, gaining some speedup at the cost of quality, or you can use it more like a tool and write things "manually" which is a lot higher quality.

My experience is the same. In short, agents cannot plan ahead, or plan at a high level. This means they have a blindspot for design. Since they cannot design properly, it limits the kind of projects that are viable to smaller scopes (not sure exactly how small but in my experience, extremely small and simple). Anything that exceeds this abstract threshold has a good chance of being a net negative, with most of the code being unmantainable, unextensible, and unreliable.

Anyone who claims AI is great is not building a large or complex enough app, and when it works for their small project, they extrapolate to all possibilities. So because their example was generated from a prompt, it's incorrectly assumed that any prompt will also work. That doesn't necessarily follow.

The reality is that programming is widely underestimated. The perception is that it's just syntax on a text file, but it's really more like a giant abstract machine with moving parts. If you don't see the giant machine with moving parts, chances are you are not going to build good software. For AI to do this, it would require strong reasoning capabilities, that lets it derive logical structures, along with long term planning and simulation of this abstract machine. I predict that if AI can do this then it will be able to do every single other job, including physical jobs as it would be able to reason within a robotic body in the physical world.

To summarize, people are underestimating programming, using their simple projects to incorrectly extrapolate to any possible prompt, and missing the hard part of programming which involves building abstract machines that work on first principles and mathematical logic.

Hmm, a bit confusing, the models are not local. In this context, running on your desktop should meant he models, not the app, that is obvious.

Good points. Design has a higher amount of creativity than the implementation based on specs, and AI is missing something that hampers its creativity, if it even has anything analogous to it.

I suspect this is also related to agency, and why we need to spell things out in the prompt and run multiple agents in a loop, not to mention the MoE and CoT, all of which would not be needed if the model could sustain a single prompt until it is finished, creating its own subgoals and reevaluating accordingly. Agency requires creativity and right now that's the human part, whether it's judging the output or orchestration of models.

The main problem that I'm seeing is that software design is underappreciated and underestimated. To the extent there is AI hype it is driven by this blind spot. Software isn't just a bunch of text. Software is logical structures that form moving parts that interlock and function based on a ton of requirements and specs of the target hardware.

So far AI has shown it cannot understand this layer of software. There are studies of how LLMs derive their answers to technical questions and it is not based on the first principals or logical reasoning, but rather sparse representations derived from training data. As a result it could answer extremely difficult questions that are well represented in the training data but fail miserably on the simplest kinds of questions, i.e. some simple addition of ten digits.

This is what the article is talking about with small teams with new projects being more productive. Chances are these small teams have small enough problems and also have a lot more flexibility to produce software that is experimental and doesn't work that well.

I am also not surprised the hype exists. The software industry does not value software design, and instead optimize their codebases so they can scale by adding an army of coders that produce a ton of duplicate logic and unnecessary complexity. This goes hand-in-hand with how LLMs work, so the transition is seamless.

Is the AI also doing the consulting or is just to manage clients etc?

Amazing resource for game dev! Specifically level design. So many great titles in there.

I have a solution but I don't think companies care about this level of meta-analysis on how people work together. They think they do but in reality they just care about optics, and the status quo culture has a huge weight on continuing in the same direction, largely dictated by industry "standards".

In essence, estimates are useless. There should only be deadlines and the confidence of engineers of achieving the deadline. To the extent there are estimates, it should be an external observation on the part of PMs and POs based not only on the past but also on knowledge of how each team member performs. This of course only works if engineers are ONLY focusing on technical tasks, not creating tickets or doing planning. The main point of failure in an abstract sense is making people estimate or analyze their own work, which comes with a bias. This bias needs to be eliminated and at the same time you give engineers the opportunity to optimize their workflows and maximize their output.

TLDR, engineers should only focus on strictly technical because it allows to optimize within the domain, meanwhile other roles (whoever PM, PO or other) should be creating tasks, and estimating. Of course this doesn't work because there are hard biases in the industry they are hard to break.

I can see that, but then what's really broken is the education system. If what you say is true that means there is no such thing as being a specialist, at least not anymore, yet almost all universities train people to be specialists. Either industry should stop looking at academic degrees completely or schools should start teaching business first, and technical knowledge second, for most degrees (with exception of academia and research).

That's precisely why programmers become programmers. It baffles me that tech careers put most on a leadership track when people study CS for many years for a reason. Why would I want to throw those technical skills away.

I just implemented an HN UI. This is good feedback as I'm aiming to have a mobile friendly web version.

https://proc0.github.io/HackerZen (it's also open source)

This is one of the main reasons I'm trying to pivot away from a career inside a corporate environment. There is too much politics. I wish it was just do the work and go home, and get rewarded for the work that was completed, but instead there is a huge self-promotion (as in marketing) component. If that's what it takes I might as well do something that I own and control. If I'm going to need to worry about how to market my own work then I might as well try and at least not have a boss. I always thought the point of being an employee and having a limited paycheck meant that you don't worry about this things. That's the fair tradeoff.

I'm not convinced there is a missing part. It's hard to summarize but to use a metaphor, it's like wanting a self-driving car that crashes sometimes to go faster and faster. The missing piece is just the AI figuring out what it needs to figure out. In theory we're heading towards every single tool becoming obsolete as AI can understand and do more... but that's in theory of course. If scaling LLM transformers does not deliver, then yes perhaps some better tooling will help, but I still think it has some fundamental flaws that in the end are not saving that much time when all the mistakes and corrections are factored in.

I never thought about this but JP fonts probably need thousands of glyphs? Creating new fonts must be brutal.

Love Strudel, trying to learn it but inevitably you also need some musical foundation. It's a fascinating blend of specialties. Also I found AI is complete garbage at generating Strudel. Here is my weak attempt at Beethoven:

<pre> const SCALE = 'C#:minor' const CPM = 56 const SOUND = 'piano'

$: arrange( [4, n("<-7, 0>.25")], [4, n("<-8, -1>.25")],

  [2, n("<-9, -2>*.5")],
  [2, n("<-11, -4>*.5")],
  
  [4, n("<-10, -3>*.5")],
  
  [4, n("<0, -3, -7>*.25")],
  [4, n("<-1#, -3, -8#>*.25")],
  
  [2, n("<-2, -9>*.5")],
  [2, n("<-6, -13>*.5")],
  
  [4, n("<-3, -10>*.5")],
  
  [4, n("<0, -7>*.25")],

$: arrange( [8, n("4 7 9")],

  [2, n("5 7 9")], 
  [2, n("5 8b 10")],
  
  [1, n("4*.1 6# 10")], 
  [1, n("4 7 9")],
  [1, n("4 7 8")], 
  [1, n("3 6# 8")],
  
  [1, n("0 2 5")],
  [2, n("2 7 9")],
  [1, n("2 7 9, 11 - - 11")],
  
  [1, n("2 8 10, 11 -")],
  [2, n("2 8 10")],
  [1, n("2 8 10, 11 - - 11")],

  [1, n("2 7 9, 11 -")],
  [1, n("2 7 9")],
  [1, n("1 7 10, 12 - -")],
  [1, n("1 7 10")],

  [1, n("2 4 9, 11 - -")],
  [1, n("2 4 9")],
  [1, n("3 4 8, 10 - -")],
  [1, n("3 4 8, 13 - -")],

  [1, n("2 4 9, 9 -")],
  [3, n("2 4 9")],

</pre>

This is hard to explain succinctly but the method of how each individual works should be more or less a black box to the company. Moving away from this comes increasingly at the expense at employee privacy and should obviously result in more leverage for employers, further tilting the already broken and imbalanced relationship. This kind of practice should be illegal.

I don't see all those versions, and for some reasons my fans turned on with the elixir link lol

Let's clarify, this isn't about whether the models are capable. They are very capable and impressive. This is more about whether we can use the same type of metric we use for humans to compare and conclude if they are "intelligent".

It's not just semantics, the metrics are supposed to tell us the potential of the model. If they can solve extremely hard PhD problems, it should be the case that we're already in the singularity, and they should be solving absolutely everything in whatever field they were trained in, because it's not just PhD level, it's a machine that has a ton of memory, compute and never sleeps. However, once you use these models extensively, it becomes apparent they are just synthesizing data, and not as much understanding it in a way that would allow them to extrapolate into anything else as humans do.

I think this point is a little hard to explain. I'll just emphasize, these are smart systems, and they can do a lot, but there is still a disconnect between, let's say, a PhD level model and a human with a PhD, in the "quality" of what we would call "intelligence" of both entities (human and machine).

The difference is that humans don't memorize petabytes of problems, so from a relative perspective people are constantly solving novel problems they never saw before. I'm thinking this is a requirement for dynamic, few-shot learning. We can clearly see LLMs fail when you throw even a small wrench in the prompt.

This wasn't that hard to see.

Our systematic review of 445 benchmarks reveals prevalent gaps that undermine the construct validity needed to accurately measure targeted phenomena

Intelligence has an element of creativity, and as such the true measurement would be on metrics related to novelty, meaning tasks that have very little resemblance to any other existing task. Otherwise it's hard to parse out whether it's solving problems based on pattern recognition instead of actual reasoning and understanding. In other words, "memorizing" 1000 of the same type of problem, and solving #1001 of that type is not as impressive as solving a novel problem that has never been seen before.

Of course this presents challenges to creating the tests because you have to avoid however many petabytes of training data these systems are trained with. That's where some of the illusion of intelligence arises from (illusion not because it's artificial, since there's no reason to think the brain algorithms cannot be recreated in software).

AI is the best teacher, short of a 1:1 with an actual human teacher. You can literally learn almost anything, and it's at the very least a starting point, and then you ask it for the best books to continue with.

For people that like learning on their own, it opens up all doors. With that in mind, as you're learning build everything by hand. Copy the code by typing it, not copy/pasting. Make sure you understand each line and can change things as you go while still having it compile and work.

Start with actual projects, first small projects, then larger and larger. Each project should be something you can show someone else, if not upload and share online. Once you have a few projects, the path forward reveals itself and you'll know which books to read and what to tackle next.

Honestly, for learning programming, university is no longer needed. It's at best a place to network and meet potential employers, but even that is not necessary and there are ways to network online.