HN user

arsenide

206 karma

gmail - ahleph0

Posts4
Comments112
View on HN

Agree. The tests are great! Playtesting reveals that gameplay breaks my assumptions on how the system should work. Where to go from here?

Telling AI to "do TDD" is a shortcut that gets me over 80% there by itself in aggregate with little effort. The last 20% is painful. More painful than the usual "last 20%" in game design and development. I think this is part of the human vs. LLM gap.

If you revert the code to before the feature was implemented, then called out the way not to do it (the failed version you're describing) - what would happen? Maybe that "80% there" turns into "90% there", or maybe it's a bit better. Or maybe a bit worse. Working with LLMs vs. humans, does the gap shrink further if you gave other people the same requirements you gave the LLM? I think it depends on who you're working with - other people have their own opinions and don't necessarily just do exactly what they're told without taking their experiences into account.

In a way I see it like playing a game of telephone. I have my understanding of the system, I translate it into words, the LLM interprets it and does what it does based on that. For some reason, it feels like this is a telephone game with 3 people: A to B to C where the translation layer of my thoughts into words is an inserted "B". In contrast, if instead I'm expressing my thoughts to another person it's A to B. This extra connection here is lossy in the same way a game of telephone is lossy from player A to player C.

I've been having a bunch of fun working on a couple games on a new platform - everything (platform and games) developed with AI.

quackwave.com

Hasn't really been stress tested yet (this is the first time I'm mentioning it publicly), but it's been fun playing with friends & family and iterating.

Party game platform, think "jackbox but no host screen" - just need a web browser open on phone/tablet/PC.

Your comment matches my experience: it has to do what can and cannot be verified. As an example, it was much easier to have AI write a large e2e test using Playwright (then add test cases and expand) than to assume it'll correctly fix bugs without guidelines like this. Also, the human loop is still important in things like screenshot verification - but the frontier models are getting even better here so I'm not sure for how much longer this will be true. The ratio of test code to production code is a bit over 2:1 right now.

I work in the slot game industry. Plenty of ex-video game devs in this space. Much better work-life balance, but it's very clearly not the same as making video games.

Probably true in general. But there are always exceptions. Hard to imagine a tool better than Excel for slot game math calculations. Maybe it can be done?

Calculations for slot machine mechanics and payouts have been in Excel for a long time. There can be a LOT of complexity in these workbooks. Sometimes it’s tricky to debug - but what’s the alternative? Code is often hard to debug too.

Simulating results (Monte Carlo) is nice but having two sets of data for validation/checking against each other is nice.

I am not aware of any alternatives.

It is useful to use the words hard and easy. As you mention, changing perspective around these concepts is the crux.

Hard problems or domains are unknowns. Working towards solving hard problems involves thinking through unknowns, which may or may not lead to understanding. An aversion to hard problems is an aversion to the unknown.

Yes! In particular, it is usually to create some concrete data/parameters to be used towards creating the final product.

In the usual Excel Solver case, for a simplified example, I want to create a biased coin where getting heads doubles your money and tails loses your money, with expected payout 90% in the long run. The parameters to change are heads/tails coin weighting, to target a value 90%. A fair coin gives a long run payout of 100%. It turns out that if a biased coin hits heads 45% of the time and tails 55% of the time, we end up with this 90% payout. Excel solver can come up with these two values.

In this example, we can come up with these 45% and 55% values theoretically. With more complex systems, we may want to see the effects of changing a subset of parameters that have an indirect effect on payout.

For example, if we extend the “biased coin game” to instead be “win your money back, 2, or 3 times your wager each with some probability” on a heads result, there are more parameters (and many solutions!) to get to 90% payout. Changing the heads probability has a further, second-order effect on the payout. Using Excel solver it’s straightforward to fix some parameters (heads/tails probability) and allow others to change (1x/2x/3x odds) to get desired results (90% payout).

If we further extend this methodology for a biased coin game, we can end up with a coin game that pays with distribution exactly like a slot machine in a casino, though it would not look like one!

The only optimisation solver I use is Excel Solver Add-In.

It’s great, but you have to be able to put your problem into Excel to use it which does not work for all use cases.

In cases where I’m trying to optimise something that doesn’t quite fit into Excel cleanly I’ll usually do some sort of hand-rolled Monte Carlo optimisation. This generally works for me in the types of problems I most often solve.

This is really neat.

As someone using Excel to do slot game mathematics calculations, sometimes the excel formulas used are very long, convoluted, and hard to grok when coming back to them. Many cells often exist as calculation cells only, used for intermediate steps which leads to even more logic complexity.

I’m excited to experiment with these to try and simplify some of the long, previously-deemed-necessary calculation methods.

Recently...

1. On StackOverflow learning more about how C# works under the hood to understand memory/CPU usage (newer to C#, used C++ in a past life)

2. Building/debugging a spreadsheet in Excel (not a software engineer by job description)

3. Debugging software issues on hardware by messing with the hardware

4. Thinking about how to code to solve the problem; drawing/writing things out, etc.; rather than just diving in head-first (need to do this more...)

It works for math at the scale I use it.

Excel is great in my industry (slot machine game/math design). It is fantastic for doing game calculations and there are reasonable ways to do error handling/checking for correctness.

I could say the same about programming a web app without writing tests/spending some time on system architecture. It looks fine until it doesn’t work.

In both instances of development (I argue constructing an Excel workbook in my line of work is very similar to programming) there are ways to mitigate risks by doing things similar to “writing tests”.

I work as a mathematician/tools engineer in casino gaming (class 2/bingo based slot machines), so I think I fit this description (math first, software second).

People that have this skillset (knowledge of Monte Carlo methods for statistical analysis/verification) are useful in this sphere.

An example problem we deal with is as follows:

We have some arbitrary histogram (list of slot game awards and probabilities), and we wish to map this to a particular game of bingo (N ball calls out of M balls in total, 5x5 bingo card). The constraint here is that when you come up with a list of bingo patterns for awards, they are ordered in the sense that when evaluating the set of bingo patterns, from top-down, the first bingo pattern that matches is awarded and the evaluation stops.

Getting the optimal fit theoretically in the general case requires exponential time, as the chain of P(this pattern given (not-that pattern and not-other pattern and not-...)) grows with an exponential number of operations in the theoretical expansion. Each time a new pattern is fit to an element of the histogram, the probabilities of all remaining possible patterns to fit changes.

There are a few ways of tackling this problem in a more optimal way: If you can solve this with good enough accuracy in linear time with a good math/cs background, you’d be useful to any company playing in this space.

I’m glancing at the documentation, and came across this line:

“Airflow is not a data streaming solution. Tasks do not move data from one to the other (though tasks can exchange metadata!)“

Many of our tools take on the order of 5-200gb of data and do either some transformation (which gets passed along to the next tool; similar size, possibly after another automated validation step) and/or validation (whereby this particular branch of workflow ceases).

The automated modules we have are self-contained; each task in our case is “data + config parameters in, data out”, then use “data out” as “data in” after choosing configuration parameters for the next step.

Would this still be a good usecase — am I misunderstanding what the above quote is about?

We have many software programs that need to be run in sequence to transform a set of data from one form (on the order of tens to hundreds of GB) into a bunch of different output file formats. At some points the tools need to be run linearly, and at other points there are a few tools that need to be run in a branched fashion and then their outputs are combined in some complicated way using the next tool. Some of these tools take on the order of up to days to run, so an improper configuration due to human error loses time (pretty common given the complexity of work). Often the time lost is on the order of days.

These tools need to be configured in certain ways depending on the business needs.

Having a nice way to look at the dataflow as a whole, configure these tools on a global level within some framework, and be able to nicely distribute the work on our internal server farm would be worth a good bit of money to the company.

If there is an analytic model, using Monte Carlo methods is extremely useful to validate the theoretical result. If both agree, it is unlikely that there is a mistake in the theoretical result: which is valuable information.

For something as straightforward as Bernoulli, where you plug values into a formula and are done, there is no reason to use MCM. However, I am in agreement that in more complicated cases (e.g. a probability distribution is based on some result of one Markov or Markov-like calculation — or possibly multiple), and the theory is slightly or much more complicated then there is a lot of value in Monte Carlo methods.

I work on Monte Carlo simulations professionally for casino slot games.

I wish that more depth was presented here: what percentage of the time do you get favorable vs unfavorable outcomes with each strategy? For further depth, use different types of players (with different exiting conditions).

Perhaps I should write such a blog post at some time..