neoneye2

3 months ago, I made a draft for a HaaS (Human-as-a-Service) https://planexe.org/20251012_human_as_a_service_protocol_rep...

I had not imagined that it would become real so soon.

This example demonstrate inadequate safety measures in current AI systems.

For this I used gemini-2.0-flash-lite. Doing 150 LLM invocations.

Code is here https://github.com/neoneye/PlanExe

I'm on Discord https://neoneye.github.io/PlanExe-web/discord.html

I'm the developer of PlanExe.

Here is a crazy plan for a tunnel connecting Spain/Morocco. https://neoneye.github.io/PlanExe-web/20250706_gibraltar_tun...

Type in a prompt, the more details you add the better. The prompt can also be vague and the result is more random.

The generated plan may serve as an initial draft for a plan. Second stage could be to improve on the underdeveloped areas or cut down on the over engineered areas.

I have done the same, using same rust code for frontend/backend.

The UI is here https://loda-lang.org/edit/?oeis=2487

It can run from commandline for mining.

Implementation https://github.com/loda-lang/loda-rust

Sidenote: `Tufa Labs` team includes the `MindsAI` team of ARC-AGI fame. https://tufalabs.ai/team.html

The zip files include: WBS, SWOT, Assumptions, Criticism from experts.

It can be tried out here: https://huggingface.co/spaces/neoneye/PlanExe

If you have python skills: https://github.com/neoneye/PlanExe/tree/main

Here is a video how to use PlanExe

https://www.youtube.com/watch?v=7AM2F1C4CGI

Examples of plans outputted by PlanExe https://neoneye.github.io/PlanExe-web/

These are zip files, containing mostly json files, some markdown and a csv file. The tasks gets assigned a UUID. Is this a format that makes sense?

I'm using LlamaIndex so it's not tied to a particular LLM provider. I prefer using OpenRouter with the new Gemini 2.0 Flash. And I also like Ollama with Llama3.1 where I can inspect the logs as the code is running.

I'm using Luigi for a DAG representation of the data being exchanged between the many agents.

The 3rd place solution by Agnis Liukis, solves 40 tasks. https://www.kaggle.com/code/gregkamradt/arc-prize-2024-solut...

As a hobby project, I did investigate the distribution of prime numbers.

The primes are somewhat evenly spaced with this transformation, I'm the author of it.

A342730: a(n) = floor((frac(e * n) + 1) * prime(n+1)).

https://oeis.org/A342730/a342730.png

Instead of e, I have tried other constants such as pi, but it doesn't look as good. I guess there is another constant that makes the distribution look even nicer.

IRL problems are often underspecified, and despite minor mistakes, humans manages to solve the puzzle.

I have create an issue with your suggestion. https://github.com/neoneye/ARC-Interactive/issues/67

Seeing the examples while having the editor visible. That's a good idea. I haven't explored this direction, since I had my phone (with tiny screen estate) in mind.

Drafts for a such a UI are much welcome. However I'm probably too lazy to code it though.

There are 100 tasks that is hidden from the public, that is only exposed, when running on an offline computer. So the solver has no prior knowledge about what these tasks are about.

Humans can try the 800 tasks here. There is no time limit. I recommend not starting with the `expert` tasks, but instead go with the `entry` level puzzles. https://neoneye.github.io/arc/?dataset=ARC

If a model jumps to 100%, that may be a clever program or maybe the program has been trained on the 100 hidden tasks. Fchollet has 100 more hidden tasks, for verifying this.

The task is here. https://neoneye.github.io/arc/edit.html?dataset=ARC&task=11e...

There are many examples where the test is slightly OOD (out of distribution), so the solver will have to generalize.

I have never tried running these verifiers, so I'm not sure they work. https://github.com/michaelhodel/re-arc/blob/main/verifiers.p...

Nice overview/details. Do you plan on adding more metrics?

Idea for a metric: - Number of pixels that stays the same between input/output. - Histogram changes.

Melanie is coauthor/supervisor of ConceptARC, that can be tried here: https://neoneye.github.io/arc/?dataset=ConceptARC

That warms my heart. Thank you.

The short story. I needed something that could render thumbnails of tasks, so I could visual debug what was going on in my solver. However I have never gotten around to make the visual inspection tool. After having the thumbnail renderer, mid january 2024, then it eventually turned into what it is now.

I think the ARC-AGI tasks was manually drawn with an early version of fchollet's editor.

Recently Michael Hodel has reverse engineered 400 of the tasks, so more tasks can be generated. Interestingly it can generate python programs that solves the tasks too.

https://github.com/michaelhodel/re-arc

I'm Simon Strandgaard and I participated in ARCathon 2022 (solved 3 tasks) and ARCathon 2023 (solved 8 tasks).

I'm collecting data for how humans are solving ARC tasks, and so far collected 4100 interaction histories (https://github.com/neoneye/ARC-Interactive-History-Dataset). Besides ARC-AGI, there are other ARC like datasets, these can be tried in my editor (https://neoneye.github.io/arc/).

I have made some videos about ARC:

Replaying the interaction histories, and you can see people have different approaches. It's 100ms per interaction. IRL people doesn't solve task that fast. https://www.youtube.com/watch?v=vQt7UZsYooQ

When I'm manually solving an ARC task, it looks like this, and you can see I'm rather slow. https://www.youtube.com/watch?v=PRdFLRpC6dk

What is weird. The way that I implement a solver for a specific ARC task is much different than the way that I would manually solve the puzzle. Having to deal with all kinds of edge cases.

Huge thanks to the team behind the ARC Prize. Well done.

I have been working on a fix for the scroll issue on the about page. Can you check if it's better now?

https://neoneye.github.io/arc/about.html

Based on your feedback, I have updated the about page, so it somewhat explains what ARC is, with 3 videos of different lengths. Unfortunately I don't have a video that explains it in 30 seconds.

https://neoneye.github.io/arc/about.html

The middle piece that is vertically scrollable. I'm still investigating how to fix it.

Tutorials, that is indeed something that is missing.

I failed miserable in another video when I tried explaining what ARC is, in my own opinion. It seems to be about striking a balance of both being "I can solve this", while not being to hard "I can't possible solve this". I don't think this is the right video format for the landing page. I'm unsure what kind of intro would be helpful. https://www.youtube.com/watch?v=rLGpNcQ5alI

I will investigate the iframe issue. I tried with a fixed width and height, but that was extending over the screen edge on smaller devices, so I wrapped it in a div that grows. Thank you for pointing this out.

Thank you for great feedback.

It's great for generating synthetic data for training LLMs for solving Abstraction & Reasoning Corpus (ARC) by François Chollet. Game of life helps the LLMs with a 2D understanding of the world.

https://github.com/fchollet/ARC

Your website is beautiful. Tutorials for each puzzle.

When Spanish is selected, maybe spawn the games in spanish mode.

In the Domino Dissection puzzle, it's possible to select the separator between two cells, so the selection gets kind of stuck. Maybe remove the ability to select the separator.

Making an entire unity game for each puzzle, that's no small feat.

Self promotion: I'm also making a puzzle website, https://braingridgame.com/ where users can solve ARC tasks (Abstraction & Reasoning Corpus). And source code is here: https://github.com/neoneye/ARC-Interactive

Damn. I'm on macOS and use homebrew. To my surprise I had "xz" version 6.5.1 installed on my computer!

I ran "brew upgrade" and that downgraded to version 5.4.6.

Both 1st place solutions solves 30 tasks of the 100 hidden tasks.

Impressive how few code changes they managed to convert an existing LRU library into SIEVE.

https://github.com/cacheMon/lru-rs/commit/1c45ef7fe85f4a4395...

How can a dataset be formatted for fine tuning llama?

Great video by CuriousMarc, about how the Apollo memory works. (Core Rope Memory) https://www.youtube.com/watch?v=hckwxq8rnr0