gauravvij137

Neo is a specialised AI/ML engineering agent that can be accessed via an MCP server inside Claude Code.

Claude code is great at software engineering but when it comes to iterative AI tasks that require long horizon experimentation and results may not come in one go, Claude limits exhaust very fast.

The crux is: We shouldn't let Claude babysit long AI workflows!

These tasks are inherently iterative and can consume hundreds of turns.

With Neo MCP, Claude Code becomes the orchestrator and Neo heavy-lifts complex AI tasks and provides direct status updates with execution logs to Claude running locally in your environment.

This way, Claude spends way less tokens and also Neo consumes upto 62% lower cost compared to direct Claude API usage, thus delivering outcomes as a combined team at way cheaper costs.

This has been our cheat code since last month and we think there is a lot of scope even now to further optimize and mature Neo.

We appreciate any feedback or thoughts from you.

They've come along pretty far now.

I remember when there was hype around GLM 5 reaching great heights on benchmarks but eventually failing on practical coding and reasoning tasks. I guess this time the hype is real.

is this different from accessing on Openrouter?

A fully automated prompt optimizer - https://github.com/gauravvij/autoprompter

Built with inspiration from Karpathy's AutoResearch + PromptFoo.

The data leaving AWS boundary kills this for any regulated workload. We've been running side-by-side evals of open models against Claude on private test suites, using Neo as the orchestration layer. Keeps everything in-house and gives us objective comparison data.

Five of the top 10 AI repos on GitHub trending this week are "skills" packs for Claude Code. The label is doing wildly different work across them.

forrestchang/andrej-karpathy-skills (~70k stars). One CLAUDE.md file, four behavioral rules, derived from Karpathy's January tweet about agent coding failure modes (silent wrong assumptions, over-complication, not surfacing tradeoffs). Karpathy didn't write the file or endorse it. The README has had a typo in the install command (andrej-karpthy-skills, missing the second "a") since launch. A second repo, multica-ai/andrej-karpathy-skills, is trending in parallel with the same content republished.

mattpocock/skills (~115k stars). Matt Pocock's personal .claude/skills/ folder, published. About 10 small SKILL.md files: tdd, to-issues, to-prd, triage, zoom-out, setup-matt-pocock-skills. Each one is a self-contained markdown prompt with YAML frontmatter declaring when it should auto-fire. Third-party writeups describe it as a reference implementation of Anthropic's SKILL.md format.

affaan-m/everything-claude-code (~175k stars; plus a second repo affaan-m/ECC at ~205k stars which is the same project under a renamed identifier). 48 agent definitions, 182 SKILL.md files, 68 legacy slash-command shims, hooks, rules, MCP configurations, npm packages (ecc-universal, ecc-agentshield), a Tkinter desktop dashboard, and a security scanner (1282 tests, 102 static analysis rules). Includes per-harness adapters for Claude Code, Codex CLI, Codex macOS app, Cursor, OpenCode, Gemini CLI, and Antigravity. Anthropic hackathon winner.

Three orders of magnitude in scope under one word.

What's underneath is Anthropic's SKILL.md format: markdown with YAML frontmatter, auto-loaded at session start. The frontmatter declares when the skill should fire; the harness picks relevant skills based on the description and injects only those into context. It's RAG-over-prompts using model-based routing on descriptions rather than vector stores. The format works well enough that you can mix skills from different authors in the same .claude/ folder without the harness caring, which is the actual reason this took off. Trending packs ship per-harness adapters on top of that substrate so the same skill content installs into Codex, Cursor, OpenCode, etc., with per-harness rewrites.

The trending list is measuring three different things under one label: small high-leverage CLAUDE.md edits (karpathy-skills, cost-to-try wins), curated personal reference sets (mattpocock, distribution-by-reputation wins), and full framework distributions (ECC, comprehensive-catalog-marketing wins). Stars are not telling us which of these are surviving on actual reuse.

The defensibility implications are uncomfortable. When a startup pitches "our agent does X better because of our prompting and workflow," and the artifact is a folder of markdown files with YAML frontmatter, that's a Notion template, not a moat. Karpathy's four rules will be absorbed into the default behavior of the next Claude release. mattpocock's TDD skill is sharp but copyable. ECC's 182-skill catalog is impressive engineering, but the prompts inside can be diffed and ported in an afternoon.

What does seem to hold value after reading these: harness ergonomics (install paths, hook plumbing, cross-tool sync scripts, MCP-server lifecycle), distribution (mattpocock-the-person is a moat, the markdown isn't), and security tooling around skill files specifically (AgentShield, ECC's scanner, is a real product even if the skills it scans aren't). None of those are the prompt.

Repos: github.com/forrestchang/andrej-karpathy-skills, github.com/mattpocock/skills, github.com/affaan-m/everything-claude-code.

Working with embeddings (RAG, semantic search, clustering, recommendations, etc.), means: - Generate embeddings - Compute cosine similarity - Run retrieval - Hope it "works"

But then I stumbled upon the issue of not being able to determine why my RAG responses felt off, retrieval quality being inconsistent and clustering results looked weird.

Debugging embeddings was painful.

To solve this issue, we built this Embedding evaluation CLI tool to audit embedding spaces, not just generate them.

Instead of guessing whether your vectors make sense, it: - Detects semantic outliers - Identifies cluster inconsistencies - Flags global embedding collapse - Highlights ambiguous boundary tokens - Generates heatmaps and cluster visualizations - Produces structured reports (JSON / Markdown)

Checkout the tool and feel free to share your feedback: https://github.com/dakshjain-1616/Embedding-Evaluator

This is especially useful for: - RAG pipelines - Vector DB systems - Semantic search products - Embedding model comparisons - Fine-tuning experiments

It surfaces structural problems in the geometry of your embeddings before they break your system downstream.

We just released an open source project for agent swarm based stock trading simulations built and tested by heyneo.so - Fully autonomous ML engineering agent.

The agent swarm self coordinates with each other via an asynchronous message bus.

There are around 10 agents with distinct roles: - 3 Analyst Agents → Generate BUY/SELL signals (SMA crossovers, volume trends) - 4 Trader Agents → Execute trades, manage $250K portfolios each - 2 Risk Managers → Validate orders, enforce stop-loss rules - 1 Reporter Agent → Aggregate P&L and generate reports

The simulation consists of capital allocation, risk checks like stop-losses and order blocking, and reporting baked into the flow. The system backtests over ~250 trading days, starts with a fixed $1M capital, and logs things like drawdown, blocked orders, and approval rates.

Repo here if anyone wants to dig into the implementation or poke holes in the design: https://github.com/dakshjain-1616/Stock-trading-Agent-Swarm-...

Extend the project in your VS Code IDE with Neo: https://marketplace.visualstudio.com/items?itemName=NeoResea...

This project implements Quantization-Aware Training (QAT) for MobileNetV2, enabling deployment on resource-constrained edge devices. Built autonomously by [NEO](https://heyneo.so), the system achieves exceptional model compression while maintaining high accuracy.

Solution Highlights: - 9.08x Model Compression: 23.5 MB → 2.6 MB (far exceeds 4x target) - 77.2% Test Accuracy: Minimal 3.8% drop from baseline - Full INT8 Quantization: All weights, activations, and operations - Edge-Ready: TensorFlow Lite format optimized for deployment - Single-Command Pipeline: End-to-end automation

Training can be performed on newer Datasets as well.

Project is accessible here: https://github.com/dakshjain-1616/Quantisation-Awareness-tra...

Founder here. I built NEO, an AI agent designed specifically for AI and ML engineering workflows, after repeatedly hitting the same wall with existing tools: they work for short, linear tasks, but fall apart once workflows become long-running, stateful, and feedback-driven. In real ML work, you don’t just generate code and move on. You explore data, train models, evaluate results, adjust assumptions, rerun experiments, compare metrics, generate artifacts, and iterate; often over hours or days.

Most modern coding agents already go beyond single prompts. They can plan steps, write files, run commands, and react to errors. Where things still break down is when ML workflows become long-running and feedback-heavy. Training jobs, evaluations, retries, metric comparisons, and partial failures are still treated as ephemeral side effects rather than durable state.

Once a workflow spans hours, multiple experiments, or iterative evaluation, you either babysit the agent or restart large parts of the process. Feedback exists, but it is not something the system can reliably resume from.

NEO tries to model ML work the way it actually happens.

It is an AI agent that executes end-to-end ML workflows, not just code generation. Work is broken into explicit execution steps with state, checkpoints, and intermediate results. Feedback from metrics, evaluations, or failures feeds directly into the next step instead of forcing a full restart. You can pause a run, inspect what happened, tweak assumptions, and resume from where it left off.

Here's an example as well for your reference: You might ask NEO to explore a dataset, train a few baseline models, compare their performance, and generate plots and a short report. NEO will load the data, run EDA, train models, evaluate them, notice if something underperforms or fails, adjust, and continue. If training takes an hour and one model crashes at 45 minutes, you do not start over. Neo inspects the failure, fixes it, and resumes.

Happy to answer questions about Neo.

Did you miss reading the official leaderboard score on openai/mle-bench page where it clearly states that Neo has the best score on mlebench?

NEO is the first fully autonomous Machine Learning Engineer.

It can reason, plan and iterate on complex data cleaning, feature engineering and model training tasks.

Neo is now the SOTA Machine Learning Engineering agent on OpenAI's MLE-Bench.

For sure, the request is going to a proprietary API for heavy lifting but the idea of this colab notebook is to extend the possibility of running 100s of experiments by calling out an API that handles the experiments as requested while the colab notebook becomes that single session of performing experiments on different models, hyperparameters and datasets, perform evals on all and get one final deployment.

Here's a free Colab notebook to fine-tune Gemma 2 2B-it model on MonsterAPI.ai

Gemma 2 2B from @Google @GoogleDeepMind is quite an interesting model as it's only 2 billion parameters in size and is able to beat a variant of GPT 3.5 Turbo. That's not a small feat. If a base model is good at majority of tasks then fine-tuning it for domain specific use-case provides a massive boost up.

And with Monster Tuner, fine-tuning LLMs like Gemma 2 2B is just a couple of clicks and the tuner comes auto-integrated with optimized frameworks like Unsloth, SDPA and FA 2 to fasten up the token processing.

This notebook has a complete workflow where it also formats the provided dataset to suit the EOS token formatting required by Gemma 2 2B IT model to deliver better fine-tuning results with a complete token distribution chart to help us get better insights about our data.

Get started with fine-tuning Gemma 2 2B on this colab notebook https://colab.research.google.com/drive/1FeFeM1viF6jNJDYgUXf...

We have developed an exhaustive prompt by prompt comparison case study between Llama 405B and GPT 4O

Overall, GPT 4o's answer seemed to be more acceptable as per the use-case but 405B was not much behind. In fact in many areas, it seems to outperform and remain context relevant.

Checkout the in-depth comparative analysis: https://blog.monsterapi.ai/blogs/llama-3-1-405b-vs-gpt4o-hea...

I have repeatedly asked this question in the chat I get by clicking on "Try 405B model" on Meta AI website and then it always tells me that it has 70 Billion params.

If that's the case then it might end up giving wrong information and we might assume it's still that big model.

Not sure if it's just me or everyone getting this?

This colab notebook uses 2 OSS LLMs: - Zephyr 7B - SDXL

And it produces 4 outputs: - Blog content - Blog title - Meta description - Header image(s)

Zephyr is really powerful and cost effective in terms of producing high quality output while constraining itself to the provided topic. It follows system prompts very well.

SDXL on the other hand is great for image generation in OSS domain. This API also supports styles, so they can be applied as per the blog theme requirements.

At the core, the LLM APIs are powered by monsterapi.ai

Finetuning Mistral on No Robots dataset leads to higher performance compared to Finetune Zephyr 7B OSS model.

Just speak and use natural language to ask for an image and this notebook will generate high fidelity images using a pipeline of 3 LLMs hosted on MonsterAPI.

This integration results in access of LLMs like Llama 2, Falcon and MPT easily and at low cost for RAG question answering on PDFs and docs.

This guide walks you through using a no-code llm finetuner for finetuning Llama 2 7B and 13B models without any complex setup of infrastructure to run it.

The only way to get rid of centralized choke points is to actually go decentralized. At Q Blocks, we're working on making this solution a reality for a lot of the ML devs constrained by the computing costs on cloud.

Similar to this approach, at qblocks.cloud we bring under-utilized GPU servers from crypto miners and data centers to use for AI training and deployments at 50-80% low cost than traditional clouds. On-demand and at scale.

In a way, it is similar to our approach at Q Blocks. Decentralized GPU computing for ML.