pranshuchittora

I am working on building an self-improving QA agent for software teams. Free and open-source. It is an agentic testing harness with batteries, includes the test runner infra (web & mobile), memory (vector store, local embedding model powered by transformers.js, self improving loop, issue reporting. More details - https://vostride.ai | Code - https://github.com/vostride/agent-qa

GitHub - https://github.com/vostride/agent-qa

Yes, I feel so. I started happening from june first weekish. I have shifted to claude code for planning things and codex for execution (as it is faster, though dumb)

Yes - https://github.com/vostride/agent-qa

Use https://github.com/vostride/agent-qa you don't need a QA team just ask coding harness to write tests and you can review the test runs.

Yes a lot. So with the acceleration of AI in the software engineering. Features are being shipped faster but causes regressions. The only way to verify is either you write tests with AI and spend hours reviewing them or you do manual QA. agent-qa aims to solve the later. First your product should work for the end user, later you can write clean test etc.

Try the OSS alternative - https://github.com/vostride/agent-qa

Try the OSS alternative - https://github.com/vostride/agent-qa

Try the OSS alternative - https://github.com/vostride/agent-qa

Try the OSS alternative - https://github.com/vostride/agent-qa

I did. Checkout the OSS alternative - https://github.com/vostride/agent-qa

Try the OSS alternative - https://github.com/vostride/agent-qa

Some digging FAST_MODEL = "google/gemini-3-flash" (fast mode primary) DEEP_MODEL = "openai/gpt-5.4" (deep mode primary) VISION_CLICK_MODEL= "openai/gpt-5.4" (the visual grounder)

fast: gemini-3-flash, falls back to gpt-5.4, 15-min run timeout, max 2 visual calls/step. deep: gpt-5.4, 15-min timeout, max 3 visual calls/step.

Why such a hard timeout, and why not latest models?

Hey, I just gave it a try and ran a quick test on booking.com. It took ~3 mins for a basic test. Do you cache the test steps so that future runs are faster and they don't call LLMs for the subsequent runs?

Also your current pricing is $300 for 1K tests which means $0.3 for each test. We tried out playwright mcp and it easily consumes 1M+ tokens for a test with ~20 steps (including image input). So with this pricing are you guys default alive?

Also is there a benchmark which you ran to prove the efficacy of your testing agent? because in the current stage it is a trust me bro kinda thing.

The word dictionary is curated with guardrails. Also the dictionary contains words which are 1 BPE token long. Under 5-6 characters

Thanks for those kind words. The landing page's demo required lots of sculpting. I would say that agent-qa is not only frontend focused. As you can run hooks in sandboxed env to test apis, so a better way to put it is with agent-qa you can test the product end-to-end not only UI.

But the issue with API testing / backend is that coding harnesses are really good at it. A product manager who writes user stories should be able to write tests for the product, and usually PMs don't care about the APIs.

Do give agent-qa a try, and consider giving it a star on GH https://github.com/vostride/agent-qa

Thanks!

It is being used in production at https://vostride.com/agent-qa The issues was agent-qa have many different kinds of files tests, memory etc and there's too much FK references which LLMs need to resolve. Using id-agent worked like a charm

V1StGXR8_Z5jdHi6B-myT 21 Characters, 14 Tokens Really really inefficient

Yes, we have the validation methods to verify the output. https://github.com/vostride/id-agent/#validateid

A random "-" separated words will fail the validation check.

Looks like it ;)

LLMs are good at predicting words, since each word in the id is ~1 BPE token. But uuids are random hex characters, this is where LLMs struggle to output the right ids.

You can use the .from method https://github.com/vostride/id-agent/#idagentfrominput-opts

To convert uuid or any text to id-agent based id. Then do the LLM inference and then convert it back to UUID.

No worries, Checkout https://vostride.com/agent-qa to see how we are using this in production.

Kinda similar, but this is token efficient. Each word is ~1 BPE token

Would love to, can you please create an issue on the GH repo.

Using "_" separator increases the token usage.

Yes, that a valid point. That's why we have a verification method which can be part of the harness to make sure the ids are not hallucinated.

This is what teams are doing today. But LLMs have a tendency to greedily write tests, which leads to hacky tricks to make the test succeed.

agent-qa is a harness where playwright works as an execution kernel and LLM works as a observer, planner and verifier.

Do give it a try https://vostride.com/docs/agent-qa/quickstart

Yes, https://vostride.com/docs/agent-qa/configuration/global-conf...

Peak unemployment ;)