tomatohs

TestDriver.ai | https://testdriver.ai | QA, Engineers, Customer Support | Austin / Remote | Full-time / Part Time 95% of companies are still wasting time manually testing due to shortcomings in Playwright, Cypress, and other frameworks. Developers rank testing as the #1 blocker to release. We've built an AI Agent that performs manual testing on it's own VM with complete desktop access. It works like a specialized "Claude Computer Use."

We're scaling our early sales and seeking QA engineers, customer support, and sales engineers.

Please DM ian [at] testdriver [dot] ai

Computer-use agent for testing: https://testdriver.ai

I started a company to do exactly this a few years ago, and got to work with amazing companies testing their developer experience.

The problem is not the docs, it's Conway's law. One team designs the API, the other team designs the portal, and another team designs the SDK. The user has a holistic experience that cuts through each team.

That, and the docs are usually written first by the most technical person around, who has a hard time sharing the world view of a noob.

It's not 2010 anymore. Most startups can't even attract "the best engineers" much less hire them.

This is the late game, why would an engineer work for a fraction of a percent of equity and a below market salary when they can take a job at FANG?

You've got to be offering something really, really valuable like remote work, an interesting problem, and/or a new experience. Otherwise the math doesn't math.

TestDriver.ai | https://testdriver.ai | QA, Engineers, Customer Support | Austin / Remote | Full-time / Part Time 95% of companies are still wasting time manually testing due to shortcomings in Playwright, Cypress, and other frameworks. Developers rank testing as the #1 blocker to release.

We've built an AI Agent that performs manual testing on it's own VM with complete desktop access. It works like a specialized "Claude Computer Use."

We're scaling our early sales and seeking QA engineers, customer support, and sales engineers.

Please DM ian [at] testdriver [dot] ai

Computer-Use Agent for QA Testing https://testdriver.ai

This is exactly our workflow, though we defined our own YAML spec [1] for reasons mentioned in previous comments.

We have multiple fallbacks to prevent flakes; The "cheap" command, a description of the intended step, and the original prompt.

If any step fails, we fall back to the next source.

1. https://docs.testdriver.ai/reference/test-steps

Would love to use this for TestDriver, but needs to support Windows :*(

TestDriver.ai | https://testdriver.ai | QA, Engineers, Customer Support | Austin / Remote | Full-time / Part Time

95% of companies are still wasting time manually testing due to shortcomings in Playwright, Cypress, and other frameworks. Developers rank testing as the #1 blocker to release.

We've built an AI Agent that performs manual testing on it's own VM with complete desktop access. It works like a specialized "Claude Computer Use."

We're scaling our early sales and seeking QA engineers, customer support, and sales engineers.

Please DM ian [at] testdriver [dot] ai

I agree with you IRT scale but not speed.

Copy and demo videos are essentially one way communication channels ("fire and forget"). The creator has no idea if the message was understood.

Also, writing copy or making a video typically takes 10 - 100x longer than consuming the same video.

A friend described calls as "high bandwidth information transfer."

An average typing speed is 40wpm but an average conversation is between 120 - 150 wpm so about 3 - 4x bandwidth.

Calls also offer sub second latency and maximum priority.

When you add video and audio in there, the pure amount of data transferred is higher.

Cool! Before building a full test platform for testdriver.ai we made a similar sdk called Goodlooks. It didn't get much traction, but will leave it here for those interested: https://github.com/testdriverai/goodlooks

TestDriver.ai | https://testdriver.ai | QA Engineers, Sales | Austin / Remote | Full-time / Part Time

95% of companies are still wasting time manually testing due to shortcomings in Playwright, Cypress, and other frameworks. Developers rank testing as the #1 blocker to release.

We've built an AI Agent that performs manual testing on it's own VM with complete desktop access. It works like a specialized "Claude Computer Use."

We're scaling our early sales and seeking QA engineers, customer support, and sales engineers.

Please DM ian [at] testdriver [dot] ai

our agent is available via NPM: http://testdriver.ai

It is very helpful to give it things like:

- A list of applications that are open - Which application has active focus - What is focused inside the application - Function calls to specifically navigate those applications, as many as possible

We’ve found the same thing while building the client for testdriver.ai. This info is in every request.

unshakable foundations in reality

Yes, you are correct that it entirely lays in the reputation of the AI.

This discussion leads to interesting question, which is "what is quality?"

Quality is determined by perception. If we can agree that an AI is acting like a user and it can use your website, we can assume that a user can use your website and therefor it is "quality".

For more, read "Zen and the Art of Motorcycle Maintenance"

This comes up all the time. It seems like it would be possible, but imagine the case where you want to verify that a menu shows on hover. Was the hover on the menu intentional?

Another example, imagine an error box shows up. Was that correct or incorrect?

So you need to build a "meta" layer, which includes UI, to start marking up the video and end up in the same state.

Our approach has been to let the AI explore the app and come up with ideas. Less interaction from the user.

We do AI E2E desktop, sent you an email.

It's absolutely not the bottleneck. OpenAI can process a full resolution screenshot in about 4 seconds.

I predict an all out war over deterministic vs non-deterministic testing, or at least a new buzzword for fuzzy testing. Product people understand that a cookie banner "shouldn't" prevent the test from passing, but an engineer would entirely disagree (see the rest of the convos below).

Engineers struggle with non-deterministic output. It removes the control and "truth" that engineering is founded upon. It's going to take a lot of work (or again, a toung-in-cheek buzzword like "chaos testing") to get engineers to accept the non-deterministic behavior.

Curious what happened to the other YC Mobile AI E2E company, CamelQA (YC W24). They pivoted to AI assistants. Could be good lessons there if you're not already in touch with them.

You're totally right here, but "debugging failed tests" is a mature problem that assumes you have working tests and people to write them. Most companies don't have the resources to dedicate full engineer time to QA, and if they do nobody maintains the test.

Debugging failed test is a "first world problem"

This is the right idea and how we do it at TestDriver.ai. The deterministic selector still has about 20% fuzz matching rate, and if it fails it trys to recover.

Oh no, it doesn’t even run on your local machine if you don’t want it. It runs on VMs.

An AI that can control the desktop. It can see the screen and use the mouse and keyboard.

We’re selling it to developer teams to help with QA

http://testdriver.ai

TestDriver.ai (https://testdriver.ai) | Growth | Full-time | Austin or Remote

TestDriver helps engineering teams move quicker by offloading QA to an AI with their own development environment. TestDriver is not just generative testing, TestDriver is like an employee with it's own computer it can see and interact with.

We're seeking someone to grow the company, including marketing, sales, GTM, etc. We're a small team so you'll wear many hats.

Some technical experience required. Startup experience required. QA experience a plus.

Please email: ian[at]dashcam.io

AI makes things up so it's not appropriate for deterministic solutions. I'm sure something is coming for your mechanical linkage problem.

As for using it more, think of it as a co-worker and not a chat bot. There was a switch that had to happen in my brain before I started realizing I could just ask AI for the answer.

Starting with, why did you prefer to ask me for this answer rather than AI? This is what AI would have said:

Great question! It sounds like you've had some interesting experiences with AI so far. There are definitely more practical and impactful ways to utilize AI beyond chatbots and entertainment.

Problem-Specific AI Tools: There are specialized AI tools designed for various fields. For instance, in mechanical engineering, there are AI-driven CAD tools that can help design and optimize mechanical linkages. You might want to explore software like Autodesk Fusion 360, which incorporates AI for generative design. These tools can take your specifications and generate a variety of design options.

Learning Resources: To get more comfortable with AI, you might find it useful to check out online courses on platforms like Coursera or edX. They offer courses on machine learning, AI, and specific applications in different fields.

Experiment with AI APIs: Platforms like OpenAI, Google Cloud AI, and IBM Watson offer APIs that you can integrate into your projects. For instance, you could use them to analyze data, generate content, or even control hardware.

Join AI Communities: Engaging with communities like AI Stack Exchange, Reddit's r/MachineLearning, or even specialized LinkedIn groups can provide you with tips, resources, and examples of how others are using AI in innovative ways.

Hands-on Projects: Try building a small project that leverages AI. For example, you could create a simple AI-powered application using Python and libraries like TensorFlow or PyTorch. This could be anything from a predictive model to a small automated system.

AI is a machine that can creates more than what you put in. That is "value creation" by definition.

If you've used AI, you know it's a massive timesaver. I find a lot of those who are skeptical haven't used it very much.

thanks, updated

TestDriver.ai | https://testdriver.ai | Austin TX, ONSITE (Flex) | Everything

We are a small team building an AI QA Agent for GitHub - on a mission to eliminate repetitive work from developers. Our v2 agent (released this week) is the best desktop automation agent on the market today.

We're a very early stage startup seeking incredibly motivated engineers, sales, and customer success to join our team.

We work from home in Austin TX and meet in person 2x a week.

Email ian@dashcam.io to get in touch.