aaronvg

why not write all code in assembly?

didn't expect someone to post us here yet, we're still working on our launch!

We have been building BAML for around 3 years -- originally it was a DSL for getting structured outputs from llms using what we call 'llm functions'

Our users told us they wanted to do more, so we made it an actual turing-complete language, still with a focus on people building AI workflows and calling nondeterministic models. This is why we added very flexible testing capabilities into BAML itself for test parameterization, dynamic tests, etc.

As we were building it, we realized humans weren't going to be the ones writing the code, so we started researching what we could do on this front. This led us to design BAML to use all fully-qualified-names for everything (classes, enums,functions etc) to name one example. We found in some cases AI agents saved 30% tokens navigating baml codebases vs TypeScript ones. There's no imports to track down, etc.

We do aim to keep humans in the loop, and we'll do some more deep dives that tackle each feature.

these painpoints seem moot in a world where AI agents are writing all the code.

kind of surprised Swift didn't launch with this by default, built in-house

this is really neat. Was using or-tools to solve my friend's scheduling problems with coworkers.

We are working on making our programming language BAML turing-complete. https://github.com/boundaryml/baml

It is a language that is embeddable in other programming languages, with the type system similar to typescript, and a runtime that is similar to Go.

People use it currently for structured outputs with llms but soon we will support orchestration and more.

We are letting some users have an early access preview! Let me know if you are interested in hacking with it!

It's 'basically, a made-up language'. It's just tongue-in-cheek because when we started this it was just a ridiculous proposition to try and make a DSL.

I'll add it in!

Boundary (YC W23) | Software engineer (compilers) | Seattle, USA (in person) | Full-time

We are building a new programming language (BAML) to build AI agents -- the "typescript" for LLMs. We are open source: https://github.com/BoundaryML/baml

A big part of this language is all the tooling around visualizing non-deterministic code, visualizing code, and getting great observability (e.g. our language has type information at runtime unlike TS).

We are looking for engineers with experience with Rust, programming languages, and/or compilers. Any amount of experience is fine.

To apply: send an email to aaron@boundaryml.com with your resume and mention you came from HN

definitely will just say picks...

You may also want to check out BAML https://github.com/BoundaryML/baml - a DSL for prompt templates that are literally treated like functions.

the prompt.yaml format (which this project uses) suffers from the fact that it doesn't address the structured outputs problem. Writing schemas in yaml/xml is insanely painful. But BAML just feels like writing typescript types.

I'm one of the developers!

super interesting to see how this is marketed:

- Created by an AWS team but aws logo is barely visible at the bottom.

- Actually cute logo and branding.

- Focuses on the lead devs front and center (which HN loves). Makes it seem less like a corporation and more like 2 devs working on their project / or an actual startup.

- The comment tone of "hey ive been working on this for a year" also makes it seem as if there weren't 10 6-pagers written to make it happen (maybe there weren't?).

- flashy landing page

Props to the team. Wish there were more projects like this to branch out of AWS. E.g. Lightsail should've been launched like this.

We're making a prompting DSL (BAML https://github.com/BoundaryML/baml) and what we've found is that all the syntax rules can easily be encoded into a Cursor Rules file, which we find LLMs can follow nicely. DSLs are simple by nature so there's not too many rules to define.

Here's the cursor rules file we give folks: gist.github.com/aaronvg/b4f590f59b13dcfd79721239128ec208

You might also find Semantic Streaming interesting. It's t he same concept but applied to llm token streaming. It's used in BAML (the ai framework). https://www.boundaryml.com/blog/semantic-streaming

I'm one of the developers of BAML.

it's kind of wild -- none of the multimillion dollar VSCode forks (Cursor, windsurf) are working properly at the moment. It seems open-vsx is quite a vulnerable single point of failure. Searching extensions gives a 503.

It's kind of insane going from 76% to 3% on the new version of a benchmark. We clearly need more rapid progress on the creation of benchmarks.

Then again, I wonder -- if a benchmark is way too hard from the beginning, would it make it much harder for people to test new solutions that actually have real-world impact, even if the new results on the hard benchmark only increased the score by 1%?

Sometimes you don't have access to a model, so this approach still works in that scenario.

We also have several users that have anecdotally told us they get worse results using constrained grammar solutions.

Thanks, we also do support swapping models at runtime!

Will incorporate this feedback.

How do you organize your prompts? Do you use a templating language like jinja? How complex are your prompts? Do you have any open source examples?

I’m genuinely curious since if we can convince someone like you that BAML is amazing we’re on a good track.

We’ve helped people remove really ugly concatenated strings or raw yaml files with json schemas just by using our prompt format (which uses jinja2!)

I found it interesting how DSPy created the Signatures concept: https://dspy.ai/learn/programming/signatures/

We took this kind of concept all the way to making a DSL called BAML, where prompts look like literal functions, with input and output types.

Playground link here https://www.promptfiddle.com/

https://github.com/BoundaryML/baml

(tried pasting code but the formatting is completely off here, sorry).

We think we could run some optimizers on this as well in the future! We'll definitely use DSPy as inspiration!

First off, this is amazing. Imagine where this will be in 5 or 10 years.

There's just so many questions: Will videogames eventually just be a single prompt (or a series of prompts / a general script) into a transformer model?

Would services like stadia be making a comeback if the value prop is that even though you have some latency, the possibilities for videogames will be endless?

Would microsoft want a piece of the $$ if the transformer was trained on Minecraft clips?

We really are in an acceleration phase

(BAML founder here) feel free to jump on our Discord or email us if you have any issues with BAML! Here's our repo (with docs links) https://github.com/BoundaryML/baml and a demo: https://boundaryml.wistia.com/medias/5fxpquglde

People have used it to do anything from simple classifications to extracting giant schemas.

One of the issues with LLMs is that each function needs to be tested and observed more thoroughly than regular functions, which leads to these UIs to figure out what the heck is actually going on.

For our DSL (BAML https://github.com/BoundaryML/baml ) we found that adding a VSCode playground to visualize LLM function inputs and outputs was a massive win in terms of debuggability and testability, so I can see why langraph is going this way.

thanks for the shoutout, we benchmarked our approach against other function-calling techniques and we've been able to beat all other approaches every time (even by 8%!) just by getting better at parsing the data and representing schemas with less tokens using type definitions instead of json schema.

You can take a look at our BFCL results on that site or the github: https://github.com/BoundaryML/baml

We'll be publishing our comparison against OpenAI structured outputs in the next 2 days, and a deeper dive into our results, but we aim to include this kind of constrained generation as a capability in the BAML DSL anyway longterm!

I doubt anyone would be confused with Traceloops the artist vs Traceloop the LLM Observability Platform

[Another BAML creator here]. I agree this is an interesting direction! We have a "chat" feature on our roadmap to do this right in the VSCode playground, where an AI agent will have context on your prompt, schema, (and baml test results etc) and help you iterate on the prompt automatically. We've done this before and have been surprised by how good the LLM feedback can be.

We just need a bit better testing flow within BAML since we do not support adding assertions just yet.

I think it depends on what you value as well, like DX. A large portion of our users switch to BAML because they actually "just want to see the damn prompt".

My bad, I think I didnt explain correctly. Basically you have two options when a "," is missing (amongst other issues) in an LLM output which causes a parsing issue:

- retry the request, which may take 30+ secs (if your LLM outputs are really long and you're using something like gpt4)

- fix the parsing issue

In our library we do the latter. The conversion from BAML types to Pydantic ones is a compile-time step unrelated to the problem above. That doesn't happen at runtime.

[Other BAML creator here!] one time we told a customer to do this to fix small json mistakes but turns out their customers don't tolerate a +20-30s increase in latency for regenerating a long json structure.

We instead had to write a parser to catch small mistakes like missing commas, quotes etc, and parse content even if there's things like reasoning in the response, like here: https://www.promptfiddle.com/Chain-of-Thought-KcSBh

Not sure it's related to function calling. GPT4 can do function calling without using the specific function-calling API just by injecting the schema you want into the prompt with directions and asking it to return JSON. It works like >99% of the time. Same with 3.5-turbo.

The problem is these libraries convert pydantic models into json schemas and inject them into the prompt, which uses up like 80% more tokens than just describing the schema using typescript type syntax for example. See https://microsoft.github.io/TypeChat/, where they prompt using typescript type descriptions to get json data from LLMs. It's similar to what we built but with more boilerplate.

We felt this 100% so we built a DSL (called BAML) to solve the "prompt transparency" problem (amongst other issues). We have a VSCode playground that always shows you the full prompt -- kinda like a markdown preview works.

We are still in beta (and open source) but feel free to check it out! https://docs.boundaryml.com/ .

Some of these frameworks like instructor use like 80% more tokens or only work with OpenAI so we aim to tackle all these problems from the ground up.