vrv

MintMCP | Founding Engineer / Founding GTM | SF Bay Area (ONSITE)

MintMCP helps enterprises safely roll out agents to the entire organization (e.g, governance around MCP tools and agents like Claude Code / Codex / Cowork). We're building an MCP gateway with abstractions useful for 1) preventing agents from doing the unintentionally bad things you hear about every week without constraining them to be useless, 2) getting the rest of your teams and organizations to adopt them with minimal friction.

We're a small team of humans with dozens of agents: our company operates on the infrastructure we build for enterprises. We're looking for agent-forward engineers and GTM folks to help solve scaling problems, maintain the highest standards for security, make agents and MCPs usable and useful.

If you have experience in infra, security, and agents, and believe MCP, while not perfect, will evolve to become critical AI infrastructure, you might find the problems we're working on exciting!

You can apply via MCP here: https://www.mintmcp.com/hiring

MintMCP | Founding Engineer / Founding GTM | SF Bay Area (ONSITE)

MintMCP helps enterprises safely do more with Agents (e.g, governance around MCP tools and agents like Claude Code / Cowork, and others). We're building an MCP gateway as the foundation with the proper abstractions that enterprises need to allow long running agents to be safely deployed across an entire organization; organizations can easily and clearly limit the blast radius of what an agent can do, while giving it the right combination of tools to be useful.

We're a small team of humans with dozens of agents: our company operates on the infrastructure we build for enterprises. We're looking for agent-forward engineers and GTM folks to help solve scaling problems, maintain the highest standards for security, make agents and MCPs usable and useful.

Problems we're solving right now: Agent Identity, ReBaC for tool resources, scalable hosting of private MCP servers, and our agent platform that vertically integrates with the foundational MCP gateway layer.

If you have experience in infra, security, and agents, and MCPs, you might find the problems we're working on exciting!

You can see more here: https://www.mintmcp.com/hiring. Your agent might find the page interesting!

Agreed, OAuth is certainly preferred for many reasons, but replace "API keys" with "OAuth access tokens" and you have the same fundamental challenge of ensuring an LLM or untrusted code never has access to the user's secrets.

If the output schema specifies an id field, the LLM can write a code snippet that references it based on the context of the subsequent request, but the LLM doesn't need to observe the underlying value unless necessary. E.g., it can pass the 'id' opaquely to another call that receives the "id" as an input. If the user specifically wants to see the "id", the code orchestration approach can have the LLM just print the content.

bfloat16 was first in DistBelief, so it actually predates TensorFlow and TPUs (I worked on both systems). IIRC the motivation was more about minimizing parameter exchange bandwidth for large-scale CPU clusters rather than minimizing memory bandwidth within accelerators, but the idea generalized.

Thanks for mentioning that Sam, I appreciate it. Speaking for just myself:

I personally don't care what tools or frameworks people use to get work done and have repeatedly suggested people use whatever works best for them.

It also wouldn't make any sense to hoard any good stuff internally if we wanted to provide a useful framework that people wanted to use externally.

We actually don't have a huge amount of tooling internally that we hold back. The only tooling I really use is the [timeline](https://github.com/tensorflow/tensorflow/blob/f488419cd6d925...) for debugging performance (not the EEG tool in the whitepaper, I've never used it). That's available externally though, as you can see.

On some of the comments in this thread in general, I'm pretty sad at the lack of scientific rigor in the community, and that goes for any person who publishes code that differs from the results they claim, regardless of affiliation. I am happy about projects like OpenAI's RL baselines, as I know others are too.

Most of the papers and articles comparing performance of frameworks have lots of bugs and aren't even comparing the same model computation between frameworks. In fact, CNTK's article points to an external benchmark showing CNTK in a good light, but those benchmarks have bugs in them rendering the comparison incorrect (we've been sending PRs to fix them). I find it disappointing that the culture of the organization promotes calling others out for being 'irresponsible' except when it suits them.

The TensorFlow team hasn't published many articles comparing performance directly to others because it's honestly a lot of hard work to verify that you are comparing fairly. Of course, I do think TensorFlow needs to improve performance out of the box for people, and the team is working on that.

I'm sorry to hear of that experience, and it's certainly not intentional. We should (and I will) always try to be better (I know some will forever view Google as an antagonistic behemoth but usually the engineers on both sides are trying to do the right thing; it just takes time to come to consensus and understanding sometimes).

Regarding internal version: we built TensorFlow to support devices as modular plugins: the CPU and GPU devices are built this way (you can read the source code to see how device registration works), and the same registration mechanism is used for the TPU code, which can't be opensourced due to internal dependencies. Internal customers just link in an additional library to get TPU support, but it still uses the same core codebase that is available in the opensource world. I know this because I wrote a lot of the device modularity and TPU binding code :)

The codebase in GitHub is pretty much exactly the same as the internal, the main exceptions being things like having to rewrite include paths for files, filesystem plugins for internal cluster filesystems, etc; and those things are modularized so that we can have equivalent implementations in the OSS build to support things like HDFS and GCS filesystems, RDMA network layer communication, etc.

We daily sync the code between the two repositories using a suite of tools we've built. I'm on sync rotation this week and you can see all of my commits and activity on GitHub as proof; I merged something like 60 commits from the community just this week. It wouldn't make any sense for anybody (Google or the community) to maintain two different versions for something that is actively being developed by so many contributors. I've also directly worked with NVidia engineers on improvements they've made (and merged) to the system; the ones I've dealt with are great, so that statement is also false.

I'll be giving a talk about all the work we do to make this possible at OSCON next week, and if you are there feel free to catch me to ask me questions.

(I made this change anyway).

wrt GPU memory: If we made InteractiveSession by default grow memory, would that work for you? Seems sensible to change the default for InteractiveSession (e.g., for use in notebooks), but not the default for Session (production training / inference) due to the potential memory fragmentation.

TensorFlow doesn't yet do loop fusion (though I believe the specific example shown in that article may already be done via constant folding). But if you have a bunch of elementwise operations, JIT-techniques can reduce the number of memory passes over a buffer. If your model is already very computationally dense (time dominated by matmul or convolution), then this won't help as much, but otherwise, JIT techniques can help.

Windows support is definitely being worked on and lots of progress has been made, so it will eventually arrive -- just lots of little details to work out, but we're optimistic it'll come soon.

For what it's worth, we're hoping to integrate these APIs into our iOS version of the TensorFlow runtime, so you can maintain graph portability but still get the benefits of the optimized implementation on the platform.

We only just got iOS support in TensorFlow last week (https://developers.googleblog.com/2016/06/tensorflow-v09-now...), but this is something our team would like to get done, so stay tuned!

FYI, TF now supports: 1) dynamic RNNs, 2) bidirectional RNNs and will soon have 3D convolution -- that comparison is a little out of date.

The arxiv paper is just a repost of the preliminary paper, which is really more a design outline and rationale -- we may work on a more substantial paper later, we've been busy building the software.

I believe our published wheels now include the code for cuda compute 3.0, so it should work out of the box now.

(as long as the images have cudnn v4 and cuda 7.5 installed, I think :)

It's on the roadmap: https://www.tensorflow.org/versions/r0.8/resources/roadmap.h...

(Do you have an example link that doesn't work? I clicked a bunch of links there and they were all working. Feel free to file a bug at github.com/tensorflow/tensorflow)

And even those numbers on the front page are out of date :) (we're even faster now: https://github.com/soumith/convnet-benchmarks/pull/96, which is from a few weeks ago.)

The field is moving quickly enough that many published benchmarks are stale within 3 months, and it's a lot of hard work to maintain up to date benchmarks, given how many frameworks there are. Also there are performance/memory/scalability/flexibility tradeoffs everywhere, so it's hard to capture everything in one number without a tremendous number of caveats.

In our issues template, we start it with:

"GitHub issues are for bugs / installation problems / feature requests. For general support from the community, see [StackOverflow](link)"

However, we still get plenty of issues filed that are more appropriate for StackOverflow, so not sure there's anything technical you can do without integrating more closely with StackOverflow :)

Can you point me to "current memory utilization" numbers you're referring to?

Indeed we are. Here's a recent commit from today https://github.com/tensorflow/tensorflow/commit/d6f3ebfdfc1d... :)

Keeping that up-to-date and useful is as hard as keeping benchmarks in Deep Learning up to date ;)

Yeah, those are just cards we know that work: we don't have all possible GPUs to test our 20+ changes a day on, so we can't formally guarantee it will work on older cards, but in general we try our best to keep it all working for older and even newer cards.

Feel free to ping us (TensorFlow) on github issues to get installation issues resolved -- on cudnn r4 we're doing much better, and we're soon to check in a series of changes to get us roughly on par with Torch on cudnn r4.

We'd love to get a more up-to-date representation of the state of our own progress :)

One suggestion is to improve the permissions system. For example, third-party github plugins that interact with the github system (e.g., setting labels, responding to comments) require "write permissions" which gives those systems "push" access to the underlying repo. Simply separating the git repository access control from the github UI / issues / pull requests access control system would be very helpful. I'm sure there are many other examples of where the permissions system needs some finer-grained access.

Edit: Essentially the same as reported here: https://github.com/isaacs/github/issues/268

Okay, after a bunch of last minute python 3 issues, we've updated the documentation and packages, so everything should be ready.

The original public release was great to identify many potential installation issues that we couldn't possibly test ahead of time -- hopefully this next upcoming release will address some of your issues.

Otherwise, please file an issue at github and we'll do our best to help!

There will eventually be more up to date numbers -- we can only ask so much of Soumith's time. In addition, getting on par with cudnnv2 is just the first step. cudnnv3/v4 and cuda7.5 are next up.

I've read some people have been able to do hacks to get it to work, but I agree we need to make it easier. The full configurability for other cudnn versions didn't make 0.6.0 but might be in a patch update sometime soon. It's part of our work on improving GPU performance in general.