leeab

I've heard of AWS Neuron...didn't realize they had a package called NeuronX. Tbh, I feel like many AI companies have similar names. Maybe the guys who own neuronx.io might be more concerned...

We're not for nor against Slurm. I do believe it has use cases in HPC, scientific and academic settings. We think our web UI is a bit easier to use and we do offer a competing scheduler.

Our focus is definitely more on container-first, cloud-native Kubernetes environments like EKS, GKE, AKS. Also we're way more health monitoring of the actual GPU hardware rather than just scheduling jobs.

Well, it depends on how many GPU clouds you're managing. We've talked to a bunch of companies, some startups, some enterprises and the main trend we found was the sheer number of companies with GPU clusters from multiple clouds...likely due to GPU shortage.

And yes, it's nowhere near 100% but an overwhelming majority was running Kubes for GPU workloads...mainly cuz so they'd have a unifying layer that wasn't managing each cloud separately and being proficient with their respective tooling.

Are you using something else? Slurm, docker, etc?

We collect a handful of metrics, but coming from our previous lives in DevOps, we only collect just what's needed to avoid unnecessary metrics bloat.

The main 3 are:

- GPU runtime stats from NVIDIA smi

- Running pods from Kube state

- Node data & events from Kube state

We have several screens with similar information intended for different roles. For example, the Workloads screen is mainly for researchers to monitor their workloads from creation to completion. The Reports screen shows mainly cost data grouped by team/project, etc.

Someone asked about this earlier. We have considered this and may go down this route in the future. Was there something specific that you were looking for with open source? (ie: privacy, cost, etc)

Our solution is self-hosted and your data remains on your servers. And I think we do provide a fairly generous free limit of 64 GPUs.

Ahh yes...here’s how you solve that. Just install the Neurox Control plane onto any regular Kubes cluster (doesn’t need GPUs, just needs persistent storage. ie: EKS, AKS, GKE, etc) without that last flag in the instructions: `--set workload.enabled=true` (<-- leave this out). More info: https://docs.neurox.com/installation/alternative-install-met...

Then on your GPU cluster w/o disk, you just need to install the Neurox Workload agent. In the Web Portal UI, click on Clusters > New Cluster and copy/paste the snippet there.

One of the reasons we went down the self-hosted route is to ensure that your data remains on your servers. Since our architecture allows for separation between where our control plane lives vs where GPU workloads run, we can definitely host the control plane portion for you. Then you just need to run our agent only on your GPU cluster. Shoot me an email: lee at neurox.com and we can discuss!

We have considered this and may go down this route in the future. One thing we asked ourselves was what open sourcing provides. Usually it's a desire for privacy or cost in the form of self-hosting, among other reasons.

Currently, our free version is self-hosted and monitors clusters with up to 64 GPUs. We feel this will work for many use cases, especially just to try it out. Monitoring GPUs typically requires you to deploy something where your GPUs live. Since you’re already installing software on your cluster, you might as well keep your data there too.

Haha...there is some truth to that. We do use Prometheus under the hood to collect metrics. However, our thesis is that metrics alone isn’t enough. We marry metrics + kube state + cost data to get the whole picture.

Also we're purpose built to monitor GPUs, so we have things like drilling down from a Kube cluster, down to GPU nodes, down to a GPU card.

GPU observability is broken, so we built Neurox.

When I co-founded Mezmo (a Series D observability platform), we obsessed over logs, metrics, and traces. I learned firsthand how critical app-level observability is for DevOps, cutting through logging noise and finding the needle in the haystack is everything.

But after diving into AI infra, I noticed a huge gap: GPU monitoring in multi-cloud environments is woefully insufficient.

Despite companies throwing billions at GPUs, there's no easy way to answer basic questions:

- What's happening with my GPUs?

- Who's using them?

- How much is this project costing me?

What's happening: Metrics (like DCGM_FI_DEV_GPU_UTIL) told us what was happening, but not why. Underutilized GPUs? Maybe the pod is crashlooping, stuck pulling an image, or misconfigured, or the application is simply not using the GPU.

Who's using the compute: Kubernetes metadata such as namespace or podname gave us the missing link. We even traced issues like failed pod states, incorrect scheduling, and even PyTorch jobs silently falling back to CPU.

How much is this gonna cost: Calculating cost isn't easy either. If you're renting, you need GPU-time per pod and cloud billing data. If you're on-prem, you'll want power usage + rate cards. Neither comes from a metrics dashboard.

---

Most teams are duct-taping scripts to Prometheus, Grafana, and kubectl.

So we built Neurox - A purpose-built GPU observability platform for Kubernetes-native, multi-cloud AI infrastructure. Think:

1. Real-time GPU utilization and alerts for idle GPUs

2. Cost breakdowns per app/team/project and finops integration

3. Unified view across AWS, GCP, Azure, and on-prem

4. Kubernetes-aware: connect node metrics to running pods, jobs, and owners

5. GPU health checks

Everyone we talked to runs their compute in multi-cloud and uses Kubes as the unifier across all environments. Metrics alone aren't good enough. You gotta combine metrics with Kube state and financial data to see the whole picture.

Check us out, let us know what we're missing. Curious to hear from folks who've rolled their own, what did you do?

Lee @ Neurox

As a founder, these are the top 3 things I wish someone had told me about balancing cost and scalability. For other YC founders out there, are these the same things you struggled with?

Sounds good! Yeah our site is definitely in need of some love. Just haven't had time (busy plugging tech debt and working on new stuff). Planning on major improvements next month along with onboarding.

Just created a live example: https://logdna.com/frame_tester.html

The gif was from our docs page which afaik doesn't allow js.

Hi everyone, I'm a co-founder / CTO of LogDNA. We were in Y Combinator's W15 batch and launched our cloud logging platform last year (https://news.ycombinator.com/item?id=11074537)

From the beginning, we’ve always wanted to be able to embed logs (light log viewer with live tail) onto any web page and we’ve just released this! Let us know if you have any feedback or cool use cases.

So far we’ve seen these pretty awesome uses from early testers:

1) custom internal dashboards with 1 embedded log viewer tab per internal app

2) showing customers their own logs

3) show inner workings of their app

Screenshot: https://files.readme.io/2351a3a-embedded-views.gif

I’m happy to answer any questions you may have!

Congratulations on the launch!! LogDNA is proud to be a part of the Manifold journey. Can't wait to see what's in store next!

Awesome! And definitely let us know if we can improve in any way.

Thanks JJ! Yeah, we hope to earn your Kubernetes business :)

Hmmm this may have been a bug. It should show stderr as err. Let me look into this.

Yeah one of those mysteries of life. :/ One day we'll probably go back and try this on a few older versions to see if we can repro what happened. But yeah, it was right off the bat when we started our beta, 2 different people said they had to modify the yaml in order to get it to work (something we wanted to avoid altogether).

Awesome! We're pretty responsive on Intercom as well if you run into issues post install. But definitely share your feedback if you notice areas that can be improved.

Yes that's true for many things in Kubernetes but not as frequent in the logging space for Kubernetes. Many still rely on Fluentd as a parsing filter first to extract metadata like pod name or container name. And most of the logging ones require you to edit your yaml in some way first. Ours is almost copy-pasteable, provided you have a LogDNA API key. :)

Ahh yes, thanks for catching that. I just made a note that we collect from all namespaces currently.

Yeah, we now set our agent up w/o namespaces. During our beta, we originally had it set up inside `kube-system` but 2 of our testers mentioned that the pod wouldn't install unless it was in `default`. So we moved it out of `kube-system` but we still weren't sure what caused the issue since it worked fine on our cluster. We were using Kubernetes v1.4 so it could've been an older version issue.

:) Hope you'll give us a shot. And let us know what you think!

We currently treat each line separately and we can maintain line ordering okay.

We have thought about this before and doing something like: if a line starts with tab or a fixed number of spaces a few times in a row, treat it as 1 line with \n's and store it as such. It would help with alerting and filtering alerts. Just not 100% sure if this will screw up anything.

Hi everyone, I'm a co-founder / CTO of LogDNA. We were in Y Combinator's W15 batch and launched our cloud logging platform last year (https://news.ycombinator.com/item?id=11074537)

Based on user feedback, we're happy to announce our super easy Kubernetes integration. No more wrestling with fluentd configs, fiddling with Elasticsearch knobs or following 30-step guides cutting and pasting other people's configs.

  kubectl create secret generic logdna-agent-key --from-literal=logdna-agent-key=<YOUR LOGDNA API KEY>
  kubectl create -f https://raw.githubusercontent.com/logdna/logdna-agent/master/logdna-agent-ds.yaml

Feel free to try it out. Happy to answer any questions!

Hey! Lee here (CTO / co-founder) wanted to say thanks for all the feedback when we launched in February. We're constantly adding new features weekly and if there's anything we can help with, I'm happy to answer any questions!

Not yet. We're still figuring our what's needed for this. It seems there's no auditing requirement at all.

No, sorry, I just didn't have time to set this up. We'll get this before the updates are made.

We do automatically parse most common log formats and you can query on those. You can also send us JSON and query on that too.

I'd love to see what you have worked on. :)