theo31

I use it and I love checking it in the morning to see what my team is up to, I don’t have to ping people and break their flow.

It helps us figure out what got done and where we are in our roadmap

Those frameworks are installed by default in our custom environment. There is no additional setup/configuration required from you.

At the moment, no, only the random hash gives some kind of security by obfuscation. More advanced security controls are coming soon.

We don't use TensorRT at the moment, but it is something that we are exploring.

Oh that's very interesting, how ready for production is it? It only works for TF right?

If you need a few dozen inferences per second per server, this is the cheapest way. And you're not depending on a proprietary solution whose parent company could go out of business in a year.

Definitely the cheapest way.

We've been in business for more than a year already actually :)

Sorry that's a typo, they are K80s: https://www.nvidia.com/en-gb/data-center/tesla-k80/

Yes! A more in-depth blog post is coming soon. We do host the hardware ourselves, for complete control over the GPUs. We found a great infrastructure provider that is also experiencing shortages.

The response time guaranteed is for a reasonably sized model. Bigger models (> 700 MB) will take a bit longer.

The model size is the zipped size of your model that is uploaded to Inferrd (either through the SDK or the website).

I'll fix the full screen problem right away, thank you for reporting.

We only have servers in the United States at the moment but are looking to have servers all around NA and EU very soon.

Yes you absolutely can!

Thank you!

We don't have any cold start delay! In our custom environment, you can do exactly what you are describing (running both CPU and GPU code). We provide you with access to the GPU and the CUDA libraries installed. It's basically lambda (minus the cold start) with GPU access.

We can scale a lot very quickly depending on how much you need.

Yes absolutely, you can run almost anything in our custom environment.

There is no cold start! We keep your service hot all the time.

Definitely not a CTF challenge. The product is still very early on and a lot of features are hidden/hard to find (that's one of the reasons the signup isn't open).

I'll send you some documentation soon.

That's fair, I'll come back with something to show :)

It's because I like working closely with early users to nail the features and experience instead of opening the floodgates right away.

thank you, fixed the typo :)

Those extras are only available to certain customers. At this price it's more the easy-to-use part that is expensive rather than the hardware itself.

1. No it's not open source, but I'm not excluding it from the roadmap :) The selling point is really the easiness of it.

2. Yes! We will be releasing tokens soon so that anyone can interact with the API.

3. Models can be up to 1GB, each model gets their own server , we only do real-time predictions for now. Meaning the models are constantly running waiting for a request

For anyone looking to deploy ML models a little bit easier, check out https://inferrd.com

It's a simple drag and drop to deploy tensorflow, scikit, spacy, keras or pytorch.

I couldn't agree more, I started building a ML hosting platform to solve your second point. I'm thinking of building a managed NN service as it is a common pain point.

https://inferrd.com

We don’t offer GPU accelerated inference for now. However it’s on our roadmap, sign up for inferrd to get updates!

You can definitely play with it, inferrd.com is free to use!

Thank you for spotting that, it's 100MB.

Theoretically we can support up to thousands of request per second under the entreprise plan.

We stopped at 50 request/s in the pricing table because that seemed like a reasonable number for most use cases.

Hi! We do not currently have PyTorch model but it's one of the next items in our roadmap.

If you have a more custom pipeline, we have a custom environment where you can deploy any custom code with specific package versions!

Hi! If don’t need any pre processing we have built in support for all the major librairies.

In addition we support custom pre and post processor via custom environments! Simply write your inference code into a predict.py and we take care of the rest.

(Btw I am really into CML.dev, great idea)

I broke my 16"'s screen because of this, and they asked me to pay $1000 to fix it.

This was in January right after the 16" came out.

If you like this concept, you might like https://clay.run which is spreadsheet + functions hosted in the cloud

See other comment, but I don't think we're talking about the same thing.

My bad, it's because I'm French :D