rhythmvertigo

Iterative (DVC) | Full Time | Technical Community Manager | Remote

We are the company behind very popular open-source tools for ML workflow- DVC.org and CML.dev. We are solving the complexity of managing datasets, ML infrastructure, ML models lifecycle management. We are Hashicorp for ML.

We’re seeking a Technical Community Manager to help us sustain and grow our active, worldwide community!

Requirements:

experience in data science or open source software

AND

passion for building a community of developers, data scientists and engineers

Listing: https://weworkremotely.com/listings/iterative-technical-comm...

coooool! going to try this out :)

Yep, it will work with your GPUs! You just set up self-hosted runners as usual https://docs.gitlab.com/runner/

GitHub & GitLab have both made it quite easy to use your own resources as runners. I recently met someone who was doing Actions with a Jetson Nano on their dresser :)

Yes, we're aiming to have another at the end of July! If you have a particular use case you're interested in, we'd love to know. We might be able to develop some materials around it.

Really glad to hear that, rkaplan. Let us know if we can be of any help!

Definitely- automatic model deployment would only work for a subset of applications. We think a lot more about having models available as candidates for deployment, which can then be inspected by domain experts on a team.

One of our motivations for building visual reports that appear like comments in a pull request is giving teams metrics & info to discuss when deciding if merge is right. That way, the automated part is training and testing, but the decision making is human (i.e., data scientists whose skills are better used interpreting models & data than running repetitive training scripts).

Out of curiosity, what kind of research? I'm very interested in getting DVC and Git more broadly into academic research circles, but finding a lot of barriers in my home field.

Thanks for the kind words! DevOps has had profound results and we think culturally and technically, it's the right moment to push for those practices in ML.

Yeah, we were seeing a lot of users organically asking about CI/CD. DVC had to come first to address some of the data management issues, which seemed like the biggest hurdle to CI. But we're excited to add this now.

Thanks! Hashicorp is our inspiration :D

Thanks, and good questions!

1. Yes! Let me link some reports and example repos:

- A basic classification problem with scikit learn: https://github.com/iterative/cml_base_case/pull/2

- CML with DVC & Vega-Lite graphs: https://github.com/iterative/cml_dvc_case/pull/4

- Neural style transfer with EC2 GPU: https://github.com/iterative/cml_cloud_case/pull/2

2. If training fails, you'll be notified that your run failed in the GitHub Action dashboard (or GitLab CI/CD dashboard). See here for some real life examples of failure ;) : https://github.com/iterative/cml_cloud_case/actions

3. CML reports are markdown documents, so you can write any kind of text to them. If your metrics are output in a file `metrics.txt`, you can have your runner execute `cat metrics.txt >> report.md` and then have CML pass on the report to GitHub/GitLab. Likewise, any graphing library is supported because you can add standard image files (.png, .jpg) to the report. So custom metrics and custom graphs. We like DVC for managing and plotting metrics, but we're biased because we also maintain it.

4. Yep, GitHub Actions is pretty powerful and flexible. Works with whatever external services you can connect to your Action!

5. It's not strictly a Docker technology. We use Docker images preinstalled with the CML library in our examples, but you can just install the library with npm in your own image. https://github.com/iterative/cml#using-your-own-docker-image

Let me know if there's anything else I can tell you about

Hi, I'm one of the project creators. Continuous Machine Learning (CML) is an open source project to help ML projects use CI/CD with Github Actions and Gitlab CI (https://github.com/iterative/cml).

CML automatically generates human-readable reports with metrics and data viz in every pull/merge request, and helps you use storage and GPU/CPU resources from cloud services. CML addresses three hurdles for making ML compatible with CI:

1. In ML, pass/fail tests aren’t enough. Understanding model performance might require data visualizations and detailed metric reports. CML automatically generates custom reports after every CI run with visual elements like tables and graphs. You can even get a Tensorboard.dev link as part of your report.

2. Dataset changes need to trigger feedback just like source code. CML works with DVC so dataset changes trigger automatic training and testing.

3.Hardware for ML is an ecosystem in itself. We’ve developed use cases with CML and Docker Machine to automatically provision and deploy cloud compute instances (CPU & GPU) for model training.

Our philosophy is that ML projects- and MLOps practices- should be built on top of traditional software tools and CI systems, and not as a separate platform. Our goal is to extend DevOps’ wins from software development to ML. Check out our project site (https://cml.dev) and repo, and please let us know what you think!

We have a lot in store here- we are unrolling a new tool for CI/CD soon that works with GitHub Actions & GitLab CI. Adding run-cache to DVC 1.0 is just one way of preparing for more CI/CD uses of DVC. (FYI I am part of DVC)