HN user

shcheklein

1,433 karma

Building DVC.org, CML.dev, and other ML workflow tools.

Posts176
Comments90
View on HN
datachain.ai 2mo ago

OpenAI's Data Agent and the S3 Gap

shcheklein
3pts0
om.co 2mo ago

What Microsoft's 10-Q Says About OpenAI

shcheklein
4pts0
zencoder.ai 7mo ago

Zenflow: Free desktop AI Orchestration app with multi-agent verification

shcheklein
6pts5
github.com 1y ago

DVC Extension for Visual Studio Code

shcheklein
2pts0
zencoder.ai 1y ago

Zencoder: New AI Coding Agent

shcheklein
3pts0
github.com 1y ago

Albumentations: Fast and flexible image augmentation library

shcheklein
1pts0
marketplace.visualstudio.com 1y ago

Using VSCode to track and visualize AI experiments

shcheklein
4pts0
github.com 1y ago

Using VSCode to track and visualize AI experiments

shcheklein
3pts0
marketplace.visualstudio.com 1y ago

Using VS Code to track and visualize ML experiments

shcheklein
1pts0
pixeltable.com 1y ago

Pixeltable: Store, transform, index, and iterate on data for ML

shcheklein
1pts0
marketplace.visualstudio.com 1y ago

Using VS Code to track and visualize ML experiments

shcheklein
2pts0
github.com 1y ago

Using VS Code to track and visualize ML experiments

shcheklein
2pts0
github.com 1y ago

Pixeltable: Store, transform, index, and iterate on data for ML

shcheklein
1pts0
www.getprog.ai 1y ago

Hirsch Index for Open Source Contributors

shcheklein
2pts0
github.com 1y ago

Machine learning experiment tracking for VS Code

shcheklein
1pts0
github.com 1y ago

Evidently: Open-source ML observability platform

shcheklein
2pts0
cml.dev 1y ago

CML: Continuous Machine Learning CI/CD for ML

shcheklein
2pts0
marketplace.visualstudio.com 1y ago

Using VS Code to track and visualize ML experiments

shcheklein
1pts0
github.com 1y ago

Using VS Code to track and visualize ML experiments

shcheklein
3pts0
albumentations.ai 1y ago

Albumentations: Fastest and most flexible image augmentation library

shcheklein
3pts0
github.com 1y ago

Albumentations: Fast Image Augmentations

shcheklein
2pts0
github.com 1y ago

Argilla: Build high quality datasets for your AI models

shcheklein
2pts0
www.datacomp.ai 1y ago

DataComp LM: Data Centric Machine Learning Benchmark

shcheklein
2pts0
github.com 1y ago

Evidently: Open-source ML observability platform

shcheklein
1pts0
marketplace.visualstudio.com 1y ago

Using VS Code to track and visualize ML experiments

shcheklein
5pts0
www.getprog.ai 1y ago

Hirsch Index for Open Source Contributors

shcheklein
1pts0
github.com 1y ago

Axolotl: Fine-tuning framework for various AI models

shcheklein
1pts0
dottxt-ai.github.io 1y ago

Structured text generation and robust prompting for open LLMs

shcheklein
1pts0
github.com 1y ago

Fast and scalable dataset preparation and curation tool from Nvidia

shcheklein
2pts0
github.com 1y ago

CML: Continuous Machine Learning CI/CD for ML

shcheklein
2pts0

If they have custromer feedback and focus groups like they mention how did it happen in the first place? Some overoptimistic head-of-something? Really curious. I own previous -2021 mb and had to drive the upgrade (touch buttons) once as a replacement car. UX is terrible. Period. I even checked then in the dealership what they did to S-class and mybachs - and yes, same crappy wheel, etc. Anyways, I was mostly surprised that they didn’t know this before. Something is wrong with their research / decision making.

Can it be also related to demand not catching up or even declining? If place is in high demand and prices go down shouldn't it cause even more people coming to it (compensating for a possible price change). (Note: not an expert on this, I'm just curious how it really works - besides obvious thing: more supply -> price goes down).

Probably people at some point were making same arguments about ASM and C. How many people though do ASM these days? Not arguing that for now it is relevant point, obviously Rust / C are way faster.

This is probably the right solution. It seems in reality nobody does this since it is expensive (more teachers, real attention to students, etc). Also if there is an explicit split there will be groups of people who "game" it (spend disproportional amount of time to "train" their kids vs actual natural talent - not sure if this is good or bad).

So, it feels to me ideally within the same classroom there should be a natural way to work on your own pace at your own level. Is it possible? Have no idea - seems not, again primarily because it requires a completely different skillset and attention from teachers.

On the other hand it might become a next level of abstraction.

Machine -> Asm -> C -> Python -> LLM (Human language)

It compiles human prompt into some intermediate code (in this case Python). Probably initial version of CPython was not perfect at all, and engineers were also terrified. If we are lucky this new "compiler" will be becoming better and better, more efficient. Never perfect, but people will be paying the same price they are already paying for not dealing directly with ASM.

The idea is that it doesn't store binary files locally, just pointers in the DB + meta data (SQLite if you run locally, open source). So, it's versioning, structuring of datasets, etc by "references" if you wish.

(that's is different from let's say DVC - that does copy files into a local cache, always)

A bit unrelated / meta question.

Since it's developed in UCLA - do they have resources to do the next phases of trials (humans, etc)? I wonder if in such cases pharma (those who obviously have enough money) care about these molecules at all? Since they probably can't put a patent on it and heavily monetize it later. Are there examples of drugs that were developed like this successfully?

Good to see this project on the front page! We are using Vega (specifically Vega-Lite [1] as an engine and templates spec for data-science plots / visualizations in DVC (e.g. how it looks like in VSCode extension [2]).

It allowed us to have:

- same engine in CLI (can generate HTML and open in browser), VSCode extensions, SaaS

- have a way to describe plot visualization / representation as a declarative spec that can be then used in all those products (plot spec). We were exploring plotly and AFAIU there was no easy way to do the same

- it's quite comprehensive and community is responsive, the project is maintained

To name a few downsides from our experience:

- DSL is quite complicated. It requires some time to master it. It hurts the adoption. In our case I don't see that many users doing custom plots / templates - majority is using pre-baked built-in stuff or use Python and export as SVG.

- In our case some features were missing (and are still missing) - exponential average - that is most commonly used to smooth ML training curves.

[1] https://vega.github.io/vega-lite/

[2] https://dvc.org/doc/user-guide/experiment-management/visuali...

Hi HN! We are the DVC team. We are releasing a new tool to make it simpler for ML teams to curate their unstructured data and improve the quality of datasets.

ML teams have a lot of files- texts, images, video, pdfs, etc. Those objects have information (metadata) about them (e.g. labels, embeddings, captions). Last 5-6 years building DVC, we’ve observed a need for data versioning at scale but also a very strong need to store, enrich this metadata and also slice and dice those files based on this metadata (create datasets). We’d seen teams keep building and rebuilding the same infra or glue ETLs and scripts again and again and decided that it’s better to solve this in a more systematic way using our knowledge and experience.

DataChain is a Python library. Think about DataChain as a “data frame” with a “chain” of operations that can be applied to it - filter, merge, map, etc. We don’t store (or require moving or converting) files, rather DataChain is storing references to the origin (paths + version id). It is using a database underneath (SQLite) to preserve results (datasets) and do out of memory computation. It can do parallel computations, data caching, and many more things that make it better suited for unstructured data, ML, and larger scale.

Saved datasets (Data chains) can be passed to a data loader (e.g. Pytorch) to access the original raw files + metadata from the DB.

Please let us know your thoughts and questions in the comments!

My partner does this once a week. We have an electric kids scooter. She has to drive the smaller scooter attached to her own since there is no place at school to leave a bike or a scooter. So, works fine, but still quite tedious. Schools apparently are not designed for that, probably they don't want taking care, be responsible, etc, etc.

Oh yeah. Check the Mission Bay in San Francisco. Everything is walkable and very relevant to my life. I never drive to a grocery store, day care, coffee (3 - 4 places with excellent espresso), kid activities (there a bunch of places - music, soccer field, etc). In fact the first 4-5 years, until we had 2 kids we even didn't own a car - there was no need in SF ... and we enjoyed it a lot.

Now, with my second kid going to school (public) I have to drive (it's literally 1 mile). It has changed the whole schedule. That single drive! Annoys me as hell. It's hard to imagine how convenient it is when you don't have to drive anywhere, even school, day care, etc. This makes a huge difference.

  Iterative.ai (Series A) | REMOTE, WORLDWIDE | FULL-TIME | OPEN-SOURCE
  
  Developer tools for ML engineers. We need people who are passionate about building infrastructure that help ML teams manage data (large scale), make sane reproducible workflow, track models. We are building infrastructure to deal with datasets of the 5N LAION scale. Our flagship open source tool- DVC.org (12K+ stars), SaaS product - studio.iterative.ai. 

  We are looking for senior (SaaS and open source) engineers:
https://iterative.notion.site/Iterative-ai-is-Hiring-852cb97...

It reminds me an approach to data science and ML. It's way more common and essential for ML teams to log their attempts (experiments). There is a whole set of tools for this - experiments trackers. Primarily to even being able to compare and pick the best direction, but also to ensure reproducibility (in some areas it might be required).

ML/DS always seemed to me closer to science in its nature vs software engineering. Can be because of its nature as well - in a lot of cases it's a process of incremental improvements (vs - simplifying this a lot - let's say in SE we do a button that just works or not).

Here is what has changed: we no longer trust developers ...

I have a different hypothesis on this (never checked it though). I think as those languages - Python, JS were getting more and more popular we started using them more and more to build and maintain more complex projects. What worked for a few glue-code scripts doesn't work for a Dropbox size and scale project. Thus there was a need to evolve and augment the language.

  Iterative.ai (Series A) | REMOTE, WORLDWIDE | FULL-TIME | OPEN-SOURCE
  
  Developer tools for ML engineers. We need people who are passionate about building infrastructure that help ML teams manage data (large scale), make sane reproducible workflow, track models. We are building infrastructure to deal with datasets of the 5N LAION scale. Our flagship open source tool- DVC.org (12K+ stars), SaaS product - studio.iterative.ai. 

  We are looking for senior (SaaS and open source) engineers:
https://iterative.notion.site/Iterative-ai-is-Hiring-852cb97...