just curious since it reminds me a bit. Have you / someone tried https://entire.io/ (I'm not affiliated at all, so it is not a plug).
HN user
shcheklein
Building DVC.org, CML.dev, and other ML workflow tools.
If they have custromer feedback and focus groups like they mention how did it happen in the first place? Some overoptimistic head-of-something? Really curious. I own previous -2021 mb and had to drive the upgrade (touch buttons) once as a replacement car. UX is terrible. Period. I even checked then in the dealership what they did to S-class and mybachs - and yes, same crappy wheel, etc. Anyways, I was mostly surprised that they didn’t know this before. Something is wrong with their research / decision making.
Can it be also related to demand not catching up or even declining? If place is in high demand and prices go down shouldn't it cause even more people coming to it (compensating for a possible price change). (Note: not an expert on this, I'm just curious how it really works - besides obvious thing: more supply -> price goes down).
Seems to be a duplicate with this Show HN authors made: https://news.ycombinator.com/item?id=46290617 . Let's please merge this if someone from the moderators team can do that. Thanks!
Yep, exactly, It takes time to close the gap so that more and more teams taking that tradeoff
Probably people at some point were making same arguments about ASM and C. How many people though do ASM these days? Not arguing that for now it is relevant point, obviously Rust / C are way faster.
What if its parents "motivation" to a large extent (and by gaming I meant primarily parents pushing extremely hard)? How would you draw the line?
To be clear - I personally don't have an answer to this.
This is probably the right solution. It seems in reality nobody does this since it is expensive (more teachers, real attention to students, etc). Also if there is an explicit split there will be groups of people who "game" it (spend disproportional amount of time to "train" their kids vs actual natural talent - not sure if this is good or bad).
So, it feels to me ideally within the same classroom there should be a natural way to work on your own pace at your own level. Is it possible? Have no idea - seems not, again primarily because it requires a completely different skillset and attention from teachers.
To be fair. I live at Mission Bay (SF) that has Caltrain railway nearby (and you have to cross it if you take particular ways in/out). I drive (and like it a lot!) Waymo. Waymo avoids crossing it (it takes a longer way to drive a bridge). So, they probably realized the risk and still to this are not willing taking it.
Congratulations on the launch! Amazing work, it would be great to see how it evolves into a bigger story, work related agents beyond healthcare.
On the other hand it might become a next level of abstraction.
Machine -> Asm -> C -> Python -> LLM (Human language)
It compiles human prompt into some intermediate code (in this case Python). Probably initial version of CPython was not perfect at all, and engineers were also terrified. If we are lucky this new "compiler" will be becoming better and better, more efficient. Never perfect, but people will be paying the same price they are already paying for not dealing directly with ASM.
Another alternative to consider is https://www.getdaft.io/ . AFAIU it is a more direct competitor to Spark (distributed mode).
thanks, fixed
The idea is that it doesn't store binary files locally, just pointers in the DB + meta data (SQLite if you run locally, open source). So, it's versioning, structuring of datasets, etc by "references" if you wish.
(that's is different from let's say DVC - that does copy files into a local cache, always)
This _shared_ filesystem is mounted on all Worker and Login nodes and contains another root directory, possibly of a different Linux distribution.
Is it possible to mount this shared filesystem on the root directory of the container? Then this slurm plugin isn’t needed.
An out-of-memory dataframe to wrangle unstructured data at scale - https://github.com/iterative/datachain
A bit unrelated / meta question.
Since it's developed in UCLA - do they have resources to do the next phases of trials (humans, etc)? I wonder if in such cases pharma (those who obviously have enough money) care about these molecules at all? Since they probably can't put a patent on it and heavily monetize it later. Are there examples of drugs that were developed like this successfully?
Good to see this project on the front page! We are using Vega (specifically Vega-Lite [1] as an engine and templates spec for data-science plots / visualizations in DVC (e.g. how it looks like in VSCode extension [2]).
It allowed us to have:
- same engine in CLI (can generate HTML and open in browser), VSCode extensions, SaaS
- have a way to describe plot visualization / representation as a declarative spec that can be then used in all those products (plot spec). We were exploring plotly and AFAIU there was no easy way to do the same
- it's quite comprehensive and community is responsive, the project is maintained
To name a few downsides from our experience:
- DSL is quite complicated. It requires some time to master it. It hurts the adoption. In our case I don't see that many users doing custom plots / templates - majority is using pre-baked built-in stuff or use Python and export as SVG.
- In our case some features were missing (and are still missing) - exponential average - that is most commonly used to smooth ML training curves.
[1] https://vega.github.io/vega-lite/
[2] https://dvc.org/doc/user-guide/experiment-management/visuali...
Seem to be up again, I also wonder what is was.
Hey, congrats! Are you competing / is there some overlap / what are the key differences with Roe AI (YC W24) - roe.ai (just launched recently on HN https://news.ycombinator.com/item?id=41202694 as well).
Hi HN! We are the DVC team. We are releasing a new tool to make it simpler for ML teams to curate their unstructured data and improve the quality of datasets.
ML teams have a lot of files- texts, images, video, pdfs, etc. Those objects have information (metadata) about them (e.g. labels, embeddings, captions). Last 5-6 years building DVC, we’ve observed a need for data versioning at scale but also a very strong need to store, enrich this metadata and also slice and dice those files based on this metadata (create datasets). We’d seen teams keep building and rebuilding the same infra or glue ETLs and scripts again and again and decided that it’s better to solve this in a more systematic way using our knowledge and experience.
DataChain is a Python library. Think about DataChain as a “data frame” with a “chain” of operations that can be applied to it - filter, merge, map, etc. We don’t store (or require moving or converting) files, rather DataChain is storing references to the origin (paths + version id). It is using a database underneath (SQLite) to preserve results (datasets) and do out of memory computation. It can do parallel computations, data caching, and many more things that make it better suited for unstructured data, ML, and larger scale.
Saved datasets (Data chains) can be passed to a data loader (e.g. Pytorch) to access the original raw files + metadata from the DB.
Please let us know your thoughts and questions in the comments!
It can be done, it takes substantially more time with a kid + walking back. My partner is driving my kido using electric scooter - which works fine as well.
My partner does this once a week. We have an electric kids scooter. She has to drive the smaller scooter attached to her own since there is no place at school to leave a bike or a scooter. So, works fine, but still quite tedious. Schools apparently are not designed for that, probably they don't want taking care, be responsible, etc, etc.
Not dangerous (except traffic that you have to cross a few times).
Oh yeah. Check the Mission Bay in San Francisco. Everything is walkable and very relevant to my life. I never drive to a grocery store, day care, coffee (3 - 4 places with excellent espresso), kid activities (there a bunch of places - music, soccer field, etc). In fact the first 4-5 years, until we had 2 kids we even didn't own a car - there was no need in SF ... and we enjoyed it a lot.
Now, with my second kid going to school (public) I have to drive (it's literally 1 mile). It has changed the whole schedule. That single drive! Annoys me as hell. It's hard to imagine how convenient it is when you don't have to drive anywhere, even school, day care, etc. This makes a huge difference.
There was an interesting discussion recently on (somewhat) opposite perspective https://antithesis.com/blog/is_something_bugging_you/ - how good testing systems makes everything way faster
Iterative.ai (Series A) | REMOTE, WORLDWIDE | FULL-TIME | OPEN-SOURCE
Developer tools for ML engineers. We need people who are passionate about building infrastructure that help ML teams manage data (large scale), make sane reproducible workflow, track models. We are building infrastructure to deal with datasets of the 5N LAION scale. Our flagship open source tool- DVC.org (12K+ stars), SaaS product - studio.iterative.ai.
We are looking for senior (SaaS and open source) engineers:
https://iterative.notion.site/Iterative-ai-is-Hiring-852cb97...It reminds me an approach to data science and ML. It's way more common and essential for ML teams to log their attempts (experiments). There is a whole set of tools for this - experiments trackers. Primarily to even being able to compare and pick the best direction, but also to ensure reproducibility (in some areas it might be required).
ML/DS always seemed to me closer to science in its nature vs software engineering. Can be because of its nature as well - in a lot of cases it's a process of incremental improvements (vs - simplifying this a lot - let's say in SE we do a button that just works or not).
Here is what has changed: we no longer trust developers ...
I have a different hypothesis on this (never checked it though). I think as those languages - Python, JS were getting more and more popular we started using them more and more to build and maintain more complex projects. What worked for a few glue-code scripts doesn't work for a Dropbox size and scale project. Thus there was a need to evolve and augment the language.
Iterative.ai (Series A) | REMOTE, WORLDWIDE | FULL-TIME | OPEN-SOURCE
Developer tools for ML engineers. We need people who are passionate about building infrastructure that help ML teams manage data (large scale), make sane reproducible workflow, track models. We are building infrastructure to deal with datasets of the 5N LAION scale. Our flagship open source tool- DVC.org (12K+ stars), SaaS product - studio.iterative.ai.
We are looking for senior (SaaS and open source) engineers:
https://iterative.notion.site/Iterative-ai-is-Hiring-852cb97...