neutralino1

Asking people to change their ways is pointless. When something is systemic, only a systemic solution can work.

I have become intimately convinced that engagement-based feeds are the root of many evils of our time, loneliness included.

Here are some of the perverse effects (if ever they needed be told), and how they relate to the loneliness epidemic

- they incentivize individuals from a young age to find stimulation from scrolling mindless content through short dopamine loops instead of seeking satisfaction through longer-term endeavors (e.g. projects, board games, bands, sports teams, etc.) which tend to foster connections with friends, neighbors, family, strangers

- they radicalize and polarize into extreme niche communities (political extremes, conspiracy theories, manosphere, etc) so that it's more difficult to find common ground with a random average person, giving you the impression that everyone is your enemy

- they reflect a skewed version of reality where societal standards (beauty, intelligence, success, wealth, etc) are distorted and artificial, which drives people to believe they are insufficient and ostracized

I firmly believe that engagement-based feeds should be heavily regulated, the same way that other addictive behaviors have (e.g. tobacco, gambling, etc.).

You win today.

It seems to me Rails has been doing this but better for years. It definitely keeps atomic and historical migrations, but also maintains a schema.sql file that can be loaded as a one-off (e.g. for mock DBs in tests).

Yeah sorry about that, this is a data dashboard. Not optimized for mobile viewing.

We debated doing 2D vs 3D and 3D brought a bunch of usability issues. We also noticed most SOTA embedding visualizations were 2D and already yielded good insights.

The original dataset is located at [1] (not our HF account). HN data is directly available via the HN API [2]. The privacy policy you point to does not cover HN posts.

[1] https://huggingface.co/datasets/OpenPipe/hacker-news [2] https://github.com/HackerNews/API

Oh you cannot actually view the raw embedding vector, only the corresponding row.

Any particular use-case to view the raw embedding?

There are cures for hiccups!?

Satya is buying popcorn.

Price is the advantage.

I often draw the parallel with cigarettes and alcohol. Kids need to produce an ID to purchase them. Sure they can fake it, but then they are breaking the law, and that still raises the barrier.

But that's likely not enough. In addition, there should be public health campaigns to warn against the risks.

Cigarette use has plummeted since the 90's, so something must be working.

Ideally yes, but the liabilities are likely too great at this time. So better start with the "hopeless", where no harm can be done, then go up from there.

Congrats on the launch!

I wouldn't go as far as to say business executives have good social skills. They are often ruthless, cunning, and deceiving, which makes them successful. If by good social skills you mean the ability to convince (read deceive) a lot of people, then sure they have good social skills. But who wants to have a beer with their CEO, VP of Sales, or other top exec? They are often depicted as wolves or sharks for a reason. It is true that they are good at networking with other people like them, but not really good at empathizing, helping, or caring for others.

A lot of ads on this page.

Sematic | Enterprise Sales Representative | Full-time | Hybrid with office in SF | $100k-$150k base, equity, benefits

Sematic (YC S22) is the open-source Continuous Machine Learning platform built by industry veterans (Cruise, Google, Microsoft, Instacart, CERN, Cisco). We build cutting-edge ML infrastructure to help businesses automate their training pipelines and accelerate the path to production. We are backed by Race Capital, Y Combinator, Soma Capital, and more.

Apply here: https://www.ycombinator.com/companies/sematic/jobs/PG1oHuE-e...

Thank you – You are right that these are very important topics, and we also had to expend a lot of work at Cruise to scale training beyond single node. We had training jobs running over dozens of GPU nodes for many days. For example, we had a dedicated team to optimize streaming of training data into PyTorch dataloaders. This evidently requires more infrastructure, and also many features around fault tolerance, checkpointing, warm restarts, etc.

We are a very new framework (launched publicly July 1st :-), so there is much work to be done to cover many more example use cases.

What we have found powerful about this plain function approach is that users can submit jobs on remote platforms (e.g. Spark, Google Dataflow, etc.), and use heterogenous resources (e.g. standard nodes to launch third-party jobs, then GPU nodes for training, etc.). So whatever "cloud provider X's data solutions" you have to use, if it has a Python API to submit and wait for jobs, you should be fine.

- How does it handle failure of individual tasks in the pipeline? At this time there are no handling of failures (Sematic is 6 weeks old :). In the near future we will have fault tolerance mechanisms: retries, try/except.

- What if the underlying jobs need to run outside the k8s cluster? You are free to launch jobs on third-party platforms from one of your pipeline steps. This is a pretty common pattern, for instance launching a Spark job, or a training job on a dedicated GPU cluster. In this case, the pipeline step that launches the job (the Sematic function) needs to wait for the third-party job to complete, or pass a reference to the job to a downstream step that will do the waiting.

- How does caching work? At this time there is no caching (as mentioned Sematic is very new :). We will implement memoization soon. What you can do is run a data processing pipeline separately and then use the generated dataset as input to other pipelines. This is a pretty common pattern: having a number of sub-pipelines (e.g. a data processing loop, a train/eval loop, a testing/metrics loop, etc.) that you can run independently, but also you can put them together in an end-to-end pipeline for automation. Sematic lets your nest pipelines in arbitrary ways, and each sub-pipeline can still have its own entry-point for independent execution.

You are free to launch jobs on Spark clusters from any of your pipeline steps, whether they run locally or remotely. Be mindful that if you do so, the function that launched the job needs to also wait for its completion. In the future we will provide deeper integrations with Spark.

When running on cloud instances, Sematic runs container images on your Kubernetes nodes. You are free to SSH into those and run the images manually to have the same dependency context, but any ephemeral context of the job (e.g. storage) will no longer be there.

You can certainly deploy the web app on one server, and run your pipelines on another (or the same). In this case, "locally" would mean that the pipeline and all its steps run on the same host machine. This is totally sufficient in many cases.

Kubernetes becomes interesting when using heterogeneous resources (e.g. GPU nodes for training, high-memory for data processing, etc.), but is not a necessity.

As they say, naming things is one of the two hardest problems in Computer Science :)

Your project is cool and has a cool name!

Thank you!

What we have seen done in the past is use things like pybind11 to expose C++ APIs in Python, which I guess is a similar concept to SWIG. If you are using build tools such as Bazel, you can even get the C++ compiled at run-time when submitting your pipeline.

Regarding i/o artifacts, Sematic lets user choose how they are serialized. We offer reasonable baseline defaults, but certain artifacts require serialization formats that are cross-language (e.g. ROS messages since you mention robotics).

As a last resort, users are free to serialize and persist artifacts by hand as part of their pipeline functions (e.g. storing in a cloud bucket) and only returning a reference to said artifact (e.g. a Python dataclass with artifact location and metadata).

Thanks! I was a pretty heavy user of Rails in past jobs. They have a nice mix of good abstractions (Model/View/Controller), tooling (CLI, local web server), and best practices (how to name tables, fields, lifecycle timestamps, etc.).

We think that is a good way to go: solid abstractions that experts can build on top of, but also more junior folks can get started quickly by following best practices.

Ploomber looks great too, I like the breakdown of your SaaS offering.

In the current open-source product, yes. But you can also simply use Sematic locally without any infrastructure (`$ sematic start`).

In the coming months, we will provide a fully-hosted SaaS offering to save you the hassle of maintaining your own infrastructure.

That is very true. In our careers we have repeatedly incentivized users to exit Notebooks early on.

Of course we understand why Notebooks are so appealing and useful, and we think they have their place in the toolbox (like a Python console for developers).

And you are correct that there already are many tools to build reproducible traceable pipelines. What we have found is that they are still too difficult to adopt. Which is why we are trying to greatly lower the barrier to entry.

Indeed, Kubernetes is hardly "light" :D

However, it's only required when running pipelines in the cloud. When running locally (`$ sematic start`), nothing else than Python is required.

Regarding background, we are folks with experience building ML tooling. We've built infra around TF and Pytorch, but none with Jax. That being said, Sematic is agnostic to the framework you want to use.

Absolutely. Software development has many nice CI/CD patterns to generate assets, test, deploy. It enables software engineers to work fast with a safety net and have all pipelines automated.

I think ML development is where web development was in the late 2000's. Clear patterns and best practices have not yet emerged. For example, many ML developers work without reproducibility and traceability enabled. This does not allow for fast and safe work.

Thanks for your question! Yes, Sematic has a neat feature that can help: Dynamic Graphs. Because Sematic uses simple Python to declare the control and data flow of your graph, you can simply loop over configurations (in your case different sets of board game rules) and train a model for each config, and eventually aggregate results to determine the winner.

Join our Discord in you want to discuss this further – https://discord.gg/4KZJ6kYVax

Most French people love La Vache qui Rit. They don't eat it as adults but it revives memories of their school trip lunches.

Can we pause the significant other too?