HN user
Researcher at OpenAI.
https://github.com/unixpickle/
I think we are missing the bigger point here. Licensing only matters on things that take real effort or money to produce. Who will care about licenses on software when software is free and infinite? It would be like licensing each ounce of water on Earth.
The `try_roots` example here is actually a _counterexample_ to the author's main argument. They explicitly ignore the "negative discriminant" case. What happens if we consider it?
If we take their "parse" approach, then the types of the arguments a, b, and c have to somehow encode the constraint `b^2 - 4ac >= 0`. This would be a total mess--I can't think of any clean way to do this in Rust. It makes _much_ more sense to simply return an Option and do the validation within the function.
In general, I think validation is often the best way to solve the problem. The only counterexample, which the author fixates on in the post, is when one particular value is constrained in a clean, statically verifiable way. Most of the time, validation is used to check (possibly complex) interactions between multiple values, and "parsing" isn't at all convenient.
To optimize for fast nearest neighbors, I chose 256 dims. Notably, this actually hurt some of the pre-training classification losses pretty severely compared to 2k dims, so it definitely has a quality cost.
The site uses cosine distance. The code itself implements Euclidean distance, but I decided to normalize the vectors last minute out of FUD that some unusually small vectors would appear as neighbors for an abnormal number of examples.
The "shop for random products" direction was actually fun for me too. Reminds me of amazon.com/stream a bit.
Probably the same complaint as
You definitely highlighted a shortcoming of the feature vector model in this case. Indeed it's quite a small model trained on a single Mac for about a week, so it's not very "smart".
I'd expect that this is a problem that could be solved by using larger off the shelf models for image similarity. For this project, I thought it would be cooler to train the model end-to-end myself, but doing so has negative consequences for sure.
I think it would be a useful feature. For the sake of being a fun project, I didn't use CLIP because I only wanted to use models that I trained myself on a single Mac. However, to make this more useful, text search would be quite helpful.
Yup, it's a small model I trained on my Mac mini! The model itself just classifies product attributes like keywords, price, retailer, etc. The features it learns are then used as embeddings
Ideally pose and lighting wouldn't matter as much as it currently does.
I think using a better model to produce feature vectors could achieve this, or perhaps even finetuning the feature model to match human preferences.
This should just be called "why VPNs are useful", i think?
This seems to be pretty much exactly a standard Bayesian deep learning approach, albeit with a heavily engineered architecture.
Wait what? Who actually calls trainable params "hyperparameters"? Nobody at OpenAI does, as far as I know.
I asked Nicole Avena, a professor of neuroscience at Mount Sinai who studies sugar addiction, if she believed it could be possible for food companies to engineer, intentionally or not, compounds that would make GLP-1 drugs less effective. Avena told me it was plausible.
Never really thought about this before. The food industry is a virus and current weight loss drugs are the best vaccine we have, but it'll forever be an arms race.
These lines from the diff linked above are the fix:
- /\* Log error and exit. \*/
- sigdie("Timeout before authentication for %s port %d",
- ssh_remote_ipaddr(the_active_state),
- ssh_remote_port(the_active_state));
+ _exit(EXIT_LOGIN_GRACE);Before one epoch, both the train and eval curves look pretty much identical. Quite curious
OP here. Interesting, I had not researched the used car market enough to know about the price inflation. Can you think of any other sites that have more realistic prices?
These are good ideas! It should be possible to test how photographic quality correlates with price, though one complication is that I train with data augmentation (a commonly used technique to make better use of the training data). Some of the augmentations such as random cropping and color jitter might already stimulate some level of bad photography.
OP here. Yeah, it's quite odd that the median price prediction doesn't fall into the most likely bucket, even when that bucket has more than a 50% confidence. I'm not sure how to interpret this. One theory: maybe the median prediction head hasn't converged yet and naturally has a bias towards lower values due to the network's initialization.
I'll know more in a week or two once all the models have converged, and can update here!
Before writing this post, I asked chatgpt for examples of positive transfer from auxiliary losses in the literature. It pointed me to this paper:
It's definitely not just a regularizer in my case, because the gap appears even before a single epoch. The gap does also appear for two very different model architectures.
One explanation is that price labels are super noisy. If there is enough noise in the primary labels, you could imagine that adding in the more predictable target variables could help reduce gradient noise and speed up training. That's my current hypothesis, but I'm very open to others. If I had more time I'd try to do more experiments on this.
Apparently my router's built-in DNS server doesn't like the domain name `ai.` or `ai`. Switching to Google's fixes the issue for me.
We can inject targeted taps continuously with a standard deviation of as low as 14.6 x 19.2 pixels from the target area, a delay of less than 0.5s and a distance of up to 40mm
Seems like 40mm isn't a very far distance. I understand that this is a PoC, but it seems like making this work from a distance is more "important" than having a low delay or super accurate precision.
I also wrote a similar thing years ago...
I think you are conflating "Transformers" and "autoregressive models". Transformers are a general purpose architecture for transforming sequences into other sequences with self-attention. AR models / GANs are frameworks for generative modeling. The model architecture is almost entirely orthogonal to the generative framework.
You can use transformers as part of GANs [1], and you can even use them as discriminative models for images [2].
[1] https://arxiv.org/abs/2102.07074 [2] https://arxiv.org/abs/2010.11929
Author here!
This project includes a boatload of 3D models I've created in code and 3D printed over the past half year or so. It also includes a more recent ray tracer, which can produce some cool pictures like [1]. I've finally started stabilizing the API, and I'm excited to see what others do with it.
[1] https://github.com/unixpickle/model3d/blob/master/examples/r...
A neural network would have some nice properties for sure, although it'd also have plenty of bad ones. For one thing, I'd guess that the NN wouldn't be great for rendering in real-time, since it's liable to be orders of magnitude slower than the JS function it's approximating. Granted you could use a GPU--but then again, you could probably compile the JS function into a kernel and get a speedup there too.
If you sample the boolean function enough to train a neural network on it, you've likely sampled more than enough to build a very accurate mesh using marching cubes or other algorithms. Meshes are especially easy to render, plus they can tell you quite a lot.
Good idea, I'll add a reference to SDFs. Didn't think about the normal-via-autodiff aspect, which would be quite useful.
It seems to me that SDFs are very hard to program from scratch, compared to simple boolean functions. Obviously they are easy to implement for compositions of simple shapes, but for things like my text example (and maybe the corkscrew example, although of course that is still a fairly simple mathematical shape), it's less obvious how to implement an SDF by hand.
This is a web demo for creating 3D models in JavaScript. It uses a variant of ray casting that can render any function mapping `(x, y, z) -> bool` in real-time.
It's easy to define very interesting 3D models as boolean functions. In my opinion, it's much easier than creating triangle meshes or composing a bunch of primitive shapes. I had fun making the examples, and I'm curious to see what other people can come up with.
I've used vanilla Ubuntu on my Dell XPS for the past year and it has served me well.
Maybe we are just preparing for the point when climate change makes going outside difficult and largely unsafe...