kolja005

This is an interesting perspective I haven't heard before. Do you have links to anyone who has articulated this further?

Think about what the implication here is for people who answered no to that question. If I were to go up to my boss and say "I'm not interested using AI because I think it's bad for society" I would essentially be saying that I'm not interested in becoming more productive and thus making more money for the company. That's a very poor reputation to be carrying around and most people are going to avoid it. I believe that this, more than any specific actions by AI companies, has contributed to the sense of inevitability that this technology is taking over whether we want it or not.

I would be inclined to agree with this except that my "most common needs" keeps expanding and increasing in difficulty each year. In 2023 and 2024, most of my needs were asking models simple questions and getting a response. They were a drop-in replacement for Stack Overflow. I think the best open source models today that I can run on my laptop serve that need.

Now that coding agents are a thing my frame of reference has shifted to where I now consider a model that can be that my most common need. And unfortunately open models today cannot do that reliably. They might, like you said, be able to in a year or two, but by then the cloud models will have a new capability that I will come to regard as a basic necessity for doing software development.

All that said this looks like a great release and I'm looking forward to playing around with it.

I just refactored a bunch of our microservices into a monolith. Fortunately the business justification was pretty straightforward because it was clear to all of us that the service architecture was weighing us down.

Since one of microservice's benefits is solving a coordination problem, now that teams are getting smaller due to AI, I wonder if we will see monoliths make a resurgence in some cases.

Since the database is used by everyone, it becomes cared for by no one. Startups don’t have the luxury of a DBA, and everything owned by no one is owned by infrastructure eventually.

This post was a great read.

Tangent to this, I've always found "best practices" to be a bit of a misnomer. In most cases in software and especially devops I have found it means "pay for this product that constrains the way that you do things so you don't shoot yourself in the foot". It's not really a "practice" if you're using a product that gives you one way to do something. That said my company uses a very similar tech stack and I would choose the same one if I was starting a company tomorrow, despite the fact that, as others have mentioned, it's a ton to keep in your head all at once.

My company has the github page for it blocked. They block lots of AI-related things but that's the only one I've seen where they straight up blocked viewing the source code for it at work.

Hey pretty cool! I recently added a similar feature to my neovim setup. I can press a keymap and cycle through all of the unstaged git hunks with each shown in a preview window. I can also change the base branch to one that, say, I'm trying to merge into so that I can have the same workflow when doing code reviews.

People like to make fun of these models for sounding like a broken record, being over complementary, etc, but I'm actually starting to think that models having a very recognizable style is a good thing because it makes identifying AI-generated content in the wild really easy. Sure, the verbosity is annoying when I'm just trying to get a straightforward, simple answer from it. But I like that I can have a pretty good sense of when content on the Internet is low-effort AI spam. If models become too good at emulating the personality of a real human, then that gets lost.

What are folks doing who were just using it for CI/test/dev environments? Just build the image yourself? Use Garage as some have suggested? I'm curious what people see as the pros and cons.

This is pretty neat. Similarly, I feel like we are really close to smartwatches being able to replace smartphones for all essential tasks. I try to rely on my apple watch with cellular as much as I can and leave my phone at home. I can't wait until agents get a bit better at navigating the web and someone makes a killer UI for the watch. I'll be able to do everything I need to do with a much more ambient device that doesn't suck attention the way a smartphone does.

Funny to see this today.

I'm a rank and file dev at a non-big tech company and I got a call from a Windsurf sales rep this week who I had connected with on LinkedIn the day before (I never gave them my number). They told me my company was in talks with Windsurf about a licensing deal but that they would give me a 30 day trial of an enterprise account for use on personal projects to let me try it in advance. I guess the idea for them is to build enthusiasm among devs in the company?

Is this a standard sales strategy for products like this? It seems pretty aggressive to me but I'm just an engineer so I wouldn't know.

As it pertains to Python in particular I think OOP is great for libraries but of limited usefulness at the application layer. Things like pytorch's nn.module IMO is a great abstraction, but every time I've tried to map some concrete business concept to an OOP construct I've regretted it.

Do we work at the same company?

You put it much better than I could have. Do you know if polars at all solves the problem of having opaque, mutable objects everywhere? I feel like there's a good market for having a dataframe library that's easier to reason about in your editor. It could even be a wrapper around pandas that adds rich typing sort of the way FastAPI does with Pydantic for Starlette.

I recently watched a talk by the author of uv that was surprisingly fascinating [1]. He goes into a few of the more notable hacks that they had to come up with to make it as fast as it is. The most interesting thing for me was that package resolution in python given constraints defined (eg. in requirements.txt) maps to a boolean satisfiability problem which is NP-complete. So uv uses a custom SAT solver to do this. I totally under-appreciated how much goes into this software and I'm bummed I have to use Poetry at work after having watched this talk.

[1] https://www.youtube.com/watch?v=gSKTfG1GXYQ

edit: NP-complete not NP-hard

Unrelated but does anyone have any recommendations for good resources on learning how to write tests/testable software?

Does anyone have a good heuristic for when a dataframe library is a good tool choice? I work on a team that has a lot of data scientists and a few engineers (including myself) and I often see the data scientists using dataframes when simple python classes would be much more appropriate so that you have a better sense of the object you're working with. I'm been having a hard time getting this idea across to people though.

Related: I found out about a vim/neovim plugin called Molten[1] that tries to be a replacement for Jupyter Notebooks in the terminal. It was a little rough around the edges ~5 months ago when I tried it, but looking at the repo it seems its still being actively developed, so maybe that's changed. IIRC it uses an ipynb server to keep track of each cell's outputs. I quite enjoyed using it as someone who doesn't need notebooks very often and loathes leaving my terminal setup.

[1] https://github.com/benlubas/molten-nvim

Is there a lot of innovation happening in this space for military applications? I ask this as someone totally unfamiliar with the technology in general. It seems like it could greatly benefit pilots, people on the battlefield, and anyone else who needs access to some kind of visual information while still having both of their hands.

I'm curious in what sense you find Python difficult to deploy? My company has tons of Python APIs internally and we never have much trouble with them. They are all pretty lightly used services so it it something about doing it on a larger scale?

You might find it interesting to know that for track cycling the penalty due to lower oxygen uptake is less than the advantage due to less air resistance. In other words track cyclists, at least in some of the longer disciplines, will go faster at higher altitudes despite there being less air to breathe.

Uber has a phone number that you can call to hail a ride. https://techcrunch.com/2023/05/17/you-can-now-hail-an-uber-b...

I use an app called Jomo to block YouTube, as well as about 10 other distracting applications. It runs on every device I have and makes it very challenging to use the app when I feel the urge to. For a long time I convinced myself that I needed to keep some access to YouTube because there is obviously some meaningful content on the platform. After having it completely blocked for a few months I have come to realize that there is nothing on the platform that is absolutely essential for me to be able to view, and that the downside of getting distracted was not at all worth it.

There needs to be an updated CLIP-like model in the open-source community. The model is almost three years old now and is still the backbone of a lot of multimodal models. It's not a sexy problem to take on since it isn't especially useful in and of itself, but so many downstream foundation models (LLaVA, etc.) would benefit immensely from it. Is there anything out there that I'm just not aware of, other than SigLIP?

Been following Jon Barron et al’s work for a little while now. The speed of improvement given the complexity of these types of systems is mind-boggling.

I wonder how long it takes before Google street view gets replaced by some NeRF variant.

From the interviews with him that I have seen, Sutskever thinks that language model is a sufficient pretraining task because there is a great deal of reasoning involved in next token prediction. The example he used was that suppose you fed a murder mystery novel to a language model and then prompted it with the phrase "The person who committed the model was: ". The model would unquestionably need to reason in order to come to the right conclusion, but at the same time it is just predicting the next token.

What has me scratching my head is the fact that Altman has been on a world tour preaching the need for safety in AI. Many people here believed that this proselytizing was in part an attempt to generate regulatory capture. But given what's happening now, I wonder how much Altman's rhetoric served the purpose of maintaining a good relationship with Sutskever. Given that Altman was pushing the AI safety narrative publicly and pushing things on the product side, I'm led to believe that Sutskever did not want it both ways and was not willing to compromise on the direction of the company.

Additionally, OpenAI can just put resources towards both approaches in order to settle this dispute. The whole point of research is that you don't know the conclusions ahead of time.

I was a little confused about this too. The authors say in the paper:

"The outputs of the ViT image encoder before pooling form the visual tokens, which are linearly projected and prepended to the embedded input text tokens."

I took a look at the HuggingFace implementation of ViT [1]. After the ViT encoder blocks there's a layer norm and then a pooling layer (line 595), where the pooling layer involves taking the first token output from the layer norm and running it through a dense layer. So, it looks like in PaLI-3 the tokens are the hidden states output by the layer norm after the ViT encoder blocks.

[1] https://github.com/huggingface/transformers/blob/main/src/tr...

I probably could have guessed that contrastive pre-training works better for downstream vision-language tasks than image classification pre-training, but it's nice to see this hypothesis thoroughly tested here.

Also, hacks to get LLMs to generate structured output seem to be mostly getting the job done. I'm less optimistic about this approach for traditional vision tasks where language is the interface, however. Are we going to get models to output a pixel-wise segmentation mask as text? I want to doubt it but seeing how LLMs are about to output long sequences of structured text leaves my mind open.

The purpose of this research is to compare large vision-language models where the vision component is pre-trained using different techniques, namely on image classification versus unsupervised contrastive pre-training (see OpenAI's CLIP). PaLI-3 also isn't an instruction-tuned model, so comparing it to Llava would be a little apples-to-oranges.