empiko

China is building incredible 200x times more ships annually compared to the US. You can imagine what the military implications are.

Doing AI just because everybody else is doing AI is not exactly a strategic vision. Going into AI without a clear plan is exactly what they should NOT be doing.

Yeah, but the juiciest tasks are still far from solved. The amount of tasks where people are willing to accept low accuracy answers is not that high. It is maybe true for some text processing pipelines, but all the user facing use cases require good performance.

Considering their budget, their research is a bit underwhelming. At this point, anybody is able to match their models. No technology moat whatsoever despite infinite money glitch.

You can implement a singleton class that works properly quite easily. The advantage is that most people are familiar with singleton as a pattern, and it is a self contained chunk of code. The cache solution you provided works, but its functionality is not obvious and it feels very hacky to me. Somebody's going to initialize Whatever in another way down the line without using the cached function...

I started doing NLP in 2014. First, I was using SVM and feature vectors, then word embeddings, then handcrafted neural network models, then fine-tuning transformer encoders, then working with LLMs. In that time I worked with huge number of technologies, libraries, and algorithms. A hiring manager recently asked me what my experience with AI agents are, and I had to say that it's basically zero.

Okay, he was obviously very new to the field and had no idea, but it illustrates how the field progressed in the past 10 years, and a person who is just joining has very similar starting line to old-timers. The breadth of knowledge I have is of course extremely useful and I am able to get new concepts really fast, as there are many similarities. But the market in general does not care that much really.

Have you actually read the link you posted? There is like dozen different ways Arab population is discriminated described there.

This is true for every subfield I have been working on for the past 10 years. The dirty secret of ML research is that Sturgeon's law apply to datasets as well - 90% of data out there is crap. I have seen NLP datasets with hundreds of citations that were obviously worthless as soon as you put the "effort" in and actually looked at the samples.

I also wonder why we do not expect radio towers, television channels, book publishers etc to make sure that their content will not be consumed by the most vulnerable population. It's almost as if we do not expect companies to baby-proof everything at all times.

Totally agree. Editors were used by publishers for a reason, and they very often managed to cut down texts by significant amounts. Most bloggers could use an editor that would make their writing more snappy. The same goes for many of the nowadays popular bestseller non-fiction books that are very clearly stuffed with repetitive writing and random anecdotes to hit some page limit.

It's human toll everywhere. AI used for peer review effectively forces researchers to implement suggestions between revisions, AI used by managers suggest bad solutions that engineers are forced to implement, etc. Effectively, the number of person-hours that is spent following whatever AI models suggest is increasing rapidly. Some of it might make sense, but uncomfortably many hours are burned in vain. There is a real cost of lost productivity in the economy by command chains not being ready to filter out slop.

Even in Ukraine there are many cases when the official Western position has changed over the time or is obviously not correct. For example, due to political reasons, Germany still cannot admit that it was Ukrainians who destroyed Nord Stream, although the evidence is pretty strong by now. There is a ton of other similar cases, as the information war vaged from both sides is enormous in volume.

It is impossible to solve this problem because we cannot really agree what the desired behavior should be. People live in different and dynamic truths. What we consider enemy propaganda today might be an official statement tomorrow. The only way to win here is to not play the game.

I strongly believe that this will be a really important technique in the near future. The cost saving this might create is mouth watering.

Yes, it is indeed to mitigate bad press. Unfortunately, the discussion about AI is so ridiculous, that it is often considered newsworthy when a product generates something funky for a person with large enough Twitter audience. Nobody wants to answer the questions about why their LLM generated it and how they will prevent it in the future.

AI generated reviews are a huge problem even at the most prestigious ML conferences. It is hard to argue against them, since the weaknesses they identify are usually in well formulated, and it is hard to argue that subjectively they are not that important. ACL recently started requiring Limitations section in their paper where authors should transparently discuss what are the limits. Unfortunately, that section is basically a honeypot for AI reviews as they can easily identify the sentences where authors admitted that their paper is not perfect and use it to generate reasons to reject. As a result, I started recommending being really careful in that particular section.

Observe what the AI companies are doing, not what they are saying. If they would expect to achieve AGI soon, their behaviour would be completely different. Why bother developing chatbots or doing sales, when you will be operating AGI in a few short years? Surely, all resources should go towards that goal, as it is supposed to usher the humanity into a new prosperous age (somehow).

Is there any other field where they give you random brain teasers for an interview? My friends outside of IT were laughing their heads off when they hears about the usual interview process.

Yeah, Wikipedia, but also many news sites use this style. It is mostly not a call to action, but additional optional information you can check out. That's why it feels wrong to use it in these cases. I think that some of the examples are outdated compared to how people format the web nowadays.

To be honest, I expected the punchline to be about how randomly drawing lines is the same nonsense as using simplistic mathematical modeling without considering the underlying phenomenon. But the punchline never came.

Predicting AI is more or less impossible because we have no idea about the its properties. With other technologies, we can reason about how small or how how a component can get and this gives us psychical limitations that we can observe. With AI we throw in data and we are or we are not surprised by the behavior the model exhibits. With a few datapoints we have, it seems that more compute and more data usually lead to better performance, but that is more or less everything we can say about it, there is no theory behind it that would guarantee us the gains for the next 10x.

Tracing the fiber back is possible only in extremely favorable conditions. The light must hit the cable just right and there cannot be too many cables from previous runs on the battlefield.

Agreed completely. There is a ton of research into how to represent text, and these simple tokenizers are consistently performing on SOTA levels. The bitter lesson is that you should not worry about it that much.

I actually think that their main problem is the belief that they can learn everything about the world by reading stuff on the Web. You can't understand everything by reading blogs and books, in the end, some things are best understood when you are on the ground. Unironically, they should go touch the grass.

One example for all. It was claimed that a great rationalist policy is to distribute treated mosquito nets to 3rd-world-ers to help eradicate malaria. On the ground, the same nets were commonly used for fishing and other activities, polluting the environment with insecticides. Unfortunately, rationalists forgot to ask people that live with mosquitos what they would do with such nets.

The problem with your argument is that what you call agent is nothing like what Minsky envisioned. The agents in Minsky's world are very simple rule based entities ("nothing more than a few switches") that are composed in vast hierarchies. The argument Minsky is making is that if you compose enough simple agents in a smart way, an intelligence will emerge. What we use today as agents is nothing like that, each agents itself is considered intelligent (directly opposing Minsky's vision "none of our agents is intelligent"), while organized along very simple principles.

I completely agree with you and I am surprised by the praise in this thread. The entire research program that this books represents is dead for decades already.

I wonder to what extent this is caused by the writing style LLMs have. They just love beating around the bush, repeat themselves, use fillers, etc. I often find it hard to find the signal in the noise, but I guess that it is inevitable with the way they work. I can easily imagine my brain shutting down when I have to parse this sort of output.

Obviously it can't handle heavy impasto. With heavy impasto, people usually literally sculp the surface using some material to math the rest of the painting and then paint on top of this. Even with minimal impasto, they will often try to mimic the texture by imprinting various tools into the material, mimicking for example brushes used by the original author. This printing method can't do this at all, and the filter might struggle when the painting is too uneven.

Otherwise this might be an interesting technique, if the result can match the color and texture of paint perfectly. I can see it being used for some low priority paintings. There are much more paintinga that need restoration than people with necessary skills, so this could save of them, as it will be more viable to fix them. The infilling is usually just a small part of the entire process, but usually the most difficult wrt how skillful the conservator must be. You must be able to match colors and style perfectly, and there are huge differences in how fast this process is depending on the skills of the painter.

Two differences. (1) You can do more targeted attacks. The furthest IRA got was usually to park a van in front of a building. With drones, you can target individuals, cars, planes, etc, even when they are physically separated from public spaces. (2) Since you can do targeted attacks, you don't need that much explosives. This makes you harder to identify and track. Breivik had to buy a farm to have access to the amount of fertilizer he wanted to use.

I like to observe how organization affects how a company operates. As soon as you create a department, that department will start to generate reasons why it should remain being a department, as a sort of self preservation instinct. If you establish a design department, they will start planning complete redesigns sooner or later -- they need to have something going on to justify their existence. When I see this type of redesign, I can't help but wonder whether it is something that was cooked so that the design department can have a place at the table.

As a tangent, HR departments are very often affected by this as well. As soon as you have large enough HR, they will start generating ideas about how to waste other teams time. They have to justify their existence by organizing some events, trainings, activities, even if they actively harm the bottom line.

I agree that AI is an extreme example, but similar pressures exist in other popular fields and subfields, especially in STEM. Peter Higgs famously said that he would probably not be able to do a PhD nowadays.