CompleteSkeptic

constrained decoding tends to make models dumber - this is why it's rarely used

I helped train some of the first "magic" models at OpenAI[1] and it was a wild ride. We were a pretty sane + skeptical team and we weren't totally convinced the models were as general as they seemed, but the query that convinced me (and later got included in the paper[2]) was "Why is it important to eat socks after meditating?" (something that almost certainly did not appear on the internet before).

An interesting follow up would be when did you realize GenAI wasn't as good as you thought in that "oh shit" moment

[1] co-author of InstructGPT/RLHF/ChatGPT

[2] https://arxiv.org/pdf/2203.02155

TypeSafe AI | Software Engineer | ONSITE (San Francisco) | Full-time

TypeSafe AI (https://typesafe.ai/) is a well-funded stealth mode company making a new class of AI models which enable a reliable way to embed composable intelligence into traditional software, founded by a co-author of ChatGPT/RLHF/GPT4 (answering the questions "If AI is so smart, why are we barely automating any work? Why does today's AI always need a human-in-the-loop?").

We're especially looking for people to join our Synthetic Data research team - no previous data experience needed (synthetic or otherwise). We need strong engineers who care deeply about the end use cases and whose instincts are to NOT trust LLMs blindly and look at the reality of their generations (much more investigator in the spectrum of investigator vs optimizer).

Short 5 min video (of me) describing the company: https://www.youtube.com/watch?v=LE3bGTaAgOE

See open roles here: https://typesafe.ai/careers#opportunities

Is this the first time OpenAI has published comparisons to other labs?

Seems so to me - see GPT-5.4[1] and 5.2[2] announcements.

Might be an tacit admission of being behind.

[1] https://openai.com/index/introducing-gpt-5-4/ [2] https://openai.com/index/introducing-gpt-5-2/

There actually is some work (https://arxiv.org/abs/2003.13630) claiming that FLOPS are a poor measure of real-world performance - with some of the more recent FLOP-efficient models actually running slower than older models.

This isn't directly relevant to PP-YOLO, but I'm surprised roboflow is still promoting "YOLOv5" - despite that model not having an associated paper and it not being made by the authors of the previous YOLO's.[1]

The ML community has been asking the authors of that model to rename their project[2] because they are basically stealing publicity by making it seem like the next version of YOLO, despite its performance being worse than that of YOLOv4.[3]

Roboflow has deflected this in the past by claiming they don't know if "YOLOv5" is the correct name[4], but by continuing to promote it, they are directly supporting it. In fact, I wouldn't be surprised that their claim of not being affiliated with Ultralytics to be either false or a half truth, given that all the top pages about "YOLOv5" were done by roboflow, including the first official announcement.[5]

[1] https://github.com/AlexeyAB/darknet/issues/5920

[2] https://github.com/ultralytics/yolov5/issues/2

[3] https://github.com/AlexeyAB/darknet/issues/5920#issuecomment...

[4] https://blog.roboflow.ai/yolov4-versus-yolov5/

[5] https://blog.roboflow.ai/yolov5-is-here/

An appropriate quote: "If you can't intelligently argue for both sides of an issue, you don't understand the issue well enough to argue for either."

There are many people for whom the declarative paradigm is a huge plus. I would say there are at least 2 major approaches in running fast neural networks: 1. Figure out the common big components and make fast versions of those. 2. Figure out the common small components and how to make those run fast together.

Different libraries have different strengths and weaknesses that match the abstraction level that they work at. For example, Caffe is the canonical example of approach 1, which makes writing new kinds of layers much harder than with other libraries, but makes connecting those layers quite easy as well as enabling new techniques that work layer-wise (such as new kinds of initialization). Approach 2 (TensorFlow's approach) introduces a lot of complexity, but it allows for different kinds of research. For example, because how you combine the low-level operations is decoupled from how those things are optimized together, you can more easily create efficient versions of new layers without resorting to native code.

I think he was referring to the fact that the request completes in finite time isn't a tight enough bound.

State-of-the-art results no longer require unsupervised pretraining with autoencoders or RBMs, but back when unsupervised pretraining was more popular, top researchers were rationalizing that it was consider more biologically plausible than the standard nets trained with back prop, since brains generalize through observing a large amount of data over their lifetime to quickly recognize new objects and since the nets aren't trained for a specific task, they would hopefully generalize better and be a step closer to general intelligence.

You may be interested in DBpedia, which gets its data from wikipedia but presents it in a structured format (for example, the page for Clojure: http://dbpedia.org/describe/?url=http://dbpedia.org/resource...), though for this case, it seems the data is already on wikipedia.

Yup! We were about to implement something like that, but we realized Reagent's model of having atoms store the state worked pretty well with javelin's FRP, and once you break down the state into atoms, the cursors aren't needed anymore.

I wouldn't go so far as to say that Reagent's model is perfect. Definitely pretty good, but we've had some issues that were somewhat non-obvious.

If you're worried about it being maintained, there was actually a great discussion of this post on the clojure subreddit (https://www.reddit.com/r/Clojure/comments/2jq0cu/om_no_troub...) where others talk about Reagent having a community-maintained fork with new features.

Out of curiousity though, what kind of helper methods would you be looking for? I found it pretty comprehensive on its own (though I do have some utility methods for some niche uses involving hierarchical atoms).

If any of my post sounded as if I was complaining about the lack of syntactic sugar, I apologize as that was not at all my intention. The best practices and syntactic ugliness didn't matter to us at all after the learning curve. We wrote some macros to solve the problem, and didn't have to think about it anymore. We used sablono as well, so the syntax was almost the same as that of Reagent.

The biggest problems that we faced with Om were how normal Clojure-y things just didn't work with it, and how the app state had to be kept in a tree instead of a DAG (mostly the latter), which created issues with consistency, performance, or both.

trying to put mutating state into the application state object

How you're supposed to use Om is put all the mutable state into a global atom. I didn't mean putting reference types in the global atom or anything horrible like that, but the problems with cursors being pointers still exist.