lrei

tech debt generator - another job AI is taking away from humans!

Warning: This is AI generated, probably a low end model as some of the content is outright nonsense eg: """ concept of MoE is quite prevalent (refer Outrageously Large Neural Networks: the Sparsely-Gated Mixture-of-Experts Layer), with Langchain’s high-level implementation of an LLMRouterChain, and notable low-level integrated examples """

No, that's not what I meant. I meant that in its reinforcement learning phase, GPT saw examples of "fix this text" style requests and was rewarded for doing a good job. That's different from seeing examples of typos and still predicting the right word which happens during the language model self supervised training. Both likely help it be good at it.

Yup totally plausible. Things like word (token) dropout and inserting random uniform noise into embeddings or just edit distance perturbations to the tokens are all well known but still Figure 1 looks extremely impressive.

GPT-4 was clearly trained to fix typos and handle not well written written requests. That much is visible directly from just using it within chatGPT UI in normal usage and fits common user scenarios (eg fix my bad draft). We know it was trained on social media data from Reddit much of which is not great writing either. Now I'm wondering if it was trained on (imperfectly) OCRed data too...

Transformer is the architecture. "Generative Pretrained" is just a term made up by the author to mean what everyone called for decades before and will call for decades after "Language Modelling". It was just a new way of saying "Language Modelling Transformer" that sounded cooler to the author and gave it cool initials. Coming up with cool names for models is hard.

These ToS, CSAM, moderation, etc are very orthogonal to the app itself. Why conflate them? Most non-hackernews people are completely unaware of them, so it won't be a factor in their choice of platform. Are we even sure other apps aren't doing similar things? Or have equally dangerous practices around password changes?

Yikes! good luck, hope it works out. This is scary. Gmail is still, by far, the best email app. Something like this could happen to me and, I suspect, a lot of other people.

Exactly the same reason why I use Iosevka and 79chars. However I use WQHD monitors and fit 4-5 splits.

I don't know what being in "tools for thought space" means but if it just mean "Roam or similar user", I can give my personal answer: I set up 2 or 3 templates when I started. The most used one I updated once since then. That's it. The time I spent doing this is essentially 0% of my Roam use time.

I can see how someone who checks out blogs/youtube about Roam-like tools might get the wrong impression but those are usually a business so they have to output content.

It also comes down to personality, I suspect some people have a tendency to optimize tooling as a form of procrastination.

Must've been quite the statistical masterpiece to disentangle it from the World Wars and the Great Depression. Off course extrapolating conclusions from early 20th century education and career progression to the 21st will be an even greater achievement.

Research shows that doing the recall is the important part. The effect is by now well accepted. Off course if you do the cards yourself you can guarantee both their quality and their relevance to you. Ive heard of research pointing that learning things that are relevant to you helps. (If you are learning Spanish to speak with a relative or friend a flash card with an obscure literary word is probably more annoying than helpful). early episodes of Learning Scientists podcast talk about using vs creating flash cards if you want to know more.

I would if I could - I have an external GPU at home. Unfortunately Apple is (not without reason) angry at nvidia so they dropped support for Nvidia in Mac OS. I’d have to use Windows which is a big no no for me. Obvious pytorch can’t support it.

Probably, they were the type that had no problem killing. Someone of a different type was in charge of justifications. The latter would’ve also been more the type that wrote, now historical, documents.

This is true of every field of science and even social science I’ve known. Also the same terms have different meaning in different fields. Also the standard use of a term within a subfield can be different from its usage in another subfield. There are even variations in usage within communities of the same subfield (e.g. inside different ML communities).

I have plenty of experience with Mercedes, Audi, volvo, jeep and BMW cars - driving, renting, test driving them or being the passanger. I would say Mercedes is at least 5 years behind the competition at approx same price point. Disclaimer: I'm obviously not qualified to talk about suspension technology, combustion engines etc. But tech (e.g pilot assist/autopilot), Driver interface features and multimedia, yeah Mercedes is very very very far behind.

Cool - I didn't know this project. I might take your offer on the free with survey next time I work on a research project with economists or finance people. (If the offer is still available)

In a eU research project we used this http://eventregistry.org/correlate + regression to predict oil barrel prices. We got better initial results than an information market from our partners but project ended and I'm not sure we'll get around to a more thorough analysis and publication of results.

Good luck - I like your idea.

trivial processing power but news organizations tend to want traffic on their websites where you can see ads and click on them and other content. Not sure for how long they will even support RSS let alone make an effort to support websockets. They begrudgingly push stuff into social media only because it significantly pushes traffic to their sites.

There is a free tier with the number of API requests limited.

Correct. Main author of the project is Gregor Leban:

https://scholar.google.co.uk/citations?user=5pAxBWsAAAAJ&hl=...

The original crawler (newsfeed.ijs.si) paper is from

Trampus, Mitja and Novak, Blaz: The Internals Of An Aggregated Web News Feed. Proceedings of 15th Multiconference on Information Society 2012 (IS-2012).

I used it only for research (1 for media bias analysis related to the refugee crisis and 1 at the intersection of social/media and economy).

I know the people who started EventRegistry (same department/institute as me) so I didn't care so much about finding alternatives (I.e I got free access, I think most research users do have free access)

Data quality was mostly ok but some artifacts of the article content extraction (e.g. Something like readability) were present like bits of social media share buttons. Also some updated articles were not marked as updates but rather as new articles (after some threshold of changes).

The API has been improved since I last used it - but I found it a bit unintuitive and complicated at the time (almost a year ago). From my understanding it is a bit engineered in order to prevent users from accidentally doing expensive queries or pulling in too much data. kind of makes it harder to use. But still simple - you get up and running Withings minutes and just refine your queries as you go. API limits were generous and you could always resume work after the limit reset (I was getting historical data rather than following the stream)

I got access for free. You should really ask them (they are responsive) I think it depends on org type/size - but the figures I heard unofficially were in the netflix (per user) price range.

Afaik there is no meta data scrapped (technically I think some HTML meta tags are scrapped but not sure if used).

There is machine-learning in the annotations - categories rely on a (cross lingual) text classifier, entities rely on matching to Wikipedia articles, maybe a bunch of small other things to - I don't know all the details - there are some papers published about it.

Hi - I've used this API several times. Ask me anything (though there have been been changes since I last used it)

Code is only easy to replicate when they give you or publish the code

No. Graduate ML students can implement the papers they read w/o a reference implementation - just search github. As I said, I implemented PV w/o the reference code. Many others did the same even before I did.

dismissing a 3% difference is actually reflective again of how delicate understanding ML results

Not really. I understand very well results in ML (Otherwise I would be a pretty incompetent graduate student). But does a 3% increase on say imdb translate to an increase on a another text classification task? possibly - but usually not. If it does translate well across text classification datasets, you will almost certainly see the different datasets and the results in the paper.

Almost nobody writes tests for experiment code. You're proving my point :)

It's a good point but in my experience, the kinds of mistakes that I've usually found with my own or others experimental code would not be possible to catch with a software test. Only with analysis of the results do they become obvious.

I'm surprised (and concerned) that there haven't been any retractions or errata published for the paper cited

there is a subsequent paper by Mikolov and Mesnil at ICLR with the correct results. And code to replicate them.

Retraction? You want to retract a great paper because there is a 1-3% accuracy discrepancy for a result on a more or less random text classification task?

about machine learning: how hard it is to actually trust results

I find the opposite true: code is easy to replicate and the datasets for algorithm comparison are open (e.g. imdb used in the PV paper). If you show very good results (especially with a simple approach such as PVs) people will immediately implement your algorithm and if their results don't match your published results, it will be known. PS: I implemented PVs shortly after it was published - though I don't care so much for the 1-3% or wtv accuracy discrepancy on the imdb dataset, the idea is great.

Graduate students almost never write tests for their code

1) I doubt a standard software test would've helped here (probably cross-val would've caught it); 2) Who writes tests for experiment code? 3) The graduate student story is concerning: either a) someone doing a lot of the heavy lifting for the paper w/o being credited or b) this someone doesn't exist

Energy is energy, a calorie is a calorie in gasoline, food or electricity. However when talking about nutrition labels, those are generally an estimate of calories potentially extracted by humans (which I think is what you mean with nutritional calories). I'm not confusing anything. But some people seem convinced that the car analogy is exactly how the human body works - you put food into your mouth and it gets "burned" by exercise or stored (calories in, calories out) and no amount of evidence or science seems to convince them otherwise.

Eat (drink) 4000 calories of gasoline. wont make you fat. point is: only calories you can convert into "human fat" will make you fatter. that process can not happen without certain chemicals (hormones) in your body, like insulin mentioned above. it's not as straightforward as that which was op's point.

Good one.