_pastel

The tradeoff is highly contextual; it's not a tradeoff an agent can always make by inspecting the project themselves.

Even within the same project, for a given PR, there are some parts of the codebase I want to modify freely and some that I want fixed to reduce the diff and testing scope.

I try to explain up-front to the agent how aggressively they can modify the existing code and which parts, but I've had mixed success; usually they bias towards a minimal diff even if that means duplication or abusing some abstractions. If anyone has had better success, I'd love to hear your approach.

Bike is the one that gets me - when the full Ode to Joy hits I tear up every time.

You could fine-tune the embedding model to reduce cosine distance on a more specific function.

I work with an extremely effective machine learning engineer, and the biggest thing I've learned is how far you can get with vibes, even in a more traditional ML situation.

He invests most of his time visualizing the inputs and outputs of his systems very carefully, instead of focusing super heavily on metrics; this is a lot more effective than I thought it would be, particularly in the early stages of a project.

Why is max_seq_len set to 2048 [1] when the model card says the context size is 8k [2]?

[1] https://github.com/meta-llama/llama3/blob/14aab0428d3ec3a959...

[2] https://github.com/meta-llama/llama3/blob/14aab0428d3ec3a959...

Tooling around embeddings has improved. Creating and fine-tuning custom embeddings for your tabular data should be easier and more powerful these days.

100% agree.

One thing that helps is hooking metabase up to its own database and building queries on your queries, e.g.:

    select *
    from report_card 
    where dataset_query ilike '%' || {{query}} || '%'

We also try really hard to keep the Collection directory structure clean and consistent. But it's still really hard.

Not only is it not constant time, it's not even polynomial - it's psuedo-polynomial. Also it'll fail on negative numbers, right? You'll need something like `10000 * log(time + min(time) + 1)` to be linear in the bits used to represent the inputs.

In computational complexity theory, a numeric algorithm runs in pseudo-polynomial time if its running time is a polynomial in the numeric value of the input (the largest integer present in the input)—but not necessarily in the length of the input (the number of bits required to represent it), which is the case for polynomial time algorithms.

https://en.m.wikipedia.org/wiki/Pseudo-polynomial_time

Some ways to make recipients feel more comfortable:

- You can suggest some other contribution. "Would you mind bringing snacks? / Would you mind handling music on the drive? / Would you mind giving X a ride?"

- You can allow them to reciprocate in less expensive situations, like taking the check when you are at a cheaper place.

True in this situation, but note that intermediate activations and gradients do take memory and in other contexts that's the limiting factor. For example purely convolutional image networks generally take fixed-size image inputs, and require cropping or downsampling or sliding windows to reach those sizes - despite the convolution memory usage being constant for whatever input image size.

Ah, the celery semantics.

Interesting how Gitlab repeatedly emphasizes it's based on "cost of market", not "cost of living".

So in the sparse grams explanation, what are the bigram weights?

Is it inverse frequency, so common bigrams get split last? And the goal is to be able to search on a larger gram that covers the more common trigrams as often as possible?

From https://acoup.blog/2019/07/26/collections-war-elephants-part.... I highly recommend the whole series.

At Bagradas (255 B.C. – a rare Carthaginian victory on land in the First Punic War) [...] the elephants disorder the Roman line. In the spaces between the elephants, the Romans slipped through, but encountered a Carthaginian phalanx still in good order advancing a safe distance behind the elephants and were cut down by the infantry, while those caught in front of the elephants were encircled and routed by the Carthaginian cavalry. What the elephants accomplished was throwing out the Roman fighting formation, leaving the Roman infantry confused and vulnerable to the other arms of the Carthaginian army.

[...]

Elephants could also be used as area denial weapons. One reading of the (admittedly somewhat poor) evidence suggests that this is how Pyrrhus of Epirus used his elephants – to great effect – against the Romans. It is sometimes argued that Pyrrhus essentially created an ‘articulated phalanx’ using lighter infantry and elephants to cover gaps – effectively joints – in his main heavy pike phalanx line. This allowed his phalanx – normally a relatively inflexible formation – to pivot.

[...]

Thus the war elephant wasn’t a ‘battle winner’ so much as a dangerous complication thrown in the way of the enemy’s plan of attack. And at that, they were awesome.

Did you target cloud infrastructure from the beginning, or fall into it by chance? Do you think that's an easier entry point than general programming?

I had a graduate algorithms class that was graded like this, but the teacher didn't explain and I was clueless. I spent the whole semester super stressed and convinced I was failing, then was shocked and befuddled by an A. "Emotionally trying" is an accurate description of how it felt.

Props to this professor; struggling in confusion is a lot more fun if you know that's the game you're supposed to be playing.

I see this when giving and receiving form advice in a few physical disciplines. "Just relax your shoulders here and move naturally."

Physical mastery often looks relaxed, natural, and simple, because all extraneous effort has been removed. When you're training hard to reach that state, the "just" can really sting. It feels like: "not only are you bad at this, but it's simple to not be bad at this".

Thanks for all your work!

If you don't mind some off-topic feedback on the Dominion website - have you considered making the Tables screen default to "New" only? At busy times, that screen consistently lags for several seconds while displaying hundreds (thousands?) of running and finished games. It's always a pain to wait for it before de-selecting "Running" and "Post-Game".

Word of warning - if you write your own task queue at a startup, you will spend the rest of your tenure justifying this decision to every new data engineer who joins.

Also, am I crazy or do the celery docs not even clarify their delivery semantics? Isn't that table stakes for a queueing system? As best as I could tell you can get close to "at least once" with

  acks_late=True, task_reject_on_worker_lost=True

If you're interested in browsing creative prompts, I highly recommend the reddit community at r/dalle2.

Some are impressive:

  - www.reddit.com/r/dalle2/comments/uzosy1/the_rest_of_mona_lisa
  - www.reddit.com/r/dalle2/comments/vstuns/super_mario_getting_his_citizenship_at_ellis

  - www.reddit.com/r/dalle2/comments/v0pjfr/a_photograph_of_a_street_sign_that_warns_drivers
  - www.reddit.com/r/dalle2/comments/wbbkbb/healthy_food_at_mcdonalds
  - www.reddit.com/r/dalle2/comments/wlfpax/the_elements_of_fire_water_earth_and_air_digital

Because anytime I search for literature on basic tweaks to the structure of decision trees, I find nothing.

Another example: modern GBM implementations all use binary trees. How would they perform with ternary trees? Or k-way trees for larger k plus some additional soft penalty that encourages minimizing the number branches, unless the information gain is really worth it?

(You can simulate ternary trees with binary, but the splitting behavior is different because ternary can more easily identify good splitting regions in the middle range of the histogram values.)

This seems like such a basic structural question, but the only relevant search result was this downvoted Stack Exchange question from 5 years ago: https://stats.stackexchange.com/questions/305685/ternary-dec...

There are lots of papers on ternary trees in old-school contexts like Ternary Decision Diagrams etc., but nothing relevant to the context of modern tree ensemble performance. Or maybe I'm just bad at searching?

(I implemented this and saw a small accuracy increase from ternary on large datasets, but much worse training speed because you get less mileage from the histogram subtraction trick. Maybe the accuracy would be even better with a more clever choice of soft penalty.)

It's baffling to me how little research attention there has been to improving tree-based methods, considering their effectiveness.

For example, LightGBM and XGBoost allow some regularization terms, but the variance/bias is still mostly controlled by globally setting the max depth and max node count (and then parameter searching to find good settings).

Surely there must be more powerful and sophisticated ways of deciding when to stop building each tree than counting the number of nodes? If this was neural nets there would be a hundred competing papers proposing different methods and arguing over their strengths and weaknesses.

I'm not sure whether the problem is that neural nets are just fundamentally more sexy, or that in order to make SOTA improvements in GBMs you need to dive into some gnarly C++. Probably both.

Note that greedy probability maximization or information gain are not quite optimal. Not without some kind of search or tiling heuristics.

For example, suppose you've eliminated all the other ships and are only looking for the 2-ship. If you overlay a checkerboard, it's best to only shoot on squares corresponding to a single color on the checkerboard. This guarantees you won't waste shots adjacent to previous misses once the search area fills up.

These numbers focus on reads. How does writing speed to cache, main memory, or disk compare? Anyone have some ballparks to help me build intuition?

Maybe there's survivorship bias in that sample. Are ancient authors we still read more likely to be obsessed with legacy-building than the ones we've forgotten?

[16] Design as a team; implement as individuals.

This is pithy. I'm curious - does anyone here follow this philosophy? How do you actually design as a team? A meeting for every single design decision?

I'm used to the team finding general alignment on the problem in meetings, but then a single person coming up with the design, writing a doc, and circulating it for comments.

  // [The eighth byte of LINENO_MID] changes in meaning over the course of
  // the program. It does indeed represent the billions digit most of
  // the time; but when the line number is getting close to a multiple
  // of 10 billion, the billions and hundred-millions digits will always
  // be the same as each other (either both 9s or both 0s). When this
  // happens, the format changes: the hundred-millions digit of
  // LINENO_MID represents *both* the hundred-millions and billions
  // digits of the line number, and the top byte then represents the
  // ten-billions digit. Because incrementing a number causes a row of
  // consecutive 9s to either stay untouched, or all roll over to 0s at
  // once, this effectively lets us do maths on more than 8 digits,
  // meaning that the normal arithmetic code within the main loop can
  // handle the ten-billions digit in addition to the digits below.

Kaggle competitions rarely produce interesting algorithmic results.

But I highly encourage you to read the winners' solutions. They are full of clever data insight, augmentations, regularizations, feature engineering, and preprocessing and postprocessing tricks.

But above all, compared to the academic literature, it's shocking how much time and creativity they spend on validation. Maybe I'm reading the wrong papers, but the flashy new neural architectures rarely even mention their validation setup; Kaggle winners sometimes devote half of their explanation to it. It's part of their secret sauce.

Two personal favorites:

(1) https://www.kaggle.com/c/severstal-steel-defect-detection/di.... The "random defect blackout" was a really clever data augmentation.

(2) https://www.kaggle.com/c/ieee-fraud-detection/discussion/111.... Particularly how they reduced overfitting with adverserial validation. They trained a separate model to distinguish between train and test sets, and then dropped features that ranked highly in feature importance on that model. That's probably a well-known technique in some circles, but I had never seen anything like it before.

Yes! It just takes practice.

I started out finding a comfortable position and relaxing with a timer set for 15 minutes. After years of practice, I now hit REM almost immediately and wake up feeling rested without a timer in 15-20 minutes.

It's easier for your body to learn if you have a specific place (e.g. a coach), time (e.g. immediately after lunch), and position (I cross my ankles and steeple my hands on my chest). These make it feel qualitatively different from night sleeping and easier to wake up naturally.

You also want to pay attention to sound and light. I use noise-cancelling headphones at work and earplugs + white noise at home. I choose a well-lit area, preferably with natural lighting, and then cover my eyes with a cloth or sleep mask. It's easier to feel refreshed when you uncover your eyes to bright sunshine, rather than waking up in a fully dark room.

Honestly, the main challenge is feeling self-conscious when napping at work. Originally I would lock myself in a shower room or go out to my car, but I'm finally self-confident enough to sleep on a public coach.

Have you tried Pandora?

In other ways they are completely outclassed by Spotify, but their mood-respecting recommendations are the best I've seen by far.