spyrja

It really wasn't a bad operating system. In fact it kind of blew its (lame) Win9X predecessors out of the water! I ran on Win2000 for years before finally switching to Linux. Of course Microsoft ended up going a different course with its newer "offerings" and I have nothing but pity for those who still have to use their products on a day-to-day basis.

Welp, looks like they're back up. Home page and notifications are loading just fine now.

Actually, the introduction of Old Norse in Britain began in the early 5th century and by the time of Beowulf, Old English was still more than 90% a Nordic language. The Norman invasion of 1066 changed that dramatically (as Old French became the "new" official language) and to this day Modern English could reasonably be considered mostly French, then Latin, and only in small part Nordic (although the most common words used are by majority derived from Old Norse counterparts.) Oddly enough, only a few Celtic words made it into the language.

When the Angles, Jutes, and Saxons arrived in England around fall of Rome in the early 400s, the "official" language was Latin (as Britain had been a Roman colony since 43 AD) whereas locals spoke various Celtic dialects. The newcomers brought with them Scandinavian tongues and for the next 500 years or so Old English developed with minor changes (with sparse inclusions from Latin and Celtic influences). That all changed in 1066 with the Norman Invasion where Old Northern French became the new official language. (French also began as a Nordic language, but over time only a few hundred or so words remained, with the rest being mostly Latin-based.) As far as Modern English is concerned, while only ~20% is based on Old Norse, those words form more than 80% of what is most commonly used on a day-to-day basis.

Not really. It does however help drive home the point that such interjections were unlikely to be used by speakers of Nordic languages in order to begin a tale. (On the other hand in Latin and Celtic traditions, interjections were widely used in story-telling, eg. "Ecce!" and "Féach!" respectively). Old English speakers would have been more inclined to used interjections in a responsive context. For example, to the statement "The boat is taking on water!", one might respond "How?!". But to begin a conversation with an interjection, that just isn't consistent with what we see in any of the speech patterns found in languages which developed from Old Norse.

Nevertheless I do think it is safe to say that such interjections were used, at least on a day-to-day basis. (Bear in mind that relatively few Old English texts survive to this day and almost certainly not many were produced to begin with. Old Norse itself was for the most part a spoken language, unlike Latin for example, and its predecessor which developed in England only started to be written down because of external influences.) Point is, all of the Nordic languages employ interjections akin to "Ah!", "Oh!", "Why?!", "Indeed!", "How?!", etc. So there really isn't any reason to think that such things wouldn't exist in OE as well.

I agree with Dr. Walkden here. While it was used as an interjection at times (just as it is today when someone exclaims, "What?!") in the context of the opening line of Beowulf "hwæt" is more likely being used to reformulate a statement in order to convey a sense of emphasis. An example in modern English would be something like, rather than saying "That was a gorgeous sunset!", one says "What a gorgeous sunset that was!". (Notice that the verb has now moved to the end of the sentence. In fact if you look at the last word of the line in question, we have the verb "fremedon" which means "performed", so indeed the placement of "what" at the beginning of the line facilitates the restructuring of the sentence in such a way that makes it "sounds right".)

To be fair, the origins of "hello" go back much further than 600 years. Variations of it appear in Old Icelandic from almost 1000 years ago, and if you look at Old English texts from hundreds of years before that you will find greetings such as "Wes þū hāl!" (or roughly, "May you be well!"). In other words, all are based on salutations which have most likely been in use in one form or another for at least two millenia (if not longer).

Most LLM's these days tend to be strongly "left-leaning". (Grok being one of the few examples of one that leans "right".) Personally I'd prefer if they were trained without any political bias whatsoever, but of course that's easier said than done given that such lines of thought are present in so many datasets.

FWIW it's a fairly straightforward algorithm. In C++:

  bool balanced(const string& text, const string& open, const string& close) {
    size_t length = text.size(), brackets = open.size();
    assert(close.size() == brackets);
    stack<char> buffer;
    for (size_t index = 0; index < length; ++index) {
      char ch = text[index];
      for (size_t slot = 0; slot < brackets; ++slot) {
        if (ch == open[slot])
          buffer.push(ch);
        else if (ch == close[slot]) {
          if (buffer.empty() || buffer.top() != open[slot])
            return false;
          buffer.pop();
        }
      }
    }
    return buffer.empty();
  }

Next up: 64-bit booleans for the win!

And for the love of God, please don't do 0[src].

I couldn't get it to do much. Also noticed a subtle bug here:

  next = compile(read(buffer), cons("halt", NULL));

Or maybe we can know them equally well? The function f(x) = x(0^(sin(πx)^2)) for example "requires" infinities, but only returns integer values.

OK, but surely only because the exact value of 1 exists in the first place.

Ah, good catch. I forgot to scale the point properly! How does the updated sketch look?

EDIT: Plotting it out as a point cloud seems to confirm your suspicion.

Without scaling: https://editor.p5js.org/spyrja/sketches/7IK_RssLI

Scaling fixed: https://editor.p5js.org/spyrja/sketches/kMxQMG0dj

That's a neat approach! So basically something like this: https://editor.p5js.org/spyrja/sketches/eYt7H36Ka

Kind of? If you want the points to be more randomly-distributed, something like this would probably be a better approach: https://editor.p5js.org/spyrja/sketches/eYt7H36Ka

In this particular case it was simply a matter of not enough corner cases defined. I was however using property-based testing, doing things like reversing then un-reversing the UTF-8 strings, re-ordering code points, merging strings, etc for verification. The datasets were in a variety of languages (including emojis) and so I mistakenly thought I had covered all the bases.

But thank you for the link, it's turning out to be a very enjoyable read! There already seems to be a few things I could do better thanks to the article, besides the fact that it codifies a lot of interesting approaches one can take to improve testing in general.

It's just a convention I use for personal projects. Back when I started coding in C, people often just opted to go with one or two character variable names. I chose three for locally-scoped variables because it was usually enough to identify them in a recognizable fashion. The fixed-width nature of it all also made for less eye-clutter. As for function arguments, the fact that they were fully spelled out made it easier for API reference purposes. At the end of the day all that really matters is that you choose a convention and stick with it. For team projects they should be laid out early on and, as long as everyone follows them, the entire project will have a much better sense of consistency.

Well it is a pretty old codebase, the whole project is written in C. I haven't done any Rust programming yet but it does seem like a good choice for modern programs. I'll check out the link and see if I can glean any insights into what needs to be done to fix my ancient parser. Thanks!

Just as an example of what I am talking about, this is my current UTF-8 parser which I have been using for a few years now.

  bool utf_append_plaintext(utf* result, const char* text) {
  #define msk(byte, mask, value) ((byte & mask) == value)
  #define cnt(byte) msk(byte, 0xc0, 0x80)
  #define shf(byte, mask, amount) ((byte & mask) << amount)
    utf_clear(result);
    if (text == NULL)
      return false;
    size_t siz = strlen(text);
    uint8_t* nxt = (uint8_t*)text;
    uint8_t* end = nxt + siz;
    if ((siz >= 3) && (nxt[0] == 0xef) && (nxt[1] == 0xbb) && (nxt[2] == 0xbf))
      nxt += 3;
    while (nxt < end) {
      bool aok = false;
      uint32_t cod = 0;
      uint8_t fir = nxt[0];
      if (msk(fir, 0x80, 0)) {
        cod = fir;
        nxt += 1;
        aok = true;
      } else if ((nxt + 1) < end) {
        uint8_t sec = nxt[1];
        if (msk(fir, 0xe0, 0xc0)) {
          if (cnt(sec)) {
            cod |= shf(fir, 0x1f, 6);
            cod |= shf(sec, 0x3f, 0);
            nxt += 2;
            aok = true;
          }
        } else if ((nxt + 2) < end) {
          uint8_t thi = nxt[2];
          if (msk(fir, 0xf0, 0xe0)) {
            if (cnt(sec) && cnt(thi)) {
              cod |= shf(fir, 0x0f, 12);
              cod |= shf(sec, 0x3f, 6);
              cod |= shf(thi, 0x3f, 0);
              nxt += 3;
              aok = true;
            }
          } else if ((nxt + 3) < end) {
            uint8_t fou = nxt[3];
            if (msk(fir, 0xf8, 0xf0)) {
              if (cnt(sec) && cnt(thi) && cnt(fou)) {
                cod |= shf(fir, 0x07, 18);
                cod |= shf(sec, 0x3f, 12);
                cod |= shf(thi, 0x3f, 6);
                cod |= shf(fou, 0x3f, 0);
                nxt += 4;
                aok = true;
              }
            }
          }
        }
      }
      if (aok)
        utf_push(result, cod);
      else
        return false;
    }
    return true;
  #undef cnt
  #undef msk
  #undef shf
  }

True. But then again, backward-compatibility isn't really such a hard to do with ASCII because the MSB is always zero. The problem I think is that the original motivation which ultimately lead to the complications we now see with UTF-8 was based on a desire to save a few bits here and there rather than create a straight-forward standard that was easy to parse. I am actually staring at 60+ lines of fairly pristine code I wrote a few years back that ostensibly passed all tests, only to find out that in fact it does not cover all corner cases. (Could have sworn I read the spec correctly, but apparently not!)

I really hate to rant on about this. But the gymnastics required to parse UTF-8 correctly are truly insane. Besides that we now see issues such as invisible glyph injection attacks etc cropping up all over the place due to this crappy so-called "standard". Maybe we should just to go back to the simplicity of ASCII until we can come up with with something better?

It doesn't appear to, but you could always add this to the included common.lisp file:

  (define Y (lambda (f) (lambda args ((f (Y f)) . args))))

Well exactly. The only variability would be on a per-resource basis, so the server-side calculations would likely be quite manageable. The RESOURCE_ID could be a simple concatenation of the name, size, and last-modification-date of the resource, the ITERATIONS parameter would obviously be tuned to by experimentation, and the MEMORY_COST needed based on some sort of heuristic.

The real question is whether or not it would really be enough to discourage indiscriminate/unrestrained scraping. The disparity between the computing resources of your average user and a GPU-accelerated bot with tons of memory is after all so lop-sided that such an approach may not even be sufficient. For a user to compute a hash that requires 1024 iterations of an expensive function which demands 25 MB of memory might seem like a promising scraping deterrent at first glance. On the other hand, to a company which has numerous cores per processor running in separate threads and several terabytes of RAM at it's disposal (multiplied by scores of computer racks) it might just be like a drop in the bucket. In any case, it would definitely require a modicum of tuning/testing to see if it is even viable.

I have actually implemented this very kind of hash function in the past and can attest that the implementation is fairly trivial. With just a bit of number theory and some sponge-contruction tricks you can achieve a highly robust implementation with just a few dozen lines of Javascript code. Maybe when I have the time I should put something up on Github as a proof-of-concept for people to play with. =)

Yep, exactly. NAN's truly are a great example of just how weird an edge case can get too!

  #define isNan(X) ((X) != (X))

Another approach. Require a hash(RESOURCE_ID, ITERATIONS, MEMORY_COST) for each and every resource request. Admittedly that might get a little tricky considering that you don't want to bog down actual users with sluggish page loads. But if carefully tuned to the highest tolerable level it might actually be sufficient. (Maybe.) It's a hard problem....

Way back when I actually perused through quite a bit of Stepanov's STL code and stumbled on that very thing, the comparison operator semantics. I remember thinking it very clever of him to approach it that way and it hadn't even crossed my mind at the time that it might lead to some kind of undefined behavior. Thanks for pointing out such an interesting tidbit!

Doing a bit of digging seems to confirm that, considering that git actually does remove a lot of redundant files during the garbage collection phase. It does however store complete files (unlike a VCS like mercurial which stores deltas) so nonetheless it still might benefit from a download-the-current-snapshot-first approach.