HN user

exebook

212 karma
Posts12
Comments75
View on HN
Large Enough 2 years ago

Tokenization can be bypassed like this; (mistral large 2)

How many letters R are in the word "s-t-r-a-w-b-e-r-r-y"?

The word "s-t-r-a-w-b-e-r-r-y" contains three instances of the letter "R."

How many letters R contain the word strawberry?

The word "strawberry" contains two instances of the letter "R."

I'd guess these poor teenagers were mentally sick of something else before the social networks thing came to be. And they will be sick of another thing after social networks get replaced.

Important thing is that we have developed rules that parents teach to kids in regards to many things, but we've developed those rules for centuries, now with internet we need generations before kids will be ready to dangers of internet as good as they are prepared for the dangers of the street for example. Damn we still have bullying in schools, because schools are around just recently, like 100+ years, we have not adapted yet.

I made my own proportional font Variable.ttf and refined it for over a decade. I am using it daily. Over many iterations I refined space width and each important coding character's shape and size. Added some ligatures in the last few years. It's very good I consider it a work of art, and I cannot think about going back to fixed fonts. Only me and my friend use it.

https://github.com/exebook/variable

It is also about performance.

I've watched a video recently from a guy who compiled entire Linux kernel in under 1 second I believe, he used TinyC and also noticed that something like 90% of compilation is tokenization of headers that are included many times, like there are headers that are included thousands of times in almost every C file, so he ended up caching tokens. So a big reason to have a separate tokenizer is that tokenization is a simpler task and can be optimized with all low level approaches, like perfect hashes, crafted nested switch/if tries, branchless algs, compiler intrinsics etc.

The good tokenizer is about as fast as a speed of writing the output array of records into memory. Which means it is important to choose right memory layout for your tokenized data so that when parser reads tokens it has as little cache misses and indirect memory access as possible. Tokenization can be thought of as a sort of in memory compression.

Exploration seems like a natural way for civilisation to live, but that's because exploration is needed to expand knowledge and grow more intelligent, we as a civilisation mostly became obsessed with intellect dozens generation ago, if not less, and gathering knowledge and exploring for us seems so important right now. It might be less important if important at all for species who have been intelligent for a long time and gathered orders of magnitude more knowledge than we do.

Related question is do species really need intellect at all and how much. It could be that the main purpose of intellect is to increase chances to survive, but what if civilisation already figured survival, will they even need to stay intelligent? Maybe their goal is to be happier or maybe have billion orgasms per second.

Another more obvious reason to explore the universe is lack of resources, at least Earth was mostly explored to be exploited. If resources are solved, one reason to explore less.

Are you an author?

I have read the sci-fi short novel when I was a kid, and that was 80ies. The future earth have found signs of life on some distant planet and the science ship was sent to investigate, they got there but could not find anything on the empty rocky planet, the somewhat detective story leads them to realize the life is on the star of that system, not on the planet, and intelligent creatures are plasma based life forms. Unfortunately I barely remember this story, and cannot remember the name or author, I've read hundreds of sci-fi back then.

I also read some random popular science post about subatomic processes in cells, it gave some examples of which I recall one that fascinated me, it appears that there is a gate in a cell that opens or closes by a single electron to let a water molecule in or out I guess. This ignited my fantasy, what if there are more subatomic stuff going on in out bodies? How far it can go in reality and in fiction? What if our memory is subatomic? Could we build something on subatomic level, a logic gate for example? Can a life form be fully subatomic?

Also there is a galaxy on the Orion's belt, and the Orion is a cat, they briefly and somewhat humorously explored the idea in Men in Black which I watched many times.

There is quantum realm in the Ant-Man series. If it existed I guess super civilisation would try to go there or stay there to be more energy efficient.

There is also dr Manhattan, whose blue body I believe is photons. Imagine he was not the only one in that experiment, and our entire human race followed him, what would our civilisation be? No buildings, no machines, no crops, no transportation, would we even stay on the planet? What we would do freed from servicing our fragile bodies most of our time? I guess we would just gather together, have fun, party, make new life, teach each other endlessly. So maybe that's what stars are, each just have trillion of dr Manhattan offsprings and that's why we cannot see any signs of life cause we're looking for meatballs.

Maybe first thing super civilisation would do is to upgrade their bodies, I'd first thing get rid of 50-100 kg of useless meat, bones and blood, I guess then the entire consciousness, memory, emotional sphere can be fit in something size of a microchip die, next step might be finding a way to base consciousness on something like quarks, that would solve transportation, and each person would need tiniest amount of energy. Then I guess they just gather together and form a star. They normal day in life would be our microsecond. So possibly every star in the universe is already a super advanced civilisation, they just do not care to contact us, who would want to contact someone who is billion times larger, slower and depend on clumsy molecules.

Do you want a bug report? The lowest row doesn't fit into width of my phone's Chrome. I can only see few millimeters of the rightmost and leftmost cubies. Pinch zoom seems disabled so I cannot adjust. Tried switching to PC mode, but everything became too tiny. Rotating the phone also doesn't help. So for me it's unplayable.

Wanted to play, but after the tutorial a first touch makes the ship disappear. When I reload the page, I can see the ship again, and the landing curve, then again if I touch the screen everything disappear except for the stars.

Values seem to be nan-boxed, which means numbers are simply doubles and therefore will still be located in the same block of contiguous memory. When the higher 8 bits are FF the remaining 48 normally are an indirection pointer. So right, for anything other than numbers your cache misses increase, but storing dynamically sized objects like strings or tuples or maps continuously in array is a hard problem anyway not to mention storage in maps or in tables.

Since this is seemingly a byte code or threaded code based VM, your code execution probably also contributes to cache exhaustion.

#line directive gives you exactly that, you step through original code, it's similar to source maps in JavaScript. #line directive has two arguments the file name and the line of the source in your language i.e. nim, nelua, koka whatever. Debuggers support it for decades. Having said that, printing variables and making debugger aware of the type system of the original language is another story maybe someone can comment on that.

What was your language? I read two of his books when I was a kid in Russian. Also built the matchbox machine but I think it was playing hexapawn which was too simplistic compared to tic tac toe and a bit disappointing. I couldn't find enough beads and had to use cut matches and pieces of paper as well. And it was quite a story to convince adults to buy me so many matchboxes, I think it was 21. I think this machine and Conway's Life also explained in that book have transformed me.

What are the bugs in tcc you have discovered? I am using it almost every day for many years and yet to encounter a single bug. It generates much slower code than gcc but the point is it does it at least an order of magnitude faster. The code it generates is similar to code generated by Delphi mentioned in a sibling comment. I wish they added linear scan register allocator and inlining of small functions to tinyc wich would not slow down compilation I believe but would make the code much faster.

Regardless how fast zig compiler self is, as far as it outsources code generation to llvm and linking to ld it will always be much slower than tinycc. Although recently I've noticed that if you ask llvm to generate and execute something they call bitcode, the code generation seemed faster.

Why OCaml? 4 years ago

OCaml is one of those languages that has it's own compiler backend. I might have forgotten but I think it even uses it's own linker. It is also backed by organisations, for example people from INRIA university and Jane Street company constantly improve it. I remember it also compiled small programs in a few milliseconds, something LLVM based languages cannot do.

Am I the only one who after seeing the title thought they are building Nintendo switch inside Factorio? Which makes me think can it at least run something like Atari 2600 at decent speed?