jgehring

There's not that much Uranium actually that's economically sensible to extract. The NEA says in their 2024 report on Uranium [1]:

Considering both the low and high nuclear capacity scenarios to 2050 presented in this edition, and assuming their 2050 capacity is maintained for the rest of the century, the quantities of uranium required by the global fleet – based on the current once-through fuel cycle – would likely surpass the currently identified uranium resource base in the highest cost category before the 2110s.

Their "high" scenario assumes having a bit more than double of today's capacity by 2050; today we have about 4-5% supply from nuclear energy worldwide.

[1] https://www.oecd-nea.org/jcms/pl_103179/uranium-2024-resourc...

That's what happens in the very last layer. But at that point the embedding for "was" got enriched multiple times, i.e., in each attention pass, with information from the whole context (which is the whole novel here). So for the example, it would contain the information to predict, let's say, the first token of the first name of the murderer.

Expanding on that, you could imagine that the intent of the sentence to complete (figuring out the murderer) would have to be captured in the first attention passes so that other layers would then be able to integrate more and more context in order to extract that information from the whole context. Also, it means that the forward passes for previous tokens need to have extracted enough salient high-level information already since you don't re-compute all attention passes for all tokens for each next token to predict.

"I don't care about cookies"?

Indeed. For bad news, e.g. crime, the (German) media takes great care to mention whether suspects or convicts are immigrants or of direct immigrant descent. Attaching this information to good news is done less frequently but would be required to paint a more realistic picture.

I think your parent paints a rather extreme picture. The two examples they picked, Wiehre and Herdern, are the upscale neighborhoods with plenty of mansions, and there's rather only one area with large concrete housing blocks (Weingarten). As usual, you pay a premium for the top locations but in terms of quality of living and "greenness" there are many other good spots.

If you don't, then like in many south-German cities (located in mountain valleys where space is scarce), you're living in a place without any green in sight

While this is generally true, the newer districts like Vauban also feature city-owned housing that is specifically targeted to be rented by lower-income families. If you're a student there are again lots of options all throughout the city.

Nice to see my neighborhood (Vauban) featured here :) It's an awesome place to raise kids as it provides an almost village-like environment, being right at the edge of the city and hence the edge of the black forest. But it also comes with the immediate benefits of a nearby regional hub and with plenty of next-door daycare/kindergarten institutions, playgrounds and a ton of other kids. This was one of the main factors that made us move back here after two years in SF.

As elsewhere, affordable, or even available housing is now an issue, in particular for families. Prices have been on a sharp increase over the last years, and although the there's quite some construction (and plans to add a whole new district) it's not keeping up with demand.

Agree, it's so convenient for grabbing fields that I ended up writing a bash script that generates an awk script since the '{print $1}' is cumbersome to type, and I can never remember how to properly output multiple fields.

At some point I thought that had the ideal use case for awk (a git --graph filter) and spent an evening desperately putting it together because, as other commenters mentioned, it's hard to find good documentation and examples online. Sure, I have a fast and mostly-working filter now, but the code is also hard to understand or even debug. On the other hand, the examples linked in the article are actually a lot more readable than I expected, so maybe it's something to consider for small but frequently-used log parsing scripts.

You're right, these issues can also be tackled independently. Transfer learning can help, but my first guess would be that it's hard to get reasonable accuracy (= usable for applications) without hundreds of hours of conversational data. You could also attempt to directly modify the audiobook data by manually adding noises or performing other distortions.

In any case, for read speech in particular there are several corpora out there already, including the moderately large LibriSpeech corpus (1000hr). The state-of-the-art accuracy on read speech is also very good -- for example, domain-specific dictation systems have been commercially viable for quite some time. So while it's true that Audiobooks are a large untapped source, I think that there are other large-scale and richer options like YouTube or movies (i.e. videos with speech for which subtitles are available) that would be more useful to make progress towards good speech recognition systems.

When training speech recognition systems you want to use data that closely matches your target domain. Models trained on audiobooks read by professionals will not perform very well for transcribing conversational or spontaneous speech or if there is background noise.

Yes! The API feels very much like using PyTorch from Python, and implementing models and working with tensors purely in C++ is very convenient. We're using it for our research platform for StarCraft: Brood War (https://torchcraft.github.io/TorchCraftAI).

[disclosure: I work at FAIR]

Yes, ByteNet v2 outperforms LSTMs on characters but not on word pieces. It would be interesting to see how our model performs on characters, especially when scaled up to the size of ByteNet (30+30 layers) and also how ByteNet performs on BPE codes. I think that character-level NMT is definitely interesting and worth investigating, but from a practical point of view it makes sense to choose a representation that achieves the maximum translation accuracy and speed.

There is no online demo but you can run the pre-trained models on your local machine: https://github.com/facebookresearch/fairseq#quick-start. CPU-only versions of the models are available as well.

For a comparison with other translation services, keep in mind that our models have been trained on publicly available news data exclusively, e.g. this corpus for English-French: http://statmt.org/wmt14/translation-task.html#Download .

Yes, there have been a couple of attempts to use CNNs for translation already, but none of them outperformed big and well-tuned LSTM systems. We propose an architecture that is fast to run, easy to optimize and can scale to big networks, and could thus be used as a base architecture for future research.

There are a couple of contributions in the paper (https://arxiv.org/abs/1705.03122) apart from demonstrating the feasibility of CNNs for translation, e.g. the multi-hop attention in combination with a CNN language model, the wiring of the CNN encoder[1], or an initialization scheme for GLUs that, when combined with appropriate scaling for residual connections, enables the training of very deep networks without batch normalization.

[1] In previous work (https://arxiv.org/abs/1611.02344), we required two CNNs in the encoder: one for the keys (dot products) and one for the values (decoder input).

Yes, that's pretty accurate. Step 3 (attention) is repeated multiple times, i.e. for each layer in the decoder. With each additional layer, you incorporate more of the previously translated text as well as information about which parts of the source sentence representation were used to generate it. The independence of the current word from the previous words applies to the training phrase as a complete reference translation is provided and the model is trained to predict single next words only. This kind of computation would be very inefficient with an RNN: it would have to run over each word in every layer sequentially which prohibits efficient batching.

When generating a translation for a new sentence, the model uses classic beam search where the decoder is evaluated on a word-by-word basis. It's still pretty fast since the source-side network is highly parallelizable and running the decoder for a single word is relatively cheap.