HN user
Does it expose any sort of "device id"? Such ids are usually asked for by advertisers and iOS and Android gladly give it to them. I'm sure there are device-id to phone-number maps out there, and anyone with money can get access to them.
Maybe I don't understand something about these models. If the model was trained to mimic Trump tweets, it means that someone spent days of GPU time to find the weights of the model. Now if we want it to mimic HN comments, we'd need to spend the same amount of GPU time to find different weights. This is what I meant by "from scratch".
Regardless, this is a bad PR and they are losing potential customers. No one wants their car to be trimmed down just because the manufactures chose so. No, it doesn't matter what's written in the fine print.
The model needs to be retrained from sctratch for different types of texts. One can release a model trained to generate Trump tweets, but it's of not much use for generating fake news on a specific topic.
Luckily, uBlock Origin can block all portal tags entirely.
This sounds reasonable, but I think in practice the capacity of T1 won't be enough to capture long patterns and the F2 sequence is supposed to help T2 to restore the lost info about the longer pattern. The idea is to make T1 really good at capturing small patterns, like speech in pop music, while T2 would be responsible for background music with longer patterns.
Don't we already do this with text translation? Why not to let one model read a printed text pixel by pixel and the other model produce a translation, also pixel by pixel? Instead we choose to split printed text into small chunks (that we call words), give every chunk a "word vector" (those word2vec models) and produce text also one word at a time.
Can we diff spectrograms to define the "distance" between two chunks of sound and use this measure to guide the ML learning process?
Would it help to decompose sound into subpatterns with Fourier transform?
Afaik, there is a similar technique for recognizing faces: a face picture is mapped to a "face vector". Yet this technique doesn't need the notion of "sequence of faces" to train the model. Can we use it to get "sound vectors"?
T1 indeed contains all the info needed, but T1 also has limited capacity and can't capture long patterns. T1 would need to have 100s of billions weights to capture minute long patterns. I think this idea is similar to the often used skip connections.
That's fine. The goal is to map "similar" 1 second chunks to similar vectors. I'm sure this can be done and uniqueness of sound won't be a problem.
I'm not an ML guy, so can't say if this is an autoencoder.
We can combine multiple sequences in any way we want. Obviously, we can come up with some nice looking "tower of lstms" where each level of that tower processes the corresponding F[i] sequence: sequence F1 goes to level T1 which is a bunch of LSTMs; then F2 and the output of T1 go to T2 and so on. The only thing that I think matters is (1) feed all these sequences to the model and (2) have enough weights in the model. And obviously a big GPU farm to run experiments.
1 second of sound. Or a few seconds of sound.
We can think of a ML model that takes 1 second of sound as input and produces a vector of fixed length that describes this sound:
S[0..n] = the raw input, 48000 bytes per second of sound F[1][k..k+48000] -> [0..255], maps 1 second of sound to a "sound vector". F[2][k..k+96000] -> ..., same, but takes 2 seconds of sound as input
Now instead of the raw input S, we can use the sequences F[1], F[2], etc. Supposedly, F[10] would detect patterns that change every 10 seconds. It's common in soundtracks to have some background "mood" melody that changes a bit every 10-15 seconds, then a more loud and faster melody that changes every 5 seconds and so on, up to some very frequent patterns like F[0.2] that's used in drum'n'bass or electronic music in general.
This is how music is composed by people, I guess. Most of the electronic music can be decomposed into 5-6 patterns that repeat with almost mathematical precision. The artist only randomly changes params of each layer during the soundtrack, e.g. layer #3 with a period of 7 seconds slightly changes frequency for the next 20 seconds, etc.
Masterpieces have the same multilayered structure, except that those subpatterns are more complex.
The same way we map words to vectors or entire pictures to vectors. We'll have another ML model that would take 1 second of sound as input (48000 1 byte numbers) and produce a say vector of 128 float32 numbers that would "describe" this 1 second of sound.
To illustrate more this idea, let's use soundtrack v=negh-3hi1vE on youtube. Such soundtracks consist of multiple more or less repeating patterns. The period of each pattern is different: some background pattern that sets the mood of the music may have a long period of tens of seconds. The primary pattern that's playing right now has a short period of 0.25 seconds, plays for a few seconds and then fades off. The idea is to split the soundtrack into 10 second chunks and map each chunk to a vector of a fixed size, say 128. The same thing we do with words. Now we have a sequence of shape (?, 128) that can be theoretically fed into a music generator and as long as we can map such vectors back to 10 second sound chunks, we can generate music. Then we introduce a similar sequence that splits the soundtrack into 5 second chunks. Then another sequence for 2.5 seconds chunks and so on. Now we have multiple sequences that we can feed to the generator. Currently we take 1/48000th second slices and map them to vectors, but that's about as good as trying to generate meaningful text by drawing it pixel by pixel (which we can surely do and the model will have 250 billion weights and take 2 million years to train on commodity hardware).
Unheard of. This is why in the US we have the 1st amendment backed by the 2nd amendment. The freedom of speech can't defend itself.
What needs regulation is ISPs and companies that deal with personal data.
That CodeNet would be the SkyNet, essentially. What's shown here looks impressive, but it's the same good old text generator that can produce something that looks very similar to the dataset used to train it. It can't go beyond the dataset and generate something new. From the mathematical point of view, that generator interpolates samples from the dataset and generates a new sample.
To give an idea how big is the gap between MuseNet and CodeNet, we can consider a simple problem of reversing a sequence: [1,2,3,4,5] should become [5,4,3,2,1] and so on. How many samples do you need to look at to understand how to reverse an arbitrary sequence of numbers? Do you need to retrain your brain to reverse a sequence of pictures? No, because instead of memorizing the given samples, you looked at a few and built a mental model of "reversing a sequence of things". Now, the state of the art ML models can reverse sequences as long as they are using the same numbers as in the dataset, i.e. we can train them to reverse any sequence of 1..5 or 1..50 numbers, but once we add 6 to the input, the model instantly fails, no matter how complex and fancy it is. I don't even dare to add a letter to the input. Reason? 6 isn't in the samples it's learnt to interpolate. And CodeNet is supposed to generate a C++ program that would reverse any sequence, btw.
At the moment, ML is kinda stuck at this pictures interpolation stage. For AI, we don't need to interpolate samples, but need to build a "mental model" of what these samples are and as far as I know, we have no clue how to even approach this problem.
Impressive. Would this model benefit from something like "dilated attention"? Instead of feeding it raw sound samples, we could split the input into 16 sec, 8 sec, 4 sec and so on slices, assign each slice a "sound vector" serving as a short description of that slice and let the generator take those sound vectors as input. This should supposedly let it gain global consistency in output.
Now an unpopular opinion. I'm not an ML expert, so take my words with reasonable skepticism. This fancy GPT2 model diagram can impress an uninitiated, but we are initiated, right? There is really no science there and it's still the good old numbers grinder: an input of fixed size is passed thru a big random pile of matrix multiplications and sigmoids and yields a fixed size output. We could technically replace this nice looking GTP2 model with a flat stack of matmuls and tanhs, with a ton of weights and given enough powerful GPUs (that would cost tens of millions), train that model and get the same result. It just won't make an impression of science. How are these GTP2 models designed? By somewhat random experiments with the model structure. The key here is the GPU datacenter that could quickly evaluate the model on a huge dataset. The breakthru would be achieving the same quality with very little weights.
If this idea worked in practice, go-ipfs/js-ipfs wouldn't have so severe problems with connectivity.
How could you possible make such a DHT? We live in the world of NATs, especially symmetric NATs, where each mobile phone user gets assigned a random ip:port every time it makes a connection. DHT, on the other hand, needs every node to have a persistent address that can be contacted any time. In other words, with the NATs, a DHT node cannot cache a bunch of peers and contact them later because those peers are no longer available at those addresses, so every time a node re-joins DHT, it needs to restart the bootstrapping process from scratch, from those initial hardcoded bootstrap servers. Effectively this makes this DHT a fully centralized system. WebRTC cannot solve this problem.
It would. As it was mentioned earlier on HN, a woman can get 100k in tax-free alimony payments from a guy making 250k. She can even get his house.
This is not a roadmap, but rather a wishlist. There is a fundamental problem that IPFS needs to solve first. This problem is called an efficient WebRTC-based DHT. In order to change the web, IPFS needs to become usable in the browsers. Since the backbone of IPFS is DHT, there need to be an efficient UDP-based solution for "DHT in the web". Right now this isn't possible and reasons are not just technical, but political. The IPFS team would need to convince all major players that enabling this DHT scenario is a good idea.
An airplane usually spends 1-2 hours in the airport between flights. During that period it needs to be recharged.
If we look at this from a buddhist's point of view, then there is no luck: the author did something in past that allowed him to use this opportunity and what he did could be completely unrelated to his work, e.g. he wrote a blogpost long time ago that he didn't consider important, but it really helped a few people.
Honestly speaking, since it's an anonymous forum, what's the difference between L5 and L7? Like if I was asked the same question - the difference between L3 and L5 - I'd say that it's just experience with a few interesting known projects that's paid so much better, while the cognitive ability is the same.
And this is irrelevant also, because we can introduce sampling the way I described and can continue driving just fine.
This is smart. Unethical, but smart.
It's great that eyes can detect polarization and orbital momentum. My point is that none of that info is needed to drive a car.
"Senior manager" isn't a remarkable achievement. It's just a role or position in a company that needs to be filled in. She happened to be the one who got picked (for various reasons). If she founded a company with $1B profits a year, it would be a different story.
A simple abstraction here: we put two high-quality monitors before our eyes and get all the data from them. Can we continue driving this way? Absolutely. This means that no matter how our eyes actually work, an array of RGB pixels is enough. I'd go further and say that a 64 x 64 grayscale camera at 10 fps is enough to drive a car. The real magic happens in the brain that reconstructs the 3D model and predicts what happens next.