opprobium

Encoder in the T5 sense doesn't produce a fixed vector, it produces one encoded vector for every step of input and all of that is given to the decoder.

The only difference between encoder/decoder and decoder-only is masking:

In an encoder, none of the tokens are masked at any step, and are all visible in both directions to the encoder. Each output of the encoder can attend to any input of the encoder.

In the decoder, the tokens are masked causally - each N+1 token can only attend to the previous N tokens.

You are missing the speech decoding part. I can't speak to why the clients you were working on were doing what they were doing. For a different reference point see the cloud streaming api.

This is a good public reference: https://research.google/blog/an-all-neural-on-device-speech-...

Possibly confusions from that doc: "RNN-T" is entirely orthogonal to RNNs (and not the only streamable model). Attention is also orthogonal to streaming. A chunked or sliding window attention can stream, a bi-directional RNN cannot. How you think of an encoder and a decoder streaming is also different.

At a practical level, if a model is fast enough, and VAD is doing an adequate job, you can get something that looks like "streaming" which a non-streaming model. If a streaming model has tons of look-ahead or a very large input chunk size, its latency may not feel a lot better.

Where the difference is sharp is where VAD is not adequate: Users speak in continuous streams of audio, they leave in unusual gaps within sentences and run sentences together. A non-streaming system either hurts quality because sentences (or even words) get broken up that shouldn't, or has to wait forever and doesn't get a chance to run, when a streaming system would have already been producing output.

And to your points about echo cancellation and interference: There's many text only operations that benefit from being able to start early in the audio stream, not late.

I just went through process of helping someone stand up an interactive system with whisper etc and the lack of an open sourced whisper-quality streaming system is such a bummer because it really is so much laggier than it has to be.

This is exactly what Google ASR does. Give it a try and watch how the results flow back to you, it certainly is not waiting for VAD segment breaking. I should know.

Streaming used to be something people cared about more. VAD is always part of those systems as well, you want to use it to start segments and to hard cut-off, but it is just the starting off point. It's kind of a big gap (to me) that's missing in available models since Whisper came out, partly I think because it does add to the complexity of using the model, and latency has to be tuned/traded-off with quality.

It is not streaming in the way people normally use this term. It's a fuzzy notion but typically streaming means something encompassing:

- Processing and emitting results on something closer to word by word level - Allowing partial results while the user is still speaking and mid-segment - Not relying on an external segmenter to determine the chunking (and therefore also latency) of the output.

Streaming for TTS doesn't matter but for speech to text it is more meaningful in interactive cases. In that case the user's speech is arriving in real time and streaming can mean a couple levels of things:

- Overlap compute with the user speaking: Not having to wait until all the speech has been acquired can massively reduce latency at the end of speech and allow a larger model to be used. This doesn't have to be the whole system, for instance an encoder can run in this fashion along audio as it comes in even if the final step of the system then runs in a non-streaming fashion.

- Produce partial results while the user is speaking: This can be just a UI nice to have, but it can also be much deeper, eg, a system can be activating on words or phrases in the input before the user is finished speaking which can dramatically change latency.

- Better segmentation: Whisper + Silero is just using VAD to make segments for Whisper, this is not at all the best you can do if you are actually decoding while you go. Looking at the results as you go allow you to make much better and faster segmentation decisions.

Unfortunately it's not, this is some supplementary video. The full thing is 44 minutes long and covers a lot of different angles. The ongoing story about the Czech Xenon clinic in it that he covers over a longer time period is especially crazy.

Edit: Yes the full one is that second link.

It is in fact psychoactive and a good anesthetic:

"How does xenon produce anaesthesia?" https://www.nature.com/articles/24525

It is not typically used in most countries because it is very expensive compared to alternatives, but it is approved in many countries for anesthesia: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3626616

I found this article very weak, but Xenon is quite a real drug.

The Xenon episode of Hamilton's Pharmacopeia is one of the very best episodes of that series, I don't know the best way to find it on streaming currently, it was available I believe on Netflix (or maybe it was Hulu?): https://www.vicetv.com/en_us/video/xenon-the-perfect-anesthe...

The full episode takes very disturbing twists and turns, worth a watch.

You asked "why would someone hide prep", you got one example. It is by no means an exhaustive list. For example, a big fear is being perceived as having HIV (since prep drugs are also part of HIV treatment).

If you do actually have interest in this topic you could read about it:

https://www.aidsmap.com/news/feb-2020/your-partner-not-angel...

"Men were able to initiate PrEP without discussing it with their partners, whereas some women said they needed to get permission. Discussions around starting PrEP could raise questions about trust and infidelity and act as a barrier to PrEP use."

"the men are forced to confront the fact..." No they aren't, they simply never confront it. People go to their graves denying that they have HIV, denying that they ever tested positive, denying that a positive HIV test has anything to do with illness.

"Otherwise the men are setting themselves up for a lose:lose scenario regardless of what the wife does." - Of course.

I mean, the most rational win:win thing to do is to get an HIV test and get treated if positive. They then both don't get sick and die and can't pass along HIV. Many people don't do that either. What's the mindset that explains this behavior? You can't work backwards from the most rational thing to do to what people actually do.

[flagged]

Because they can get HIV from their husband, but also cannot discuss HIV, refuse sex, or take prevention steps for themselves without reprisal.

Perhaps I found the article clearer because of familiarity with the subject.

On the "retroviruses must be harmless" virology: He's a denier of viral involvement in cancers in general, not just that HIV must be harmless. He is way outside mainstream consensus on all kinds of things.

For instance, he argues that Kaposi sarcoma, a very common AIDS related cancer was caused by drug use and not opportunistic infection. It is now very well established that all KS, which also affects (typically older) HIV- people, is caused by HHV-8 infection.

He claims that Hep-B/C can't cause liver cancer.

He claims that HPV doesn't cause cervical cancer (https://www.academia.edu/31617237/What_if_HPV_does_NOT_cause...)

The core thing he does on all of these topics is just to ignore or deny anything that doesn't agree with him, eg: Hemophiliacs treated with tainted blood get AIDS, HIV viral load directly corresponds to disease progression which is clearly halted by dropping HIV load with treatment, the HPV vaccine demonstrably prevents cervical cancer, etc. He is far off in quack territory.

https://en.wikipedia.org/wiki/Peter_Duesberg#Consequences_of...

He was investigated but it was dropped as being protected by his academic freedom: https://www.science.org/content/article/berkeley-drops-probe...

He also, that I know of, still supports this position. To this day, you will find people getting into this particular conspiracy and rejecting treatment. It doesn't go well for them.

I do think that freedom of speech is important, and that many attempts to squash "misinformation" are misguided, but some speech has consequences. Personally I find Duesberg utterly reprehensible and morally culpable.

You can read a lot about it here: https://en.wikipedia.org/wiki/Duesberg_hypothesis

The mainstream consensus is the he was wildly wrong about HIV specifically, that HIV causes AIDS, and that his influence in South Africa to not deploy anti-viral medications killed hundreds of thousands of people before the policy was reversed.

Part of his hypothesis was that viruses in general, not just retroviruses, were not connected to cancers, the consensus view is that this is completely wrong. We have a very large body of evidence on many virus caused cancers now.

Even at the time he was arguing this, it was clear that the retrovirus HTLV was disease causing in humans https://en.wikipedia.org/wiki/Primate_T-lymphotropic_virus

So, the two known human retroviruses both cause disease and retroviruses cause diseases in animals. Duesberg held on to and promoted this concept long after it should have been clear to him that there was zero empirical support for his idea.

This injection isn't a vaccine, it's an anti-viral drug being used as pre-exposure prophylaxis. The first approval of this approach was in 2012, but using an oral pill with a short half-life taken daily.

That drug is still in use and also highly effective, the new improvement is to provide the same approach with a longer acting injected drug. One reason there has been great interest in this, despite the already effective oral PREP, is that there are thought to be socio-behavior advantages for cases like women in Africa as in this study. For example: the woman does not have to keep a supply of daily pills that a partner can find. Also possibly improved adherence with no missed doses.

The drug itself is not thought to be more biologically effective than the oral drugs, which are basically already at close to 100% effective assuming the patient actually takes them as scheduled.

South Africa (study was in SA and Uganda) has an adult HIV prevalence of 18.3% and 210k new infections per year. It is easy to select a high risk group in which you would expect to see new HIV infections during the course of the study without intervention.

Not just efficiently, can't solve.

No, not exactly. "the era of sending excess rooftop solar generation to the grid is over" is the OP's statement I disagree with. "They can't take any more solar input", yes they can, by adding more grid battery, which is currently happening.

If you look at that graph for a whole day you will see a huge kick-in of power company owned battery storage in the evening. There is no reason the grid cannot operate batteries at scale to absorb excess rooftop generation. NYT: https://archive.is/t1uPj

Recent example of a proof regarding theoretical limitations of Transformers: https://aclanthology.org/2023.tacl-1.31.pdf (also extended to cover SSMs https://arxiv.org/pdf/2404.08819)

Literally not being at home to eat home made food is obviously a case where other factors dominate.

However, I really don't think that's the general case people are thinking of when trying to argue that it makes some kind of economic sense for people to eat a fast food diet. The average person does go home at some point, especially so for families.

Or more specifically, it's the claim "I can't go to the grocery store and make this meal for less than a fast food meal", that is absurd.

Maybe what we have here with you and with many others in America is just a lack of food knowledge.

This is exactly what's happening. This is why I find it so upsetting to hear a source like NPR mindlessly repeating the myth that fast food is cheaper. People have gotten totally out of touch with basic home economics type stuff and the food industry is all too happy for them to stay in the dark.

Menu cost, I downloaded the app to look up the price, I was not offered a coupon.

But the exact same thing applies to the grocery prices I quoted. I gave the non-sale prices from my area's more expensive grocery store, that's what I consider the starting point, you can definitely do better.

When I say "cooking from scratch", I specifically mean the super fast and easy stuff. Starting from raw materials doesn't mean you do anything complicated to it.

For the burger example: buying pre-formed burger patties is still massively cheaper. Throwing a pre-formed burger patty from your fridge in a pan and putting it on a bun with a slice of cheese will take you ten minutes. Microwave small potatoes while you fry. You are done. There is no prep, you have made 1 easily washed pan and bowl for potatoes and your plate.

Is it the exact same thing taste-wise as your fast food meal? No, the potatoes aren't fried, sorry. Does it hit all the macro nutrients for far cheaper, and probably less time than even going to the fast food place? Yes.

I used for cost comparison a pre-formed hamburger patty from Safeway. If it takes you more than 10 minutes to pull this from fridge, heat a pan and fry, something is wrong. You put it on a bun and put things on it and you eat it. There is no prep area to clean. Wash a pan and your plate. This isn't even a scaling issue, this is negligible time and cleanup for anywhere 1 to 4 people. This is a real side by side comparison as a McD's quarter lb also has nothing on it requiring prep.

I understand what you are trying to say about "fully loaded cost". It's also wrong. The fully loaded cost is still much lower for home vs fast. Unless you insist that you really desire specifically something like deep fried french fries, a specific cooking method that is extremely scalable and well suited to restaurant production and very inconvenient at home. But it is emphatically not true that a meal of similar ingredients/macro nutrition (burger and potato) is in general ever cheaper in fast food form.

If you want to promote the myth that fast food is cheaper, you should cite any other source than that you vaguely remember there being articles 10-15 years ago.

You can just look at your own grocery store and your local McDs. I also live in CA and for a quarter lb with cheese comparison I looked up:

My local McD's: $6.39 My local Safeway (not a budget option, no sales, you can do better than all of this): 1 1/4 lb beef patty $1.69, 1 slice cheddar cheese $0.37, 1 hamburger buns $0.22 = $2.28, misc condiments are negligible but let's say $0.25 total = $2.53

That's less than half the cost. The time and resources cost of frying that patty in a skillet and throwing it on a bun with cheese and ketchup comes nowhere close to doubling that, it's not even close.

How long do you think it takes to grill a hamburger patty?

To your second point: This is where exact apples to apples comparison breaks down. The sane home cook skips deep frying at home and associated hassles unless it's a special occasion. Microwave the potatoes or boil. Fast, minimal cleanup, and now it isn't junk food either.

This simply isn't true and you should cite some source for this "known fact". It seems to be a "known fact" passed around by people who "don't know how much anything costs at a grocery store."

Example: A quarter lb with cheese at McD

Average price at a US McDonald's, $6.65: https://www.fastfoodmenuprices.com/how-much-mcdonalds-quarte...

Average price according to USDA for home cooked: $2.17 https://www.ers.usda.gov/data-products/chart-gallery/gallery...

Fast food is typically much more expensive than home cooked from scratch and people have very confused ideas about this.

2 lb of ground beef is way more beef than a happy meal. A kids meal patty is only 1/10 of a lb, you've given enough for 20 kids meals.

The rest of your numbers are similarly off: - you are giving each happy meal 1.25 lbs of potato!? - apple serving size is 1.2 ozs - An average apple is 8-10 ozs, 4 apples = 26 happy meals minimum.

You realize how expensive fast food is if you are at all used to cooking at home from scratch all the time.