oneshot908

And even as an employee, if you tell them over and over again that customers keep asking for that feature, they disagree with you over and over again.

Time is the only nonrenewable resource. And you just lost some reading this response. Act accordingly.

I know of a team at an AI-oriented company that uses Reinforcement Learning to find cheats and exploits in 2600 games as their happy hour activity. No need to work late (though one could), just start that training run and hope for the best.

Not remotely true. TPUs and GPUs are neck in neck with each other right now w/r to overall efficiency, check out https://mlperf.org/press#mlperf-training-v0.6-results for more details.

GPU advantage: more refined ecosystem and you can buy them for $<1000 or get laptops with them built in, and if NVDA has sweat more software engineering blood and tears than GOOG into your model's functions, it will run better on them

TPU advantage: Colab has a free tier that lets you play with them at no charge and if GOOG has sweat more software engineering blood and tears into your model's functions, it will run better on them.

All IMO of course. And deep down it can get more complicated than that, but I salute GOOG for being the first company to ship competitive AI HW, doubly so at scale.

Also agreed, when hedge funds don't silo their quants, instead of seeing 50 different strategies from 50 quants, they get 50 variants of the same strategy, source:

https://www.amazon.com/gp/product/1119482089/ref=ppx_yo_dt_b...

You might want to look into the story of Harris Fogel. He was a tenured professor who was unceremoniously terminated for seemingly correctable or even unintentional behavior. He's suing.

https://www.insidehighered.com/news/2019/04/04/former-photog...

Stallman OTOH didn't even have tenure. The bigger story for me is if repeated complaints of sexual harassment (leg grabbing etc) went nowhere until now. That's unacceptable and I suddenly side with the Twitter mob in that case because if so he's had his due process already. And in that case, not only is he an ass, but he probably drove a lot of women out of the field and that can't be undone.

I think your understanding of the tech industry is rather unlike mine. I have been a manager previously and there is quite a bit of documentation and process involved in terminating anyone, so much so that a lot of really bad apples can jump teams without getting terminated if they time it well.

Even when they're caught, they get put on a Performance Improvement Plan (PIP) which is shorthand for giving them 60 days to find a new job or to turn their life around internally. Usually, it leads to the former, sometimes the latter. I've seen both.

It's an imperfect and biased process. But the attempt is usually made because HR fears unjustified termination lawsuits despite the "at will" employee status of just about everyone.

Unless they've tried to hack the company's servers for private data, I've never seen anyone fired on the spot without the above process unfolding. Maybe your experience differs?

PS I also think Charles Manson and The Unabomber were unambiguously guilty. That doesn't change my opinion that they deserved the trial they got.

PPS If as amyjess seemingly suggests that female professors at MIT repeatedly filed complaints against him and nothing happened, well then carry on Twitter mob, good job, seriously.

That business card is wildly inappropriate I agree and I love the bit about the plants. Is there a story I'm missing here where complaints were filed and nothing was done? That would change my viewpoint 180 degrees here if so. Because that means the guy's behavior was repeatedly and officially pointed out to him and he IDGAFed the advice. It also seems like an even bigger story than Stallman himself on par with GOOG's behavior the past 5 years.

PS The Kelsey Merkley talk is fantastic: https://www.youtube.com/watch?v=1Y0FuH5FNCo

That's one of the purposes of HR. Watch any corporate harassment training video if you don't believe me.

TLDR: the accusation of harassment is 100% determined by the accuser. The determination of whether harassment occurred OTOH is decided by HR after judging the merits (or lack thereof) of the case.

I see no reason why MIT shouldn't have proceeded similarly. And while you might argue that's not 100% impartial, that's a lot better than a Twitter mob (to me at least).

Given the piles of video and text of Stallman being Stallman, and the "Hot Ladies" bit on his office door, do you really think they would have high-fived his conduct and told him to carry on? I'm cynical, but I'm not that cynical.

Or let's put this another way. The Unabomber and Charles Manson got their due process. Are you saying Stallman is worse than both of them? So I'm guessing you guys downvoting me no longer believe in our legal system? That'll end well I'm sure.

You assemble the evidence before as impartial a committee as you can, and you let them make the decision. I think in this case Stallman is an offensive personality who says offensive things on company time using company equipment. That's going to be a no-brainer.

That said, I have a nagging worry from watching some videos of his behavior that he is mentally ill and there might be a backlash from that.

But why we can't allow a process like that to transpire before passing judgment is beyond me. Why wishing such an impartial judgment upon him is downvote worthy is really worrisome to me. That's not what western democracies are about as I understood them up to now.

I Just do not believe we should make career-ending decisions like this based on the rage of a mob on social media, that's literally a Black Mirror episode (and a really bad episode of The Orville as well). I believe their role is to raise awareness of situations like to the point where the above should transpire. Does holding that viewpoint now make me subject to "cancellation" as well?

Without "due process" Trial by Twitter(tm) is going to end like the Reign of Terror at the end of the French Revolution.

Stallman had said more than enough to merit his termination IMO, but he still deserved due process before such judgment was passed.

The California Assembly recently passed AB-1482 which will lead to statewide rent control if signed by the governor. I personally believe more in increasing the supply than attempting to control the existing supply, but no matter what, signing this without demanding SB-50-like concessions to make it easier to increase supply near transit was a real miss IMO. SB-50 got pushed into 2020.

https://leginfo.legislature.ca.gov/faces/billTextClient.xhtm...

https://sf.curbed.com/2019/5/16/18617019/transit-housing-bil...

That discussion was reasonably settled a very long time ago and court cases established reasonable laws around it. Then one side demanded a do-over, in much the same way they are seeking do-overs for a lot of previously seemingly settled issues.

I quite enjoyed the cognitive dissonance of the PETA director being a Type 1 Diabetic.

https://www.humanewatch.org/person/mary_beth_sweetland/

That's what robot lovers are for silly goose!

Yes, you got it. I made up the "Dark Tower" remark because they sat the Google+ team on the top floor of the only semi-highrise on the main Mountain View campus. This separated them both literally and figuratively from the rest of the Googlers and in 2011, that was just not "googly." And yes, this was a Vic Gundotra move.

I love a good skunkworks project. But Google+ needed Google to succeed. Apparently Vic felt otherwise and the rest is history, no?

I was an IC6 at Google when Google+ launched...

From my perspective, I think partitioning the Google+ team into their own Dark Tower with their own super-healthy cafeteria that was for them and their executives alone was the biggest problem. IMO this even foreshadows separating off Google Brain from the rest of Google and giving them resources not available to anyone else. Google was at its best a relatively open culture and 2011 is the year they killed other cultural icons such as Google Labs and (unofficially) deprecated 20% time. I think the road to the Google we see today started then. It's also the year they paid too much for Motorola and started pushing Marissa Mayer out the door.

Then there was the changing story of the 2011 bonus. When I hired in, we were all told our 2011 bonus would be tied to the success of Google+. That's a fantastic way to rally your co-workers, except... Once they launched Google+, the Google+ Eliterati (so to speak) changed their minds and announced that any Google+ bonus was for Google+ people alone. Maximum emotionally intelligent genius IMO. Now your own co-workers have been burned. Also not very "googly."

Finally, there was "Real Names." The week of its launch everyone I knew wanted an invite and I used up every single one of them and continued to do so as more were made available to me. Then "Real Names" happened and people stopped asking for invites overnight. That's the moment for me when the tide turned against this thing.

I really liked the initial Google+ UI personally, but the UI ran head-on into the nonsensical "Kennedy" initiative wherein some brilliant designer seemed to decide that since monitors are now twice the size they used to be, they should add twice the whitespace to show the same amount of information as on a much smaller screen. Subversives within the company took to posting nearly blank sheets of printer paper on walls with the single word "Kennedy" in a tiny font you'd only see if you got close to the things. That said, my godawful company man manager would repeatedly proclaim how beautiful he thought the Kennedy layout was in our office for all to hear whenever they updated GMail or Search to use it.

Of course, there are other reasons beyond my tiny perspective here, but I did have a front row seat for this and it was really disappointing to see a potential Facebook killer die of a thousand papercuts like this.

Imagine a world where low information sorts interpret a sampling of possible hi-res reconstructions from low-res security videos as ground truth. That to me is far scarier than the OpenAI and MIRI fear-mongering about GPT-2.

I relocated out of the bay area in 2016-2017 and it was the happiest year of my adult life. I think Silicon Valley has a zero sum culture and having returned here for personal reasons but otherwise against my best judgment, I cannot wait to GTFO for keeps.

I am really really good (perhaps top 10 or so worldwide) at one thing, and pretty good at a couple other things. If I didn't have that, I think I would feel utterly worthless in 2019. If I extrapolate to most of America, wow, I get why they elected who they elected.

Something has gone very wrong here. And I am stymied as to how to fix it because neither political party, which gets to set the agenda every 4 years, seems to grasp what has gone wrong.

But when I travel abroad, I am happy. I see can-do cultures that have far less than western nations, but are so inspiringly optimistic to do more, that I don't want to come back to America. I want to set down roots and help them knock it out of the park. And I am getting close to doing exactly that. Doing so would shatter my personal life, but I suspect if 2020 continues what we started in 2016, I will follow through on what my inner voice is telling me to do.

On the contrary, I think this is one of the biggest emerging blockades to progress in ML/AI research, especially in academia. It has always been more cost-effective to run ML algorithms on consumer HW such as GeForce GPUs and gaming CPUs. It's frequently even faster than contemporary cloud offerings when the consumer HW gets ahead of existing enterprise HW. And it's so effective that HW companies starting changing their EULAs and crippling previously available aspects of APIs to herd AI back into the datacenter where they seem to think it belongs.

And that IMO is a reinvention of the "Walled Garden" of academic HPC (ask any grad student begging and pleading for supercomputer time) which has always sucked and its new commercial incarnation is even worse because it's unclear how to get commercial cloud time on government grants.

OTOH it's fine for large shops like OpenAI, DeepMind, AWS AI, FAIR, MS Research etc because they have deep deep pockets. So if you're content with most future groundbreaking research coming from a small tribe of market leaders, well great, but I suspect innovation is already slowing down because of this.

In my experience so far, "Amazon's Choice" is rarely what I'd choose. So I'd guess by now that its value (probably calculated using some sort of ML model trained on sales data) has been gamed to death by motivated substandard sellers (so it probably needs a refresh like Google refreshes its scoring function for web pages in response to SEO).

I've also noticed that sponsored products (which I don't want ever) are better targeted towards my queries by EBay. The difference seems to be that Amazon deletes keywords until it has something both sponsored and irrelevant to show and hopes for the best. That must have survived A/B testing, yuck.

What I think it comes down to is I'm an oddball that wants organic search results. I understand why Google messes with that to deliver ads, but for the life of me, I'm trying to help Amazon take my money and they're getting in my way and my larger purchases (>$100) are migrating away now because it's becoming impossible to find what I want, occasionally worked around by searching the Amazon catalog, ironically, from Google.

Who's going to connect this to CleverBot and let the hilarity ensue?

How do sponsored products now topping organic search results, comingled fulfillment centers mixing real with counterfeit items, and dropping keywords to add inaccurate results to spearfishing queries serve to put the customer first, especially those who paid for prime membership? Shopping on Amazon used to be nearly effortless, now I have to make sure I'm really getting what I searched for, and my return rate in 2018 went from close to 0% to about 10%.

Sure, that's a nice approximation, and it will reduce performance somewhat IMO, it'd be nice to quantify how much, no?

The failure rate of GeForce is higher than Tesla (though it's not that much worse in my experience these days). OEMs worked around this by burning the GPUs in for a couple days running HPC or DL code before sending them out to customers and then they binned the failed GPUs into gaming machines where bitwise reproducibility didn't matter as much. Also, it's usually the memory controller that's defective on 1080TIs so it's pretty easy to spot early.

NVIDIA was not amused.

Indeed, off-topic, but I find it fascinating that I can buy a year of Kosher, Vegan, Gluten Free Survival Food on Amazon from ~$1469.

https://www.amazon.com/NorthWest-Fork-Gluten-Free-Emergency-...

That drops to $1149 if I don't require Vegan, non-GMO, Kosher:

https://www.amazon.com/Healthy-Survival-Food-Emergency-Prepa...

SNAP benefits at anywhere from $134 to $192 a month are both more than sufficient for this. So why are ~42 million people starving in America? Answer (IMO): Cloud providers reduce the friction of GPU adoption the same way grocery and convenience stores reduce the friction of food acquisition, both in exchange for profiting off of it.

What are those performance limitations, really?

Memory? Because if you can spread your model across multiple GPUs, and you've implemented Krizhevsky's One Weird Trick to switch between reducing the smallest of either parameters or deltas, you're golden.

I thought tensor cores and NVLINK would end up Tesla differentiators, and really great ones at that, but now they're both in the Turing consumer GPUs so I am really scratching my head here.

That said, the EULA is just stupid. I cannot use CUDA 9.2 or later at work because of it. No one is going to audit our computers for any reason ever, period, full stop.

It's been this way since day 1. NVLINK remains the only real Tesla differentiator (although mini NVLINK is available on the new Turing consumer GPUs so WTFever). But because none of the DL frameworks support intra-layer model parallelism, all of the networks we see tend to run efficiently in data parallel because doing anything else makes them communication-limited, which they aren't because data scientists end up building networks that aren't, chicken and the egg style.

I continue to be boggled that Alex Krizhevsky's One Weird Trick never made it to TensorFlow or anywhere else:

https://arxiv.org/abs/1404.5997

I also suspect that's why so many thought leaders consider ImageNet to be solved, when what's really solved is ImageNet-1K. That leaves ~21K more outputs on the softmax of the output layer for ImageNet-22K, which to my knowledge, is still not solved. A 22,000-wide output sourced by a 4096-wide embedding is 90K+ parameters (which is almost 4x as many parameters in the entire ResNet-50 network).

All that said, while it will always be cheaper to buy your ML peeps $10K quad-GPU workstations and upgrade their consumer GPUs whenever a brand new shiny becomes available, be aware NVIDIA is very passive aggressive about this following some strange magical thinking that this is OK for academics, but not OK for business. My own biased take is it's the right solution for anyone doing research, and the cloud is the right solution for scaling it up for production. Silly me.

In 2007, I had an architect design the outside both for aesthetics and to smooth negotiating with the Santa Cruz mafia planning department, but his floor plans were abysmal, and I would have hated life, stubbing my toes daily.

So I took over that task, drove my girlfriend at the time absolutely nuts iterating over it, then one day showed her something she said was bleeping perfect. We built that plan relying on a neighbor who was a general contractor at the top of his game (early 40s). I still live in that house, but the girlfriend and I broke up mid-project, oh well.

General observations:

1. The town will extract maximum value from you for the tiniest changes so get as much as you can in the 1st draft (e.g.: another $5000 to authorize making windows openable at floor level vs not)

2. Get a contractor you trust, things can go non-linear if you don't, and even if you do, you'll get some flakes. Do not be nice to flakes. Flakes suck.

3. Rent a nice place elsewhere during the process. If you don't have the $$$ to do this, you probably shouldn't be doing this at all. It will break you. The movie "The Money Pit" is IMO mostly documentary and only part comedy.

4. I have 10 Gb/s Cat 6 hard-wired Ethernet in my walls. Local contractors didn't even know that was possible in 2007. Do your research. This serves me well when I train DL models in my house.

The next project for me is a solar power system to power all my DL servers. Since they each eat ~1.5 KW, I'm going to need 10 kW overall for all 4 servers plus household requirements (but don't call it a datacenter or NVDA will audit you, call it "A House of Ill Compute"). To that end, I have nicknamed my home as the house of 200 TFLOPs (16 Pascal GPUs across 4 servers). It's about to become the house of 2 PFLOPS (RTX 2080TI GPU upgrade pending).

1. Yes 2. No, what are you smoking? I want some.

A "viable" Holodeck is a technology sufficiently advanced compared to the present day so as to be indistinguishable from magic. Ergo, no Holodeck anytime soon. No Arnold Judas Rimmer hard light holograms either because WTF is a hard light hologram in the first place?