kommissar

They use aws for some things. For example, https://aws.amazon.com/blogs/machine-learning/facebook-uses-...

It’s free this year, too. Might as well register if you’re interested.

No coincidence. It’s reinvent season. https://reinvent.awsevents.com/

We use Glue extensively, but we have a rule of thumb not to use any of the 'special sauce'. That means is using it purely for 'Spark as a service'

This is spot on IMO. I use Glue internally (opinions are my own) and still believe that the best course of action is Glue should only run managed Spark. We provide an empty Scala "script" that does nothing, and load a compiled JAR file with the Scala code that actually runs our job as a library and have Glue exec into that.

We can version the ETL in git, run local tests outside of the Glue data plane, prototype in the Spark shell, and much more.

Click though to https://folding.extremeoverclocking.com/team_summary.php?s=&...

I'm still not sure what this is. Who is ec2spot??

My guess is that maybe Amazon installed the Folding client on some part of ec2 capacity in an idle pool waiting to be launched?

I'm not sure. That would use quite a lot of power... and setting up networking and getting approval to do this would be a ton of work.

I'm not sure how you can push an update that takes multiple services down for 17 hours. Unbelievable.

Actually, a website already exists to track Amazon prices. They could have just searched for it:

http://camelcamelcamel.com/

Oh, cool. I knew the colon meant a Ruby symbol, but I did not know that you could use & to convert a Proc to block.

What does the &: do in Ruby?