HN user

smuser

28 karma
Posts0
Comments9
View on HN
No posts found.

My understanding (not an expert) is a lot of problem domains have very sparse / infrequent rewards - imagine if the only reward you gave a minecraft agent was when it mined a diamond, it would take a lot of gameplay for it to randomly do that and get a reward. So researchers spend time tuning the reward space (oh you mined some dirt, here's a tiny reward. Oh you mined rock, a greater reward, etc) but it's kind of akin to hand crafted feature detection from the pre-neural network days. The Q* mystery is did OpenAI 'solve' reward modelling the same way neural networks solved feature detection.

Yupp relevant bit from there:

Internet traffic via a geostationary satellite has a minimum theoretical round-trip latency of at least 477 ms (between user and ground gateway), but in practice, current satellites have latencies of 600 ms or more. Starlink satellites would orbit at 1⁄30 to 1⁄105 of the height of geostationary orbits, and thus offer more practical Earth-to-sat latencies of around 25 to 35 ms, comparable to existing cable and fiber networks[51] (although transmitting a signal halfway around the globe takes at least 67 ms at the speed of light).