HN user

tmarthal

204 karma

web analytics / math / data / java / python / awesome

los angeles, ca

Posts1
Comments129
View on HN

There are only two ways to change society's behavior: policy or technology. No use arguing individually: court cases are dealing with the policy aspect and technically there's zero recourse on information being disseminated/copied that is published online.

^ those are the job reqs for the central recruiting across the company. Thanks 'CFShopify' for posting. =)

I'm a Dev Manager in the Shopify Fulfillment Network. If anyone is interested in talking about the culture or what working at Shopify is like, feel free to hit me up on twitter @tmarthal or email tom.marthaler@shopify

seem to have hit one of these areas of toxicity yourself

Have you considered that the original poster's experience is actually the norm and that your experience is the one that is the anomaly? I was 1 for 2 in organizations with shitty leadership, and the organization that was run properly had zero open headcount. Everywhere people are hiring into is not one of the "good ones".

Check out the old-fart tool, 85% of the company has been at Amazon for 3 years or less. Do you think that if the normal/average organization/team was a great place to be, that there would be so much attrition?

collision probability of 1.3 percent, with the two satellites coming as close as 190 feet (50m)

LEO orbits have speeds around 7.8 km/s (rounding up to ~8000m/s for quick calculations) - this avoidance detection is saying that the two satellites both traveling at 8000m/s would be in the same 50m box at the same second.

A quick calculation shows that the collision avoidance is operating at least the millisecond level to predict this collision (50m/(8000m/s) ~.005 seconds.

One thing someone once mentioned to me is that space is big and things travel fast. It's hard to believe that the two satellites (most likely each <1m in diameter) came "close" to colliding, when a half second later they would be 8000 meters apart.

By that logic, surely every interview practice is justified?

In a sense, they are. As long as the interview practice is well understood by the applicants, it only leads to filtering the applicants that cannot pursue the requirements for that practice. Granted, the FAANG interview process and filtering only works because the SWE jobs are desirable enough.

I understand that the required practices and filtering also lead to a certain type of applicant. This is the main fault of the system (the lack of diversity potential).

The median tenure of engineers at Amazon is 1 year. That means that the new engineers need senior and principle engineers to guide them on what tools exist. If an engineering organization happens to have strong senior Amazon engineers then they can guide their teams/org to use the tools that exist, because they do exist.

However, everything (and I mean everything) at Amazon depends on the team (and organization) that you land in. Some organizations do not have senior technical leadership; service ownership is handed off to teams without long tenured Amazon engineers so they do not get exposed to the types of tools to use (nor do these teams get time to discover, learn, and on-board to the tools that do exist). This is how an engineer can have the experience written about in the article.

The article is anecdotal, and definitely not the norm for the "majority of engineers"

Further down the article she mentions that she solved the dependency by "I ended up rewriting the Lambda in Java", noting that the use-case doesn't care about the known JVM warmup times.

The POM packaging and jar based deployment seemed to make the dependencies work.

I think what you're actually seeing is the output of a Kalman Filter temporal model. The zestimate is the mean of the filter output, but the error bounds of the filter are more important than the actual mean/reported value.

What happens is that the model uses trends to extrapolate from each "real" data point (in this case, a house sale in your neighborhood). The problem is, and what Kalman Filters help manage, is the uncertainty propagation between each house sale. When it has been a long time from when a sale has occurred, it is unclear what the real/actual price is of a home. This means that on the estimated price the error bounds are large, and zestimate still just reports the mean value of this huge uncertainty.

What then happens is that a house is sold in your area, a new data point is recorded, and the filter re-adjusts itself and collapses its uncertainty/error bounds in the time of that measurement around the measurement. And you get correction. This is why the "old zestimate" is updated.

Most likely because if you are doing ML and do not have a PhD (or previous experience), you are just looking at calling a library function that you do not understand. The majority of 'machine learning meetups' (not in the Bay Area), are attended by programmers that are looking to figure out how to call an R package to give them recommendations or similar items in a list (clustering).

edit I just read the other replies to this post. I believe that most startups with Machine Learning teams are doing more than just calling R-libraries; most development work that I've done for myself and teams has been for tooling and operationalizing data infrastructure (i.e. data engineering, not data science). However if you need a simple recommendation for an app then calling the library methods without 100% understanding may be enough (but calling library methods without underlying understanding is a bad trait in a programmer (e.g. calling the sort() function without understanding quicksort)).

There are a bunch of 'fork-n-go' github jekyl sample applications that wrap a webpage with a Google Spreadsheet datasource.

Here is a sample (from a github employee) that I've used in the past. Source: https://github.com/jlord/hack-spots and live site: jlord.us/hack-spots/#info - not sure how they are caching the spreadsheet, but I've never seen it not perform (not sure about concurrent users or whatever though).

This is a different solution, and less manual - but OralB has a bluetooth enabled toothbrush that will auto-record your tooth brushing in their app (as well as metrics like 'time used too much pressure' and 'total brush time'). You need to manually record flossing and tongue brushing via the app though (using a button on your smartphone like others have described). It's not bad, it's not good, but it's a start.

Right. And when someone is searching for facts, Google is using their dominant position to just display the facts, instead of just a link to a web site that has the facts.

The problem is the definition of what are 'facts'. Are birthdates? Are consumer good prices? Are classified postings? Airline flight times?

That Feist v Rural case is interesting. The Feist Corp were not able to copyright telephone listings. I wonder if the telephone company itself could have laid claim to their copyright though.

Can you list the reasons that you're thinking of not storing data on Github? I would assume that you have a (paid) user or organization account and keep the repo private, unless it was for a published paper.

I was thinking more along the lines of data that you'd want to graph or preset for analysis, as in a notebook or a paper. I would (hopefully) assume that your simulation data would be generated and used in such a way that it wouldn't need to be stored permanently. At least, when I was doing simulation based analysis, I wasn't necessarily concerned about any individual run, but rather a combination of a bunch of simulations (all of which were ephemeral).

Are you saying that people aren't storing data on github? How large of datasets are you talking about?

It's a shame that Github's large file storage (LFS) mechanism isn't publicly released yet, because I am sure that will appease your problems. (This is assuming your datasets don't fit into a csv (note that the current hard limit for github files is 100MB, and a 100MB CSV file is a lot of data)).

I am actually working at a startup to do just this, but using the 'Choosing Wisely' guidelines for care practitioners.

Right. The one thing that (tangentially related) is the quote "The prospect of substantially increased financial risk would discourage universities and other patent holders lacking extensive litigation resources from legitimately defending their patents".

Do universities really see themselves as the ones "lacking extensive litigation resources"? I think they don't understand the nature of 'small inventors'.

Whenever you say "that made no sense", I think that you are using too much bias and not giving enough credit to what the data is telling you.

If you look at the most "controversial" data science paper from 2013 where a study correlated intelligence to Liking the Facebook pages "Curly Fries" and "Thunderstorms" (here is a summary: http://www.wired.com/2013/03/facebook-like-research/), there were a lot of proponents saying that there was no causation, and the correlation was not founded, etc.

Of course, you would say the study "makes no sense". Intelligence can't be predicted by Facebook Likes. There is no correlation there, etc. But why not? If you read the paper (http://www.pnas.org/content/110/15/5802.full.pdf) their logic is sound. Is the marketing campaigns that the company bought based on the TV Stand<>DVD Player connection any different than other marketing campaigns? Facebook does all of their ad display based on similar data analysis as above, and it seems to be working for them.

Note: There is the not-so-hidden machine learning feedback loop now (explained better here: http://www.john-foreman.com/blog/the-perilous-world-of-machi...), where people Like the 'Curly Fries' and 'Thunderstorms' pages because of the research.

Right. Using pandas may allow you to process more data than an excel spreadsheet, but if the file is on your laptop then it's inherently not very big. It's a great tool, but it is a data exploration tool, not a data processing workflow tool.

IMHO, there are too many 'data scientists' nowadays that are taking averages and calling it 'analytics'. If the call you are making exists in a library, more than likely it's not "sophisticated".

The other poster is right. That is typing, that is not static type checking.

You can write code like:

List<MyObject> mylist = new HashSet<Integer>() {{ add("string"); add(1L); }}

You will get groovy.lang.GroovyRuntimeException with constructor issues.

I don't think it is the culture of 'speed' necessarily, but rather one of 'feedback loops'. With javascript in the browser you can see and experience code-reload after an F5†. There is no compilation time; no real detriment to importing another non-used library if it's not compiled into your code.

Opposed to using a JVM language, where everything is statically compiled into a single container, including another library (or upversioning one) will impact compilation times and oftentimes trigger a complete re-build, often taking minutes before you can see the output of your code.

Minutes versus seconds when dealing with code correctness really helps, and is what is needed most when trying to 'hack it together'.

† Note: This is the same with test suites: karma tests are often instantaneous and spock/junit tests sometimes take 30-40 seconds before they start to run, depending on environment.

There are only a handful of Yann LeCuns in the world. You can't just duplicate certain technical expertise.

This isn't about developer X using framework Y; this is about research level PhDs (and equivalent) that cannot continue work for a US-based company because of visa issues.

This is almost akin to a celebrity getting upset and punching a paparazzi. It happens due to stupidity, human emotion and rash behavior. Is it right? No, but it does shed light on some issues.

Reporters and journalists can and do scrutinize people. However, what has become of journalism is that for sites like Pando Daily/Tech Crunch/? etc. they are gossip magazines about Silicon Valley business people and software developers. Not only are they talking about the companies, they are talking about the personal lives of the people that make up the companies.

Given the work/life balance of most startups and successful companies, it is tough to judge the differentiation between a tech company's executives. There are reasonable legal precedents being set for common tort law with celebrities, however when do the executives of a company become a 'Public Figure' and lose their right to privacy? Why is it okay to follow someone and publish everything they say and do?

The same thing, of course, can be said for journalists. The lines are being blurred all around.