HN user

ajones

86 karma

A software engineer in NYC. Focus on infrastructure, data, and backend engineering.

http://atjones.co

Posts16
Comments27
View on HN

It isn't an error. If you click into one of the individual data sources (links available on the home page), you see messages like this:

This page is being updated. It will post records of White House visitors on an ongoing basis, once they become available.

This page is being updated. It will post records of White House staff salaries as the data becomes available for the Trump Administration.

LearnVest | New York (NYC), NY | ONSITE

Infrastructure Engineer - http://app.jobvite.com/m?3sglUhwJ

We are currently a team of five that focuses on the construction, maintenance, and automation of LearnVest's web operations. We are expanding the team in order to have more bandwidth for the exciting projects that we have in our pipeline. We spend a lot of time working with Docker, Kubernetes, Jenkins, Python, and AWS.

Some problems that we are trying to solve include:

* Migrating our monolith to a microservices model

* Leveraging our centralized logging infrastructure to quickly discover anomalies from deployments or environment changes

* Creating Ansible plays that help us achieve an immutable infrastructure

-

If you have any questions, please reach out to me at ajones(at)learnvest(dot)com

LearnVest | New York, NY | ONSITE

Software Engineer - Developer Infrastructure (http://app.jobvite.com/m?3O1EThw8): I'm building out a team that is focused on internal tools and our build & deploy systems. This team sits inside of our infrastructure group and is building tools that are used across our rapidly growing organization. Our ideal candidate has experience with continuous integration, Python, and Ansible.

Infrastructure Engineer (http://app.jobvite.com/m?3sglUhwJ): My group is also adding an infrastructure engineer to help us build the next generation of our systems. If you're in NYC and passionate about building highly available and scalable systems, let's talk! We're using Kubernetes, AWS, and Ansible to push LearnVest to the next level.

Depending on the technical abilities of the person using the solution, I would recommend one of the below two systems:

- Chartio: A good tool for someone who isn't very technical to discover insights. Drag-and-drop interface.

- Periscope: A good tool for someone who is able to write SQL to build visualizations from their queries.

You inflated numbers from your initial calculations, but I don't believe you inflated them enough. How do you expect a team of 20 to recreate an entire company in one year? Here's some things that I believe you missed:

- You need a team of product and project managers to put in time to plan the product and a high-level road map. The more granular product decisions are able to be performed in parallel to the build, if building the product in an agile fashion.

- Etsy has an amazing engineering culture that cannot be constructed if the developers are just focused on delivering the product. This culture has lead to many internal tools that help with site reliability.

- Etsy has already built a team of strong contributors. If Amazon were to build their own Etsy-like marketplace, there would be recruiting costs and false positives which will slow down delivery.

This list could go on and on, but is meant to illustrate that there's more to this sort of project than engineering, marketing, and infrastructure.

The tech and talent at Etsy would bring quite a bit of value to any company that were to acquire them. Their tech culture is top notch. I've followed their blog (https://codeascraft.com/) for a couple years now and have gained a ton of knowledge from it.

Some selected blog posts:

- Leveling Up Your Organization with System Reviews (Their latest post. https://codeascraft.com/2015/12/21/leveling-up-with-system-r...)

- Fault Injection in Production (http://queue.acm.org/detail.cfm?id=2353017)

- Q3 2015 Site Performance Report (Posted every quarter. https://codeascraft.com/2015/11/10/q3-2015-site-performance-...)

TLDR pages 11 years ago

TL;DR a collection of TL;DRs for common *nix commands with clients to display the shorter docs

The American crocodile is mostly found in Central America and the Caribbean, but also exists in the Everglades. I'm pretty sure it is an endangered species.

Stammy's articles are always a great read. As a developer, I appreciate his perspectives on the intersection of programming and design. I recommend his three-part post on designing and developing a responsive, retina-friendly site.

http://paulstamatiou.com/responsive-retina-blog-design/ http://paulstamatiou.com/responsive-retina-blog-development-... http://paulstamatiou.com/responsive-retina-blog-development-...

By this sort of thing do you mean using it as a data warehouse? That's what Redshift is branded as and the performance gains are definitely similar to what the blog outlines.

In my opinion, Redshift is the best data warehouse solution for a team building a small to medium-sized warehouse. This covers most use cases. For those building a data warehouse above a petabyte in size, you're going to have to look at different solutions. Redshift is powerful and relatively cheap compared to its competitors.

Thanks for the response. I agree with your thoughts on hiring. I'm one of those guys who came in and learned Scala. The initial reason for my team using Scala was, "Why not?" Since that time, my colleague who hired me has left the company and I have been the lone engineer on the team.

We're currently looking to hire a second engineer and have also been rethinking the direction of the data engineering team. Changing the language to Python was a suggestion for the hiring factor and because it is the popular language for data engineering. Scala has been a fun language over the last year and a half and I wouldn't mind sticking with it because it has been fun. For that reason, I have spent time comparing the two languages for data engineering and haven't come to a consensus on which one is ideal for my team.

Maybe it's just me, but it seems that several data engineering organizations are picking up Scala. Is there a reason for this outside of Slide 13 in the presentation that I am missing?

My team has plans of moving away from Scala and towards Python primarily because of the job market. There is a feeling that it will be significantly easier to find a good data engineer who uses Python than it will be to find the same who uses Scala.

I'm not very familiar with the DC market, but I did a quick search on AngelList to write this comment. Here's a few company's that jumped out to me:

- FiscalNote: Real-time open data analysis - Contactually: CRM - Vero Analytics: Analytics platform

I know that LivingSocial is based in DC. I believe they may be the biggest startup in the city (size and popularity).

When I left college, I joined a big bank and eventually made the jump to a startup. Here's why:

- I spent my first of two years as an applications developer and didn't push more than 20 lines of code into production. My days were wasted with planning meetings and release management. I spent more time in excel spreadsheets than anything else.

- In my second year, I worked on a new product. Things seemed much more interesting, but something was still missing. I cover a lot of that in my blog post about my transition from a big bank to a startup. https://atjonesblog.wordpress.com/2014/03/27/my-transition-t...

- In the article, it is mentioned that Goldman works on problems in machine learning, data mining, and cloud computing. I'm sure this is 100% accurate, but what they fail to mention is these interesting projects are usually strictly for PhDs or engineers with decades of experience.

- As a technologist in a bank, you're a second-class citizen. The executives may say things like, "we're not a bank, we're a tech company!", but I saw no actions to back up the statements. Majority of the recognition goes to the bankers, ignoring the infrastructure laid by the tech and operations side of the business.

After all that, I'd say I had a decent experience, but I don't think I would have missed much by jumping straight into a tech company.

I only got through one class, but have heard my concerns echoed by many who are only continuing the program because the have invested so much into it.

The program is based on weekly discussion board posts and a few exams per course. For the most part, I find myself attempting to put fluff into my posts rather than meaningful analysis.

There are "sync sessions" throughout the courses, but they don't occur every week. This is much unlike a MOOC, which provides a set of weekly lectures to view and gain understanding.

There was zero project work in my initial class and projects at or below the level of undergrad projects in more advanced courses.

I managed an A average while spending the absolute minimum amount of time on the class.

When paying 4,000 dollars per course, I had hoped that the instruction would be at the same level or better than that of a set of MOOCs. In the field of data science, I believe the time can easily be directed towards side projects for greater knowledge and similar employment prospects.

I spoke with Berkeley when deciding where I was going to apply and got the sense that they plan on leaning on their more established programs to provide instruction. While this may sound great, you won't get the specific data science instruction that these curriculums require.

After an awful experience with Northwestern's online analytics program, I don't want to touch any of these programs.

I agree with the statement about Astoria. I live next to the neighborhood on Roosevelt Island and spend many Saturdays/Sundays walking the neighborhood because of the great food and people. The NQ line that runs through Astoria will take you to the heart of Silicon Alley.

I also recommend checking out East Harlem. There isn't a ton going on up there, but rent is cheap and you can hop on the 4/5 and be at Union Square in ~20 minutes.

[dead] 12 years ago

Agreed. I load a few GBs of data into memory for analysis without any issue. This is a huge difference from my old 2GB MBP.