blister

Hah, I literally just fought this for the past month. We run a large esports league that relies on player ranked data. They have the data, and as mentioned above, they send it down to the browser in beautiful JSON objects.

But they're sitting behind Cloudflare and aggressively blocking attempts to fetch data programmatically, which is a huge problem for us with 6000+ players worth of data to fetch multiple times every 3 months.

So... I built a Chrome Extension to grab the data at a speed that is usually under their detection rate. Basically created a distributed scraper and passed it out to as many people in the league as I could.

For big jobs when we want to do giant batches, it was a simple matter of doing the pulls and when we start getting 429 errors (rate limit blocking code they use), switch to a new IP on the VPN.

The only way they can block us now is if they stop having a website.

Give one of the commercial VPN providers a try. They're usually pretty cheap and have tons of IPs all over the place. Adding a "VPN Disconnect / Reconnect" step to the process only added about 10 seconds per request every so often.

Holy Moly! The author of this paper makes a very scary argument further shifting the burden of proof onto the employers.

The problem with hiring is that there is a hugely disproportionate supply/demand ratio. We might have 1 position to fill and 100+ applicants. How can any company correctly protect themselves against allegations of illegal hiring criteria in this type of environment? Right now the best protection is to say "we hired the most qualified candidate", but even that argument is shaky. What if the most "technically qualified" candidate performed the worst in the interview process?

Hiring is scary enough already without bringing the law into the picture. :/

I've noticed the same thing and have been actively working over the past 5 years trying to make introductory robotics approachable.

I've been a software developer for a long time, but robotics always felt like a mystery to me until I had someone show me the basics. I recently started a small company trying to teach the basics of robotics. The biggest challenge is that robotics have a real price attached that normal software development doesn't have associated.

With Javascript, I just need a text editor and a browser, but with Robotics I have to make the leap and purchase a kit and trust that the kit will do enough to teach me what I need to know to get started.

We designed and built a low-cost kit and have been using it in middle-schools throughout the southeast teaching kids the introduction, and so far that's been working really well.

Hah, I've been using Unix-based OSes in a professional capacity for over 15 years and I've never heard of apropos. :)

Unfortunately it seems like it's really hit or miss depending on which worker handles the order.

Some days I get exactly what I ask for really quickly, when other days I'll have my shopper switching 25% of my requested items with alternates (some of which are wildly off base).

My biggest frustration is that it seems like the shoppers aren't intimately familiar with my local grocery stores. I'll request an item that I know exists, they'll swap it out for something completely different and not what I want, yet if I personally go to the store, I'll easily find the item I requested.

One cool approach to a solution would be to use data to save the day. Millions of people fly through the carriers each year. The airlines could analyze the statistical heights of their average airline passenger and design the seats accordingly. Then assign seats on the flight based on passenger height. You could have a percentage of your seats configured for short people, a bigger percentage configured for average height, and several set up for tall outliers. Then everyone will be statistically more likely to get a comfortable experience.

Needs more Metallica midi auto-playing: https://www.youtube.com/watch?v=WL9qdhRkcbk

Twitter has caused serious harm to myself and my family. I'm a big fan of the platform, but I've been "doxxed", harassed, and was even SWATed once by different people trying to get me to give them my username.

I'd release it, except that'd be almost as big of a pain. I'd have to release a new edition of my book, change all my websites. Ugh. People are dicks. I wish I'd picked a less desirable handle.

We used a shop exactly like you described: http://www.e3embedded.com/

We used them to design and build a custom board for our Sumo robots that had our V1 built from an Arduino board and a custom board that plugged into the Arduino. We couldn't really afford to pay them a big chunk up front, but we negotiated a pretty pleasant royalty deal that comes out to a few dollars per board (which we get manufactured for less than $10/board).

They did excellent work and I highly recommend them for that type of "prototype -> production" path.

Doesn't this use case eventually screw over any future Twilio customer that selects that number? I see several inbound calls to my Twilio number from random numbers that I have to pay for, even though I'm only using Twilio in a development capacity right now. Do you guys "expire" numbers from your pool that receive aggressive amounts of Inbound traffic (voice, fax, or SMS)?

At some point a few years back, my VW dealership sold my phone number to people for vehicle service contracts. Even though I sold that car 4 years ago, I STILL get at least 2-3 calls per week from different companies offering to sell me an extended warranty. How does Twilio mitigate that for us?

(Off topic, but seriously, thanks for Twilio. Ping me if you want to know how we're using it in our new startup [email in profile]. It's pretty rad.)

Anecdote of one: I was diagnosed with Stage 4 Hodgkins Lymphoma two years ago. My _only_ symptom at the time was pretty severe back pain, which is unusual for lymphoma.

After treatment and remission a year later, I still had the severe back pain. The cancer pain management doctor I was seeing was perfectly happy to keep prescribing godlike levels of opiods. (50+ mg of oxycodone on top of 30+ mg of slow-release morphine pills).

After a few months, I lamented that I was tired of taking pills and dealing with all the side effects. He offered to send me to Physical Therapy, but explained that most of his patients preferred to stay on the pills and that PT rarely worked.

After 3 weeks of twice-weekly 30 minute PT sessions, my back felt amazing. No more pain and I was able to quickly wean off the drugs.

At some point, my cancer had caused back pain, which I'd continued to worsen by avoiding exercise and sitting at weird angles to alleviate the pressure.

As a humorous aside, I had to pay for several weeks of PT out of pocket because my insurance did not consider it important. They would prefer to shell out $500+ dollars every month to see a pain management doctor and continue taking pills.

Dude, you absolutely should be trying to be a manager in 3 years, if you want to go down the management track.

Depending on company size, you should probably be a team-lead or something by then.

Junior developers that remain "solo" workers after 3+ years in the industry is a bad sign for value. If you've been developing for a while, at the very least you should be put in a leadership position from a project perspective. Management/leadership experience doesn't just pop up out of nowhere. It's a skill you have to work really, really hard at. Take every leadership opportunity you can grab. You don't have to be "the boss" in the formal sense of the word, but you absolutely should be working towards a leadership role.

I think you may be missing the brilliance of this product from a sales perspective.

Even if this product isn't a massive success, the units they do move will have a huge potential impact to their retail clothing lines.

Now, instead of suggesting clothing items in the store based on other items you've looked at, they can now suggest clothing items in their sales material based on what you already own and look good wearing.

If I had this product and Amazon started sending me curated outfits that they knew would look great on me and matched my style, I'd probably start using Amazon exclusively to fill out my wardrobe.

As it stands now, I buy almost everything EXCEPT clothes through Amazon and prefer to let my wife buy me things she wants me to wear.

This product, on it's own, is a game changer in the fashion industry. Now Amazon has a compelling and unique product that is going to make it increasingly difficult for other retail outlets to compete in the same space.

And once we start seeing "apps" for this camera that auto-instagram/snapchat your pictures, I think we're going to see a huge demand in the teen/millennial female market for what is kind of a geeky niche product.

I think it's your third paragraph here. You claim not to have insulted anyone, but you literally claimed that GitLab employees are worse than Hitler.

That's:

A) Super inflammatory B) Extremely disrespectful C) Adds nothing of value to the discussion

You're absolutely allowed to have a difference of opinion. Even saying something like, "I really hate the GitLab interface and it feels super janky" would have probably gone through this unscathed.

But you definitely were rude and openly hostile.

Hrrm... what if you COULD spin up factories like an EC2 instance.

What would be needed? A line? Programmable robots? Inputs and outputs?

Interesting idea.

Don't look for jobs when you're desperate. Look for jobs while you're happy and comfortable in your current position.

It's like investing. Don't buy while the market is going up. Buy after a crash. Buy when the deck is stacked in your favor.

I (mostly) only ever actively look for positions when I'm comfortable in my current position. If a new employer really wants me, they'll bend over backwards to get me. I recommend everyone do the same.

Sometimes you may not have that leverage. In those situations, absolutely take whatever job you have to take to keep your house and keep the electricity flowing. But don't stop looking for a better job.

You are literally the only person on the planet that is going to look out for your best interests. The difference beteween a 100,000 and a 110,000 salary over 30 years is freaking $300,000 (not even counting for investment potential). Maximize your career for yourself.

Either they're bluffing completely or you're extremely junior.

We often spend $10k-$30k on each new hire. If you make it through that pipeline and they try to strong-arm you on the first day... they're going to have a rough time.

I had to trick my brain into just enjoying it as a strange engineering discipline. I like to go through life imagining everything as a system to be programmed, so to me, business politics was just a different type of system in a programming language I had yet to learn.

Now I like to think that I can bend those systems to my will as well. :)

Good luck with your small business endeavors.

This is just my perspective. I've been in (increasingly) challenging engineering leadership roles over the past 5 or so years.

When I first started, I felt exactly like you do. That my job was to enable and empower my engineering team to build awesome product and that all the "management" stuff was a huge waste of time.

I received some advice from my boss that I was missing 50% of the job. You're being paid and expected to do the engineering leadership side on top of being the interface with the company for progress and business objectives. Any talented engineer with a modicum of people skills can usually do a halfway decent job of leading a small team to build a product, but it takes some business acumen to understand how your team fits into the bigger operating environment of your business.

That was the skill they expected you to bring to the table, and you not only despised it, you blew up at them in a meeting.

Granted, that company may very well have had a terrible meeting process and been looking at all the wrong KPIs for how things should work. But knowing how to positively influence that process inside the environment and doing so in a political way that preserves culture and existing team dynamics is crucial.

I hope you give that type of role another shot. The world needs better engineering leaders that can help businesses understand the value of a strong engineering culture.

ISC Consulting Group | Software Test Engineer | Augusta, GA | REMOTE, SECRET CLEARANCE REQUIRED

We're looking for a full-time software test engineer for a major military contract. Any type of software engineer will do, but this position is going to involve writing and building automated tests for our existing software and system development teams. This individual will officially be an engineering member of the QA department and will be lightly disassociated from the actual engineering team.

This individual must have experience with writing automated tests (functional, regression, etc) and experience with any of the AWS stack is a plus.

Unfortunately, we absolutely must have someone with either an active permanent or interim SECRET clearance.

If you're interested, email (me at ericharrison dot info) with your resume and I'll pass it along to my QA lead. Thanks. :D

Shit, I can't even find a SINGLE qualified mid-level software engineer with a clearance for a job that's already started and fully funded. Good luck bringing on 50 people in a short time frame. You might be fully staffed in a year if you have the world's most talented internal recruiting team.

This is a REALLY hard problem to solve and the government does not make this process easy. If we miss milestones, we don't get paid, so that $1,000,000 in salary that your team was paid for the last 90 days of work suddenly doesn't get paid. Your $100,000 in profit you forecasted suddenly turns into a $1,100,000 loss for that quarter.

When I started working in government way back in the early 00s, I too thought "I can just automate all these worthless idiots out of a job." If it was that easy, someone would have figured it out by now.

Most companies have to spend $30k - $50k just writing a proposal in the hopes they'll win some work. The problem at this size and scale is astronomical.

If you can think of a better way, I really would love to talk to you. Email is in my profile. :)

Yeah, and having already had a clearance in the past definitely helps speed up the process to "reactivate" it. The big bummer for companies like mine is that our contract says that we have to have employees with an active clearance. So to activate the clearance, we have to hire them, keep them on overhead for several months and they can't start working until their clearance is approved.

It's a painful $20-$40k hit to the budget. :/

It's not just the $20B revenue companies that provide support and services to the government. The big companies make the headlines with their billion dollar contracts, but the bulk of the support, services, and infrastructure is provided by Service-Disabled Veteran Owned Small Businesses, 8(a)s, and a bunch of other tiny business acronyms. We're all trying to kick ass (because we love the country) for pennies on the dollar.

Let's do some math.

So, this is a $927,000,000 five year contract. Right?

So let's tell the government not to pay talented companies to provide services and instead hire federal employees to build out infrastructure to do the same thing.

Assuming we have kind of a $200,000/year base salary for the types of people we need (so like a GS-15 or so), how much does that cost the government over the same time span?

$1,000,000/employee.

So you would prefer the government hire 927 people full time for five years to completely replace MS? Do you think they'll do a good job? Is $200,000 salary a good deal, or will you end up hiring bad software developers at that rate? Can you hire better developers for $400,000/year? Can 464 people build a MS replacement in 5 years?

Also, who will support it? Do we need to factor that in as well? Will we have to replace this new OS/tech stack in 5 years? Is that another billion dollars?

I suspect that you're looking at the price and being blown away by the number. The DoD spends $580 billion each year (roughly). Are there any comparable companies with a $580 billion/year run rate? Expand your mind, friend. This is a bargain price for the government.

You should do some digging and try to find out how much DISA is paying for their own version of AWS. I suspect that number will blow your mind as well.

Also, DISA not only has AWS instances, they have their own freaking installation of AWS! As in, they literally own a replicated version of AWS that only they can use and control. The government is a bloated, ugly beast, but they know what they're doing and they squeeze the vendors for every drop of value. Margins in this space are tiny.

What is their option? 1.5 million Macs? 1.5 million RedHat installs? They've tried going that route in the past and then paid through the nose just to get basic desktop applications available to their workforce.

Oh dang. I estimated tens of thousands. I was off by an order of magnitude and I was the one trying to defend this contract to HackerNews. Lol. Yeah, DISA is freaking huge. We're doing some work for them right now and the scope is massive.

Speaking of which... if any of you have a Secret clearance and want a job... :)

Holding them ransom? They chose this at the highest levels. I was in DoD for the transition from Sun to MS on the desktop. You're off by orders of magnitude. My one, tiny little facility had an IT department of several hundred to support the crazy infrastructure stuff we had to deal with.

And don't assume that they're not also using other services and providers as well. How would you provide desktop services and solutions to 80,000 people distributed all over the world?

If you have a solution that doesn't cost $1bn, please write the white paper. I'll gladly write the RFP response and we can enjoy our profits. :D

The numbers may seem staggering, but this is how much shit costs in the government. Most of our competitors in my space are offering services with 2%-4% margins because the government is so stingy about how the work is performed. DISA alone is probably at nearly the scale of AWS in terms of compute power and system requirements. And they probably have 5-10x the number of employees. It's not a trivial problem to solve, and I've been working for contractors for 10+ years now.

DISA is a big group. They have mountains of RedHat instances already as well. I would bet they've probably got a billion dollars or so invested in the RedHat infrastructure.

If I had to hazard a guess without seeing the SOW/PWS, I would bet that most of this MS contract would cover desktop support services.

DISA supports IT services for a huge portion of the DoD. This is basically a Fortune 100 company in terms of employees and scale.

What would you suggest as the alternative? Have DISA write their own tech stack, operating systems, and cloud services? This is a freaking great deal for the government.