dpenguin

There are a lot of arguments about how DoH with TLS 1.3 will give us privacy etc by the proponents of DoH(not this article).. but it’s basically moving the trust from ISPs to CDNs. There are fewer major browsers and fewer major CDNs than ISPs, I suppose.. so not sure if it’s a good move.

It was not a counter argument to anything per se. Just pointing out that even with episodes of failed estimates, there is reason to continue estimating.

Just woke up. Totally agree with this article, even though it was published 4 hours ago in the early hours of the morning!

Got it. Better to stick to tried and tested “boring” stuff and actually code small functionality yourself than chasing a thousand line library for trivial stuff.

At work, someone used a 3rd party library to send stats to a server in json format and “standardized” on it. 100s of developers had to now learn this new library and it’s APIs and one bug caused everyone a bit of hassle. It probably made the original dev’s job just a little easy but made it a bit hard for everyone else. These things do happen. The point is to learn from this and make better decisions over time, not give up and say we can’t get better. Trying to provide an estimate forces you to think more carefully and become better over time.

Would you say SpaceX launch is novel? Did they do it on schedule?

Software is not rocket science. Rocket science is not rocket science.

That’s cheating. No, that’s not what happened in that org though.

Yeah. The reason I mentioned it is to highlight that the business often needs to know what something will roughly cost and cannot just headfirst five into it. Estimation is a form of communication that addresses this.

Looks like tech debt to me.

It’s understandably a big project. But you can still start breaking it down, correct?

Break it down by services first. Then do estimates for say 2 services to get an idea of what it takes to estimate the whole thing. Part of the process is also to communicate the cost to business so that they can see if the feature is worth it or not. (Read “Shape Up” by 37signals on this. Free ebook but you get way more than what you pay).

Can’t comment too much without knowing your codebase but it is engineering. Most tasks are not supposed to be this hard. If they are, then it’s time to start paying back tech debt so you have things under control again.

Ah! xml.js

Code organization (part of Architecture) matters a lot here. Especially in large code bases. Software is meant to be “soft” or pliable easily. To do that, one should be able to alter a module without having to understand the whole system. If a small task takes too long, there’s a problem with the software architecture or code organization.

Let me clarify: when I said “they are not qualified for the job” I meant they are not qualified to produce the estimate yet. They can always go back and learn what’s needed and come back with a more educated estimate (still an estimate) or they call it out as a risk item and address it ASAP. More often than not, this is because they are not ramped up (manager’s or team lead’s fault) - why else would you ask a complete newbie, regardless of seniority, to estimate?

It could be because you are working on a brand new area every single “Sprint “(or equivalent). That’s not prudent assignment of resources(the manager should know this).

I have worked in companies where ownership is somewhat static(same few folks work on a given area of the product) and things get done within 10% of estimates (a lot of innovation too).

I have also worked in terms where ownership is fluid and “resources” get assigned to tasks. Nothing is now “estimatable”. Your experience is probably in line with the latter.

It only means you didn’t do your homework(no offense, everyone falls into this trap too and conclude the issue is that estimation itself is not possible ). You can’t give an estimate based on what the task would take at your previous job.

You need to call out what you don’t know(Now, don’t give me Rumsfeld wisdom.. Most software is not that complicated)

How often do you really do novel/unique/innovative stuff? Honestly.

99% of teams that need estimations are not in a phase where they are designing something that is hard to estimate. And those that are usually are prudent enough to not bother R&D with project planning(yet).

99% of engineers are working on things that CAN actually be estimated fairly accurately. New CRUD APIs? New ETLs? New React component? New integration with a product that has a published API? New service? New message format? New protocol? Whatever. Just break it down enough until it’s very clear what it takes to get it done. There will be a few unknowns but you can call them out as risks. Try to eliminate risks first and raise a flag so your manager/PM can adjust his forecast.

The thing is, everyone wants to be “working on the next big thing” so much that they actually believe they are working on something groundbreaking that cannot be estimated. Either that or they are not qualified for the job yet(hence they need to learn something new, which can be unpredictable). They just need to be grounded and it’s a fairly easy journey. They are all probably emulating the symptoms of those in 1% by believing they cannot estimate their work.

Potentially somewhat controversial take on this: why do college grads deserve a better job than, say, high school grads if the subject they studied in college is not particularly relevant to the society at this point(which the lack of jobs for that qualification is indicative of)?

There was probably a point in time when learning hunting was like going to college and along comes farming to make all those hunters jobless. That’s how the wheel of “progress” rolls.

The idea that going to college will make you more bucks or guarantees you a job in your field of study is very arcane at this point.

It’s basically modified Parkinson’s law at play - work will expand to most available workforce.

With increased automation, either new jobs are created(thereby increasing the overall output of the economy) OR expectation for individual productivity is reduced, decreasing individual stress.

History is your proof. Automation has been constantly increasing forever and there’s no job loss en masse because of that. Empirically speaking there are almost always more jobs created.

Not saying anything about quality of life.

Exactly what I was thinking as I was browsing this thread. If I can understand the code by visiting as few files as possible and reading as few lines as possible and not having worry about finding those, job done.

There should be a metric that indicates aggregate LoB for a codebase. Simplistically, cluster references of all entities (variables, functions etc depending on the language), calculate the overlap of those clusters and aggregate them. Track this metric over time and when it breaches a certain threshold, refactor until it’s back to where you want it to be.

Cool! You could provide a read line-based command line interface with server or client driven completions to make it even more ready to use.

Agree with another who said it’s always the people that end up being a problem and not tech. Unfortunately a lot of tech geared towards fixing the problem ends up creating more of this because there is a new team that has to be created to add the new tech solution to the mix. Given that, I’d like a devx stack that lets a developer go as vertical as possible without involving other people. Once you decide to add a feature, you should be able to go all the way to deployment (and iterating) without involving anybody else.

My ideal devx stack(some parts exist but some don’t)

1. Heroku like deployments everywhere in a Unix like standard fashion. If the standard exists, I’d not worry about vendor lock-in and can choose the right provider for my scale/cost etc. I’ll also be able to have the same workflows for private DCs , dev setups etc without running huge bills.

2. Heroku CI/flow/pipeline, again standardized so I’m not stuck with one vendor.

3. GitHub + VsCode (or vi or emacs or pycharm) delivered as an offline first service. With templating capabilities so you can onboard new team members with a click.

3. Expo.io like capabilities across platforms. I get it that it’s already available for iOS, Android and web. But what if I have a native app? What about a server I wrote in c/c++? Docker probably. But the updates are still not handled for me automatically.

4. As a historically backend person, I find it hard to believe it took this long for the front end world to get to the component framework. I’d like to be able to create frontend pages by just pointing to an openAPI crud spec. I want to be able to add special buttons or forms in one line (no more) by just specifying a non-crud openAPI spec. We are almost there but not quite.

5. Since Saas is the future, All user management, subscription, engagement tracking, support, Customer pipeline etc should be a few line setups.

So I’d get up one day having dreamt of a new feature, walk to my home office in pajamas, code up the backend, specify how to expose it in the frontend, add a few test cases and click submit. It goes through testing locally, in staging, to production, gets distributed a set of people, and sends me some data on how good the dream can be by the time I finish my coffee. If I am happy I click “roll out “ And the entire world benefits from my dream.

Some companies have painstakingly built this stack. One man shows can actually do this fairly easily but mainly because the User base is small. But nothing that works for medium sized companies.

Too much to ask?

Sorry “typo”. I meant, as easy to use as Heroku is.

Can fully relate to the author. We have been struggling with effective k8s at our company for 2 years now. Too much to learn to get your first service in production. You will end up writing wrappers after wrappers that make you think “we can’t be the first guys to solve this” and after googling, you find that every issue you find is described as “just” one of the downsides of using k8s and here’s how we overcame it.

I wish someone can make a deployment orchestrator for your private DC that is as simple to use as Heroku is. Edit: fixed typo.

There are several nuances. I’d say these:

“don’t bring me solutions.. (for approval). Implement them and show me the results.”

“You are allowed to bring me problems that you have tried and decided you cannot solve”(But that’s going to affect how I view you - over time if you keep bringing me the same type of problems, I’ll decide you’re not good enough for your job).

I’m generally talking about tech problems here.

Great answer along the same lines I myself look at tech debt as well.

Another important thing to keep in mind is that while you can leverage tech debt to move the business forward all you want, be extremely aware of your tech debt and reduce it before you go bust. It’s very easy to develop a belief of “this has worked for 4 years so it’s solid and doesn’t need to be looked at anymore” when in fact, you could be teetering on a total collapse of the system within 3 months because some aspect of the system/business started gaining traction non-linearly.

PS: I have worked at very large, medium and small companies that grew big. Haven’t worked at a failed startup so far - so a bit of selection bias in my opinion.

I agree 99.9% of products do not need a micro service architecture because 1. They will never see scaling to the extent that you need to isolate services 2. They don’t have zero downtime requirements 3. They don’t have enough feature velocity to warrant breaking a monolith 4. They can be maintained by a smaller team

I also agree that the way to build new software is to build a monolith and when it becomes really necessary, introduce new smaller services that take away functionality from the monolith little by little.

Microservices do have a good usecase even for smaller teams in some cases where functionality is independent of existing service. Think of something like LinkedIn front end making calls directly to multiple (micro)services in the backend- one that returns your contacts, one that shows people similar to you, one that shows who viewed your profiles, one that shows job ads etc. none of these is central to the functionality of the site and you don’t want to introduce delay by having one service compute and send all data back to the front end. You don’t want failure in one to cause the page to break etc.

Unfortunately, like many new tech, junior engineers are chasing the shiniest objects and senior engineers fail to guide junior devs or foresee these issues. Part of the problem is that there is so much tech junk out there on medium or the next cool blog platform that anyone can read, learn to regurgitate and sound like an expert that it’s hard to distinguish between junior and senior engineers anymore. So if leaders are not hands on, they might end up making decisions based on whoever sounds like an expert and results will be seen a few years later. But hey, every damn company has the same problem at this point.. so it’s “normal”.

This notion of invasion of privacy is all relative. If they ask you for your mother’s maiden name or first pet or the city where you had your first kiss, you are okay typing it in a form for them.

If they try to infer the active port numbers on your computer to see if there’s a Remote Desktop installed by a bot, you’re not okay.

What’s the alternative? Do you want them to disclose everything they do in a marketing article even though 99.99% of people will have no clue what that means and 10 of the 100 people that bother to read and understand will use it against them. To what end? To gain your trust? You - who has already given them your credit card number, mothers maiden name and city where you first got intimate with your first partner?

Is “behaving like cyber criminals” bad though? Just because it’s doing something out of the ordinary in terms of tech doesn’t mean it’s bad. Fighting cybercrime is always a cat and mouse game akin to counter terrorism, counter espionage or even plain cops and robbers. You need to think like your enemy, have informers, etc etc, while not harming the good citizens. That’s what is going on here.

Are there cops who misuse their power? Absolutely. Are there spies who use information for personal gains? Sure. There need to be checks and balances that make it bad for such people to go rogue.

Privacy acts aim to do some of that. They bring accountability but also an ability to opt out (the latter is hard though - akin to ostracizing oneself from a community).

The argument here is again “if it doesn’t work for you, just move on”. Unfortunately, tech world is full of examples that do something slightly evil in order to provide some kind of instant gratification and once it catches on, no one can really opt out without coming across to the rest of the world as a weirdo or a Luddite or an oldie. Don’t have an Instagram account because you don’t want to keep scrolling through pics? You are a Boomer (FYI I am not - see what happened there? I felt the need to clarify). Don’t have a Smartphone? What age do you live in? Etc etc.

It’s not like making Nukes kind of tech where the harm is very obvious and people restrain themselves from making it in general.

It’s like farming/industrial revolution/social media kind of irreversible change where people actually welcome it and realize the evils a few (or a few hundred) years later.

That said, power to you if it catches on because If this succeeds and you make a ton of money, you will not be needing to be part of any room while you will be enjoying the money you earned by making the few productive office workers’ day less productive and more annoying while making countless believe-anything office workers‘ “feel” they are more productive.

Just keep it in a different room.

Great analysis In the original article BTW. Hope to see more of these on HN.