Ask HN: How to address horrible practices at new job?

https://news.ycombinator.com/item?id=9766993

by remyp • 11 years ago

23 20 11 years ago

news.ycombinator.com

I recently (~2 weeks ago) started a new position that ostensibly was "running the development team (3 devs)" for a SaaS firm. I am working remotely, so it has been difficult to get the CEO to keep meetings or spend the necessary time (> 5 minutes) to discuss much of anything.

I have been getting up to speed on the codebase -- which is 10 years old -- and in doing so I have noticed the following problems so far:

- No documentation, a few in-code comments only

- No automated testing whatsoever

- No monitoring (a critical API went down today and we found out because our customers told us)

- No development server

- Development environment not standardized or even documented (3 hour screen share with the only guy who knows how to get set up)

- Code is committed directly to master, not on branches

- Developers manually push code directly from local environment to production through undocumented process

I brought up these problems and the solutions to them and was brushed off as "slowing down our process" and "solving problems we don't have". My current dictated priority is a coding up a feature addition that will take me a month.

This is a small firm, so there is only one decision maker. Do I try to get him to see the light, ignore it and do as I'm told until he trusts me, or start calling the other firms I turned down to take this (extremely well-paying) position?

Nadya• 11 years ago

I assume their code base has been working for 10 years? Not every owner cares about scaling or quality, sometimes speed is their selling factor.

That's why we have McDonalds and 5-star restaurants instead of only 5-star restaurants.

You can try to educate him on why documentation would help speed up development, would make on-boarding new devs easier/faster (increasing product quality/stability and possibly decreasing churn for when your critical API fails on a customer...)

But his business is clearly working for him. So why change things? Why spend a lot of time (and time=money) on fixing problems that haven't ailed his successful business for 10 years?

Einstalbert• 11 years ago

Sometimes these businesses work because of people like the OP, who shoulders the burden of a bad decision maker for years. We're celebrating one of our big company milestones and we're no different. The decision makers think we're here but by the grace of God sometimes. I say we're here but by the grace of Todd and his addiction to energy drinks.

remyp3• 11 years ago

This is definitely the case currently. There is one developer that has been around for several years and he has clearly been bearing the brunt of the work. This firm's bus factor is equal to 1.

gt565k• 11 years ago

Run, and run fast.

Since you've only been there for 2 weeks, you've yet to experience the ridiculous amount of production issues that will arise due to the lack of a solid software development process. Wait long enough, and when the only guy that knows the system inside-out leaves, you'll be left with the fire extinguisher trying to put out fires every other day.

Nothing will change. I can tell you from experience. If the place has been running like that for 10 years, there's a reason for it, poor management. I'm sure a lot more people before you have brought this up.

Run, and run fast!

hga• 11 years ago

Sounds like a case of responsibility without authority. You signed on for the job of "running the development team" but they've demonstrated they're not going to give you any authority to actually do that.

And your CEO has no respect for you if he can't find more than 5 minutes (total or at a time) to bring you on board.

I'd leave ASAP, this is going to be nothing but pain, and very high stress; almost certainly not worth the money.

brudgers• 11 years ago

Your question imples a large amout of technical effort and presents zero business value creation in return. After two weeks on the job, you're arguing with your customer over their prioritization of the backlog.

My advice: learn why the current methods are working for the current team in their pursuit of creating customer value and why the YAGNI of reality doesn't conform to Uncle Bob's best practices. Note that I am not saying this is your dream job. but it is a chance to gain experience beyond theory and more nuanced judgement such as only experience can provide.

Good luck.

degenerate• 11 years ago

This is the best advice in this thread.

SideburnsOfDoom• 11 years ago

I would seriously look at the terms of your employment contract. there is usually a "probation period" of a few months during which time you can be let go at very short notice because it's not working out for whatever reason.

It's not always kept in mind that "it's not working out" cuts both ways. If you aren't happy there (and I really would not be) then it's time to decide - as they say "if you can't change your company, then change your company."

Some of these practices, YMMV. For instance, "Code is committed directly to master, not on branches" gets a big shrug from me. What does it matter so long as only 1 person is working on a repo at a time, or if changes are isolated and merges are frequent?

Others, like no testing, no build automation, no live monitoring, give me the creeps. There is actual harm from them "an api went down and we didn't know" so there is leverage there to pitch you can make for mitigation of recurrences of actual problems and for proven industry practices. But it looks like a lot of uphill struggle.

gus_massa• 11 years ago

Assuming the stress/money ratio is good enough, you can try this "Getting Things Done When You're Only a Grunt" http://www.joelonsoftware.com/articles/fog0000000332.html

I think that monitoring is an easy target because it's isolated and it will not "slow down" the rest of the team.

The next step could be automated texting for your new code and later for all new code (and much later, for old code).

(Documenting the development environment is still in my todo list :) )

NathanKP• 11 years ago

There is a pretty simple order of operation you can target if you want:

1) As you onboard and get your environment setup simultaneously write a Vagrantfile so that you can provision a local dev environment in an automatic fashion.

2) Set up an automatic monitoring solution. This shouldn't take longer than 30 mins but will be all it takes to provide proof that lack of tests and deploys directly from local to prod are endangering the stability. It will also allow you to pinpoint outages coinciding with deploys.

3) Write an extremely basic test script. Even if it is nothing but a bash script that uses curl commands to do auth, the top 5 transactions, and that's it it will still be better than nothing. You don't need 100% coverage for tests to be useful. Most of the time there are a key ten to twenty tests that can cover 80% of the core stuff that you don't want to break.

4) Any time you figure out how an endpoint works, or have to ask one of the other engineers something take 5 minutes to commit this meatspace knowledge you just learned to a bitspace markdown file somewhere in your repo.

The nice thing is that these are all things that take minimal time, but have a huge multiplicative effect on your productivity and that of others. You should be able write better code faster than your coworkers and the results will speak for themselves.

tacone• 11 years ago

Nice advice.

They won't allow you to change everything bottom up, but they probably will not care about small adjustments.

Little by little, so much can be gained.

remyp3• 11 years ago

Agreed. This is the advice I will plan to follow -- its simplicity is Gandhi-esque. Thank you, Nathan!

davismwfl• 11 years ago

Its a small firm and these are not uncommon issues. The size of the firm means that likely the focus of the CEO is to bring in the next client, not worry about whether there is a development server setup properly. A lot of the times, these are the exact reason he has hired someone that knows their job. He isn't going to want to know about all the little details, he just wants everything to work so he can sell. And the feature he has you working on may be critical, as he sees it, to land a client or give existing clients something new.

My 2 cents. Don't judge everything just from 2 weeks. All of your issues are valid, but give yourself a little time to acclimate to the team. Gain their respect, gain the CEO's respect by showing you can do the job and take care of details. Take some of these issues you highlighted and fix them. Add automated testing for your new feature, add documentation for it, setup a quick monitor to alert when the API goes down and learn the development setup so you can fix that process. In fairness most of these issues show a lack of development leadership so be the leader. Don't try to convince the CEO why you need to do these things, just do them and the results will speak for themselves. Do yourself a favor first, get the respect of the team by playing along and getting the feature done and do it with proper process and documentation. That'll go a long way.

In the end, you may hate it and need to leave, but give yourself a chance to see if it is systemic issues, or just a lack of development leadership. I have seen many times when a small business CEO tries to control development because he doesn't have anyone strong to do it. Usually this is because he doesn't trust the results from the team totally because of things like an API going down and it takes a customer call to fix it. Why else would he have hired an "outsider" to run the team? He doesn't know what he doesn't know. Show him through doing and see if it changes, if not, move on.

an0nymoose1• 11 years ago

Is your name Nate?

remyp3• 11 years ago

Nope.

segmondy• 11 years ago

Demonstrate competency first. Solve the main problem you were brought there for. Solve it right, solve it fast, employ all the ideas you wish to see, documentation, testing, monitoring, dev server, peer reviews. Earn their trust and respect, you are 2 weeks in. Focus on the job at hand, and in 2-3 months, you will know who can be your allies and you can then start figuring out how to get these changes out.

PhilWright• 11 years ago

It sounds like you have not been in the position of leading a development team before. As a result you are suffering from the same mistake that I made myself when moving into that situation.

Stop asking your boss for permission to do every technical task. As a developer this was appropriate but when running a team it is not. It sounds like your boss is viewing the development team as he should, it's a sausage factory. He cares about the output but has no real interest of what happens to produce it.

As he will not let you dedicate fixed time now to make changes you need instead to gradually introduce them. Set a goal of getting all the process changes you want made over the space of the next year. For example, add some automatic monitoring. Don't ask for permission, just go ahead and go it because it is a small task. Once it is working well you then get the three other developers to start using it. Once everyone sees the benefit over the next few weeks you can then make another change.

jbdigriz• 11 years ago

This is generally a problem in various forms at firms of all sizes, so running away is simply conceding that you lack the desire to develop some of the most important skills required when moving into an elevated lead or management position: the ability to compel people without forcing them and the ability to compromise.

Looking at your list of gripes, some are clearly critical issues (ie. Automated production monitoring and alerting of customer facing system) while some are obviously nitpicking (ie. Comments and documentation? Lol). Drop the "nice to have" stuff and instead pursue the "need to have" items. No CEO worth his salt is going to accept the liability of customer loss or lawsuit simply because no one thought to build some simple monitoring jobs. At the same time, I cringe at the prospect of a developer even mentioning comments or version control minutae to an executive level - definitely NOT a point to be explicitly bright up beyond some quantified metric on a regular report. It's distracting noise for him and definitely your problem to solve. Given that you mentioned being very well compensated, well that's a pretty good hint that the task ahead of you will be challenging and often not in the way you expect, as evidenced by this case.

On that note, be sure all these things are actually important for that specific business. I once worked on an in-house, Windows based C++ trading system with over 600k lines of code and there wasn't a single unit test to speak of. Granted, it was developed following SDLC and included 4 levels of testing (dev integration, ba functional, user demo and qa) but the system was running continuously for almost a decade and reaped profits in the high 9 figures. Some shops produce high quality software with deeply knowledgeable long tenured senior technologists and don't necessarily require such testing frameworks as they perform that task themselves and often far better than some pre-canned block of code ever will. I've also been in shops where testing and coverage minimums were required and tracked precisely in real time, preventing code release if standards were not met. The end result were loads of trivially passing, nonsense tests and a culture where developers often felt compelled to create even those for only the lowest hanging fruit to meet the minimum requirements for a deadline or risk a monetary impact to their bottom line. So don't just rotely follow (let alone impose) paradigms without realizing that quantifying their value is hard in many scenarios, not to mention could be construed as stepping on toes...

As some have mentioned, the best managers are those who are able to gain the trust of their teams and this is something that takes time and strings of successes along the way. At the same time, lead by example and instead of imposing your will, let the results speak for themselves. Add tests in some minor modules and if they fail, use the opportunity to demonstrate how they saved potential headaches or worse. I've found that people invariably flock to those who demonstrate not only ability and intelligence, but also civility and geniality. I can't even begin to explain how small humorous exchanges have vastly strengthened my network. I've had to lead teams of juniors and offshore folks without any official management title and its forced me to adapt and develop various methods of building repoir and ultimately compelling people to get things done, with few sour grapes to speak of and many successes (and comp increases) along the way.

Regardless of all of the above, you're still getting paid, right? By your admission, paid WELL. So why would you leave that because you believe you're not being allowed to do your job (most of which actually requires more work and maintenance)... 2 weeks in?!? Perhaps after 6-12 months, you come to the conclusion that the culture is just not for you. So, leave then - I assure you there will be just as many, if not more opportunities then, except at that point you can be confident you've shed any insecurities and gave it your best shot and have some quirky behavior tolerance to boot. That sort of techno "battle scar" is quite visible as a sort of confidence backed by wisdom and experience and makes prospective employers wet their pants. Or maybe you just kick ass and prove you're worth every penny - smiling all the way to the bank. Either alternative is far more appealing than tucking your tail and running. Food for thought

deeteecee• 11 years ago

why do other people say this is "a general problem" in any way? sounds extremely fishy and rare to me.. i guess just do what you're told if that salary is such a big bonus for you but ask him if it's okay to slowly tackle those problems you considered after you get the main task at hand finished.

JSeymourATL• 11 years ago

it has been difficult to get the CEO to keep meetings or spend the necessary time (> 5 minutes)...

It turns out a lot Small Company CEOs have limited bandwidth. Learn his communication style and how to manage him. Can he take a 7am call on his commute into the office? Relative to your performance imperatives, help him understand how these things move his agenda forward. Otherwise, his problems aren't going away.

Ask HN: How to address horrible practices at new job?

Related Stories