Ask HN: Do you regret building your own software as a non-technical company?

I've already seen several internal tools becoming a burden for the company that has created them. Development is costly and the resulting product is often buggy, unreliable and demands constant work and attention.

Is that a common thing? Is it a bad idea by default for a non-technical non-SaaS company to hire developers and build anything custom?