Pinbenterjamin

Best advice I can give, as a manager, is self-discipline. It's a core tenant of what divides my good workers, from my standard workers.

Sure, learning Java, or C#, or Python is nice. But what really makes an effective worker to me is someone who has strategies for tackling problems, and I believe that the highest abstraction of that is self-discipline.

When you are tasked with a problem in the real word, whether it be enterprise, start-up, or side project, you will need to look at the problem wholly. You will need strategies for laying a path that you, and you alone can follow.

There's so much to writing software, but every developer wants to type. Do you have a good method for discovery? Understanding unfamiliar project landscapes, or code flows? Are you practicing reading code? If a manager gave you a project in a project you've never seen before, how do you know where to start?

Do you have a strategy for note taking? For flow charting as you discover? For documenting minutiae as you come across it during development? If you do, are you consistent? You need to be diligent and habitual in your process. If you present problems in a consistent manner to your manager, you will developer an easy means of communication, allowing you to solve things faster, and get what you need faster.

Are you consistent in your coding style? Are you implementing according to the problem, or according to whatever new tool you most recently discovered. That's engineering discipline.

I didn't really get there until I was in my late 20s, but it took some seriously trivial changes in my life to get started on that road. Like, I started making my bed every morning. Reading before bed, making a small breakfast, and coffee before work. Walking twice a day. Gym at lunch.

Having a schedule, being habitual, helps you get in the mind set of optimization. Optimizing your own processes will lead you to consistency, and sharpening your consistency will force you to be disciplined!

I'm going to try and be optimistic about this;

10b in research and development? I can't imagine that it's going to be for 'weapons' only right? Some of this has to go into the future of humankind. I would imagine they have to be interested in colonization, as it's generally in the interest of humankind to expand to the reaches of outerspace, and to do so and be confronted with opposition would require such an agency.

I'd like to know too what the overlap in responsibility between NASA and Spaceforce will be. Are they both scientific ventures? Is Spaceforce actually interested in intergalactic safety.

I'm cautiously optimistic.

It is nice to have tools that don't require 30 seconds to boot up, just so you can view some xml (Looking at you visual studio). But the recommendation of vi or emacs...just why? I scratch my head when people recommend 30 year old tech.

Those who use vi and emacs to their full potential are that generation of developers who grew up with those tools. Nowadays, why wouldn't you use something like VSCode as your main text editor? Atom? Or even n++? Unless you are confined by your job to an sshterm or something. If you want to LEARN a text editor you might as well pick one of the modern ones with all the nice modern QOL updates. The tech-flex of developing in a terminal is so dumb.

And VI? Why not VIM? VI is like recommending ANSI C over C18.

You definitely should. I remember when I started doing serious side projects, how invaluable having a document was.

It should be living though. Start with something small, describe goals, and outlook. Put a feature checklist on it. As you work through early problems and refactors, treat it like a development blog. Include build notes.

You will probably put this down, and pick it up a few times on your journey. Make sure you leave plenty of breadcrumbs for yourself so that you can be productive each time you do.

And, should you decide to put it down for good, you'll have a nice little project history for lessons the next time you try it.

It's all in what you want as a reader.

For some, it's comparing 'holding a physical book and turning the pages' to the more active, not taking readers.

I don't think the leaders in the ebook industry really understood what the target market was when they took on the endeavor.

For me, ebooks made a lot more sense. Using exclusively the kindle app, I can keep track of my progress across multiple books, score and rate them, make notes (which are preserved and easy to access), and share insightful quotes from my favorite books. If you're a social reader there's really nothing like it.

If you're more of the page turner type, or someone who enjoys the catharsis of reading a book, nothing is going to compare to the feeling of turning a physical page, sipping on some tea in just the right amount of lighting, etc.

I think ebooks set out to capture the first market, and they did it, but people expected it to dominate the book market, which it hasn't and won't.

There are impossibly diverse options for learning today.

Ideally, you'll want to find which methodology (or -ies ideally) works best for you, and continue to branch from there.

The idea of 'how do you learn' extends above answers like 'books' or 'youtube' or 'coursera', and fits better in the categorization of 'I like to Listen', 'I like to try', 'I like to watch'.

Ideally, find which of these, or which combination of these is most interesting to you, and then find the appropriate tool to leverage that.

Personally, I like to Read, and I like to Do.

I have a brainstorming meeting with my Team once every 2 months where we spitball home project ideas, and have a show and tell from work we've done in our spare time. Once I've settled on an idea, I apply the above concepts to learn as much as I can about it.

For example, the last project was to build and solve a Rubiks cube. I read about the mathematics of a Rubiks cube, and built models alongside. I know that having reference material, and an instance of CLion open next to me is my recipe for successful learning.

Process Automation / Business Rule Optimization is a nearly untouched field for large, existing software driven companies.

For goal N, where N is an amalgam of small processes, what can we do to isolate, automate, and optimize the process?

Everyone is hyper focused on generic ML solutions, but there's it has been a hell of a lot easier to hand-code solutions to problems in existing mid-to-large scale businesses that are looking to alleviate a process pain point. If you have the brains to determine some basic ROI ballparks, you can make back engineering hours fast if you can either increase the output of your business, or decrease the required people to accomplish that output.

There's some fields that touch upon this, like Dev-Ops, but this is a more strict definition of 'operations', in that I'm referring to the non-technical people.

The impact of this practice is amplified by two major factors; 1. Quantity of work 3. Difficulty of work

If you have high quantity of N, small optimizations are compounded across all the work centers. If you have highly difficult work, reducing the time it takes to 'handle' or 'create' output can be reduced.

This idea ties in to a fantastic book, 'The Goal' by Eliyahu M. Goldratt.

Being in 'hot tech' is cool, but mastering old concepts is valuable to businesses rigid to change. As the number of established companies that leverage tech for a majority of their work grows, the need for brains that are capable of process improvement with a scalpel, rather than a chainsaw.

Kind of unrelated, but I recently tested out a scenario for the Dotnet Environment that worked really well;

I created a 'Random' service that lives for the length of the execution of the application.

This service has an instance of Random that persists with the object, and exposes simple methods with min/max parameters.

I register the service in a unity container, and then immediately resolve it, causing the Random type inside of the random service to instantiate.

Then anywhere I want to generate a random number, I inject that service.

This works because, as long as you persist a single instance of 'Random', two calls to 'Next' or 'NextDouble' won't result in the same number.

That's the general direction I'd like to take. When we capture the inputs for the scrapers, I'd like to persist everything. Mouse jiggles, delays, idle time. I think it would definitely help advance the software.

We have an enormous legal team that communicates constantly with end points to ensure they are aware of our scraping. And as I said in another comment, we store no results other than what is already available to anyone else using the web.

We've had this division for many many years, and before my time we paid another company to do this. There's no legal issues.

According to the NDA with my company I can't reveal anything about the architecture beyond the fact that it is hosted locally on a homebuilt distributed system that randomly chooses from a pool of 120 residential IPs.

We do have human emulation routines that helped avoid most detection, and that library is decoupled in such a way that we can edit behavior down to the individual site.

Some sites are just so damn good and detecting us and I just don't get it.

Well, when that option is available, as in the case of something like SAMBA WEB MVR, we absolutely opt for that instead, and pay our dues.

I still don't agree. The whole reason this business exists is to remove the cost from all the industries that need to run background checks.

I think the extent and reason for the checks aren't apparent. So I'll give a few examples where we have high volume and I hope that will enlighten you as to the reason why there are so many players in the industry.

The highest volume checks are around the medical and teaching fields. We often run 6-month, to one year recurring checks on teachers and doctors to ensure licenses and certifications are still active. As well as necessary immunizations to work in their environments.

Do you expect a low margin industry like teaching to staff a full time employee to do nothing but run background checks? They want them done and the schools have access to the information, it's just much easier for them to pay us a few dollars an employee and get a nice report than do the legwork themselves.

Additionally, incurring the cost of access for the relevant data is a barrier for companies without a bunch of cash laying around.

We don't solicit companies with incriminating information about their employees, it's a necessary part to a safe environment.

I don't have a perspective on the ethics of easier background checks. We run employment checks, the ultimate decision of whether to hire falls to the customer ALWAYS. I've seen plenty of former criminals get hired. It's a workplace culture 'thing'.

The right to be forgotten is alive and well most of the time, 90% of our clients don't observe information further back than a few years. I feel like that is a fair assessment of someone's behavior.

There are a number of ways we do this.

First, the process of automating a source is not as simple as 'grab data, send to person that creates the case'.

We have many many layers of precaution and validation both by humans and other automated systems, that helps guarantee accuracy.

On top of this, even public records has reporting rules in the industry. There are dates, specific charges, charge types (Misdemeanor/Felony), and a battery of other rules that the information is processed through in order to ensure we do not report information that we are not allowed.

We always lean to the side of throwing a case to a human. In the circumstance that anything new, unrecognized, or even slightly off happens, we toss the case to a team that processes the information by hand. At that point, we are simply a scraper for information and we cut out the step of having a human order and retrieve results.

We do not go back 40 years. Industry standards dictate that most records older than 7 years are expunged from Employment background checks. And most of our clients don't care about more than 3 years worth, with exceptions like Murder, Federal Crimes, and some obviously heinous things.

We also run a number of other tests, outside of public records to provide full background data. We have integrations with major labs to schedule drug screens, we allow those who are having a background check run on them to fill out an application to provide reasoning and information from their point of view to allow customers to empathize with an employee.

We also have a robust dispute system. The person having a background check run on them receives the report before the client requesting it in order to review the results and dispute anything they find wrong. These cases are always handled by a human, and often involve intensive research, no cost spared, to ensure the accuracy of the report.

There's a plethora of other things I'm missing, but if you have any specific questions, I'm happy to answer.

*EDIT

To clarify, there is a lot of information in public records. It isn't unclear or ambiguous at all. Motor Vehicle and Court records are extremely in-depth and spare no detail.

I don't empathize with your viewpoint because, whether it's a web scraper, or a person, the work is exactly the same. There's no additional volume, or extra steps. We just emulate a worker.

We measure the value in FTEs, and when a researcher quits, we do not replace them if the appropriate FTEs have been reached with projects.

It's a major benefit to the business not only because we don't have to pay another employee, but we can reduce training costs, and costs incurred by mistakes. We can also adjust execution of one of these agents, which normally would require rearrangement of work instructions, and retraining.

These are public records, 90% of them do not have integrations for automated systems, and those that do, we utilize. They are typically search boxes with results. We are not circumventing any type of cost that would otherwise be incurred.

We do not log any of the results, store them locally, or maintain any of the PII with each search. If a case was searched 20 minutes ago, and comes up again, we rerun the entire thing just as a human would.

Finally, to your point about 'help me with my homework', I consider posting on the HN forums homework for this type of research. There are a diverse set of talented developers on here with esoteric experience. The fact that an article related to the work I do came up on here, I thought, was an excellent opportunity to seek advice and perspective.

I run the division at my company that builds crawlers for websites with public records. We scrape this information on-demand when a case is requested, and we handle an enormous volume of different sites (or sources as we call them). We recently passed 700 total custom scrapers.

Recently, we have seen a spike in sites that detect, and block our crawlers with some sort of Javascript we cannot identify. We use headless Chrome and selenium to build out most of our integrations, I'm starting to wonder if the science of blocking scraping is getting more popular...

I don't think what I'm doing is subversive at all, we're running background checks on people, and we can reduce business costs by eliminating error-prone researchers with smart scrapers that run all day.

I don't want to seem like the bad guy here, but what if I wanted to do the opposite of this research? Where do I start? Study the chromium source? Can anyone recommend a few papers?

Understandable, as IBM has always targeted the enterprise solutions market. Having control over (I think still?) the most popular enterprise linux installation is a nice add to their business.

It opens greater flexibility for their servers, as they have 'ownership' of the distro now. Possible bid for a speed benchmark as a selling point? Some of the giants out there are going to be hard to beat for that though...

I don't know that they are doing this specifically for the cloud market.

Hey Ben! (Also) Ben here.

I work in New Jersey, as an Enterprise App Full-stack dev with 0 college experience.

I started as a developer 4 years ago @15/hr, and I recently breached the 6 figure mark. I've told my coworkers that I have a fire lit by a sense of inadequacy. I've always been behind classically trained developers, that is what keeps me pushing forward. Always playing 'catch-up'.

Now that I'm involved in my company's interview process, a few thoughts on what having a college degree does for our offers;

1. Because we hire through an agency, we already have a single layer of vetting that helps remove unqualified persons (both college-level and not), which ensure we have a decent, homogeneous pool to conduct face to face interviews with.

2. Whether we hire a candidate with or without a college background, our the offer range isn't enormous. If we have an offer in mind, (say 75k), having a college degree doesn't automatically grant you the high end of our scale. I can't think of a single instance where we cared about their degree once they were in the Face-to-face. Their performance in the interview dictates their offer, and we aren't asking questions like 'How do you implement bubble sort'. We ask some hard skill questions, sure, but we also ask just as many communication and general problem solving ones as well.

3. The range of starting pay is really small for us (think 75k mid, 70k min, 80k max). So that 3% gap makes a lot of sense.

To summarize, a degree may put you in a position to interview, but the range you get paid is largely depending on a wide array of skills, some of which are unrelated to development. Missing some of the hard-skills won't disqualify you for a position as much as missing the communication skills will (for us).

The important bit here is that, they will remain individual applications, with a unified underlying architecture.

It is a smart move, internally, to leverage a single framework for dealing with similar services. Less training, and greater lateral agility for employees. Developing a new feature for Whatsapp, and need some extra man power? Shift some workers off one of your other messenger services. Right now that may not be possible.

I decline to believe at this point that there is another corporate reason other than that.

I somewhat agree with this short article, with the exception of the 'Prestige' portion.

"Certainly there is some level of prestige that comes with having a degree. This comes from the level of communication required to simply navigate a degree. Also it is, in a way, a validation stamp that says… “Yes - you have the skills to be a Software developer”."

You can hold yourself in high esteem for completing a degree, but as we are all aware, having a degree does not guarantee you a job.

I think between a person with a degree, and someone completely self taught, the degree holder may have a better chance at procuring a job, but what will really get you a leg up is forming habits that make you successful in both facets.

I do not have a degree, and I've won out opportunities against diploma holders, but I know when I see someone who is classically trained, and exercises effort to continue through self-teaching.

I do not believe these are two means that are at odds each other, but two complimentary ideas.

A better article may have been 'The value of school; Having multiple avenues for learning will put you ahead'

Not sure if this is the correct use of 'Glass Ceiling'.

Anyway, have you tried fundraising sites like GoFundMe, or Kickstarter? Have you shown your project on various social media sites, like Github/Reddit/HN? Have you tried Grant Search Sites like Grantspace? Have you considered if your technology would be useful to a local business? I have had success in this route myself, I had a Local VFW contract my services to build a local website to track their dart-league scores, it was a great opportunity. Have you hosted a trial for a select group, and received feedback? This can be great fodder for bargaining.

There are a lot of options, but patience and perseverance are your best friends here. It's not easy. Period. You have to work hard and fail through many avenues before you can rightfully join the ranks of 'failed startups'.

Very bizarre...

Just reviewing some of the categories and runners up;

Application & Data Tool of the Year Javascript was pitted against frameworks made in Javascript? It was also judged against HTML5? Against Java? Against MSQL? All of these tools can (quite literally) be used in the same application, cleanly.

Advice for next time; If you're going to join the hordes of writers ranking technologies, research your categories better, and include some sort of metric that they are judged against, other than your own experience. Because some of these tools would be supremely useful over the winners, in the right environment.

Solid self-awareness here by the team at Streetmix.

It's touched upon in the sentence;

"Discord, unlike Slack, is not as well-known in the corporate world. Although marketed to gamers as a voice chat alternative, its core service (much like Slack’s) is the chatroom."

Slack is a pretty solid solution for large enterprise teams, needing support for different parts of their organization. It's akin to a 'living wiki' of sorts.

Discord is much more ad-hoc, and less feature rich, which makes it a better place for smaller teams with a need purely for communication and coordination.

There are great collaboration tools that I could envision discord working with, for smaller teams. Like VSCode's Code Share.

My critique of these services in general comes in the same flavor that my 'open office' argument does. Lines of communication are critical, but should be used judiciously. Everyone available all the time for conversation and questions, without recourse, can lead to the type of environment where everyone holds knowledge in their head, instead of writing it down, or documenting it.

I've always believed that the best tool we have now are scheduled, web-based team meetings with screen sharing. They have to be scheduled, and one person is always leading a topic. It's targeted, and there is usually a goal. Concise. I've always liked that.

Google, and S.C. as a whole are really the 'Grandchildren' to the 'Grandpa' American Political system.

Changing the status quo in that environment is like steering a barge through honey. They're so resistant to change, it's stunning.

While this is marketed(?) as a anti-hacking tool, this is just a reverse proxy. It's a good step, and it doesn't smell too badly of 'google'. They aren't offering to 'host' political party data, as far I can tell, and they aren't running voting machines. They're just stepping in as the 'kid next door that is good with computers' so that Grandpa stops getting Russian viruses.

Exploring the use cases for 'AI' nowadays is more popular than defining it, I would like to see more dialog around discussing what constitutes 'Artificial Intelligence'.

As with most of us in this field, I hate the often-overused term. More aptly, I believe these are 'Decision Making Machines', they really don't do Critical Thinking, or Abstract Thinking, they simply have a set of given inputs, and a function that decides which set of those given inputs matches a given output.

There's no better example of this than Watson, and Medical Diagnosis. That is not AI. We have an enormous database full of years of medical history. From that history, we draw conclusions about symptoms. For a patient's given symptoms, we are often able to crawl back up that ladder to the diagnosis. Yet we still call Watson AI.

Sometimes, these decisions are chained, giving the illusion of Critical Thinking, but at the end of the day, they are goal-driven algorithms.

I believe there's a really interesting parallel between Facebook and TPB.

TPB did everything in it's power to push the stance that, 'We run the site, what users use our site for is not our responsibility'.

Whereas Facebook takes responsibility for its' users, and the impact is has on them.

The interesting thing about this model is that; When a site shirks responsibility for its' users, it has no interest in moderating who joins / who can use it TPB doesn't even have mandatory 'accounts'. You would think, if Facebook was truly interested in supporting a healthy community of users, it would impose restrictions on who joins.

It would be the most evil win/win in technological history. Facebook running background checks on potential users before joining. All that data? They could vet potential 'threats' to their 'community', and sell what their allowed to retain to marketing companies. Now you have a heavily moderated social networking site that lives up to your 'The Good Place' reality.

I agree wholeheartedly with your points.

Responsible, experienced engineers should teach newcomers that experiencing self-doubt is part of the process in maturing. I had the fortune to work under a mentor in my original position in this company that wasn't afraid to pull his assigns in to help solve problems he was working on.

Seeing that people of his caliber still acknowledge the value of second opinions and eyes really helped me understand that this business is not about the individual bodies entombed in cube walls, but a complex effort of many teammates.

After I embraced that, I learned where I could, and contributed where I could, and I personally feel much more productive as a member of my new team because of that. Do I still feel like I don't know things? Definitely. I also know that it's a big field and we all won't know everything.

My next goal is to find my way to a leadership position so I can perpetuate this kind of mentality downwards.

In our field, unknowns and chaos are unavoidable. This profession offers a lot to be cynical about.

Establishing clear lines of communication and trust is hard work. As hard as development. People are unpredictable and diverse. It's like writing a different language for every part of the application. The syntax you use to communicate between people changes.

As the world continues its descent into reliance on technology and software, more people are faced directly with generating requirements for developers to implement, and right now the gap between the two world is still rather large. The best way to help close those two worlds in, is not to point a finger at those around us and complain they aren't capable in this environment, it is to help build the bridge so that the person who is in your position next does not have to face those same realities.

| But if they do owe it, they do have to pay you.

That should be true, I agree, however it is not. Here is a source outlining the laws in each state.

http://ask.legalsolutions.thomsonreuters.info/cc-nto-vacatio...

For example, in my state, the law is; Unused vacation will not be not considered wages unless an employment agreement, union contract, or company policy provides that vacation will be cashed out on termination of employment, in which case an employee may have a claim for vacation pay if the terms of the agreement, contract, or policy are not followed (Chrin v. Cambridge Hydrodynamics, Inc., 2003 WL 25754809 (N.J. App. Div. Dec. 30, 2003); see New Jersey Department of Labor & Workforce Development: Wage and Hour Compliance FAQs).