HN user

maxFlow

143 karma
Posts12
Comments49
View on HN

Coding in the flow state has always been appreciated, it's just been surpassed by a new technology.

Your brain can still "just click" with agentic coding. But it will have to be at a higher level of abstraction. Perhaps the "click" feels different, and will take some adjusting to.

Hi Peter. Thanks for your AMAs, always a source of great value.

A couple of questions from me:

1. Which would you say are the top spececializations within tech that employers are most willing to sponsor visas for nowadays?

2. Would you say the willingness to sponsor tech professionals has lessened somewhat as of late? Given the economic climate, opportunities to hire remote globally, etc.

3. Are there any impactful immigration reforms we should keep a watchful eye for, vis a vis the 2024 US presidential election?

Thanks & all the best.

It would be useful to give more clarity around your thoughts on the "Data Engineer" role. Is it also in decline? Is the market as a whole in a relative decline?

My thoughts: the tech job market as a whole has been in decline, as unanimously observed. There may be some signs that the slowdown is abating though. Next 6-12 months will be key to see how DS and DE rebound (or not).

Most businesses' data engineering needs have been solved or will shortly be solved by managed services that 10 years ago would require endless and extensive self-built ETL pipelines, databases and tools. For the exceeding majority of businesses, this means they can and should focus on building capacity for business logic, analysis and predictions instead of data engineering.

Could not disagree more with your take of "DE demand will decline due to DE needs being already solved for most businesses". Apologies, but have you ever worked as a data engineer or even close to one? Pipelines break, requirements change, businesses expand, and infrastructure needs to be managed and optimized, etc. ETL processes, in the wild, are decidedly not one-off affairs.

Why write? 3 years ago

Thanks for mentioning this. I don't get to do much writing in my time off work, since that time is devoted to (more) coding to try and make my side project into a business. So what I do to compensate is I make an effort and write all commit messages, descriptions, code review comments and documentation as if I were writing for an audience of top engineers auditing my work. I find this approach makes me enjoy the process more, be more deliberate about my actions, and how I communicate them. Even a dull runbook can be beautifully written.

I'm in a similar situation to yours, and I've kind of settled on a similar approach: put out the most pressing fires and reassess 3-6 months down the line. At first I had thoughts to refactor but now I just don't want to deal with the complexity, not to mention I have end-users pressing me. I'll have to tolerate the spaghetti, but at least make it produce correct output.

Best of luck with the migration to DO :)

I've found that Statistics is one of those topics that changes your world view about everything. You can consider pretty much any issue statistically, and that will enrich your perspective significantly. In that sense, Statistical Rethinking will help. However, it's a book on Bayesian stats, it's quite dense, and examples are coded in R. It may be overkill for web app metrics interpretation. For that you may be better served with basic stats & inference, frequencies, descriptive statistics, percentiles, basic distributions, data visualization (e.g., trend lines, scatter plots, boxplots, histograms), etc.

To be clear though, Statistical Rethinking is a beautiful piece of work. You can check out the author's lectures[0] and see how much they suit your needs.

[0] https://www.youtube.com/playlist?list=PLDcUM9US4XdPz-KxHM4XH...

I speak four languages out of necessity, not by choice.

I also speak four languages and mostly agree with your take. I'm native in Spanish; English I learned gaming, and reading Tolkien as a kid. The other two, German and French due to a combination of self interest, education and travel. While I often fantasize about picking up a couple more (namely Norwegian and Japanese), I quickly become disappointed as I go through the motions all over again. It's a huge mental effort for a seemingly low _tangible_ ROI.

Sure, listening to music or reading in the target language and understanding most of it is quite the magical experience, probably similar to what cracking a secret code feels like; but there is no practical gain to it afterwards. It's a bit like reading/writing poetry: an intense but ephemeral enjoyment. More of an art form than anything else really. Unless, of course, you find yourself immersed in the language by way of relocation, then it truly does make sense to learn it. I do get your point with Dutch though: now you've got to figure out a fifth system for conveying an idea you're perfectly capable of saying in four other systems; it gets tiring.

I've been comparing it with programming languages lately. The question often pops up in HN: "what's the best programming language to build a backend in?" -- imagine you already can build a great backend in Python/Go/TS but you start picking up Rust only for the purposes of building said backend, what's the point? Just use whichever language you know best and build the damn thing already. Simple enough right? As is often the case though, this type of analysis is superficial; you may build a fantastic backend in say, Clojure, but then miss out on the opportunities a more popular language with a larger community may have to offer (e.g. Python). Writing Python may not necessarily provide general cognitive advantages over writing Clojure, but it will give you easier _access_ to the entire ML ecosystem, for instance. Does being capable of using more powerful tools help develop cognitive advantages?

I only read the abstract, but even if _Bilingualism Affords No General Cognitive Advantages_, learning a second language, English specifically, has unquestionably changed my life.

We are already looking to move to other places and created a university-wide google sheet where we compare different countries and cities around the world for those kinds of categories.

Could you share a few places at the top of your list? As you yourself pointed out, it looks like the same story is playing out everywhere one looks, especially in the West.

Given the current situation, the more I think about it, the more I conclude that going fully remote from a low-COL location may be the better answer. Unfortunately I think this complicates things for founders to raise.

Also, this makes me think how utterly unprepared Germany is to receive all those immigrants needed to keep the economy working.

Best of luck with your plans.

These days I do 20 minutes a day of mind control which involves nothing more than counting down from 9 to 1 repeatedly.

If it works for you that's great. However, I would say approaching meditation from a "mind control" exercise such as "counting down from 9 to 1 repeatedly" goes counter to its purpose and will hinder its full potential. If you're willing to experiment with other methods, try letting your mind wander and don't let it stick; just experience the ebb and flow. Twenty-minute sessions sound about right for me as well.

And also the monster investor happens to be a university. Why does a university need to manage such a huge endowment ($150bn of assets)? Further, why does a university end up investing in the rental market? Seems to me these are both standalone loop holes of our current economic system that will eventually need to be dealt with if we want to fix both education and housing. Until then we'll keep witnessing all sorts of distorsions and crashes.

I'm a Python dev, learning Rust out of interest, but also considering its viability for Data Engineering. It would be helpful to know the equivalents of the following libraries: Pandas (Polars), Prefect/Airflow/Orchestration, Flask, SqlAlchemy, psycopg2, requests, BeautifulSoup.

Thanks!

1. Could you share a bit about your stack? I'd be specially interested about the data engineering side of things if possible.

2. As a SWE, how deep into biology/genetics concepts have you had to go during your tenure?

This was one of the worst written, saddest pieces I have ever read. It's full of idiotic commentary, unjustified praise and just plain ignorance. Out of the dozen+ remarkably stupid quotes I found I'll share this excerpt:

That’s when SBF told Sequoia about the so-called super-app: “I want FTX to be a place where you can do anything you want with your next dollar. You can buy bitcoin. You can send money in whatever currency to any friend anywhere in the world. You can buy a banana. You can do anything you want with your money from inside FTX.”

Suddenly, the chat window on Sequoia’s side of the Zoom lights up with partners freaking out.

“I LOVE THIS FOUNDER,” typed one partner.

“I am a 10 out of 10,” pinged another.

“YES!!!” exclaimed a third.

What Sequoia was reacting to was the scale of SBF’s vision. It wasn’t a story about how we might use fintech in the future, or crypto, or a new kind of bank. It was a vision about the future of money itself—with a total addressable market of every person on the entire planet.

“I sit ten feet from him, and I walked over, thinking, Oh, shit, that was really good,” remembers Arora. “And it turns out that that fucker was playing League of Legends through the entire meeting.”

Lo and behold. The intricate, behind-the-scenes scheming of the highly educated financial elite, revelead at last. All predicated upon the grand vision of... buying a banana with your "super-app". No wonder we're going through a much needed correction, some people need to be weeded out of decision making roles ASAP.

Back in early 2021:

As the newest senior leader on Shopify’s engineering team, Polinsky will lead Shopify’s efforts to build its e-commerce infrastructure and will lead the company’s efforts to double its engineering team in 2021 by hiring 2,021 new technical staff. [0]

“Engineering hiring is probably the biggest limiter to Shopify’s growth,” Harley Finkelstein [President of Shopify] said in an interview after the results. The company had hoped to hire 2,021 new engineers this year and current conditions mean “obviously, it’s hard to do that,” he said, without disclosing the number that have been hired. A tight market for talent is a worry for many industries but none more so than technology. Universities are not producing enough new engineers and the entire sector is competing for graduates, Finkelstein said. [1]

As of today they only have 10 open positions in the "Engineering & Development" department, the majority of which are for Staff/Tech Leads. [2]

Perhaps it would behoove Shopify to put a bit more thought into their hiring strategy; rather than just wanting to inflate engineer numbers as a vanity metric via jingly campaigns (2021 on 2021). "Just a number on the page".

[0] https://techcouver.com/2021/01/06/shopify-to-double-engineer... [1] https://archive.ph/RMzlm [2] https://www.shopify.com/careers/search?search%5Bkeywords%5D=...

It's similar to data engineers thinking they will do real ML some day and their data job is just temporary...

This is unfair to DEs, not to mention inaccurate as of 2022. The statement assumes DEs are yearning to do "real ML" while suffering in their "data job" and waiting for a chance to move "upwards". Data engineering is a terminal career path by itself which may or may not support a dedicated ML function. In fact many of my DE colleagues switched from DS/ML/DL citing "model fatigue".

Now, "ML engineers" spending 90% of their time producing a workable dataset to get to the "real ML" is a different discussion.