Tackling Webdev as a Bioinformatician: why is it so hard?

https://jessimekirk.com/blog/web_for_biofx/

by LifeIsBio • 6 years ago

139 188 6 years ago

edoceo• 6 years ago

It's awesome to see an article get to the five why's. This is the Way.

LifeIsBio17• 6 years ago

This has been my first experience with it and it seems like a great tool to ensure structured thinking. It's easy to give up a level or two deep and just say, "Oh, it's probably because..."

cpach• 6 years ago

Very thoughtful article. There is so much accidental complexity in software.

LifeIsBio17• 6 years ago

It's kind of ironic. While I've been thinking about the complexity of software/webdev for a while, I was feeling a little unsure about posting this particular link because I wrote it in a hasty, semi-embittered fit of annoyance. Regardless, thanks for the read. :)

tejtm• 6 years ago

hopefully this observation won't be taken too harshly...

I have also worked in programming for the sciences my entire career, multiple decades, would not want to do anything else. There is an awful lot of actual complexity out there especially in the life sciences which is inherently messy ... a system of exceptions ... Sometimes looking at the stuff that has to happen now, just to give information away for free it is staggering. The only conclusion I come to is complexity envy, Because I can't believe all this inevitablely became necessary. Yes there are bad actors and good security is very hard, and much more so if you are transacting money, but in other cases you can't really steal what is being given away but now there is the expectation that all this extra stuff is par for the course it you are putting on the web.

MattGaiser7• 6 years ago

Documentation is frighteningly bad across the industry and nobody has a good solution yet other than hoping that someone else will do something about it.

And if you think Django is murky, try Spring :). Django is one of the easiest frameworks to work with simply because it includes so many functions you would otherwise need to somehow integrate. And even then there are still these issues.

Good post.

bayesian_horse2• 6 years ago

I think Django's documentation is fantastic. 90% of the time I have to look up something, google redirects me there anyway.

MattGaiser7• 6 years ago

Django's documentation is great for developers who know their way around documentation like that as it is very comprehensive. It drills down nicely and is quick and easy to understand if you otherwise know what else is going on.

What it misses is mapping that documentation to the specific problem a person wants to solve or the feature they want to build. When I was learning Django a couple years ago I found the examples and best practices sparse.

Larrikin• 6 years ago

I was so disappointed with other frameworks since Django was my first. I've come to realize this is widespread as I've learned other languages, but every time I start a new project and setting up the important boilerplate I'm constantly thinking in my head "Django would have just included this or there atleast be a battle tested plugin"

MattGaiser7• 6 years ago

I think it depends on who you are as a dev. A lot want to create from scratch or have it be a handcrafted product, which I can respect.

I just like producing cool stuff and don't want to be manually running SQL queries in both dev and prod.

LifeIsBio17• 6 years ago

Django has been a joy to work with. As other's have mentioned, I'm a huge fan of their documentation. I think they do a great job of letting you quickly hop between Tutorial and Reference docs. Lots of links help me quickly dive into more detail when I need, then jump back out.

aserafini• 6 years ago

Is there a better documented framework than Django? I am not aware of one.

MattGaiser7• 6 years ago

My original comment misses some nuance.

bayesian_horse2• 6 years ago

Trying to make the opposite transition, my quibble with web development is that much of the work seems so pointless.

I hope that after this crisis we will care less about making people click ads than about learning how stuff works in biology.

MattGaiser7• 6 years ago

We still run into the problem of how to pay for content on the web. Do we really want everything to either be volunteer or sponsored by someone?

Mati_Roy• 6 years ago

it's a public good that will likely be underfunded if we stick with volunteers and sponsored

except maybe if status is a sufficient payment?

mayyybe good if altruistic ideas are more likely to propagate under those incentives?

LifeIsBio17• 6 years ago

I heard there's a great new site that can help you learn about bio if you're interested... You know, if the site will let you login, that is. ;)

TheOtherHobbes2• 6 years ago

[Looks like you've read 0.1% of this page! Enter your email into this popup to receive emails you don't want about things you're not interested in. Also turn off your ad blocker, because we're not a charity, you freeloader.]

Animats• 6 years ago

Trying to make the opposite transition, my quibble with web development is that much of the work seems so pointless.

That's the problem with web development today. Something that not only should be straightforward, but used to be straightforward, is now very complex. Partly because it involves connecting a large number of complex and constantly changing packages.

The last time I wanted to do something with an API, I coded up the API in Go and ran it from FCGI to get multiple copies. Easier than all those "frameworks". Probably faster, too, since Go is hard-compiled.

realharo3• 6 years ago

used to be straightforward

As someone who remembers having to do weird CSS tricks for really basic layouts (negative margins? hidden extra elements that don't make any sense, but cause other elements to move?), manually modifying the UI made up of jQuery plugins that require a very specific prescribed HTML structure to even work, or doing significant adjustments for 3 or 4 different browsers, I definitely dispute that claim.

Web development right now (at least the frontend for sure) feels like a breeze compared to about 10-15 years ago.

Steps 1 and 2 of the whole process may have gotten more difficult, but steps 5 through 100 are a lot easier these days.

juskrey• 6 years ago

Welcome to real world, academic Neo.

LifeIsBio17• 6 years ago

As my PI used to say, "If it were easy, it wouldn't be fun. And we wouldn't have jobs."

CameronNemo2• 6 years ago

looks like it got hugged

https://web.archive.org/web/20200507060013/https://jessimeki...

LifeIsBio17• 6 years ago

Oh no! It's github pages and totally static, so it should recover pretty easily, I hope. Thanks for the archive though!

saagarjha5• 6 years ago

Looks fine on my end. Good choice on using GitHub Pages by the way; had you not I’m sure half the comments would have been about how you would have it so much easier and moral it would be had you used that ;)

CameronNemo2• 6 years ago

Must have been a blip on my end then.

saagarjha5• 6 years ago

All programming has a barrier, but I think web development is a unique situation where there’s a lot of complexity because it’s so open to abuse. If you don’t do HTTPS right bad things can happen, so setting that up is hard. If you process payments bad things can happen, so setting that up is hard. If you send people email bad things can happen, so setting that up is hard. With things like game programming the worst that can happen is your game is slow or buggy or laggy but usually you’re not exposing it to the huge, often malicious audience. With web development you make one mistake and someone is stealing thousands of dollars from you and running cryptomining on your VPS while defacing your website.

jamil74• 6 years ago

Yeah and it results in so many small, seemingly insiginificant things that can bite you. Like the last point in which I guess the author had hardcoded their redis uri rather than using the environment variable provided by Heroku. How would anyone know about that unless they had been told by someone or been bitten in the past or had some understanding of how Heroku worked underneath.

Edit: from the comments it's clear that OP didn't actually hardcode the connection string.

korijn• 6 years ago

To be fair, haven't we all (eventually) learned to support environment configuration by messing it up in similar ways at some point?

golergka• 6 years ago

Can't really agree with that. Although most of my experience and expertise is with mobile game development, I think that I've read at least 5 blog posts and articles that explained that all connection strings, usernames and passwords (aside from something really trivial for local testing) should always be read from environment variables and never be hardcoded, and arguments were convincing enough. So, even if I haven't worked with Heroku before, I would look for a way to get this url in a service-provided environment variable.

Also, I would look to configure all components to fail loudly, so that if something wouldn't connect to Redid, my server just wouldn't launch.

I'm not trying to say that I'm better or smarter than OP: I've actually tried learning bioinformatics, but dropped out of uni 14 years ago, realising that my brain is just not created for anything biology-related. Rather my point is, many of these problems (and problems I've seen junior developers struggling with) are not some arcane knowledge that you should just learn by heart (like all of those awful genus names in Latin, or all the enzymes in the dreadful ATP scheme that I still have nightmares of sometimes), but a logical extension of more basic and universal principles that actually make total sense once you think about them.

29athrowaway• 6 years ago

If you redirect users, bad things can happen. If you consume third party dependencies, bad things can happen. If you handle sensitive or personally identifiable information, bad things can happen. If you are not aware of privacy protection laws, bad things can happen. If you configure S3 buckets, bad things can happen. If you deploy static assets, better exclude sensitive information like version control data or bad things can happen. If you construct database queries, bad things can happen. If you manipulate files, bad things can happen. If your site can be put in an iframe, bad things can happen... and the list goes on and on.

astrobe_• 6 years ago

With things like game programming the worst that can happen is your game is slow or buggy or laggy but usually you’re not exposing it to the huge, often malicious audience

Not my domain of expertise, but cheating and lags in online real-time games (like FPS games) can ruin a game quickly. Also, in-game shops, currency, micro-transactions probably are targets for crackers and exploiters.

saagarjha5• 6 years ago

Generally that’s not something you run into until you’re actually much deeper into the field.

LifeIsBio17• 6 years ago

This is a fantastic point. Payments in particular have been intimidating for me. Nobody wants to pay for something and then have some goofy mistake on the backend mess up processing it. Or even worse, make them feel like their credit card is susceptible to theft.

I don't know how to explain it, exactly, but worrying about this definitely made development more difficult, even when initially setting up and testing out.

saagarjha5• 6 years ago

If it makes you feel any better, I have been programming for almost half a dozen years, the latter portion of which I spent learning how to do so formally, and I am deathly frightened to work on any of the things you had trouble with. I’ll take securing an ancient obfuscated firmware binary any day over trying to secure your website :)

ensiferum• 6 years ago

So because the webapp is exposed to the public over the internet and there's a risk of hacking the the developer story can be poor? That doesn't sound right. The developer story is just poor because the frameworks and developer Tools are poor, badly documented and don't work well together holistically. Part of this is just due to the continuous churn in web development that creates insane amount of busywork and makes sure that almost all of your available documentaton is always put of date. i find that most of this "complexity" that the author mentions is just man made.

mytailorisrich• 6 years ago

I don't think that this is what makes it hard.

Setting up https, or online payments, etc. is actually quite simple. People with engineering or scientific background have most likely faced much more complex tasks. Security and safety aspects can be more stringent in many other industries, including in software development.

The difficulty of the web, I think, is the proliferation of standards (or not so standards) and technologies: networking, https and friends, APIs (payments, etc), html, css, javascript, php, ruby, java, aws, react, vue, redis, SQL, etc. And they are all evolving independently all the time. This is a right mess and can be overwhelming.

Cthulhu_• 6 years ago

But things like HTTPS, payment processing, emails / account management and even game programming, all of these have already been made easier - nobody has to build any of that on their own anymore, at best they have to configure it.

To make a safe website, stop wanting to do everything yourself. Use a webhost that doesn't give you SSH access, or a service like Netlify.

prox4• 6 years ago

I just checked the author’s website, and he could have just used an installer to get him a wordpress website with an elearning system plugin attached to get to the same result. Buy an SSL and he would have been done. Any other path is just making it harder if it is not his field of expertise.

AdrianB1• 6 years ago

And linking all these heterogeneous pieces together and configure them to work is like cat herding: each component is so different and the developers are so proud of their "innovative" way of doing things that one cannot master it. A few years ago I started to believe that modern web developers are more about showing off and hipstering than having a genuine desire to make things better.

ml_basics• 6 years ago

A great read!

KaiserPro3• 6 years ago

I 100% feel your pain.

I used to work for a very large financial news company. They were all javascript. I though to my self: I learnt JS in 2008, this should be easy (it was 2014). Apparently very hard.

HTML isn't HTML anymore JS is forced everywhere, and everything has a different name.

Anyway, I disappeared back into HPC/VFX/machinelearning, to re-appear into webdev in 2020.

I was forced to do some front end work as part of a bootcamp. It took the best part of a week to make a bit of text a link.

The template library was so far removed from anything googleable, there was no documentation, worse still no style guide or example style guide.

I am glad I don't do web GUIs, they are hard and nasty.

woutr_be• 6 years ago

Honestly, they're only hard and nasty if you make them hard and nasty. A text link is as easy as <a href="">...</a>.

You can of course use React with 2 different libraries to achieve the same goal, but why would you do that.

realharo3• 6 years ago

React by itself isn't difficult either. Anyone who has an hour or so to spare can go through the tutorial https://reactjs.org/tutorial/tutorial.html.

They need to rewrite it to use hooks at some point, but it's still a pretty good intro.

LifeIsBio17• 6 years ago

JS is forced everywhere

I think this has a lot to do with it, interestingly. The web has become so ubiquitous, and the major players are so resourced, that the barrier to entry can seem really high. Maybe I'm wrong, but my perception is that potential customers wouldn't take a second glance at my sight if it were plain html with minimal CSS and didn't render perfectly on mobile.

I am glad I don't do web GUIs

Same.

noahtallen• 6 years ago

You can make a very lovely site with just HTML and CSS! CSS is extremely powerful by itself. As long as you don’t need any dynamic content or dynamic UI, you can likely avoid JS entirely :)

onion2k5• 6 years ago

It took the best part of a week to make a bit of text a link.

This highlights the problem with web dev quite nicely. Not doubt you weren't really just making some text in to a link. I know the sort of thing bootcamps do, so you were probably installing and learning React, displaying a set of components, implementing React Router, and figuring out how to update app state based on the browser's History API. All with a template library that was a terrible choice for learning (because it didn't have docs). You could summarize that as "making a bit of text a link", but that's glossing over the details.

What you were really doing was learning component orchestration, reactive state management, and a browser API. Why wouldn't that take a week?

So the problem with web dev is actually that it's really, really easy to trivialize. You can make hard things sound like they should be very straightforward because they look a lot like something simple from the past. And when people find that they're not really doing the simple thing any more they complain because the new, harder way of doing something (which is actually doing something completely different eg reactive state management and in-browser routing vs asking the browser to do a GET) is hard.

dynamite-ready4• 6 years ago

And there's the problem...

"...component orchestration, reactive state management..."

Webdev tool makers are sadists, and many of the 'thought leaders' in the field have Stockholm syndrome.

Some aspects of web development certainly are hard (security especially).

Frontend development however, does not need to be as hard as it currently is. At the very least, I think there's an opportunity to simplify that layer of the stack, and give the control back to design professionals.

Dreamweaver to my mind, represented a more productive direction than React currently does.

bokwoon• 6 years ago

If a simple link to another page requires component orchestration and reactive state management I'd say it is a bit over-engineered.

KaiserPro3• 6 years ago

The actual task was this:

There is an Icon, and some text that reads: "you are on a trail find out more" The find out more is a link, make sure all the text _and_ the icon are included in the link.

So its not creating a new state or widget. its literally moving a href tag.

Well it would have been, if it wasn't for the template engine, and what ever the state engine was/is.

Now, if this was QT creator, I'm pretty sure I could have done this task in about 4 hours (I don't know C++ very well and I've used QT about 3 times before, each for less than a day.)

Hasu2• 6 years ago

It's funny, when I was first learning programming 15 years ago, the thing I heard everywhere was "webdev is easy and for noobs", and sure, it was 15 years ago and webdev has come a long way... but when I interview the fresh-out-of-college candidates for junior positions, there's still the elitism of "If you aren't managing memory and flipping bits you aren't a REAL programmer."

No, most web developers aren't implementing the most complex and intricate algorithms or 'close to the metal' or whatever other gatekeeping people want to claim, but that doesn't mean it's easy work. I've never had a job where I didn't see some green engineer laugh off a feature request with "Oh, that's easy, it's just calling an API and putting the information into some HTML, what's so hard about that?" and then getting stuck when their first implementation fails QA, or the product manager explains that the customer needs it to be ever so slightly different, and the complexity of those edge cases turn out to not be so easy to deal with as the world of perfect computer science might suggest.

Is it hard in the same way? No, I will fully admit that coming up with a novel algorithmic solution requires a different mindset and brilliance that very few, if any, really good web developers I've met really have (although I'm certain there are people who excel at both, they're rare enough that I haven't been lucky enough to encounter them). But being really good at building a solid, secure, modern website isn't easy either, and it's also valuable.

This post focused on the complexity of the tooling, which yes, is definitely a thing, but if you don't know web development, and you know that you don't know web development, why are you attempting to build something using complex high level tools designed for and used by advanced web developers? Then again, if you don't know web development and you don't know what you don't know, and you've just heard 'Django is good' how would you know to maybe pick Flask or something simpler to understand?

I don't have an answer or a larger point other than this: writing software is difficult and complex and none of us should look down on anyone else because what they do seems easy on the outside.

guggle3• 6 years ago

I wouldn't count Flask as simpler than Django. These 5 lines hello world examples are incredibly misleading.

eximius• 6 years ago

Django is an enormous project that dwarfs Flask in size and complexity.

padthai• 6 years ago

Exactly.

You end up cobbling together half a dozen plugins (auth, forms, orm, admin) and adding dependencies that, albeit useful, are complex themselves: Celery, SQLAlchemy, etc.

travolter• 6 years ago

Http is just clearly not made for the current state of the internet. It was obviously made for much simpler uses cases but has organically grown together with the internet. This has led to numerous hacks and workaroudns which often became the defacto standard to do things on the web. It's not necessarily an easy problem to solve, client server communication is hard. But http protocol seems like the least effective way to do it, but almost impossible to replace now.

TheOtherHobbes2• 6 years ago

Applies even more so to CSS and JS.

All the above are quick hacks that never really did the job properly in the first place. So they're propped up by a creaking pile of frameworks and other support systems that try to mitigate their awfulness, but mostly just add extra awfulness of their own.

Meanwhile backend systems - core server and db infrastructure, user authentication, computing resource allocation, payments, media and content management, privacy, and project build systems - are even more byzantine and non-standardised and more or less have to be retooled from scratch for every project - a huge job which duplicates an incredible amount of work across the entire industry.

tannhaeuser2• 6 years ago

Http is just clearly not made for the current state of the internet

As a corollary/reformulation of that sentence you get that what businesses are trying to do in the web is clearly something it wasn't made for. Yet businesses want to free-ride on the web's success, and are ruining it.

maelito2• 6 years ago

or the “right” framework to choose, changes from year to year

I've been using React in all of my projets since it came out in 2013. That's 7 years. React's (+ node for backend if needed) been one of the recommended for at least 4 years.

And lots of things have changed ! Using modules and building complex apps now with Webpack & co is so much easier than before. Not needing to support IE is also a blessing.

The piece doesn't say how mobile dev would be easier. Or how embedded dev would be easier.

It just says : dev is hard. Yes it is, it's a a full job.

Hopefully nocde solutions will reduce the gap for some use cases.

simongray• 6 years ago

React - as a library, but especially as a UI paradigm - has clearly won the frontend war for now, taking the mantle from JQuery. I don't even use React directly, just its DOM renderer through a ClojureScript wrapper (reagent) and the high-level concepts are still basically the same.

Still, when I glance at the JS frontend world people are still looking for the next hot frontend framework all the time, even if many are doing something similar to React. Going by github stars, Vue is the biggest one now. There's also Angular, Svelte, Ember, and Aurelia.

maelito2• 6 years ago

You're right. React and Vue are hard to compare though. The fact that React has no router, while react-router has 40k stars makes it a very imprecise metric.

tannhaeuser2• 6 years ago

Is the author asking why putting up a paywall for Django hosted on Heroku with Stripe integration on what should be static content is difficult (for stories related to overcoming idiosyncrasies in using Python for bioinformatics, or all things)? I'd like to give a rebuttal, but the site (the Minimal Errors theme for Jekyll?) does click-jacking (blocks copy/paste) so I won't bother. I guess the answer is right there: webdev (and bioinformatics) is full of incidental complexity and zero-sum "solutions" advertised to you when the original purpose of the web is just publishing structured documents via HTML/SGML (even though I'd agree Python is great for learning, and Heroku has at least a reputation for being the easiest cloud hosting option out there).

LifeIsBio17• 6 years ago

The site that's giving me problems isn't the site you were on, but another site I'm building:

https://mycodestories.com/

It's not static; it's a platform for helping people (life scientists) learn to code.

In general, Heroku has been a great platform. The fact that you can just `git push`, and your dyno rebuilds and restarts and your changes are live is so cool.

bpizzi• 6 years ago

And it looks like you've just scratched the surface, you're not finished encountering walls yet.

Hang in there, there's a relative important chance that the fog will clear for you in some months/years.

The hard truth is, software engineering today is made of layers upon layers of abstractions. You have to dig threw those layers, understanding sufficiently the inherent and different complexity of each one before jumping to the next. And then finally it'll make sense, and new layers (there's a new layer every year btw) will be easier to grok.

If it helps, we've all had to go threw this learning process. All of us. However I don't think everyone is made for enjoying this process, that's another hard truth.

The real difficulty is sticking to the learning process while everything and everyone is screaming at you "Go! Build something! Ship Something! Show something you lazy b".

valera_rozuvan• 6 years ago

I start getting my friends up to speed by giving them just enough JavaScript + HTML to write a Tetris clone. Of course - I guide them along the way, answer any questions they have, and push them towards the right direction if they get stuck. If they can get through it - then they can make it in the world of WebDev, and software engineering in general.

The main thing to remember is that you can only show the way - the person in question must actually step forwards and be motivated enough to go through the learning process. Otherwise - you need to think of another career. Programming is not for everyone IMHO.

LifeIsBio17• 6 years ago

A little tangential, but learning to code through video games has always seemed like one of the best ways to do it, particularly with friends.

ertian• 6 years ago

I feel like interacting with there real world, and with users in particular, just inevitably leads to orders of magnitude more complexity.

You can see that even in simple cases. Writing a basic single-propose script is trivial; it may end up being just a line or two. As soon as you decide to make it general and reusable, though, complexity explodes: positional arguments are confusing, better name them. Users are gonna expect a familiar format. Need a library for that. Reasonable and understandable names are tricky! What arguments are required versus optional? What if _this_ option is specified without _that_ one? Gotta add proper errors for every bad state! What if the user starts using globbing? Or wants to specify multiple inputs and one output, or vice versa?

Your three-line script is now 100+ lines. And you're not even worried about hostile or abusive users yet!

Writing code for the web is all that--and much more besides.

gexla2• 6 years ago

You start with HTML. Write your HTML in an editor. You now have what we started with and I won't complain about your un-styled site. Right now I'm working on a simple personal thing with unstyled pages and a static site generator (Hugo.)

The first Google search for hosting should be fine. There should be a simple tutorial for uploading your HTML file.

That's all there is to it. Beyond that, maybe you're just trying to bite off more than you can chew because you think you need things. It would be like going to ACME engineering supply and grabbing a bunch of stuff off the shelf because it sounds good. Then you get home and you have no idea what these things do.

A lot of this stuff looks deceptively like it all goes under the category of "web development." Sending an email isn't web development. That's dealing with a different protocol and different server software. You don't need to go far down that path before you start to wonder how any of this mess can work at all. I'm not even talking about web development, I feel this way all the way down to the metal. Yesterday I spent hours trying to troubleshoot an issue with Ryzen 3rd generation chips apparently having a problem when you load up all the slots with RAM (I threw in the towel because I can get by with half the RAM.)

That has always been the case. Software hasn't gotten worse, it has always been bad. But functional. Maybe it has become more complex.

Cu3PO42• 6 years ago

Anecdotally I have a Ryzen 3600 on a B450 board with all RAM slots populated and have yet to notice an issue that isn't there when using only half the RAM.

gexla2• 6 years ago

Probably the wrong place for the discussion. Do a search and you'll find all sorts of people with this issue. I had the problem after I first built the computer. Then I was able to boot by lowering the speed of the memory in the BIOS, now I have the problem again after a reboot. I don't know what changed. There's hundreds of threads which cover a lot of ground on this issue. I'm just going to make this a long term project to understand the issue until I finally cave and buy something which has already been built.

Edit: Also 3600 with B450 board. Kingston DDR4-2666 Ram (8x4) which is all the computer store here in smallish city in the Philippines had in stock.

goatinaboat2• 6 years ago

Because it's artificially made hard by people who enjoy complexity for its own sake. The article is about implementing a shopping cart, as far as I can make out. Well people used to write those kinds of websites, that perfectly well enabled users to browse products, fill a basket, place the order and make a payment using nothing more than notepad.exe.

Some software is legitimately hard to write, even some websites. But this isn't an example of that. The author has been sent on a wild goose chase.

LifeIsBio17• 6 years ago

I'm curious where you got the idea that the site is a shopping cart?

CodeStories is a site to help life scientists learn to program by giving them problems, then letting them submit scripts and then auto-graded their scripts for correctness.

goatinaboat2• 6 years ago

CodeStories is a site to help life scientists learn to program by giving them problems, then letting them submit scripts and then auto-graded their scripts for correctness.

Because it’s a pretty generic e-commerce application that could have been built on the web 25 years ago. It should not have been complicated to do.

mattlondon3• 6 years ago

Tackling bioinformatics as a webdev: why is it so hard?

"I am a professional in Field X but I am doing work in Field Y despite having minimal skills or experience and I ran into loads of problems and issues that I don't know how to fix! Field Y is total bullshit and people need to make it easier so people with no skills or experience can do it! Wah wah!"

... or to be less facetious, modern web/frontend development is a genuine skill set that benefits from experience and knowledge.

Should we expect anyone to just jump in cold with minimal experience and not find it difficult? E.g. in the same way would we expect people to start bioinformatics cold and be productive immediately?

I'd doubt it.

On the past web dev has been looked down upon as somehow "lesser" than "real" engineering, with sneering criticism from people more used to traditional development approaches (particularly criticism of JavaScript). It is just some web pages right? HTML and CSS? How hard can it be?

Well guess what: it is a genuine skillset and people should not expect to just be able to jump into something without knowledge training or experience and be proficient at it, just like we'd not for brain surgery, car mechanics, concert piano, structural engineering, or indeed bioinformatics.

</rant>

hmwhy• 6 years ago

I second this.

I have some experience in both web development and scientific computing in various fields (not bioinformatics). Things felt just as hard at the beginning in all cases—I don't recall things being any better in scientific computing compared to web development.

I agree that documentation is terrible most of the time, but that's not something limited to web development in my own experience. (On the contrary, check out the documentation of the website in question in this thread: https://news.ycombinator.com/item?id=23031408, it's absolutely fantastic).

Some things become second nature when you have developed expertise in that field, and it's easy to criticise something that you have comparatively less knowledge in and pass it off as "too difficult", particularly when the web development part seems to be a means to an end (which seems to be the case here).

Edit: missing conjunction, tidied up the last sentence for clarity.

LifeIsBio17• 6 years ago

Oh, interesting. I just wrote a reply saying the exact opposite. I remember learning scientific computing as being much less frustrating. Sometimes you'd have some error spawning out of some ancient C/Fortran library. But that was rare for me. Even after a couple of years on the web, I still feel like I encounter issues where I just don't even know how to start.

zhdc12• 6 years ago

Same, and they're not.

If anything, programming for academia is more of a mess than web development. Case in point - I spent the better part of a day rewriting what's essentially a parallelized for loop because the library I'm using in Julia to open HDF5 files has a well known memory leak. I have the impression that something similar in any popular web dev framework or library would have been ruthlessly purged a long time ago.

throwaway713• 6 years ago

It is a genuine skillset and people should not expect to just be able to jump into something qithout knowledge training or experience and be proficient at it

Sure, but this also applies to understanding the U.S. tax code, which I think is a needlessly overcomplicated, poorly designed system that requires layers of inefficiencies to extract value from, much like web development.

fastball• 6 years ago

You can still make a webpage that is just pure, simple, structured HTML. In fact, the HTML5 standard arguably made it easier to write a simple HTML webpage.

The complexity in webdev exists for good reasons much (most?) of the time.

If you want interactivity, you need an actual programming language. If you're using an actual programming language, then there is a lot of tooling that can be used to make your life easier in the long run. Not intially, but definitely from a maintenance / bug-fixing / etc perspective. If you want to work on a large, complex project with a large team, this "complexity" is invaluable.

That being said, JS as a language kinda sucks still, but "complexity" has been added to attempt to ameliorate that too, such as with TypeScript / CoffeeScript / etc.

WilliamEdward3• 6 years ago

Webdev is more like a trade than a science; you benefit more from remembering a large body of facts and details that comes with experience. There's no theory to learn, you just have to get practise in things like databases, browsers, HTTP, frameworks, etc.

This is probably why the author is struggling. This is a real world industry tackling real world problems. It is not the neat and cut world of science research.

LifeIsBio17• 6 years ago

It is not the neat and cut world of science research.

You had me up until this last sentence. Everything I've experienced in science research is the opposite of neat or clean cut. Actually, I think I do my best programming and debugging with a treat it like a science experiment.

seer• 6 years ago

On the contrary, learning the fundamentals of CS has made me a much better dev, more so than any fact or tidbit.

Its a testament to the awesomeness of the web however that so so many people even without any formal training can go about creating value.

You _can_ be a webdev tradesman, fixing things till it works, kinda, or you can delve into the fundamentals and be much more productive. But you don’t have to and I think thats awesome.

For example I still remember that conference that introduced Ruby on Rails to the world so many years ago. Most webdevs did not know there was an underlying theory and standard (REST) that can be used to make web apis bearable! After rails, we kinda figured huh, so this RFC for the web was sitting there all those years gathering dust and only weirdos and graybeards talked about it. But it was actually very very useful.

LifeIsBio17• 6 years ago

You're absolutely right. Webdev is certainly its own fully fleshed out fields. Oddly, I think I can play off your facetious quote and turn it into a more analytical way of framing what the post was about.

"I am a biochem major but I decided to work in bioinformatics/scientific-programming despite having minimal skills or experience and I ran into loads of problems and issues that I was able to steadily solve because the ecosystem felt sane, the tracebacks were clear, and the debugging usually had a clear path forward. Then I was a bioinformatics professional doing work in webdev [...] I ran into loads of problems [...] that felt consistently more opaque and magical than when I was learning scientific programming"

I think that's the heart of it. The transition to the web feels so much more chaotic than the transition to bioinformatics.

fxtentacle2• 6 years ago

There used to be a time when web frameworks strived for simplicity.

I fully agree with the article's opinion that things have become needlessly complicated.

The new best practice seems to be to install NodeJS, then use the npm repository manager to install bower, then use that to download your packages. And 5000 dependencies and 500 MB in downloads later, you have bootstrapped a Hello World template.

Or name me one person who can, from memory, correctly configure the rails sprockets asset pipeline :p

There's no denying that most of the tutorials and contemporary web development advice boils down to "follow these instructions carefully + magic happens"

dalke• 6 years ago

Fun bit of trivia: the first two major web libraries - cgi-lib.pl and CGI.pm - were written by bioinformaticians. Steven E. Brenner and Lincoln Stein, respectively.

pixelbash2• 6 years ago

Is anyone still using bower?

Also webpacker for rails is somehow both insanely complex and alright to use.

tarsinge• 6 years ago

This 100%. The vast majority of websites (not webapps) don't need anything more than HTML, CSS and some JS/JQuery.

Or name me one person who can, from memory, correctly configure the rails sprockets asset pipeline

I feel that one, it's insane that the most basic "I need to execute a few lines of very specific JS on this specific page" must devolve into a DRY absolutist abstraction madness, previously with sprockets, now with webpacker.

unclebucknasty3• 6 years ago

Your second paragraph neatly sums up the state of things. It's gotten downright cartoonish.

I remember when Rails first hit the ground. It was touted as a revolution in Web simplicity (and was relative to then, when what we expected of our apps was relatively simple).

But, someone got it in their heads that web apps should be near-indistinguishable from desktop apps. But, let's not address the impedance mismatch there. Instead, let's create a bunch of layers out of new tools and concepts to obscure the fact that these things don't fit together.

So our projects/apps now demand more and have just grown extremely complex. Turns out that making the Web act just like a native app (while battling all its Web semantics) is hard. We want it to be X, when all it wants to be is the Web.

Add to this all of the other disparate pieces that we want to work together: countless technologies, languages, libraries, 3rd party services, etc. And if you're talking full stack (and who's really not fullstack these days?), the number of universes that you must make coexist is insane. And, much of this is to layer over existing technologies (like Rails) that weren't intended to be used in this way. Enter entire concepts like transpiling, webpacking, etc.

Then to deploy, You're probably dealing with no less than 6-10 cloud provider services, each with its own set of concepts and requirements to be satisfied. Even if you're not the one deploying, you have to understand their implications on your code.

Web development is now less about writing logic and much more about wiring a bunch of existing stuff together, with discrete chunks of logic serving as the business/domain layer glue. I mean the app still has to serve some business purpose, so the code that supports that must be written. But that's barely the start these days.

simion3145• 6 years ago

IMO because of the poor building blocks of web dev (we are trying to build applications with document markup) things that should be simple are too complicated. As an example say I give you an url that will respond with a json of an array of persons and you want to view this json in a table( just view some raw data, I do not want fancy themes or features).

So if we ask web developers how to solve the above problem we get at least 25 different ways to sovle with with JS , and some other 25 ways if we use some backend PHP,Python, Java. If you ask some Qt/Flex developers you will get the same response, Drag and drop the DataGrid and set the data source to your data (and you get a ton of fancy features like adjustable columns, sorting,,,) and you don't have to install 300 dependencies.

mattlondon3• 6 years ago

But Qt/Flex is just one way of building desktop UIs :)

Comparing the entire gamut of web development against a single library seems unfair.

What about asking the same question of .net developers, VB classic developers, golang developers, electron developers, python developers, Java developers, c++ developers, swift developers? They'd all do it differently than QT developers would because there are different GUI frameworks that need to be used.

This problem is not unique to web dev.

prox4• 6 years ago

As a webdev, that is actually the beauty of the web, you get to solve / create / build in all these ways. Neither are wrong.

Another point I really like is that I can view a webpage from the 90’s and it will look fine (maybe not great)

And there are frameworks, platforms and plugins which allow you to do what you described without little coding. You can go up and down on the stack, switch stacks and still get to the same end result.

at_a_remove• 6 years ago

Yes. I think I was first paid for a website I developed around 1997. As I continued along the path, it was clear that we're trying to implement a kind of remote generic programming environment on top of systems that were not originally designed as such, and that this happens on almost every level.

HTML, at heart, is still a markup language designed to separate a stream of text into semantics and display them. CSS ... I am not even going to go into my feelings about CSS except to say that it suggests that people who look at design principles like C.R.A.P. were not near the discussion and were not in the room for some time. A programming language named after Java, in which it no way resembles, that had to be pushed out in almost no time at all given the irrational deadlines, dominates the landscape ... and we must transpile to it. And we're pushing video over HTTP.

All of the first-mover advantages have locked us into this path and many very intelligent people are busy using their prowess to perform work under some very arbitrary conditions, like not being able to use the letter "e" in writing. While it is a challenge and stretches the abilities, no respite is offered, and so there's a tremendous amount of waste occurring as these initial conditions, propagated down through time, must be constantly worked around.

emsy4• 6 years ago

This shouldn’t be the top comment. Yes, switching fields and expecting to just work professionally is naive and insulting towards the other profession. Has web dev a lot of legacy cruft and suffers from over-engineering? Absolutely. Even for seasoned programmers starting web dev is harder than it needs to be. Front end development has JS frameworks, CSS or compile-to-CSS languages, templating, web packing, JS transpilers, tree pruning, multiple package managers and what not. But in the end it’s just a bunch of text, images and buttons.

goto112• 6 years ago

But you don't need any of that. A static HTML page and a bit of CSS for styling and you have a web site.

If you don't think package management or transpilers makes your particular project simpler, then don't use them. They are tools.

lucideer• 6 years ago

Every field has cruft. Part of gaining experience in a field is not only learning how to do things, but also learning how to navigate the field and differentiate the cruft from that which is of value.

Even for seasoned programmers starting web dev is harder than it needs to be.

Maybe, but is it harder than it "can be", or is it significantly different to other fields in this respect. I don't believe so.

If you think the extra layers of unnecessary complexity noone needs is either (a) unique to web dev or (b) easily removable, then I think you may have a slightly naïve perspective on humans & society in general. A perfectly organised and navigable field is pretty utopic.

stupidcar• 6 years ago

You can reduce any field to "it's a just bunch of...": All coding is just a bunch of binary numbers. All chemistry and biology is just a bunch of molecules. All construction is just a bunch of concrete, bricks and steel. It's nonsense, and is exactly the kind of naive and insulting statement you talked about.

Complexity is emergent. Look below any complex system and you'll find a simpler one, but that doesn't mean the higher-level complexity is unnecessary. All the things you complained about have good reasons to exist, invented by people who actually work as professional web developers and need to solve problems.

The real problem is "seasoned programmers" who don't work in web development, but who expect to be able to jump in an master it in a day, based on the fact that they put together a GeoCities homepage twenty years ago, and are enraged and disgusted to find that things aren't so simple anymore. It's absurd. Developers wouldn't expect to be able to jump into kernel development, or database development, and find it to be a breeze.

agumonkey• 6 years ago

To me there's hard and hard. (reminiscent of simple vs easy).

Hard concepts are ok for anybody with adult desire to learn. It's the ad-hoc transient social choices (package managers, conventions, ecosystems) that are overwhelming. It's draining.

saati• 6 years ago

Why should you decide which comment should be the tlc?

azangru• 6 years ago

100 percent this.

And yet, even within the web development community, we hear a lot of complaints about how this or that isn't beginner-friendly; or how such-and-such is gatekeeping, and so on. This has been baffling to me. Is it the same in other disciplines? Are engineering, bioinformatics or medicine, for example, expected to be beginner-friendly?

mettamage• 6 years ago

IMO web dev isn't that hard, given one has a CS background. I came out of college, immediately started to teach a bootcamp (they had a lot of faith in me). What I learned about the difficulty of web dev:

- Finding out the right curriculum by yourself is hard

- Once you have the right curriculum, then you can learn it yourself provided one has the right background. In fact, they even can teach that curriculum to people that don't know how to program or do web dev yet

When I was done teaching NodeJS/ReactJS, I went on to freelance for a company that had a microservice backend in Java. Since my CS degree was a Java school, I picked up Spring Boot quite quickly. I needed to build on top of code and not create it from scratch. That really helped, starting from scratch would've been more difficult and does require a lot more experience (so altering an open-source project might be a good way to practice). After 3 months the job was done and I left.

Now, I know that I don't have a lot of experience, but I honestly haven't seen any hard challenges yet when it comes to web dev at the 3 year old startup level. There were no scalability or security challenges to take care of. The web dev rabbit hole can go deep (and that is hard), but in practice it mostly doesn't (unless you work at Google scale, I presume). Deployment with IBM and Kubernetes was just: read the docs and know how to debug and troubleshoot.

It's not easy per se, but if you start with the right background (CS and a lot of programming courses in such a CS curriculum) and get a good coding school curriculum after that, then I would argue that web dev isn't that hard. There are enough startups where you'd be productive.

Though, to be fair: if I couldn't use a debugger, I'd be totally lost. Using a debugger makes it not that hard. This is also why I had a harder time with learning Webpack.

What I found way harder is wrapping your mind around concepts that seem completely alien. I have this still a bit with functional programming (not covered in my CS degree) or back in the day with databases or certain forms of math.

Reading API documentation isn't part of it which is what I feel web dev is mostly nowadays. Even when you learn another programming language, it's all procedural/object oriented, so you just need to learn the quirks of that particular language. When I first started out I also had to deal with emotional fear of being inadequate. But after your fifth language of feeling that way, you know it's part of the process and you shouldn't take it to mean that you are inadequate.

JeremyNT• 6 years ago

I would suggest that it's really easy to envision some much less painful abstractions for web development.

Sure, learning an entire field from first principals is going to be hard, and to do science you need to understand those principals. But the general concept of "make a dynamic web page" is not computer science! What most people (and the author of the blog post) need is a clean abstraction that allows them to glue a few APIs together with some custom forms.

Taikonerd• 6 years ago

That's the premise of low-code, right? "I don't need a fully general language with configurable everything; I just need a simple tool for this one common use case."

I'm excited about what Dark (https://darklang.com/) is doing with this idea. As you said, "a clean abstraction that allows them to glue a few APIs together with some custom forms."

rsynnott4• 6 years ago

... or to be less facetious, modern web/frontend development is a genuine skill set that benefits from experience and knowledge.

I mean, there's a strong argument that it has become too complex for no particularly good reason. It is _far_ more complex today than a decade or so ago, and the results in many cases are arguably worse for the user.

If you compare the _average_ modern react-y website to a Ruby on Rails one with minimal or no Javascript from 2007 or so, the react-y one is far more difficult (and slower) to work on, but is it really delivering extra value to the end user? There are exceptions, of course, but I think for most CRUD web apps things have gone in a very bad direction.

bitexploder• 6 years ago

Funny thing is, those rails and simple Django apps are just as effective today as they were 10 years ago. In fact, not letting your app get bogged down in the mess that is modern front ends is likely a competitive advantage these days. Keep it simple. Django and Rails are the best they have ever been. Most software is about data, not front ends. It takes dedicated teams to tame “modern” front ends. Let the data show you the way and avoid front end stuff until you absolutely can’t. Maybe, after you need something to do with a lot of VC money or you can justify a certain UX in your app.

mattkrause3• 6 years ago

Should we expect anyone to just jump in cold with minimal experience and not find it difficult?

My take on this is that the author had some (relevant) programming experience and thought it would transfer better, hence the frustration.

That was my experience, as someone in the same boat (computational scientist trying to put together a webapp). I have a CS degree and my job involves a lot of diverse coding, from low-level stuff to interface with hardware to graphics/ML/data stuff, not to mention gluing together other people's code.

I nevertheless found it very frustrating to get started with web dev. There seem to be a million slightly different ways to do things and people have strong opinions about which ones are best, so it's hard to settle on authoritative source, especially one that's up to date. Many tutorials bounce between basic web stuff (which I wanted) and basic programming (which makes me zone out). Plus, there's a lurking fear that I'm gonna screw something up in a way that exposes a bunch of data or costs me a fortune in hosting (etc) while trying to make something that I sketched out on three pieces of paper.

This seems a little different from bioinformatics, where a lot of the complexity is baked into the subject matter. For background, textbooks certainly vary in how they present information, but they all pretty much agree: Molecular Biology of the Cell isn't going to tell you that DNA is where it's at, with RNA being a fad vs. The Cell claiming that DNA is outmoded and it's all RNA, baby!

hysan• 6 years ago

OP’s comment still seems pretty on point though even with your example. Taking the exampling to another field in programming that’s also diverse like gamedev and you run into a similar “too many choices” problem. The solution of course is to just pick a popular tutorial and start. You find and fill in the gaps along the way. Chasing for the elusive best practice is just setting yourself up for failure as a beginner.

downerending• 6 years ago

If you have any doubt that software takes real skill, wade into the muck that is software written by scientists. You will discover classes of errors at all levels that you didn't even think possible. Testing? What's that? Exponential run time? Does that matter? And why does my program give different answers now than it did last year? Those numbers worked for my paper--make the program give them again!

doteka• 6 years ago

I am a software engineer who works closely with a team of bioinformaticians. Your reply suggests you only have one side of the story.

A bioinformatician is someone who has quite serious programming skills on top of their biology knowledge. They are frequently bottlenecked by compute and so tend to have a good understanding of data structures and algorithms. They usually know 3 or more languages and are no strangers to databases, version control, containers, et cetera.

And yet I recognize the feeling from the article: they get stuck in the heap of complexity we (as an industry) have created around a bog standard web frontend. The majority of us are not doing rocket science - the applications I’m working on today do similar things to the ones I worked on five years ago. Yet somehow, the ‘how’ of it, the whole Rube Goldberg machine, tends to get more convoluted every passing year.

I often get the feeling a nonzero percentage of it is introduced purely so web developers can feel they’re “real engineers” whatever that may be. There’s nothing to prove, now stop piling on junk and reinventing wheels.

jldugger2• 6 years ago

Why’s there so much data in the url? A common refrain I’ll use for the remainder of the post is, “I don’t know”. One of my biggest gripes with web programming is the number of rabbit holes to fall down. There’s just not enough time.

I mean, bioinformatics is _notorious_ for spending just enough time working on software to publish a result and no more. If we want to talk about rabbitholes I nominate BioPerl.

here are a list of terms that you’d never come across as a bioinformatician: ["X-Forwarded-Proto", "proxying", "requests", "TLS connection", "HTTP", "loopback/localhost", "whitelisted proxy servers", "gunicorn", "nginx", "Apache/httpd", "HAProxy", "Django", "SECURE_SSL_REDIRECT", "SecurityMiddleware", "adapter", "URI"]

This list is more than a little disingenuous:

Django: This is like, the framework OP built their software on top of. I don't know how one gets the point of filing this bug without knowing the term Django

requests: literally, a user's web request.

HTTP/URI: this is the web developer equivalent of a biologist not knowing the term DNA. Or DOI for that matter.

whitelist: the opposite of a blacklist

Much of rest comes down to the simple fact that job of web developers is to build a network service. This is primarily done with layers; each layer only needs to know how to interface with the adjoining layers above and below, like a lego brick. This is actually a _solution_ to the complexity, allowing us to constrain the design of each layer to interact only with those above and below it. This layer abstraction lets us reason about each layer locally, independently of the system as a whole.

nginx, Apache/httpd, HAProxy: in this context, mentioned as possible proxies forming the outermost layer. in this context, accepts HTTP/HTTPS connections and makes HTTP connections to gunicorn

gunicorn: the thing that runs the Django app. Accepts HTTP connections, makes WSGI calls to the Django app

Django: Accepts WSGI function calls, routes them through middleware, then to the app as you define in urlpatterns

SecurityMiddleware: a class that potentially modifies inbound requests and outbound responses

Finally, OP is in the unhappy situation of bridging two network protocols: HTTP and email. The normal trick of using a protocol relative URI will not work here because you've gone outside the browser request-response loop. This puts them in the unhappy state of the lowest layer needing to know the details of the highest layer, hence the discussion of absolute uris. This is a direct violation of the local reasoning constraint, and what makes all of this hard.

X-Forwarded-Proto: An HTTP header, added to incoming HTTP requests designed to indicate to layers below which URI scheme was originally used, HTTP or HTTPS.

SECURE_SSL_REDIRECT: a setting for SecurityMiddleware that will stop processing the request and send a response to the browser telling it to resend the query over https (an HTTP 302 redirect), if the incoming connection is seen as coming over not http.

tl;dr: email is the thing that sucks.

teekert2• 6 years ago

In a related article: "Tackling bioinformatics as a Webdev: Why is it so hard?"

Maybe Webdev is just complex? But tbh, as a bioinformatician (in Next Generation Sequencing mainly) I actually find Webdev easier. I mean, we face Bam files, Sam files, Cram files and annotate them with Bed files (with many different poorly documented standards) or GTF files depending on your Ensembl or UCSC references persuasion... Some file types are 0-based inclusive, some are 1 based exclusive, some call chromosome just ints (1,2, and then error when X, Y if you really assume they are ints) some call them chr1, chr2, chrX etc. We have fastq files with different quality encodings depending on their age, I recently had some strange errors, turns out the R2 of a fastq file is reverse complementary from the standard (why?!)... It's a mess and many people that spend years in the field still don't know more than just "choose one side and stick with it"... We also have a huge amount of frameworks for data analysis pipelines, Snakemake, Bpipe, Broad's WDL, etc, etc, etc. Not to mention the insance amounts of read mappers (BWA, RSEM, Tophat 1 and 2, Kallisto etc). Need I go on?

LifeIsBio17• 6 years ago

That's super interesting. If someone asked me if bioinformatics was complex, I'd say "Yes, massively so." But then, I would take it in a totally different direction. To me, the complexity of bioinformatics come from the biology. How do you think about weird biological edge cases? What do your distributions look like and how do you choose thresholds? How do you even form a good question?

Thinks like [B|S|Cr]am files are complex in their own way, but mostly they're just kinda annoying to have to figure out. You can dig in as deep as you need pretty easily.

On the other hand, maybe I'm over-simplifying because I feel more comfortable in this domain, which I think is your point.

teekert2• 6 years ago

Well, I come from Biology, always had Webdev as a hobby and then moved into bioinformatics which now turns towards software development. So Biology feels most familiar, but it is indeed also always complex.

Btw, the files are ok and with effort can be understood and maybe some could even be considered elegant. It's just the shear amount of "standards" or rather the lack thereof.

yread• 6 years ago

Hah bam files are the pinnacle of engineering compared to the clusterfuck that is VCF: GT or PL? Ploidy? Enumerating all possible ALTs but some with 0 AF? And then you have mutations in cis, try to merge them. Which assembly - you can guess because it doesn't have to be specified. You also don't have to specify your transcripts and of course you have better defaults than everyone else. How many different formats for annotating mutation effects are there 10? 20?

With webdev I can at least stick with jquery and do no harm

collyw3• 6 years ago

With webdev I can at least stick with jquery and do no harm

The amount of time saved by doing this would be immense based on my experience inheriting maintenance projects in Anglaur / React / whatever flavour of the month framework was used. They may seem productive when developing, but trying to fix bugs in them can be an absolute nightmare.

rgoulter• 6 years ago

I don't think the author is saying "why front end application development hard?" so much as "why is setting up a website hard?".

On one end of the spectrum, simply putting content on a website isn't so hard, right? On the other end, making a web application is understandably difficult.

Features like users, payment forms, and email confirmation (so e.g. the website can be used for a course) sounds like they ought to be as simple as the former, but are closer to the latter.

Is the value proposition from the website building websites like Wix that they take care of the hard stuff?

namelosw• 6 years ago

It is no brainer for professionals, but to be frank, it's too hard for everyone. It's time for a new PHP.

IshKebab• 6 years ago

Backwards compatibility.

It's possible to design coherent easy to use UI systems. Flutter and Qt are great and both much easier to develop for than the web IMO. But they don't have to worry about running in web browsers.

sradman• 6 years ago

I love these kinds of retrospectives as they give insight into the learning process. We tend to focus on the acquisition of specialized knowledge but there is often a body of shared knowledge, the context you might call it, that we pick up indirectly.

You have to read between the lines but what we have in this post is a Data Scientist skilled in the Python programming language struggling to acquire the shared knowledge required to build and deploy Web Apps. One toolchain (Python), one framework (Django), one backend architecture (12-Factor Apps) but the narrative surrounding URLs, HTTP, DNS, TLS, REST APIs, JSON, CORS, and www-forms is missing.

We seem to be missing a simple way to learn the shared knowledge needed to build apps in a world reshaped by Web, Mobile, and Cloud technologies.

LifeIsBio17• 6 years ago

I’m not sure exactly what it is about this comment, but I really like it. “Context” is a great word for this idea. There are a bunch of small/medium complexity things that need to play well together to get things to work, and it can feel trickier than necessary.

not_a_moth• 6 years ago

This is common among academics, who expect full transparency and understanding in what they're doing, instead of a "get it working and go" attitude.

The problem is the OP didn't test anything: He launched and users couldn't log in.

Ideally I'd like to gain 100% visibility into everything related to my tasks in webdev, but it's impractical and counter productive.

dade_• 6 years ago

It is hard to get the hang of Django is because the polls tutorial isn't helpful to actually understand Django and no one wrote this book yet, but I think Nigel has done a great job: https://djangobook.com/mastering-django-2-book/

heisnotanalien• 6 years ago

Because instead of fixing genuine problems and making it easier people invent new languages and frameworks to do things in their own way.

cmrdporcupine• 6 years ago

Honestly I've always found UI development or anything with some aspect of UI in general has the potential to devolve into a tangled mess. I've avoided web facing development for the last 10 years of my career, and have transitioned to embedded work. Every time I traipse through job postings and look at the tech stack companies are using ("wanted: full stack developer") I just shake my head. I am sure most of the choices are justified, but it all seems so brutally overwrought from the outside looking in. But beyond a fetish for novelty in our industry, I think that's just endemic to user facing development.

walleeee• 6 years ago

Related: institutions and labs need to recognize that 'research software engineer' is an increasingly necessary role and attract/fund more of them.

thrower123• 6 years ago

Yes, web development is hard and stupidly complicated and slapdash and we all hate it. It sucks, and the whole thing should be burned to the ground, the ashes sown with salt, and pissed on by donkeys.

Unfortunately we're stuck with this Rube Goldberg collection of excrement, and it pays the bills. So there's nothing much you can do except lean in and embrace the suck. Maybe hit yourself in the forehead a few times with a clawhammer or start carrying a hipflask to keep yourself at happy-drunk.

6510• 6 years ago

People who understand the mess are much more influential and they are moving the show forwards. Making things easy for newcomers is just not very high on the agenda (anymore)

Perhaps things have to get complicated enough for it to get priority, perhaps cool things will happen all of a sudden. I was highly amused by the spreadsheet programming tool a while back. It seems a giant leap in the other direction.

Tackling Webdev as a Bioinformatician: why is it so hard?

Related Stories