davidkell

Agreed and well put.

Lack of a decent Rails/Django option for JS/TS is part of the issue IMO, since that ecosystem is so popular atm.

I also wonder if you have to get burned a few times rolling your own and that's just part of the journey :)

For junior developers reading this: frameworks are great, learn to love them, they will save you much time and stress when shipping real products.

If you like Python, I can recommend this:

  - Start with SaaS Pegasus https://www.saaspegasus.com/
  - Deploy to Heroku
  - Get comfortable with "X-in-HTML" frameworks - Tailwind, HTMX, Alpine
  - Deliver your mobile app as a PWA https://web.dev/what-are-pwas/

(Nowadays I'd replace Turbo/Stimulus -> HTMX/Alpine)

Special thanks to Cory for his excellent guides on Django + JS: https://www.saaspegasus.com/guides/

Happy SaaS Pegasus customer here! Reasons to love it:

- Cory’s writing on low JS alternatives to single page web apps was game changing for us. And that’s all built in. 10x faster to develop.

- You get access to a wonderful community of Django devs. Any question gets 3+ independent responses.

- The framework is non-opinionated. You get the same examples in both HTMX and React + APIs, many versions of CSS frameworks (Bulma, Tailwind, Bootstrap), multiple deployment methods, etc. This gives you a valuable toolkit to fit your solution around the needs of the problem.

Thanks folks, much needed! Where is the list of destinations today?

Many of your integrations talk about “syncing” rather than event collection, which to me sounds like what Fivetran is doing. Does that distinction make sense and how are you thinking about that?

+1 for SaaS Pegasus. In particular, Cory has done a wonderful job in introducing a more pragmatic approach to Javascript with Django [0]. Plus he's built a strong Slack community and is always around to support and listen to any feedback.

We re-built our SaaS product using this boilerplate and wrote about the experience [1]. TL;DR: would recommend!

[0] https://www.saaspegasus.com/guides/modern-javascript-for-dja...

[1] https://davidkell.substack.com/p/davids-opinionated-guide-fo...

+1 For this recommendation, an exceptional book. In particular chapters 2-3 demystify how databases work, and provides an intuition on when to use different kinds of databases and query languages without the typical hype (spoiler alert - use an RDBMS!)

OK, that's an interesting POV.

To be honest, I was surprised that you advocated writing so many tests for each view, easily 10-20 per view.

Part of the beauty of Django's various class based interfaces is that you can be confident that if you add a validator to a model field, then it will be validated by the corresponding ModelForm in the corresponding CreateView.

But then again, you advocate against OOP in python including CBVs. If you do write all your endpoints as functions, it makes more sense you need to test it because it's easy to forget to include a line of validation or whatever it is.

FWIW I love the fact that various Django classes are like DSLs. DSLs are less powerful by construction, so less buggy. It is almost like using low-code. But I do see your point about tracking inheritance and control flow, it can be a challenge at scale.

We recently started working with "traditional" Django + Hotwire + React wrapped into web components for interactivity.

We've previously tried React+Hasura/Postgraphile and React+DRF/GraphQL, but this is next level productive and a joy to use, and bugs are rare. Particular love for Django CBVs.

As the sibling comment says, while I disagree with specifics this article is full of gems.

@Alex3917 What do you think of using Django-rendered templates for views that don't require significant frontend interactivity?

You seem to advocate the "everything is an SPA + Rest API" approach, which I thought was interesting.

Has anyone found a frontend JS framework + backend combination with the developer ergonomics of say Rails, Django or Phoenix? Would love to hear.

Any Snowflake developers reading this - the current snowflake-connector-python is pinned to 0.17, almost 1 year out of date now. Would be great to get that bumped to a more recent version :-)

Fivetran has built all the custom OAuth flows for their 150 custom integrations and you can build it into your own (internal or external) applications, it is neat. @goergewfraser When do you plan to add the ability to configure connectors that need extra config after the initial connection, e.g. choosing reports from Google Analytics?

Do you plan to support one-click integrations? (= Stitch Connect JS [0] / Fivetran "Connect Card" [1])

[0] https://www.stitchdata.com/docs/developers/stitch-connect/ja... [1] https://fivetran.com/docs/rest-api/connectors/connect-card

For Postgres https://pgtune.leopard.in.ua/#/

You can do the same thing with the jupytext extension. But sometimes it is helpful to have the rendered results in version control, eg internally we use it to discuss data science findings on Gitlab.

IMO no one has done more to make deep learning accessible than Jeremy + fast.ai team. Thanks for the amazing work!

My question is about the coding style - @jph00 I’ve read your fast.ai style guide and worked with APLs like q/KDB (written by Arthur Whitney who you cite).

My experience is that brevity is great, until you need to collaborate or have individuals working on small parts. That was my experience as well trying to write an extension to the fast.ai code (where I had to read large amounts of source to understand how to implement a small change).

Given that a key motivator for literate programming is collaboration/communication, how do you think about this?

Thanks, appreciate the response.

Top of my wish list would be a python <-> VSCode experience at the typescript level of quality.

Interesting that mypy is a Dropbox (GvR’s previous gig) whereas Microsoft has a competitor python type checker, pyright [0]. Anyone have experience with it?

[0] https://github.com/microsoft/pyright

What's the secret of the performance? Especially interested to see that it is written in Go.

How is Hobbes support for relational workloads? My favourite feature of KDB is mixing SQL-like queries with list manipulations. I see support for tables in the Hobbes docs [0], but an array of tuples is row-oriented whereas KDB is column-oriented (flipped dict of lists). Am I missing something?

Nice job btw.

[0] https://hobbes.readthedocs.io/en/latest/language/types.html#...

The architecture you've proposed here has many overlaps with q/KDB (or Jd, Shakti), which does not (AFAIK) have a decent open source alternative. Looking forward to see what you build!

Thanks for sharing graphique, that is very neat.

How do you feel about using SQL for everything, eg using Postgres policies for RBAC, or PL/pgSQL to generate passwords?

In principle, I love the idea of Postgraphile but this is what turned us off.

The reactive programming model (or whatever you call it, honestly don’t know) is at the core of so many great tools and products - eg Terraform/Pulumi, Kubernetes, React, Mobx, Observable...

At the same time, we’ve struggled to find decent reference material or libraries for building a reactive framework for X (for our use case, X = data science workflows). Most of these libraries seem to implement all the primitives from scratch.

I’d be interested to here other people’s thoughts on this!

In my experience, GraphQL enables smaller, less experienced teams to build complex web apps faster. (As long as you use decent frameworks and tooling.)

There are definite downsides versus eg REST (notably performance, which becomes harder to reason about), but it’s an acceptable trade-off for us.

I’m also optimistic about the great tooling that is improving all the time - eg Hasura, Postgraphile, Graphene-SQLAlchemy all solve N+1 today.

The two places I’ve seen this working best today:

- Open source data science/scientific computing ecosystems. Notably python, where all the libraries interop seamlessly via numpy/pandas/arrow and Jupyter is the visual coding platform. But also R/tidyverse and Julia.

- Modern “no-code” tools, where the visual coding is Notion/Coda/Bubble, interfaces via Zapier/Integromat/Autocode and data models in Airtable/Sheets. (Many of these tools use the word “block” as part of the UX)

And ofc, we take it for granted but the concept of a “file” is the ultimate building block for applications.

In my experience, commercial disincentives aside, the main trade off for this power/flexibility is the complexity. It is intimidating for new users, and hard to design well for because of the combinatorial explosion of interactions. Users need to be strongly motivated to get over this complexity hump - whereas most users, most of the time want a single happy path. Personally I don’t see this as a negative thing - you are essentially coding best practices into the tool.

As an aside, the instant feedback coding in python looks fantastic! Could be a fantastic extension eg for Jupyterlab or VSCode.

I’m not familiar with those solutions in particular.

For type safety, many backend frameworks generate OpenAPI specs automatically, and you can generate Typescript stubs based on this. Ditto for gRPC and gRPC web. We use these.

But I’ve not seen a replacement for the “application data graph” (but would be interested to learn about them!). The link from @jefflombardjr [0] explains it nicely - it is a great abstraction (in certain situations). Modelling your application data graph is like modelling a good database schema - when you get it right, the rest of the application follows naturally. It’s magical when it works, and I’d happily do it even if it’s just me working on the project.

And GraphQL has a great ecosystem, that is the advantage over niche tools. Example - last week we added auto-generated GraphQL types and relationships for Postgres JSON fields, with the help of [1]. No more malformed JSON breaking our app.

Note this is all in the context of a web app. Reading other comments, the tooling seems to be less developed on other platforms. And again, without decent tooling (especially for the server) I wouldn’t touch it.

[0] https://medium.com/@JeffLombardJr/when-and-why-to-use-graphq...

[1] https://github.com/graphql-python/graphene-pydantic

We’ve trialled combinations of REST, GraphQL and gRPC-web across a bunch of different products. GraphQL has reliably won for us in trading off DX, UX and feature speed. Reasons -

- Auto-generated, type safe entities from source to client, including relationships = fewer bugs

- Ability to unify different backends (eg a database, warehouse, external APIs, cloud storage)

- The “application data graph” concept always brings huge clarity to the architecture design - you get to build your mental model in code

- GraphiQL is excellent

Having said this, most of the benefits come from the incredible tooling, eg for our stack Graphene, Graphene-X bridges, Typescript, Apollo. I would never consider writing a GraphQL server from scratch, and we’ve had bad experiences with instant database -> GraphQL solutions (not really keen on writing my application logic in SQL). It’s also not an either or - our current app uses REST for file uploads and stream large datasets to the client. And ofc, you can achieve many of these benefits with other solutions.

Wundergraph looks like a great addition to the ecosystem, it would already remove boilerplate from our app.