HN user

Mortiffer

300 karma

I am doing Data things

Posts10
Comments118
View on HN

The R community has been hard at work on small data. I still highly prefer working on on memory data in R dplyr DataTable are elegant and fast.

The CRan packages are all high quality if the maintainer stops responding to emails for 2 months your package is automatically removed. Most packages come from university Prof's that have been doing this their whole career.

I have not found a single prof that thinks the trend have having higher and higher % of university staff be administrators is a good thing.

I wonder if it is possible for them to connect funding to a maximum allowed ratio of admin to prof / lectures

Cloud Egress Costs 2 years ago

Pls also consider the egress network speed. I got burned by scaleway when they changed their blob store pricing. Trying to my grade out to Hetzer would have taken months at the network speeds they were providing

core element is https://postgrest.org/en/stable/ . I use this in production in large corporate projects on k8s. For a large number of use cases you can put logic into stored procedures SQL. PG can also do JS or Py stored procedures but you get a better developer experience if your logic code is deployed through regular CI/CD containers or functions (we use both extensively together depending on cost trade offs either one.)

Supabase suggests you to use their DENO serverless functions which is cool and all but i think most people would rather deploy node functions on cloudflare for webprojects.

That being said the target customer group are those that want to have 99% of their logic in JS frontend. Backend just does CRUD and Auth.

Based on recent experience with large deployments at S&P500 types, IMHO the value snowflake providers to most companies can easily be deployed in house with either old school Hadoop,Spark, Hive or old school Impala, Hbase or more recently Trino.

The clients I had where we installed snowflake every is super worried about any additional copy operation because everyone is keenly aware of the unlimited cost. This makes getting shit done harder.

Clients that have a decently sized k8s cluster running Trino or old school Spark/Hadoop cluster on fat VM's you know what you got you know what your paying for you make estimate of how much ram / cores you need for certain workloads and once they are purchased your engineers get really good at squeezing as much work out of the given resources as possible. And no constant complaining in meetings about what potential extra cost this additional computation may have.

Also if other people working in other parts of your org don't run snowflake themselves you gotta pay for their snowflake usage on your bill or you pay for copying data back out to S3/ADLS/sFTP so that other departments can get to the results of your computations. And snowflake really doesn't like it when you do this, they even gave exporting data a new name, they call it "unloading" making you feel like your undoing something which you probably should not undo.... On that note, Snowflakes data export options are significantly underdeveloped in contrast to Databricks, Cloudera and also the original open versions Spark, Trino and Impala.

</rant> :)