oppositelock

Why does anything need to replace anything?

Containers, VM's, physical servers, WASM programs, Kubernetes, and countless other technologies fill niches. They will become mature, boring technologies, but they'll be around, powering all the services we use. We take mature technologies, like SQL or HTTP for granted, but once upon a time, they were the new hotness and people argued about their suitability.

It sounds like another way to distribute software has entered the chat, and it'll be useful for some people, and not for others.

This comes down to your philosophical approach to API development.

If you design the API first, you can take the OpenAPI spec through code review, making the change explicit, forcing others to think about it. Breaking changes can be caught more easily. The presence of this spec allows for a lot of work to be automated, for example, request validation. In unit tests, I have automated response validation, to make sure my implementation conforms to the spec.

Iteration is quite simple, because you update your spec, which regenerates your models, but doesn't affect your implementation. It's then on you to update your implementation, that can't be automated without fancy AI.

When the spec changes follow the code changes, you have some new worries. If someone changes the schema of an API in the code and forgets to update the spec, what then? If you automate spec generation from code, what happens when you express something in code which doesn't map to something expressible in OpenAPI?

I've done both, and I've found that writing code spec-first, you end up constraining what you can do to what the spec can express, which allows you to use all kinds of off-the-shelf tooling to save you time. As a developer, my most precious resource is time, so I am willing to lose generality going with a spec-first approach to leverage the tooling.

I've been building API's for a long time, using gRPC, and HTTP/REST (we'll not go into CORBA or DCOM, because I'll cry). To that end, I've open sourced a Go library for generating your clients and servers from OpenAPI specs (https://github.com/oapi-codegen/oapi-codegen).

I disagree with the way this article breaks down the options. There is no difference between OpenAPI and REST, it's a strange distinction. OpenAPI is a way of documenting the behavior of your HTTP API. You can express a RESTful API using OpenAPI, or something completely random, it's up to you. The purpose of OpenAPI is to have a schema language to describe your API for tooling to interpret, so in concept, it's similar to Protocol Buffer files that are used to specify gRPC protocols.

gRPC is an RPC mechanism for sending protos back and forth. When Google open sourced protobufs, they didn't opensource the RPC layer, called "stubby" at Google, which made protos really great. gRPC is not stubby, and it's not as awesome, but it's still very efficient at transport, and fairly easy too extend and hook into. The problem is, it's a self-contained ecosystem that isn't as robust as mainstream HTTP libraries, which give you all kinds of useful middleware like logging or auth. You'll be implementing lots of these yourself with gRPC, particularly if you are making RPC calls across services implemented in different languages.

To me, the problem with gRPC is proto files. Every client must be built against .proto files compatible with the server; it's not a discoverable protocol. With an HTTP API, you can make calls to it via curl or your own code without having the OpenAPI description, so it's a "softer" binding. This fact alone makes it easier to work with and debug.

Random UUID's are super useful when you have distributed creation of UUID's, because you avoid conflicts with very high probability and don't rely on your DB to generate them for you, and they also leak no information about when or where the UUID was created.

Postgres is happier with sequence ID's, but keeping Postgres happy isn't the only design goal. It does well enough for all practical purposes if you need randomness.

I'm a systems nerd, and I found working with it quite challenging, but rewarding. It's been many years, but I still remember a number of the challenges. SPE's didn't have shared memory access to RAM, so data transfer was your problem to solve as a developer, and each SPE had 256k of RAM. These things were very fast for the day, so they'd crunch through the data very quickly. We double-buffered the RAM, using about 100k for data, while simultaneously using the other 100k as a read buffer for the DMA engine.

That was the trickiest part - getting the data in and out of the thing. You had 6 SPE's available to you, 2 were reserved by the OS, and keeping them all filled was a challenge because it required nearly optimal usage of the DMA engine. Memory access was slow, something over 1000 cycles from issuing the DMA until data started coming in.

Back then, C++ was all the rage and people did their various C++ patterns, but due to the instruction size being so limited, we just hand-wrote some code to run on the SPU's which didn't match the rest of the engine, so it ended up gluing together two dissimilar codebases.

I both miss the cleverness required back then, but also don't miss the complexity. Things are so much simpler now that game consoles are basically PC's with PC-style dev tools. Also, as much as I complain about the PS3, at least it wasn't the PS2.

Am I confusing them?

It's not difficult to implement JWT's, the concept is simple, however, with authentication code, the devil is in the details, and that's true for any approach, whether it's JWT's, or opaque API tokens, whatever. There are many, many ways to make a mistake which allows a bypass. Simple concepts can have complex implementations. A JWT is simply a bit of JSON that's been signed by someone that you trust. There are many ways to get that wrong!

Convenience, when it comes to auth, is also usually the best path, and you need to be careful to use well known and well tested libraries.

JWT's are perfectly fine if you don't care about session revocation and their simplicity is an asset. They're easy to work with and lots of library code is available in pretty much any language. The validation mistakes of the past have at this point been rectified.

Not needing a DB connection to verify means you don't need to plumb a DB credentials or identity based auth into your service - simple.

Being able to decode it to see its contents really aids debugging, you don't need to look in the DB - simple.

If you have a lot of individual services which share the same auth system, you can manage logins into multiple apps and API's really easily.

That article seems to dislike JWT's, but they're just a tool. You can use them in a simple way that's good enough for you, or you can overengineer a JWT based authentication mechanism, in which case they're terrible. Whether or not to use them doesn't really depend on their nature, but rather, your approach.

Very cool work!

I had to solve a similar problem years ago, during the transition from fixed function to shaders, when shaders weren't as fast or powerful as today. We started out with an ubershader approximating the DX9/OpenGL 1.2 fixed functions, but that was too slow.

People in those days thought of rendering state being stored in a tree, like the transform hierarchy, and you ended up having unpredictable state at the leaf nodes, sometimes leading to a very high permutation of possible states. At the time, I decomposed all possible pipeline state into atomic pieces, eg, one light, fog function, texenv, etc. These were all annotated with inputs and outputs, and based on the state graph traversal, we'd generate a minimal shader for each particular material automatically, while giving old tools the semblance of being able to compose fixed function states. As for you, doing this on-demand resulted in stuttering, but a single game only has so many possible states - from what I've seen, it's on the order of a few hundred to a few thousand. Once all shaders are generated, you can cache the generated shaders and compile them all at startup time.

I wonder if something like this would work for emulating a Gamecube. You can definitely compute a signature for a game executable, and as you encounter new shaders, you can associate them with the game. Over time, you'll discover all the possible state, and if it's cached, you can compile all the cached shaders at startup.

Anyhow, fun stuff. I used to love work like this. I've implemented 3DFx's Glide API on top of DX ages ago to play Voodoo games on my Nvidia cards, and contributed some code to an N64 emulator named UltraHLE.

Ask your neighbors with solar who they used and if hey liked working with them, and also get a lot of estimates from lots of local contractors. You will see all kinds of system proposals when you do this. Go with the contractor that you like dealing with who has a good system proposal. A good sign for me was when someone was willing to make changes; eg, use micro inverters vs optimizers for my complex roof geometry.

You can't figure out who's good online, interview the local companies. Locals will also get you through the permitting and code compliance process too.

There are companies attempting to recycle them into new batteries, such as Redwood Materials, but from what I know, recycled lithium is more expensive than fresh lithium today.

The problem with used EV batteries is that they've started to degrade, and they degrade in chaotic ways, so you can't offer a predictable product made from old cells. Some cells may have shorts internally, others may have evaporated some electrolyte, or the electrodes may have degraded. Right now, lithium recovery is quite primitive from used cells. I've tried to reuse used batteries myself for storage, and the unpredictable wear made me give up.

Also, EV batteries, which are optimized for power density, may not be the best choice for home storage, where you want the ability to deep cycle to buffer power usage as the NYT article describes. The NMC cells common in EV's don't like to sit at above 90% state of charge (this cutoff is arbitrary, but > 90% results in fast breakdown), and they don't like to go below 20%, so you have a useful range of 70% of the capacity. You can over-provision by 30% or you can use lithium-iron-phosphate cells, which are less power dense, but much more tolerant of deep cycling.

I set my home up like this a long time ago. I use 100% of my solar and export nothing to the CA grid due to batteries. It's not cost effective to do this given the cost of storage when I set this up, but it's really neat to someone of my nerdy predisposition. My goals originally were to have solar based backup power, because I lose power quite a lot despite living in silicon valley, and it's worked great for that too.

It's a nice change for little experimental programs, but production servers need lots of functionality that third party routers offer, like request middleware, better error handling, etc. It's tedious to build these on top of the native router, so convenience will steer people to excellent packages like Gin, Echo, Fiber, Gorilla, Chi, etc.

The panics are really annoying. Sometimes, you generate routes dynamically from some data, and it would be nice for this to be an error, so you can handle it yourself and decide to skip a route, or let the user know.

With the panic, I have to write some spaghetti code with a recover in a goroutine.

The degree itself is unimportant, but there are important skills that a CS program teaches, which you can teach yourself, but that degree makes getting interviews a lot easier. Like I said, I've been doing this for a very long time, and worked with many hiring processes.A person without either experience or a recognized degree has a very difficult time getting their foot in the door. I may have misread and thought that the OP was interested in engineering.

For PM's, or management, the skill set and showing that you have it will be different.

If you have the means, get a computer science degree at a reasonable school, and don't listen to people telling it's too late, and that ML is your meal ticket. We have two kinds of jobs here in the valley, the glamorous and competitive, and the really challenging and necessary. The latter are more immune to hype cycles and economic downturns. If you do something along the lines of networking/security, cloud infrastructure, and learn something that confuses other people, say, how to use OAuth2 properly as an example, you will be able to work on the infra side in almost any company. Once you get your foot in the door, then you can learn the latest hottest fad wherever it is that you are, on the job. Infrastructure is the computer industry version of cleaning out the stables for the horses, but it's also necessary everywhere and once you prove yourself and show you are capable of learning on the job, you can work on other stuff. If you start to enjoy infra stuff, you can work anywhere. Don't get stuck doing any kind of process though, don't be the compliance guy, for example.

The biggest shortage in silicon valley is that of capable minds and hands. It's on you to make yourself marketable, but once that's done, there's tons of opportunity. I've been working here since 1996. I'm a grey haired old timer now, and I've seen this industry from big companies, startups, boring companies, and fad companies. Get your foot in the door, the first job won't be glorious, but as you demonstrate skill, pay and rank will follow. Don't go to any of the companies staffed by lots of startup bros, because a wrinkle or some greying hair is a disadvantage there, but there are plenty of other places to start.

I've been running pgBouncer in large production systems for years (~10k connections to pgbouncer per DB, and 200-500 active connections to postgres). We have so many connections because microservices breed like rabbits in spring once developers make the first one, but I could rant about that in a different post.

We use transaction level sharing. Practically, this means we occasionally see problems when some per-connection state "leaks" from one client to another when someone issues a SQL statement that affects global connection state, and it affects the query of a subsequent client inheriting that state. It's annoying to track down, but given the understanding of behavior, developers generally know how to limit their queries at this point. Some queries aren't appropriate for going through pgbouncer, like cursor based queries, so we just connect directly to the DB for the rare cases where this is needed.

Why so many connections? Say you make a Go based service, which launches one goroutine per request, and your API handlers talk to the DB - the way the sql.dB connection pooling works in Go is that it'll grow its own pool to be large enough to satisfy the working parallelism, and it doesn't yield them for a while. Similar things happen in Java, Scala, etc, and with dozens of services replicated across multiple failure domains, you get a lot of connections.

It's a great tool. It allows you to provision smaller databases and save cost, at the cost of some complexity.

In the US, you pay state tax and federal tax separately, so those are only the state taxes. If you're paying the maximum marginal rate in CA, you're also paying 37% federal rate and a 3.8% surcharge on investment income on top of that.

You pay surcharges not included in the base tax rate as income goes up. There's a 1% surcharge over $1M income that's from a passed proposition. I've never hit it, doubt that I will, but it's there.

Come to CA.

If you make over $66k, your income tax is 9.3%, going up to 13.3% if you make FANG money.

We have a roughly 10% sales tax.

You'll pay about 1.25-1.5% of your overpriced home's value in property tax annually.

And your car is going to cost you a few $hundred to register every year.

If you're a fan of any specific kind of tax, we've got them all here.

Why is 350ppm the perfect concentration? Higher is better for plants, because most are carbon limited, for example. The Carboniferous era, whose biosphere sequestered so much carbon, had far higher concentrations.

I’m not disagreeing, but that seemed like a statement out of the blue.

This framework looks wonderful.

Some people have an aversion to "goto" statements, and Java annotations are even worse, they're a "comefrom" statement, where you end up executing code before or after your function based on this annotation, so it makes the code really annoying to follow.

Java examples which show trivial code annotated with @POST or @Path are not representative of production systems, where you may have a lot more annotations for your DOM, documentation, and in some cases, you actually have more annotation boilerplate than you have code in the handler/controller.

Having annotations interleaved within your logic makes it difficult to provide good API documentation, and it's hard to automatically refactor, because your boilerplate is interleaved with real code. With an approach like this grumpyrest, you can put all your machine generated code into a package, and simply connect it to your hand written code with a little bit of glue. It makes spec-driven development much easier.

OpenAPI is very popular, and annotation based frameworks make it more difficult to integrate with it. If you generate API docs automatically from code, as with JAX-RS, it's easy to break things by accident because nobody audits machine generated docs. If you reverse the approach, and do spec-driven development, you code review the API behavior, and the code follows, which is a better model, in my opinion. Grumpyrest looks like it makes integration with spec-driven workflows quite easy.

A word of caution to the author; if this takes off, you will be inundated with issues and PR's, since people will use this in ways you never dreamed of. I'm experiencing that kind of onslaught in something I open-sourced for Go for REST API's.

A completely disagree with you on IRIX.

I'm biased, because I did work with IRIX at SGI and I did in fact touch some kernel code as well.

What I miss from IRIX, that no other system has yet replicated: 1) Realtime mode. RTLinux doesn't count. In IRIX, you could run your own code at a higher priority than the scheduler itself (this was called hard realtime), and you'd give it time slices when you could, or a core or two. 2) Frame scheduling. When rendering 3D, you could have the scheduler connected to the monitor refresh rate to make you less likely to miss a frame boundary.

Yes, Motif was very plain, and X11 was a hot mess to work with, but the thing had capabilities which are still hard to find.

Nah, for me, that wasn't sufficient. I get at least 10 hours a week of full sun on my whole arms and legs and sometimes my torso if I decide to bike shirtless; more sun than most people, and still, I was vitamin D deficient.

Monthly blood tests prescribed by a doctor at a local bloodwork lab.

I have some hard numbers on this in my own health. I'm a pasty north European living in California and I get plenty of sunshine, as I enjoy the outdoors. As I got older, I got increasingly more viral infections and strep throat was a twice a year affair. It didn't occur to me that I could be vitamin-D deficient, but I was, severely. For some reason, 10+ hours of sunshine a week isn't enough for me to make my own.

Over months, I tracked my blood vitamin D levels as I took supplements, and now I know that the 2500IU dose is enough to keep me at nominal levels - the higher doses cause it to build up too much and would eventually cause liver issues.

I know it's anecdotal, but in the few years since I did this, those recurrent sicknesses have vanished. I'm also taking vitamin B, which is another one you get less efficient at absorbing as you age.

The days of tuning software have changed. In the past, these systems were indecipherable and had no good profiling or debugging tools, so they came with a lot of hardware connected which allowed you to profile and tune and debug.

If you wanted to profile PS2 code, for example, the way you did that was with a gigantic logic analyzer connected to various traces at Sony's labs. We used their Redwood City lab. You'd give Sony instrumented code, and they'd do some kind of secret magic on it and run it on this instrumented PS3, and a few days later, you get a profile! Downside, this thing is incredibly complex, but on the upside, you had little performance penalty for profiling. Later, these kinds of tools came built into dev kits.

For the PS3, programming the Cell was quite difficult and so you had to do lots of tuning. The single PPC core (called the PPU) was really slow, so you had to offload what you could to the SPE's, which were incredibly fast on floating point math, but it was on you to interleave DMA and computation in a way to get good utilization of them. The Dev kits had tools to give you visibility into this.

Programming on a laptop only requires the laptop, as you've got all the tools you need in the OS. Only apple does debugging at the board level.

It's fun to take a stroll down memory lane.

I'm a Brown grad, who graduated in the mid 90's. In the mid 90's, Brown's tuition was an unthinkable $32k/year, if I remember correctly. As an immigrant with no family assets, I got a big financial aid package, and being a Rhode Islander, I also got a state grant, and took the rest as loans. It was just barely doable for someone who didn't have family assets. Today, it seems truly impossible.

Since I'm from Providence, I visit Brown regularly when I visit my family, since I have lots of memories from the time. Today's campus looks very little like it did in the 90's. We had dumpy dorms, crappy cafeterias, and rather plain, but functional buildings. Today, the dorms have been renovated or rebuilt to be much more luxurious, the school has built a number of incredibly opulent buildings, and there's just a whole lot more space and more wealth on display. The total enrollment went from somewhere around 8,000 then to about 10,000 today, so it's not that they're building these things to handle more students.

It seems that per-student spending has gone way up, and they're now providing a luxury education experience, not just a degree.

Lab grown fish are one thing, but simple fish farms already work really well, the issues are that the fish don't eat a natural diet and are sicker and less healthy to eat. Maybe that's an easier problem to solve than growing fish meat in a lab.

This is different than fish, as tungsten is recyclable and doesn't rot.

Read up on the Simon/Ehrlich wager. Nobody knows the future, but your professor was demonstrating a solid economic principle.

You must have worked on this earlier than me. I started with DX7 on Windows, before that I worked purely in OpenGL on workstations on high end visual simulation. Yes, in DX7 we used printf debugging and in full screen-only work, you dumped to a text file or as you say, MDA if necessary for interactive debugging, though we avoided that. DX9's visual debugger was great.

I don't remember console development fondly. This is 25 years ago, so memory is hazy, but the GameCube compiler and toolchain was awful to work with, while the PS2 TOOL compile/test cycle was extremely slow and the API's were hard to work with, but that was more hardware craziness than anything. XBox was the easiest when it came out. Dreamcast was on the way out, but I remember really enjoying being clever with the various SH4 math instructions. Anyhow, I think we're both right, just in different times. In the DX7 days, NVIDIA and ATI were shipping OpenGL libraries which were usable, but yes, by then, DX was the 800lb gorilla on windows. The only reason that OpenGL worked at all was due to professional applications and big companies pushing against Microsoft's restrictions.

I was directly involved in graphics during this time, and was also a tech lead at an Activision studio during these times of Direct3D vs OpenGL battle, and it's not that simple.

OpenGL was the nicer API to use, on all platforms, because it hid all the nasty business of graphics buffer and context management, but in those days, it was also targeted much more at CAD/CAM and other professional use. The games industry wasn't really a factor in road maps and features. Since OpenGL did so much in the driver for you, you were dependent on driver support for all kinds of use cases. Different hardware had different capabilities and GL's extension system was used to discover what was available, but it wasn't uncommon to have to write radically different code paths for some rendering features based on the cababilities present. These capabilities could change across driver versions, so your game could break when the user updated their drivers. The main issue here was quite sloppy support from driver vendors.

DirectX was disgusting to work with. All the buffer management that OpenGL hid was now your responsibility, as was resource management for textures and vertex arrays. In DirectX, if some other 3D app was running at the same time, your textures and vertex buffers would be lost every frame, and you're have to reconstruct everything. OpenGL did that automatically behind the scenes. this is just one example. What DirectX did have, though was some form of certification, eg, "Direct X9", which guaranteed some level of features, so if you wrote your code to a DX spec, it was likely to work on lots of computers, because Microsoft did some thorough verification of drivers, and pushed manufacturers to do better. Windows was the most popular home OS, and MacOS was insignificant. OpenGL ruled on IRIX, SunOS/Solaris, HP/UX, etc, basically along the home/industry split, and that's where engineering effort went.

So, we game developers targeted the best supported API on Windows, and that was DX, despite having to hold your nose to use it. It didn't hurt that Microsoft provided great compilers and debuggers, and when XBox came out, which used the same toolchain, that finally cinched DX's complete victory, because you could debug console apps in the same way you did desktop apps, making the dev cycle so much easier. The PS1/PS2 and GameCube were really annoying to work with from an API standpoint.

Microsoft did kill OpenGL, but it was mainly because they provided a better alternative. They also did sabotage OpenGL directly, by limiting the DLL's shipped with windows to OpenGL 1.2, so you ended up having to work around this by poking into you driver vendor's OpenGL DLL and looking up symbols by name before you could use them. Anticompetitive as they were technically, though, they did provide better tools.