Where Banking APIs Fall Short

https://blog.svapnil.com/p/where-bank-apis-fall-short

by svapnil • 2 years ago

27 10 2 years ago

blog.svapnil.com

adzm• 2 years ago

Even HL7 which is used for healthcare communication is still very often communicated over file drops as well. Honestly it has some great advantages in terms of simplicity. Need to reprocess a file? Put it in the folder. Fail to process? Move it to a different folder for later viewing. The biggest issue is just having good SFTP server and client solutions that are scalable on multiple platforms.

svapnil3• 2 years ago

I never knew about HL7, thanks for sharing.

I agree that simplicity and open standards will win. SFTP is straightforward and has worked well in the past. In banking and finance, SFTP is what powers most programmatic corporate banking. Eventually, I think we'll see SFTP and message queues (like SQS) used more often as processing speeds up. I believe message queues are what powers much of real time payment processing already (https://news.ycombinator.com/item?id=36805571)

PaulHoule• 2 years ago

Another approach is to treat it as a document exchange process that is done over REST or RPC. Message queues are attractive though in that they are easily bidirectional.

crowcroft• 2 years ago

The future doesn’t look like an API, it looks like better and better files.

Would be curious what the author thinks of initiatives like Open Banking, which don't seem to be a big topic in the US, but are in the EU and Canada at the moment.

My expectation is that banks should be abstracting away the 'under the hood' part, and so the future would look like an API.

svapnil3• 2 years ago

Open Banking imo === the ISO20022 standard. It already is an "API," just in a format that most web developers aren't used to

habitue• 2 years ago

Differences between files and REST apis:

- files get persisted to disk by default. Usually we don't save every rest result to disk (though obviously, you could)

- files tend not to be processed at request time, but rather the directory acts as a queue. Usually with rest, we try to process things at request time until stuff gets too slow. Files always decouple

- multiple transports are the norm: SFTP, SMTP, etc. doesn't matter since it's not going to be processed at request time, but async. Think of "dropping it into a directory" as stripping off the transport layer the same way your web framework and reverse proxy abstract away http 2/3 for you.

- formats: much more variety than json everything. Back in the day, people liked to write custom parsers for everything. Nowadays, we like to let json be the bytes to in-memory data structure, and programmers recurse over that structure rather than spending their time ensuring their parser isn't ambiguous

svapnil3• 2 years ago

Thanks for sharing habitue

It's true that REST APIs operate under the assumption of processing at request time, while not the case with files. That's what makes Bank REST APIs so tricky - processing doesn't ever actually occur at request time.

There are so many crazy formats that are supported. Simpler is better imo

nicholasjarnold• 2 years ago

That's what makes Bank REST APIs so tricky - processing doesn't ever actually occur at request time.

HTTP has provisions to enable async semantics between the client and the server. I would always advocate for teams to return 202 Accepted responses with a Location header containing the absolute or relative path to an endpoint that can be interrogated for status of the underlying async operation. This can work fine for lower-scale services (or even higher scale services with rate limits enforced at gateway). You POST'd a payment. You get 202 Accepted with some Location value like (simplified) `/payments/<identifier>` for which GET can be used to retrieve current status (e.g. pending, failed, committed, etc).

If a polling model is not desirable you can also use webhooks and inform the caller whenever a given task is finished.

Both of these strategies work fairly well in practice in my experience. However, I Must agree that simply dropping a big file off at an sFTP share and moving on with life is certainly "easier" from the client perspective. But...do you then roll your own mechanism for checking status? Why not just use HTTP semantics that exist already?

orev• 2 years ago

Batch files have been used forever (since mainframes), and I get suspicious when someone implies that APIs are the “normal” way to do things. It makes me think they don’t have enough experience outside of the app development bubble.

Files definitely have limitations (they’re almost always a single table of data—it’s hard to do data structures), and that seems to be driven by the need for someone in the chain the be able to open it in Excel.

merely-unlikely• 2 years ago

that seems to be driven by the need for someone in the chain the be able to open it in Excel.

This is key. I know from experience that the ability to fall back to emailing CSVs around and having operations folks manually process them in Excel is required way more often than your average tech oriented person would ever expect. Plus a lot of processes that might appear to be automated at banks turn out to have someone doing a ton of manual work as part of their daily process. Things gradually and haphazardly eventually get automated but don’t necessarily start that way. Banking seems to oscillate between strict standards with robust technology and random tasks developed when a specific need arose only to never change for long periods of time.

Where Banking APIs Fall Short

Related Stories