devbug

I recently built out training and inference at a FinTech (for fraud and risk) using Elixir and tried this very approach…

We’re now using Python for training and forking Ortex (ONNX) for inference.

The ecosystem just isn’t there, especially for training. It’s a little better for inference but still has significant gaps. I will eventually have time to push contributions upstream but Python has so much momentum behind it.

Livebooks are amazing though and a better experience than anything a Python offers sans libraries.

H12SSL-i or H12SSL-NT

ROMED8U-2T

Amazing.

Do you know if Telkom offered ISDN around ‘95 thru to ‘01?

My parents had “two lines” back then so we could always have access to the Internet, even while talking on the phone. I recall, vaguely, using dial up but we may have had an ISDN line. In part, because my Dad wanted to order LaserDiscs from the US while my Mom was chatting away to family.

You can LISTEN/NOTIFY. Or you can use logical replication and a custom subscriber.[1] Supabase uses the latter.[2]

[1] https://www.postgresql.org/docs/current/logical-replication....

[2] https://github.com/supabase/realtime

There’s some articles floating around that touch on the idea. Bitsquid comes to mind. [1]

To give a slightly more detailed description, you allocate two arrays of the extents. One is the dense array of whatever type, say T, and the other is the sparse array of 32 bit or 64 bit integers. The sparse array stores indices into the dense array for extant objects. However, for extinct objects, the indices instead point to the sparse array itself to encode a free-list.

To insert, in O(1), you check the free-list for an unused slot. If there is none, then you grow the sparse array (increment a counter) and use the newly introduced slot. Then you insert (or construct in place) your object at the end of the dense array. Finally, you map that slot in the sparse array to the end of the dense array.

To delete, in O(1), you swap the object you wish to delete with the last in the dense array and decrement the counter (swap and pop). Then you add the index in the sparse array to the free-list. You’ll need to know the index assigned to the object you swapped with to do this, which can either be stored on the object itself or by introducing another array the maintains an inverse mapping.

To iterate, you can treat the dense array like any other array.

This scheme poses some challenges:

- You lose pointer stability because you are relocating your objects. You must access them through indices. If you can defer your deletions, say to a frame boundary, you assume your pointers are stable.

- Indices, without further work, are not safe. They are just as safe as pointers. Usually you don’t expect more than N objects, so you can use log2(N) bits for the forward and reverse mappings between sparse and dense arrays and then use the leftover bits to identify stale indices. You can, and should, encapsulate this in a type-safe handle system. [2][3]

- High churn can cause index invalidation to fail (if you wrap a counter) and performs worse than other data structures.

- “Swap and pop” requires T to be trivially copyable, or ideally, trivially relocatable.

It’s also worth mentioning that you can reorder the dense array to maintain custom invariants. This is really useful if you have dependencies between objects and want to iterate in order.[4]

[1] http://bitsquid.blogspot.com/2014/08/building-data-oriented-...

[2] https://floooh.github.io/2018/06/17/handles-vs-pointers.html

[3]: You usually can use 24 bits for index and 8 bits for a discriminator (counter) in games. Handles (that you pass around) can be wider to exploit register widths and increase safety, since you have 32 bits for further runtime checks.

[4]: By doing more work at insertion and deletion, a constant factor on O(1), you can save work on iteration or updates, a constant factor on O(n). A great example of this being a worthwhile optimization is for scene graphs or transform hierarchies. If you partially order by depth in the hierarchy, you can guarantee correct computation without chasing indexes or pointers. If you maintain additional information about boundaries, you can trivially parallelize computation of transforms or reduce redundant work.

At a quick glance, this is very close. However, the proposal goes through several contortions to maintain pointer stability. If you preallocate sufficient storage for your worst case (which can be unmapped virtual memory) then it makes implementation a lot simpler.

Neat that the committee is looking to introduce it. Too bad we eschew the STL (and even CRT) in games.

A similar data-structure that is also very useful uses sparse-dense arrays but encodes a free-list in the unused memory. It's an ideal data structure if your access patterns are sporadic reads/writes bookmarked by frequent complete iterations of the dense array. You can insert and remove (swap 'n' pop) in O(1) and iterate in O(n) without polluting your cache and skipping over empty slots.

It's frequently used in Entity-Component Systems (ECS) which introduce safety by associating a counter with each entry in the sparse array (usually called a "generation") and incrementing them on deletion. You can also verify cross-linkage between the sparse and dense arrays like the data structure described in the article, but that comes at the cost of another indirection and fetch from memory.

This echos my experiences, especially with the move to “live services” model for games. Often the teams working on the services for multiplayer games are very small so it’s all about leverage. They’re also embedded within a culture that doesn’t value or understand the practices and tools that are taken for granted in webdev, mostly because they’re hard to transfer to the other domains in gamedev.

I’m glad to see the open source development of these tools. Personally, there’s a lot of stuff colleagues and I have built that I would love to see open sourced. Otherwise we just end up implementing the same services over and over again.

Another example is patch delivery. It’s a solved problem yet everyone keeps rolling their own, or only shipping to Steam. Of course, all managed with some Jenkins scripts written by someone that’s no longer at the company. I want Fastlane for games that makes it easy to target the various distribution channels across desktop/mobile/console.

Really cool!

I've been meaning to implement SGP4 from scratch as a learning exercise. What I found really interesting is how the USAF/NORAD tracks and reports objects in LEO: they publish Two-line Element Sets (TLEs), which are a fixed-width ASCII format derived from punch cards.[1] The format is pretty easy to parse.[2]

[1] https://en.wikipedia.org/wiki/Two-line_element_set [2]: https://www.celestrak.com/NORAD/documentation/tle-fmt.php [3]: https://www.celestrak.com/NORAD/documentation/checksum.php

Brute force search is how Scaylr does it. Here’s an article they wrote about their implementation.

https://www.scalyr.com/blog/searching-1tb-sec-systems-engine...

Windows bundles software decoders (for H.264 and H.265) that are available through MFT and DXVA.[1][2] Chromium leverages both already.[3] It's also a low effort way to support hardware-accelerated decoding as vendors (Intel, AMD, NVIDIA) register MFTs that wrap their propriety SDKs (Quick Sync, AMF, NVDEC).

These provided MFTs have to meet certain minimum requirements for certification by Microsoft, which amounts to "good enough." Results vary in practice and can be incredibly frustrating (especially encode). It's often better to use the underlying SDKs but that comes with a lot of hassle.

(I've been working on a video capture and sharing app inspired by Shadowplay.)

Windows 10 is also rolling out support for AV1, though not bundled.[4]

[1] https://docs.microsoft.com/en-us/windows/win32/medfound/h-26...

[2] https://docs.microsoft.com/en-us/windows/win32/medfound/abou...

[3] https://chromium.googlesource.com/chromium/src/+/master/medi...

[4] https://www.microsoft.com/en-us/p/av1-video-extension/9mvzqv...

Worth noting that you pass in data in a proprietary binary format. [1]

I've done it, not out of performance concerns, but because transcoding between proprietary binary formats with types is a lot saner than the alternative.

[1] https://www.postgresql.org/docs/current/sql-copy.html#id-1.9...

The key differentiator is x86 vs PPC and 1 TB/s bus.

    Location: Vancouver, Canada
    Remote: Yes (done it before), or On-Site
    Willing to relocate: No
    Technologies:
      C/C++, Python/Ruby, Erlang/Elixir, Java, and HTML/CSS/JavaScript
      MySQL/PostgreSQL, Redis, BigQuery, BigTable, ElasticSearch, RabbitMQ
      Win32/Linux/BSD, AWS/GCP, Docker/Kubernetes, Scrapy/numpy/pandas
      OpenGL, Direct3D 11, WebAssembly
    Résumé/CV: https://mtwilliams.io/#cv
    Email: me@mtwilliams.io

Experienced in building distributed systems that operate at scale. Most recently, was the only data engineer for 15 million user video game that routinely saw 1,000-10,000+ req/s per microservice.

Hi!

I really enjoy working on things that operate at scale (in users or data) or provide a lot of leverage for my teammates. I'm open to a lot of roles. Most recently I was the sole data engineer for several million MAU video game.

(I'm currently a Physics student, which means I can only commit ~30hrs/wk.)

    Location: Vancouver, Canada
    Remote: Yes (done it before), or On-Site
    Willing to relocate: No
    Technologies:
      C/C++, Python/Ruby, Erlang/Elixir, Java, and HTML/CSS/JavaScript
      MySQL/PostgreSQL, Redis, BigQuery, BigTable, ElasticSearch, RabbitMQ
      Win32/Linux/BSD, AWS/GCP, Docker/Kubernetes
    Résumé/CV: https://mtwilliams.io/#cv
    Email: me@mtwilliams.io

Latency kills: memory latency, network latency, and storage latency. In that order.

In essence, an interrupt storm[1] caused by a design flaw that resulted in the rendezvous radar reporting erroneous readings at a high frequency.[2] The computer couldn't handle all of this work on top of what it was already doing (too much work, too little time) so ultimately it had to prioritize and drop some of the less important work.

[1] https://en.wikipedia.org/wiki/Interrupt_storm [2]: https://arstechnica.com/science/2019/07/no-a-checklist-error...

I love lifting for exactly this reason. It's meditative.

I disagree.

Near realtime (say sub 1-minute) feeds can be leveraged to extract a lot of value out of the data you're collecting. Perhaps maybe not for your typical SaaS startup, but for free-to-play video games you end up leaving so much value on the table if you can't quickly (automatically) respond to spend or churn indicators.

I think a good rule of thumb for this is looking at how fast your decisions are made. If you're making decisions daily then refreshing your data every 24 hours is probably good enough. If you're making decisions every hour, then every 60 minutes is probably good enough.

Another factor is scale. When you're dealing with 6-figure CCUs and are trying to optimize conversion or retention through split-testing, figuring out which variants are anomalously poorly performing quickly can save you a whole lot of money.

I reckon there's at least a 50 titles that can benefit from streaming analytics, which immediately affects anywhere from 100-500 employees (analysts, engineers) and likely influences over 1000 (broader company). That's a significant portion of the field.

Logistics is your force multiplier. Sacrificing the (excessive) performance of 7.62 to improve the rounds you can field is well worth it. That said, 5.56 still holds up well in CQB, provided you have a sufficiently long barrel and a bullet with decent ballistics. But therein lies the problem. The move to shorter barrels means lower muzzle velocity which severely gimps a round that leans on velocity for its lethality. The move to shorter barrels has also necessitated the move to piston-operated guns like the HK416, but they don't come without negatives, especially when guys are over-gassing out of fear of stoppages.

It takes ~20 LOC to setup scaleable asynchronous I/O w/ IOCP. It's been like that since 1996. So that's the benchmark as far as I'm concerned.

It's a pretty useful attack vector since you can get an arbitrary program to load your payload under certain circumstances, so you don't even need malicious code running if you can find a vulnerable target. cough SharePoint cough

I learned something a few days ago that made me both laugh and cry:

LoadLibraryA will happily load a DLL from WebDAV through a UNC path. Something like \\example.com@80\path\to\payload.dll.

It goes without saying that this has been abused by viruses to surreptitiously fetch their malicious code.

As someone in the midst of building a game in C for 7 platforms, with WebAssembly being one of them, my main disappointment is with the lack of coroutines (or lack of control over the stack to implement them.) It hinders how wide my engine can go since I'm limited to a fork-and-join model for splitting work across threads. Poor code generation is also another pain point, but I fully expect that to improve drastically over the coming year.

Overall, I'm pretty excited for WASM and the implications of it, but it does feel like the web has regressed in the ability to deliver games.

All fun and games until you accidentally commit fraud.

I don't get the hate. I can reason about asyncio code without fuss and the throughput/latency impact is worth it. My only pain point is the derth of libraries. There are few if any palatable libraries that provide the powerful abstractions I'm used too. Things like ORMs.

Talk about timing!

I'm in the process of rejigging our live telemetry pipeline (dealing with petabytes) to improve flexibility and resiliency and have been looking for something to replace our similar systems that have grown organically. This fits the bill exactly; it's designed exactly how I imagined our new system should look.

I'm wondering if HN has their personal favorite for these sorts of problems.

And yet it still doesn't respect the Insert key.

https://marketplace.visualstudio.com/items?itemName=adammara...

Deadlifts and squats. If you're a newbie, follow Mark Rippetoe's "Starting Strength."

I recently used RCTEs in a production environment to operate on reasonably large graphs. We eventually moved to an in memory solution built on top of ETS for various reasons, but performance was not a deciding factor (at least as much as you would think); Postgres handled our queries well beyond expectation, even with heavy factoring into functions. It's definitely not slow.

The largest burden was on development and maintenance. Iterating algorithms and queries powered by RCTEs (and functions) on a changing schema is not scaleable (in effort), at least with the traditional migration scripts approach to upgrades and rollbacks. If you do want go this route, you need to change how you ship those functions and queries.

We theorized that shipping them as part of the application and performing necessary migrations behind-the-scenes would have drastically reduced the maintenance burden. We also considered composing queries completely dynamically and fully leveraging our language (Elixir) to provide a toolbag of functions that aren't functions as far as Postgres was concerned. In either case, the hassle of versioning functions and queries would no longer get in the way iteration.