HN user

imslavko

366 karma

Slava

Posts10
Comments136
View on HN

I worked on LiveGraph for a long time at Figma. We went through our own evolution:

1. first we would use the WAL records to invalidate queries that could be affected (with optimizations for fast matching) and requery the data

2. then we used the info from the WAL record to update the query in-memory without asking the DB for the new result, it worked for majority of the queries that can be reliably modeled outside of the DB

3. I believe after I left the team reverted to the re-query approach, as managing a system that replicates the DB behavior was not something they were excited to maintain, and as the DB layer got scaled out, extra DB queries were less of a downside

Thoroughly enjoyed the article since I have heard about Neon but never understood what it offers on the technical level over other PG-compatible projects.

The article mentions that a consequence of separating storage from compute is that compute nodes cache hot pages in memory and load cold pages from object storage (like S3?) when needed. Does anyone know what are the consequences of this decision. In case of a query that touches multiple rarely used pages, would that incur high latency and ingress? How does that penalty compare to a vanilla postgres running on AWS and storing pages on EBS?

Oh yeah, a pure function that accepts previous state, and returns the new state is the pattern I use a lot.

The issue is that it is hard to do on complex graph structures in an algorithm where incremental changes happen to the graph O(n) times - it ends up creating complex code and complex execution that might be slow to pass the time limit on Codeforces, let's say.

In the OCaml world maybe this is the place where you say "screw it, this abstracted function does some stateful temporary business, but it looks pure from the outside" but in Haskell it's a lot harder to pull off without going deep into monads (and I forget how those work every time).

Slightly off-topic but what's a good forum to seek help on FP practices outside of the courses like this online?

Every winter break I get back into trying to learn more FP (in Haskell) and in the past several years I have been practicing algo problems (codeforces, advent of code, leetcode).

I always get stuck on more advanced graph algorithms where you traverse a and modify a graph, not a tree structure - it gets particularly tricky to work on circular data structures (I learned about "tying the knot" but it's incredibly challenging for me) and usually the runtime perf is sub-par both asymptotically and empirically.

I only read the article once, but to me it seemed like this was an explicit requirement: produce a vector artifact that renders the same on all platforms without dependencies (basically a PDF?) and without invoking a browser in the build process.

Sort of like folks would praise Go's ability to compile a static binary without dylib dependencies besides libc.

We omitted the details for the sake of brevity (the article is already really long), but we run agents on the test devices and plug them into CI. Agents allow remote access for servicing remotely, and that's enough most of the time.

For things like system updates and taking care of the hardware - we do it manually today. The fleet is still small, so it is manageable but in the future we would like to consider a vendor, if we can find one.

Hey there

we test on macs, windows and linux laptops, it is very surprising that drawing 1 rectangle is painfully slow.

Sometimes it happens when your browser does not enable hardware acceleration or when your Linux distro does not know how to switch to the discrete GPU.

We won't be able to tell without getting more of your hw specs and debug information, feel free to reach out to the Figma support or email me at skim@figma.com - this is exactly the type of issues that elud us when looking at prod metrics in aggregation.

(author of the article)

I want to acknowledge that the load-times for Figma go up linearly with the complexity of your design file + its dependencies. It is always painful when users rightfully complain about giant design files taking a while to load and fully render.

The team is working on changing that so hopefully your experience gets better over time.

This is not my area of expertise, so I am not in the position to promise anything on this forum but I just want to say that the testing framework described in the article is also used to continuously test and measure the file load/parsing time as folks are working towards algorithmic improvements.

That's a good way to address the noise on VMs! We do something different but in a similar spirit: when we compare to the main branch, we calculate the baseline based on 1-2 weeks worth of historical data on main (we identify the latest step change with a simple linear regression). This way we approximate the baseline based on ~100 data points which also helps to address the variance.

Of course re-running the code from main and the PR on the same VM side by side would be the best, and it would cost a lot more money (especially once you factor in GPUs). We considered it but opted to the strategy I outlined above, it's mainly a trade-off between accuracy vs costs

Thank you for your comment!

WASM gave Figma a lot of speed by default for a lot of perf-sensitive code like rendering, layouts, applying styles and materializing component instances, our GUI code is mostly React and CSS.

WASM engine performance has not been a problem for us, instead we are constantly looking forward improvements in the devex department: debugging, profiling and modularization.

One of the largest challenges of the platform we face today is the heap size limit. While Chrome supports up to 4GB today, that's not yet the case for all browsers. And even with that, we are still discovering bugs in the toolchain (see this recent issue filed by one of our engineers) https://github.com/emscripten-core/emscripten/issues/20137

The challenge of the perf-testing at scale in our company is helping developers to detect perf regressions when they don't expect them - accidental algorithmic errors, misused caches, over-rendering React components, dangerously inefficient CSS directives, etc.

Hey, I am one of the authors of the article and the systems described.

The default 20% margin of error is indeed pretty wide, and it is intended to catch large obvious regressions (e.g. an algorithm accidentally becoming quadratic instead of being linear)

As we described in the blog post, we have the second system based on the real hardware. This system is on-demand. If an engineer has a suspect commit or an experimental change that might affect performance, they would schedule a 30min-1hr run on that queue, where we run selected benchmarks 9-15 times each on laptops of various strength. In this configuration, the margin of error is closer to 2-3% from our observations so far. To get more confidence, you would want to run even more trials, typically we advise 9 iterations, though.

We also do all our daily benchmarking on those laptops too.

Edit: in addition to preventative testing, we also track production metrics in a similar way as described by the sibling comment

Asana S-1 6 years ago

I never worked at Asana but I heard it was both, it had code sharing between client and server, some server-side javascript before node.js (I think it was based on JSC) and something similar to isomorphic javascript before this term was coined by Airbnb. If you look at Meteor, a lot of similar ideas migrated there.

I wanted some useable web/mobile version of org-mode synching from Dropbox for a long time! Just to confirm: is this a product non-free product?

Pointless 6 years ago

Could you elaborate? I thought the name was a reference to a "point-free style" of piping commands. But maybe I am missing something more obvious?

I am not arguing with this. In my comment my intend was to show that the distinction between the memoized top-down approach and bottom-up (the approach that is usually taught as DP in a classroom) is significant in some memory optimizations.

This is a comment to demonstrate the differences between DP and memoized recursion to people in the sibling comments.

When I was learning DP vs Memoization I thought Floyd-Washall algorithm to find shortest path length between all pairs of nodes in a graph is a good example of DP that wouldn't work the same way with memoization.

In FW algorithm, because of the order of filling up the table and discarding old values on the go, we are able to run the algorithm only using O(n^2) memory, but if we were to run it in a memoized recursive fashion, I would guess you will have to do with O(n^3) memory.

Another example is when a recurrence in DP only depends on the previous row (if we are going row by row) and you can only keep around O(n) memory swapping two rows representing the current row and the previous one. In a recursive black-boxy memoization method you will have to do with a full O(n^2) memory usage.

Finally, there is a technique to do DP on a table using O(n) memory vs O(n^2) and recovering the optimal path by recursively splitting the table into halves and only storing O(1) rows at a time. This technique is more complicated to explain in an HN comment tho.

Update: forgot the simplest example: fibonacci numbers. In the top-down approach, you would memoize results based on the index of the fib number, so you would need an array of results caching O(n) values. But if you build it up bottom-up, you can get away with only using two variables: previous and current and the do something like:

    prev, cur = cur, prev + cur

The way I solved the terminal emulator shortcommings is this: Install a gtk-based terminal emulator with gtk/gnome dependencies on you ubuntu subsystem and run it with `DISPLAY=localhost:0.0` environment variable. This works if you run XServer Client on your Windows side. I use XcSrv and pantheon-terminal.