syntacticbs

I also worked on trading systems.

Usually two methods `onMessage(long timeNow, byte[] buf)` and `onTimer(long timeNow, int timerId)`.

All output sinks and a scheduler need to be passed in on construction of the application.

Then you can record all inputs to a file. Outputs don’t need to be recorded because the inputs can be replayed but it is useful for easier analysis when bugs happen.

I have even worked on systems where there were tools that you could paste recorded input and outputs to and they code generated the source code for a unit test. Super useful for reproducing issues quickly.

But you are spot on in that there is an overhead. For example, if you want to open a TCP socket and then read and write to it, you need to create a separate service and serialise all the inputs and outputs in a way that can be recorded and replayed.

Interesting approach. I often get to the same goal by using the replicated state machine pattern. Where all inputs to a system are recorded. Both methods seem to rely on designing your application in a very specific way to be able to replay inputs and deterministically get the same outputs.

I run Linux on my laptop and desktop and have a MacBook.

I wouldn't recommend it for your laptop unless you're an enthusiast. Suspend/sleep is still janky, power saving is poor and dock support is hit and miss depending on the day. And this is after applying a lot of tweaks.

For desktop it's fantastic and I would recommend. It's not without some issues but what OSes is actually without any issues.

For server, I can't imagine using anything else.

The local scope leak seems to only happen when you drop down the call stack. See below how I can call func2 from the top level and it's fine, but if I call it from within func1, it will leak.

  #!/bin/bash
  
  declare -A map1=([x]=2)
  echo "1. Global scope map1[x]: ${map1[x]}"
  func1() {
      echo " * Enter func1"
      local -A map1
      map1[x]=3
      echo "   Local scope map1[x]: ${map1[x]}"
      func2
  }
  func2() {
      echo "  * Enter func2"
      echo "      Local scope map1[x]: ${map1[x]}"
  }
  func1
  func2
  echo "2. Global scope map1[x]: ${map1[x]}"

UPDATE: I did a bit of exploration and it turns out ANY variable declared `local` is in the scope of a function lower down in the call stack. But if you declare a variable as `local` in a called function that shadows the name of a variable in a callee function, it will shadow the callee's name and reset the variable back to the vallee's value when the function returns. I have been writing bash for years and did not realise this is the case. It is even described in the man page: When local is used within a function, it causes the variable name to have a visible scope restricted to that function and its children.

Thank you. You have taught me two things today. One is a bash feature I did not know existed. The second is a new reason to avoid writing complex bash.

Oh dear. I've been trying to get people to not use this feature for a while.

One thing that has bitten me in the past is that, if you declare your associative arrays within a function, that associative array is ALWAYS global. Even if you declare it with `local -A` it will still be global. Despite that, you cannot pass an associative array to a function by value. I say "by value" because while you can't call `foo ${associative_array}` and pick it up in foo with `local arg=$1, you can pass it "by reference" with `foo associative_array` and pick it up in foo with `local -n arg=$1`, but if you give the passed in dereferenced variable a name that is already used in the global scope, it will blow up, eg `local -n associative_array=$1`.

As a general rule for myself when writing bash, if I think one of my colleagues who has an passable knowledge of bash will need to get man pages out to figure out what my code is doing, the bash foo is too strong and it needs to be dumbed down or re-written in another language. Associative arrays almost always hit this bar.

I completed both parts of this on Coursera. It can be a very challenging course at times but it's thoroughly rewarding. One of the key takeaways was a working mental model of how the hardware and low level software that underpins your code works.

Pre-allocating up front would also have been valid way of doing things.

For the specific use cases I have worked on though, I'm not sure it would have had any additional performance benefits. The specific class of systems I've worked on usually have only a few sockets which open at the start of a day and stay connected until the end of a day. Pre-allocating would make the socket opening process faster but that is not usually the part of an application life cycle that needs optimising. If there were lots of sockets opening and closing or the speed of opening and closing needed optimisation, then what you suggested would be a good idea.

I've worked in applications where a lot of IO is required. If performance is something your application cares about then you'll probably end up using direct ByteBuffers which are off heap and you'll likely want to set a sensible value for: -XX:MaxDirectMemorySize

However if this value ever does get exceeded, you need some way of tracking down what allocations happened prior to your OutOfMemory exception.

Though the above article implies the sampling rate is once a second (I guess there's some cost to increasing that rate). Usually you won't be allocating direct memory on the reg since it's expensive to allocate and deallocate relative to heap memory so you kind of want to capture ALL allocations and deallocations. As such a sample based approach is not ideal due to possibly missing some data between samples.