abuckenheimer

excellent explanation, to add to this since I was curious about the composition, '%c' is an integer presentation type that tells python to format numbers as their corresponding unicode characters[1] so

'%c' * (length_of_string_to_format) % (number, number, ..., length_of_string_to_format_numbers_later)

is the expression being evaluated here after you collapse all of the 1s + math formatting each number in the tuple as a unicode char for each '%c' escape in the string corresponding to its place in the tuple.

[1] https://docs.python.org/3/library/string.html#format-specifi...

This is really nice! The context switching that comes from using the [official k8s reference](https://kubernetes.io/docs/reference/kubernetes-api/workload...) is a real pain. If your writing a deployment and need to check one thing about the pod spec all of a sudden you jump to a new page and lost where you were. Ontop of that this keeps track of the indentation level the spec your looking at within the context of whatever parent path your writing it for.

Maybe one nitpick would be to keep colon between the key and the type so one can copy paste multiple lines of relevant spec to be filled in your editor easily.

I've only skimmed the quickstart docs https://kargo.akuity.io/quickstart/ but this snippet immediately clicks for me as something that I'd want to try

    cat <<EOF | kubectl apply -f -
    apiVersion: kargo.akuity.io/v1alpha1
    kind: Promotion
    metadata:
      name: prod-to-${FREIGHT_ID}
      namespace: kargo-demo
    spec:
      stage: prod
      freight: ${FREIGHT_ID}
    EOF

This is clever, otherwise you have to coordinate your cache setter and your async function call between potentially many concurrent calls. There are ways to do this coordination though, one implementation I've borrowed in practice in python is modes stampede decorator: https://github.com/ask/mode/blob/master/mode/utils/futures.p...

python3 -c 'while (1): print (1, end="")' | pv > /dev/null

python actually buffers its writes with print only flushing to stdout occasionally, you may want to try:

    python3 -c 'while (1): print (1, end="", flush=True)' | pv > /dev/null

Very cool, py-spy[1] has been an invaluable tool in my development process since jvns blogged[2] about it. The power of being able to visualize where your code is spending its time is so obvious and I'm glad people are building tools to make that easier.

As a quick compare and contrast between py-spy and pyinstrument it looks like py-spy has the advantage of being able to attach to an already running process which is super useful when your program is stuck and you don't know why. I haven't used pyinstrument yet but I do like the fact that it can do its flame graph in the console, sometimes I find saving down an svg file and opening up the browser a bit arduous. Excited to give it a try.

[1] https://github.com/benfred/py-spy

[2] https://jvns.ca/blog/2018/09/08/an-awesome-new-python-profil...

It's interesting to compare palantir's risk factor on open source vs pivotals[1]. Palantir takes a defensive posture saying basically that their ability to not contribute to open source without running afoul their licenses is important:

Our platforms contain “open source” software, and any failure to comply with the terms of one or more of these open source licenses could negatively affect our business.

Where as pivotal says that being an active part of a vibrant open source community is crucial:

If open-source software programmers, many of whom we do not employ, or our own internal programmers do not continue to use, contribute to and enhance the open-source technologies that we rely on, the market appeal of our offering may be reduced, which could harm our reputation, diminish our brand and result in decreased revenue.

Like this makes sense, palantir is a closed source first kinda company and obviously a different value to its customers than pivotal. Kinda interesting to see that manifestation in the risk factors though.

[1] https://www.sec.gov/Archives/edgar/data/1574135/000104746918...

Just remembered the puck building[1] has these on the sidewalks surrounding the Houston street side, if you go into the Eastern Mountain Sports store and down the stairs you can see them from below.

[1] https://en.wikipedia.org/wiki/Puck_Building

Would highly suggest listening to the "How I built This" podcast[1] with the peleton founder John Foley, really great story with usual tech themes like doing things that don't scale and finding a niche but loyal customer base and expanding organically.

[1] https://play.google.com/music/m/Dmjr6m5i4h3j6gaysgjd47ryhci?...

Yes HFT buys trade flow from robin hood because they make more money executing against it but that's not actually to the detriment of the people on the robin hood app. The main way HFT firms make money is by making a market, they offer to buy and sell stocks cheaper than anyone else and get paid by people crossing the spread and sometimes exchange fees. The reason robinhood trade flow is valuable to HFT firms Isn't because they are trading against "dumb" millennials but because they know millennials aren't likely to move the market playing around on their smartphone. HFT firms can collect a small rent sitting in between millenials trading with one another without the risk of being on the wrong side of a trade that materially moves the price of a stock.

Matt Levine does great write ups on this stuff, would highly recommend: https://www.bloomberg.com/opinion/articles/2018-10-16/carl-i...

got it, thanks for the response!

Right what I'm confused about is that first bit, my understanding from the RFC is that the implementation should have look something like

    return pbkdf2.derive(password, email, PBKDF2_ROUNDS, STRETCHED_PASS_LENGTH_BYTES)
      .then((quickStretchedPW) => {
        result.quickStretchedPW = quickStretchedPW;
        // stretch to twice the length necessary
        return hkdf(quickStretchedPW, kw('generated'), HKDF_SALT, HKDF_LENGTH * 2)
          .then((generated) => {
            // split output into two cryptographically strong keys
            result.unwrapBkey = generated.slice(0, HKDF_LENGTH);
            result.authPW = generated.slice(HKDF_LENGTH);
          }
        );
      }
    )

    hashed_password = hash(password, 'salt1')
    hashed_auth_tok = hash(hashed_password, 'salt2')
    hashed_unwrap_key = hash(hashed_password, 'salt3')

I've never heard of HKDF before but it is really an elegant solution to this. My first guess on how to do this would have been something stupid like split the Authentication token in half and 0 pad it. But this would have significantly reduced the entropy available on both keys, reducing the search space on the authentication token and the encryption key making them much more brute force able. HKDF instead expands the key and essentially requires the server to be able to reverse HMAC-Hash to find the encryption key from the the authentication token.

What I'm confused about is that they seem to be using HKDF as a hash [1] and not as a key generation funciton. I think this is just as secure as what I was expecting but it seems more complicated and doesn't jive with the purpose of the RCA[2] as I read it.

[1] https://github.com/mozilla/fxa-js-client/blob/1d92f0ec458ace... (separate HKDF calls with the same IKM)

[2] https://tools.ietf.org/html/rfc5869

I loved this article on the elegance of deflate http://www.codersnotes.com/notes/elegance-of-deflate/ finishes with this:

It's not one algorithm but two, that dance and weave together in harmony. You can pick dictionary matching, or entropy coding, and you can select between them on a per-byte basis with no overhead. That's what I find so clever about it - not that it can do one thing or the other, but that it can choose either, and yet represent them using the same language. No extra markers, nothing to say "oh, now we're switching to entropy mode", nothing to get in the way.

[disclosure I am a sanic contributor] aiohttp is a great project, very similar to sanic, I'd say the big thing is sanic is slightly more flask like than aiohttp in ergonomics (which is something aiohttp has been working on[1]).

Sanic also makes performance a key development goal so you can feel comfortable the project will scale gracefully. But I will stress I don't have current benchmarks to compare against aiohttp

[1]https://aiohttp.readthedocs.io/en/stable/faq.html#id1

Really I think the specific the problem here with DAF's is just misaligned incentives, the DAF values the donors donation but the government is the one that has to pony up the tax rebate. Obviously there is some auditing mechanism here where the valuation can't be too far from reality otherwise the government would come down on it but in practice it seems like that is one of their benefits. I think the more interesting and subtle problem though is that the rebate is awarded at donation time instead of when the money actually gets put to use which lets the DAF accrue fees on government money.

in a highly stylized example:

1. A donor gives the DAF a $10m yacht

* DAF gives a donor $10m donation receipt

* Government gives donor <donor tax rate> * <donation receipt> rebate i.e. .30 * $10m = $3m

2. DAF liquidates yacht into $9m cash and invests it

* the DAF charges .006 management fee on this for 5 years with a 5% return

3. At the end of 5 years the DAF liquidates its investments and puts all of that money to work on a cause.

* ~$11.2m goes to work on a charitable cause

* DAF has collected ~340k in fees

This is more or less the description the DAF in the article but now but we've simplified to assume a conservative consistent return and that the whole donation gets put to work at once at the end of a set term whereas more likely it would be slowly liquidated over time. Now let's assume that instead of the government paying up the tax rebate when the donation is made that it is paid out when the money left the DAF and went to work on a cause.

1. donor gives a DAF $10m yacht

* DAF values it at $9m and gives the donor <donor tax rate> * <donation receipt> rebate i.e. .30 * $9m = $2.7m

2. DAF liquidates yacht into $9m cash, covers their $2.7m rebate outlay and invests $6.3m

* the DAF charges .006 management fee on this for 5 years with averages a 5% return

3. At the end of 5 years the DAF liquidates its investment and puts all of that money to work on a cause.

* ~$7.8m from the DAF goes to work on a charitable cause + a $2.7m from the government to match that original rebate for a total of ~$10.5m

* DAF has collected ~238k in fees

What's the difference here?

1. The government saves $300k on a rebate because the DAF has to liquidated the yacht to come up with the money to pay the rebate so the valuation is based on something the reflects the actual cash

2. The government keeps <rebate amount> ($2.7m in this case) in its bank account for the 5 years while is in the DAF. This means the government can use it for it's own operations over that time period rather than it sitting in the "warehouses of wealth". It also means the DAF can't collect fees on this money because it is not managing it.

I'm not trying to make a statement on if the government should or should not subsidize donations to charity, just point out how that subsidy seems to be leveraged by the introduction of DAFs against its original intent.

sorry for the terrible formatting

I don't know the Kirill Balunov example is pretty beautiful/simple/flat/readable

    if reductor := dispatch_table.get(cls):
        rv = reductor(x)
    elif reductor := getattr(x, "__reduce_ex__", None):
        rv = reductor(4)
    elif reductor := getattr(x, "__reduce__", None):
        rv = reductor()
    else:
        raise Error("un(shallow)copyable object of type %s" % cls)

    reductor = dispatch_table.get(cls)
    if reductor:
        rv = reductor(x)
    else:
        reductor = getattr(x, "__reduce_ex__", None)
        if reductor:
            rv = reductor(4)
        else:
            reductor = getattr(x, "__reduce__", None)
            if reductor:
                rv = reductor()
            else:
                raise Error("un(shallow)copyable object of type %s" % cls)

I think the reality of this is fairly well captured in the 4th to last paragraph

Even human readers, who may have two minutes to read each essay, would not take the time to fact check those kind of details, he says. "But if the goal of the assessment is to test whether you are a good English writer, then the facts are secondary."

I would go a step further and say the goal of standardized tests is probably not to test if your a good English writer but rather to see if your good at that test. AI has tremendous ability to streamline the grading of standardized tests which are narrowly focused measure and I think that's fine if it comes with proper checks. As many people mention here I don't think AI will be able to objectively measure good general writing and I doubt we'd fully cede artistic evaluation to a computer as a society but I agree with being weary of this.

Standardized tests in general though seem to me like one of the greatest examples of a measure becoming a target and loosing a lot of its effectiveness (Goodhart's law). I can't help but be reminded of Paul Grahm's essay on nerds[1] by this article because the effective point of school from an evaluation standpoint is these tests. However now we've built a computer that grades these test's nearly as well as its human counter parts and its glaringly obvious that this evaluation is tremendously game-able. Which resonates with Paul's argument that: "The problem with most schools is, they have no purpose."

[1] http://paulgraham.com/nerds.html

I feel the same way, although my instinct is generally to build a custom generator. Only costs a couple lines but is plain old python and quite explicit

    target = {'system': {'planets': [{'name': 'earth', 'moons': 1},
                                     {'name': 'jupiter', 'moons': 69}]}}

    glom(target, {'moon_count': ('system.planets', ['moons'], sum)})
    # vs
    def iter_moons(t):
        for planet in target['system']['planets']:
            yield planet['moons']

    sum(iter_moons(target))

FWIW no one who replied to this email thread said something even close to "no". Victor Stinner points out that startup time is something that comes up a lot and mentions some recent work in the area [1].

Python is a big ship, it may not be as nimble as a young FOSS project but it is always improving and investments in things like start up time pays dividends to a large ecosystem.

[1] https://mail.python.org/pipermail/python-dev/2018-May/153300...

+1 been using sanic for a couple years, it's an awesome easily grokable micro framework. Sanic is flask-like in a lot of places where a quick perusal of the quart code makes it look like they go an extra mile on the flask scale, but maybe take up some complexity in that process. Up to the consumer on which trade off they pick. I tend to think that since there is no WSGI equivalent for asyncio (something people are thinking about https://github.com/channelcat/sanic/issues/761) you want a slightly different model than flask anyway.

We do not control and may be unable to predict the future course of open-source technologies, including those used in our offering, which could reduce the market appeal of our offering and damage our reputation.

This is a really cool risk factor that I've never seen before in an S-1, exciting that more big public companies are willing to accept the risk reward that comes with contributing to open source:

If open-source software programmers, many of whom we do not employ, or our own internal programmers do not continue to use, contribute to and enhance the open-source technologies that we rely on, the market appeal of our offering may be reduced, which could harm our reputation, diminish our brand and result in decreased revenue. We also cannot predict whether further developments and enhancements to these open-source technologies will be available from reliable alternative sources. If the open-source technologies that we rely on become unavailable, we may need to invest in researching and developing alternative technologies.

I'm not surprised to see net neutrality mentioned as a risk factor.

Our platform depends on the quality of our users’ access to the internet. Certain features of our platform require significant bandwidth and fidelity to work effectively. Internet access is frequently provided by companies that have significant market power that could take actions that degrade, disrupt or increase the cost of user access to our platform, which would negatively impact our business.

I wonder if you could make an argument that public SAAS companies have a fiduciary duty to their shareholders to support net neutrality policy.

When people say cpython is slow they are generally pointing to two things

1) The interpreter is _slow_

2) You can't achieve real thread based parallelism

I think in general people have a high level concept of (1) as static vs. dynamic and interpreted vs compiled is an understandable trade off. CPython as an implementation generally gets type casted as slow for its default dynamism which have understandable negative performance implications[1]. However CPython also gives you lots of ergonomic ways to push your program towards the static/compiled end of the spectrum with things like pandas/numpy/numba/extensions. In general using these correctly put you within the ballpark of _faster_ languages, could you write faster assembly by hand? Sure! Is optimizing in another language worth your time? I don't know.

I've never really understood (2) the lack of threading as a problem, multi process parallelism can be accomplished fairly easily if you are CPU bound or projects like uvloop[2] make async tasks fast enough to compete with any web framework out there. Furthermore even though your cpu cycle cost may be levered to a considerable point while operating over hundreds/thousands of servers doing distributed computing right is still hard and developing something like celery/airflow/luigi/dask from scratch is not cheap either. Leaning on CPython's massive ecosystem can massively lower the barrier to entry in a lot of big problems.

I think there are plenty of examples of re-writes in go[3]/rust[4] that have worked great for people, I have no doubt that python is _not_ the end all be all language, but I think the "python is slow" worry is generally overplayed.

Specific problems require specific solutions, I'm glad Haskell seems to work for the author in genetic analysis but think this could have been a more interesting article with some specific python-haskell comparisons rather than the generic python is slow argument.

[1] http://jakevdp.github.io/blog/2014/05/09/why-python-is-slow/

[2] https://magic.io/blog/uvloop-blazing-fast-python-networking/

[3] https://web.archive.org/web/20170101002625/http://blog.parse...

[4] https://blogs.dropbox.com/tech/2016/05/inside-the-magic-pock...

fun fact Roku was spun out of netflix just so they could remain neutral https://www.fastcompany.com/3004709/inside-netflixs-project-...

So with JavaScript you've got arrow functions, the method shorthand definition syntax, the spread operator, destructuring assignments, all functional Array methods and async functions.

This confuses me quite a bit, there are nuanced differences between these ideas in the two languages but at the surface these are things that very much exist in both languages

arrow functions:

  (x,y) => { x + y } 

  lambda x,y: x + y

  MyObj = {
      foo(x,y) { return x + y }
      bar(x,y) { return x * y }
  }

  class MyObj:
      foo(self, x, y): return x + y
      bar(self, x, y): return x * y

  function add(x,y) {
      return x + y
  }
  add(...[1,2])

  def add(x, y):
      x + y
  add(*[1,2])

    [a, b, ...rest] = [10, 20, 30, 40, 50];

    a, b, *rest = [10, 20, 30, 40, 50]

    forEach(["Wampeter", "Foma", "Granfalloon"], print);

    list(map(print, ["Wampeter", "Foma", "Granfalloon"]))

    function resolveAfter2Seconds(x) {
      return new Promise(resolve => {
        setTimeout(() => {
          resolve(x);
        }, 2000);
      });
    }

    async function add1(x) {
      var a = resolveAfter2Seconds(20);
      var b = resolveAfter2Seconds(30);
      return x + await a + await b;
    }

    add1(10).then(v => {
      console.log(v);  // prints 60 after 2 seconds.
    });

    async def resolve_after_2_seconds(x):
        await asyncio.sleep(2)
        return x

    async def add1(x):
        a = resolve_after_2_seconds(20)
        b = resolve_after_2_seconds(30)
        return x + await a + await b

    loop = asyncio.get_event_loop()
    loop.run_until_complete(add1(10))

If you look at the design documents for pandas 2 there is a good illustration of how a lot of pain points in pandas 1 spring from numpy ( https://pandas-dev.github.io/pandas2/internal-architecture.h...). I think any significant development effort numpy would probably greatly benefit both libraries.

Will have to check out dplyr :) love to see how they master the magic that is multi-indexes.

I know pandas is a bit meaty for a date time library if you don't already use it but their Timestamp class is awesome. String parsing is a breeze, offsets and timezones are easy and then there's a ton of support for time series.

    In [34]: pd.Timestamp('2016-08') == pd.Timestamp('2016.08') == pd.Timestamp('2016/08') == pd.Timestamp('08/2016')
    Out[34]: True
    
    In [38]: pd.Timestamp('2016') == pd.Timestamp(datetime.datetime(2016,1,1))
    Out[38]: True
    
    In [49]: pd.Timestamp('2016') + pd.offsets.MonthOffset(months=7) == pd.Timestamp('2016-08')
    Out[49]: True
    
    In [52]: pd.Timestamp.now()
    Out[52]: Timestamp('2016-08-17 08:01:07.576323')
    
    In [53]: pd.Timestamp.now() + pd.offsets.MonthBegin(normalize=True)
    Out[53]: Timestamp('2016-09-01 00:00:00')
    

"The negative side of that story is that if there is something you’d like Swarm to do and that something is not part of the Docker API, you’re in for a disappointment."

Isn't entirely a fair exploration of the downsides of Docker Swarm compared to Kubernetes. I agree a lot of the wind is taken out of Kubernetes sails with Docker's new features but certainly not all of it.

This deserves a longer write up then I can give it right now as so few articles on the web right now really delve into the differences between these systems outside saying who maintains but for its attempt I'm pleased to see this article.