lpage

WFQueue isn't handing out &T, so Send is sufficient:

  unsafe impl<T: Send, const N: usize> Sync for WFQueue<T, N> {}
  unsafe impl<T: Send, const N: usize> Send for WFQueue<T, N> {}

https://archive.is/mGsFm

As an auction theorist who's biased towards putting things in auctions and as an operator who's been through the venture investment process several times, I'm biased in wanting a more efficient capital formation process with less adverse selection (better outcomes) for both sides. So, I'm doubly biased, but I appreciate why this hasn't been done and that it could be done.

Classical auctions aren't a good fit because they can't handle non-price factors. However, a combinatorial clock auction could handle non-price factors while also improving price discovery. But as Milgrom says (paraphrased), the best auction is the one people use, so the more realistic outcome is what OP describes: running your own (implicit) auction.

Your mention of Futamura Projections was a nice reminder of how very mathy/theoretical foundations underpin Nice Things in high-level languages, like Hindley–Milner inspired type systems and modern optimizing compilers targeting SSA form. Value Lattices in Cue [1], another config programmable configuration language, also fall into this bucket.

[1] https://cuelang.org/docs/concepts/logic/

Maelstrom [1], a workbench for learning distributed systems from the creator of Jepsen, includes a simple (model-checked) implementation of Raft and an excellent tutorial on implementing it.

Raft is a simple algorithm, but as others have noted, the original paper includes many correctness details often brushed over in toy implementations. Furthermore, the fallibility of real-world hardware (handling memory/disk corruption and grey failures), the requirements of real-world systems with tight latency SLAs, and a need for things like flexible quorum/dynamic cluster membership make implementing it for production a long and daunting task. The commit history of etcd and hashicorp/raft, likely the two most battle-tested open source implementations of raft that still surface correctness bugs on the regular tell you all you need to know.

The tigerbeetle team talks in detail about the real-world aspects of distributed systems on imperfect hardware/non-abstracted system models, and why they chose viewstamp replication, which predates Paxos but looks more like Raft.

[1] https://github.com/jepsen-io/maelstrom/

[2] https://github.com/tigerbeetle/tigerbeetle/blob/main/docs/DE...

This thread [1] covering the 2020 Prize in Economic Sciences, awarded to Paul Milgrom and Bob Wilson for their improvement of auction theory, is a good read.

Vickery Auctions are one of many in the auction format bestiary, and like all classical formats, they predate both computers and modern auction theory. What's really cool these days (and I'm admittedly biased since it's my space) sits at the intersection of computer science and economics. Economic mechanisms and auction formats that use everything from SAT solving and combinatorial optimization to machine learning and function approximation to drive better real-world economic outcomes.

[1] https://news.ycombinator.com/item?id=24753291

Joshua Bloch did a great post [1] on how nearly all standard library binary searches and mergesorts were broken (as of 2006) due to this exact issue. The punchline is that the bugs started cropping up when the need and capability to sort O(2^32) element arrays arose.

[1] https://ai.googleblog.com/2006/06/extra-extra-read-all-about...

Imandra [1] has an excellent literate programming walkthrough that hits on the same issues and plugs the holes one by one using formal methods, ultimately proving merge sort correct.

[1] https://docs.imandra.ai/imandra-docs/notebooks/verifying-mer...

The source itself is commercial AGPL v3.0. I'm guessing that this was an accident so I opened https://github.com/quickwit-oss/chitchat/issues/42 to clarify.

let's say hypothetically you create a market for "+100 FB shares, -500 SNAP shares". If everyone is competing on price to quote that combination, that creates the most competitive market

Some combinatorial auctions make this tradeoff, packaging goods either to deal with computational limitations or to concentrate bidding on a few packages. It can work if there's near total consensus on what the economically relevant packages are, but it doesn't work otherwise. US equities is an "otherwise" case given a huge diversity of needs. The chance of someone wanting the opposite side of even a pairs trade at any given point in time is vanishingly rare. It's far more likely that the person doing the pair would interact with two or more counterparties independently actively interested in or willing to for the right price sell FB and buy SNAP (perhaps conditional on hedging). The mechanism design game is more about giving every party the tools they need to communicate their value function to the auctioneer and creating the incentives to bid (close to) truthfully.

However, if there are many expressive bids with various conditions (e.g. minimum quantities, conditional on execution of another leg, etc), they may not get "implied" into creating a reasonable market, creating exponentially more arbitrage opportunities if they become locked/crossed.

And this is why combinatorial auctions are a global optimization (as opposed to implied, which are effectively an iterative and greedy approximation) and, in our case, one that seeks to find uniform clearing prices. There are formats with price discrimination and others in which mechanical arbitrage within an auction is possible, but ours is not one of them. There isn't a separate price for {A}, {B}, and {A, B} — within each auction there's a uniform price for p_a and p_b, and p_{a,b} = p_a + p_b (and so on for any arbitrary linear combination). Theoretical point: linear prices don't always exist (in practice, they do for interdependent value goods that aren't strongly sub or superadditive, e.g., capital market goods), and they're not inherently desirable. We chose linear pricing largely because it's a natural fit for how capital markets work now, and perceived fairness/simplicity is itself a valid mechanism design consideration.

Congrats on the launch! What are the main benefits of this approach compared to creating additional combo (multi-leg) products on existing exchanges?

Thanks! There are two main differences. For one, combos are (as the name suggests) predefined. That works reasonably well for products like futures and options where the 80/20 approach of making combos for somewhat structural ones like different expiries in the crude and eurodollar complex or packs and bundles designed as standalone financial instruments/hedges. An implied generational liquidity mechanism can knock out some basic structural price arbs between combos, resulting in a combinatorial auction approximation.

This approach falls apart when the combinations are very general as they are in the markets for equities, credit, and many of the assets that trade in the screens.

The CLOB/predefined bundle approach also doesn't address substitutability and non-price factors, and dealing with those is key to unlocking Pareto efficiencies.

There are already a lot of mechanisms in traditional markets that deal with revealing or concealing true demand (e.g. block trades, icebergs, etc)

The problem with block trading venues and other approaches, e.g., conditionals, boils down to incentives. Initiators of block trades are usually going in the same direction, so opportunities for direct interaction/coincidence of wants are rare. And market makers don't want to take large deltas unless they can hedge and/or know the counterparty. The net effect is not much size getting done. Conditionals are a similar story to blocks. They don't have the opportunity cost that a firm block resting on a venue does, but there's information leakage, and the surface area for interaction is still small. Market makers aren't incentivized to provide liquidity, and directional traders are worried about/behave strategically due to concerns over information leakage.

what's to stop exchanges from (1) creating more common bundles that people want to trade

I'd say that the market has already done this in the form of ETFs and index products and an entire ecosystem of ETF market making emerged around it.

(2) matching them with price-time priority so everyone gets a fair price? Wouldn't the auction model just create wider or locked/crossed markets?

I'm not sure that I follow this part entirely. The uniform price combinatorial auction that we're running results in everyone getting the same price on a symbol-by-symbol basis. And, we view time priority as a bad thing (the arms race dynamic of time priority was known to practitioners since markets first started going electronic but Budish et al. were the first to write about it in detail). Periodic auctions have better fairness and post-trade mark outs theoretically and in practice. Some of the European venues where batch auctions have made limited inroads demonstrated this.

Are you expecting to only support institutional investors or retail order flow as well?

We don't segment the market at all or exclude subscribers (beyond requiring that they're FINRA registered BDs), but the way that PFOF works means that we likely won't see retail order flow from the brokers that wholesale it. We do view Smart Markets as a win/win/win/win for retail customers, brokers, market makers, and regulators alike (cleaner routing, better transparency and price formation, better allocative outcomes, lower technology costs).

How does the expressive bidding interact with NBBO held orders?

https://news.ycombinator.com/item?id=30257586

Expanding on that a bit—we have the standard Rule 611 requirements, meaning that we clear within the NBBO on a symbol by symbol basis. We don't route.

Thanks! We found that the "UX" aspect of fitting squarely into existing workflows to disrupt without disrupting is key—as is how we message the product. Both our focus on solving speed and Expressive Bids as code are "UX" decisions aimed at slotting us into existing market structure and mind space.

A surprising (to us) takeaway was that making a product in this space sound "vanilla"/undifferentiated is a good thing. Once folks aren't concerned about an initial integration being a lift, they're happy to onboard us as "just another trading venue" (but with great story about unique liquidity and match quality). Many then get excited about adopting the lowest hanging fruit incrementally for their specific use cases. And after peeling away a few layers of the onion, they get excited about the future state in which others do the same, and what initially seems like incremental change becomes a market structure transformation.

We studied the history of adoption in other markets where it's gone well (FCC spectrum, display advertising, procurement) and poorly (OptiMark, POSIT4—great attempts, ahead of their time, killed by complexity and subtle mismatches between the mechanism and market participant needs).

This is a significant tailwind for the next wave of electronification that we're hoping to advance. US swap dealers, for example, now have a SEF reporting requirement, but most of the pre-trade is still in the screens. A Smart Market that allows dealers to control for non-price factors could change that.

How do you deal with best execution obligations for the "legs" of your trades?

Excellent question. Each leg of the trade comes in as an individual limit order with a price/quantity bound on it. This makes it easy for brokers to satisfy their 15c3-5 requirements since Expressive Bidding can only further restrict these limits. We clear the auction with a constraint that bounds the clearing region to the NBBO, snapshotted at the start of each auction. Rule 611 affords up to 1s if the market moves while we're optimizing, and away market movement has no average effect on execution quality, as auction start times are random and orders are firm.

In the future, we'll do ISO sweeps, but their usefulness is somewhat limited since getting fills from a sweep can change what's feasible for the auction. As we gather data on how folks use us, we'll hone on other regulatory options.

What market data do you publish about your order book?

We report fills to the TRF to satisfy regulatory reporting requirements, but we're entirely dark otherwise.

Thanks! We're quite fortunate in that we always knew roughly where we needed to get to from a technical, business, and regulatory standpoint, so actually getting there was just a long hike on a mostly known path, one step at a time.

Thanks! I'll say that a key design goal of ours is to make it so that PMs/asset managers/execution traders running complex books don't have to think about microstructure. Today's markets and the algorithmic trading stacks surrounding them are enormously complex. Smart Markets and multiunit auctions trade external complexity and internal simplicity (the existing double auction world) for external simplicity/usability and internal complexity (the computational challenges of combinatorial auctions).

Anything involving the phrase "better price" is a regulatory sensitivity, so please forgive the specificity of this answer. It's possible but not guaranteed that both the buyer and seller can get a better price for micro and macro reasons. On the micro front, we're a uniform clearing price auction with prices out to the sixth decimal place (NB: this is not sub-penny pricing—we only accept orders with prices in increments of $0.01). That makes it possible for buyers and sellers to split the spread, with auction dynamics dictating the split. Also, Expressive Bidding allows market makers to provide liquidity on a hedged basis, potentially incentivizing them to post more aggressive sizes and prices.

On the macro front, we're focused on finding Pareto (more accurately Kaldor–Hicks) efficient outcomes missed by simpler auction formats. Imagine that a buyer expects to move the market 5bps while executing, and a seller expects the same. If agency issues prevent them from finding each other outside of OneChronos (perhaps they only have 2bps of statistical expectation) and they trade on OneChronos at the bid/mid/offer, the reader can decide if that constitutes getting a better price than other markets or not. Multiunit auctions make market impact/execution risk known pre-trade and reduce uncertainty for counterparties, potentially reducing the cost of liquidity.

Thanks, and with you 100%. That's why we've invested heavily in a more technically challenging for us but better for the end-user approach to Expressive Bidding. It's also worth noting two failed attempts at smart(er) markets—OptiMark and POSIT4. Both were ahead of their time and a little off the mark in aligning the mechanism with trader needs, but poor usability/high degrees of user-facing complexity didn't do them any favors.

Conditional and contigent orders have been around for a decade+.

Conditional orders are a real testament to priorities shifting away from concerns over information leakage/fairness and towards concerns about how to get liquidity in an increasingly fragmented landscape. They're a nasty bandaid solution for routing opportunity cost.

Options exchanges have complex order books

CLOBs for options and implied order books for futures dealt with some limited forms of exposure risk and proved a big boon for both markets, but they're also a bandaid solution (pre-defined packages and in some cases HFT scale legging risk and no ability to deal with substitutability or side constraints). We're excited to see what gains the general approach (combinatorial auctions unlock).

Supply/Demand curves have been modeled in cryptocurrency smart contracts like Uniswap.

Clearly this idea is different and novel but borrows somewhat from all these concepts.

While I agree with you that these are not new ideas, I'd phrase that a little differently and credit the actual inventors. Milgrom, Wilson, McAfee, Cramton, Ausubel, and too many others to name pioneered the theory and practice of multiunit and combinatorial auctions and, by extension, all of these concepts. Except for conditional orders, which make mechanism designers cry. There's truly nothing that better highlights the tragedy of the commons that is market fragmentation.

To clarify, 1T isn't what we're claiming as our revenue opportunity; it's what traders are missing out on annually in the form of portfolio returns due to market friction and missed Pareto outcomes.

(FWIW and not that it's the market that we're going after per se—our strategy is mostly blue ocean—the market for US equities electronic execution services across the whole stack of technology, market data, broker algos, etc., is $18B/yr.)

Speaking more broadly than capital markets, it can result in more economically efficient outcomes, especially when some economic agents create disproportionately large negative externalities for others.

For OneChronos, we're a uniform price clearing mechanism that will eventually support submarkets, which can enable (among other things) what's effectively a mutual opt-in repeated play game of reputation and the ability to bid based on reputation. So we don't support any form of price discrimination at present, and what we will eventually support has the nuance to it that both parties chose to opt-in (and Expressive Bidding means that they can bid in multiple submarkets simultaneously without exposure risk). This induces a meta-game of sorts — a market for reputation. We're including it as a cleaner and more transparent version of existing behaviors with the efficiency gains to go with it.

Hey (I don't want to out your first name here), we should catch up! I'll email you after digging out the inbox.

Proxy Bidders are pure functions that map inputs (market conditions on other venues, metadata) to Expressive Bids in our bidding language (a bounded fragment of linear mixed real integer arithmetic logic—LIRA); certain EBs are CFMMs, and our most general solvers are SMT LIRA.

Thanks! I'll reply to you there as well.

1. There are two places where deep learning and prior-based approaches can come into play for combinatorial auctions. One is pretty analogous to AlphaZero, but substitute placing a piece on a Go board with accepting a bid, hoping that upon reaching a terminal state the set of bids accepted is feasible and close to optimal. The second is perhaps more in line with what you mentioned—using ML for hyperparameter selection in an algorithm portfolio. When we go live and have production data, our meta optimizer will measure how different approaches are doing and allocate computational resources accordingly in an online fashion. We always use a vanilla unit double auction as a baseline to measure relative performance within an auction cycle, and if the baseline is better, we use it instead.

2. There's a fun and serendipitous story here. I wrote an extremely early prototype as a tiny lisp and evaluator to go with it. We needed a very restricted and functionally pure language that we could control the execution context of, symbolically execute, and do basic formal methods on. The approach worked for a POC, but it was a far cry from real-world adoptable. We proceeded to prototype a DSL with an HM inspired type system and a more pythonic syntax, arriving at a poor man's ML. Better, but a DSL, and something limited/bespoke that would ultimately be annoying for developers. Then we met the guys at Imandra [1], who convinced us that we could have our cake and eat it too using vanilla OCaml/ReasonML and an ultra-high level theorem prover to keep code in an acceptable logic fragment. As an aside, rust is our systems PL and where we do most of the heavy lifting. Evaluating Expressive Bids isn't computationally expensive relative to the optimization problem.

[1] https://www.imandra.ai/

We're still a small enough team that we're more focused on talent than roles. As an example of what that means, our stack is polyglot (rust, OCaml, elixir, python), and we don't assume or require that folks have worked in any of those languages before. We invest heavily in learning and teaching.

It sounds like you have a very relevant background, so please email us if you're interested in discussing further!

VCGs got a real-world test in FB's ad market [1], and the results were mixed. VCG is in a class of theoretically interesting but fragile and overly game-theoretic mechanisms. Our mechanism is boring from a mechanism design standpoint—it's a uniform clearing price periodic auction without any cleaver demand reduction or tricks aimed at incentive compatibility. The complexity of what we allow for with the bidding language makes closed-form/theoretical analysis at best difficult and, in cases, impossible. Instead, we focus on giving traders a direct means to express their valuations and mechanism that minimizes information leakage and post-trade regret (situations where a bidder wishes they'd behaved differently given the auction's outcome).

[1] https://www.researchgate.net/profile/Alexander-Leo-Hansen/pu...

Some EMMs are excited to leverage Expressive Bidding to quote more size with less risk. For example, I'll bid 200 shares of XYZ @ $10.00 XOR {1,000 shares ABC @ $9.99, -500 shares (beta hedge) DEF @ 19.98}.

Folks also plan on using Expressive Bidding to enter other business lines that high startup costs or low margin (ex ongoing technology costs) previously kept them out of.

How bidders communicate bids to the auctioneer (the bidding language) is a central concern for any auction. It's pretty straightforward for unit good auctions (I'll pay $5 for A or I'll sell B for $4), but combinatorial auctions involve arbitrary packages of goods; the set of all possible bids is the powerset of the goods being auctioned. Having bidders attach a value to each package is both a computational impossibility for anything more than a few goods and a "UX" nightmare. For example, a bidder that wants to buy A for at most $5 in a market for goods A and B with free disposal (meaning you'll take something extra if it's free) needs to enter the bid ({A, $5}, {AB, $5}).

Information theory tells us that no universal bidding language (there's a representation of any package of interest) is uniformly more compact than the power set representation. Nonetheless, a good bidding language makes "common" bids compact and easy to communicate. We thought about this problem deeply and realized that functionally pure computer programs mapping proposals (packages of goods) to valuations (how much the bidder will pay or would want to receive) are about as natural as it gets. There's a direct analog in asking a human or a pricing algo for a price in a bilateral trade setting. However, our optimizer doesn't know what to do with an arbitrary computer algorithm, and exhaustively querying one to get the power set of prices out is computationally infeasible. However, using formal methods, we can (in the right setting) convert a computer program into an equivalent representation in a logic fragment called mixed integer real arithmetic. And that (via SMT solving) is something that an optimizer can work with.

You can see what Proxy Bidders (the pure functions that create expressive bids) look like here [1].

[1] https://www.onechronos.com/docs/expressive/bidding-guide/#in...

We're simply standing on the shoulders of giants for this one. Paul Milgrom and Bob Wilson, winners of the 2020 Prize in Economic Sciences, are responsible for the underlying theory and commercialization in other industries. My advisor, Preston McAfee, introduced me to the concept (Milgrom's book was the auction theory text) and was largely responsible for bringing mechanism design to ad markets. Lots of folks are applying machine learning to accelerating discrete optimization problems, and an advisor of ours, Kevin Leyton-Brown [2], pioneered applying it to combinatorial auctions for wireless spectrum repacking.

Our value add is mainly a team that understands these fields and the extreme nuance of capital markets. That's allowed us to generate novel IP and a purpose-built solution.

[1] https://www.nobelprize.org/prizes/economic-sciences/2020/sum...

[2] https://arxiv.org/abs/1706.03304

The products that still trade predominately {OTC, bilateral, non-electronic} do because non-price factors, trading conventions, and counterparty risk make it difficult or impossible to trade on a central exchange. Expressive Bidding and a mechanism that allows for matching market dynamics (OneChronos) will enable electronification and more active trading in these markets. Ten years ago, I would have said that there were commercial headwinds against this (dealers wanting to trade bilaterally). With banks increasingly focused on repeatable and less variable trading revs, such is no longer the case. The unsuitability of such products for double auctions is the limiting factor.

Thank you!