Against Optimization

https://aworkinglibrary.com/writing/against-optimization

by earthboundkid • 2 years ago

21 3 2 years ago

aworkinglibrary.com

graycat• 2 years ago

The OP is not very precise in just what is meant by "optimization".

Trying to evaluate the claims anyway, maybe a lot are correct but some are wrong.

The meaning of optimization there seems to be essentially, in some respects make better.

But there are some fields in pure and applied math and computer science where: To be short, simple, and explicit, we are at the central sort of FedEx and want to schedule the fleet at least cost, i.e., know for the next flights out, for each airplane, which cities should it go to and in what order to move the loads and make the total operating cost as small as possible.

Uh, maybe the OP would object, say that the FedEx fleet scheduling problem really WOULD have unpredictability. Okay: The planes are at the central sort, parked, and empty. The central sort is done, and from that we have data, for each city, the load to be delivered to that city. And in the evening, we get data from the cities, for each city, the load to be carried to the central sort.

We have a file that lists all the possible flights from the sort, to some cities, and back to the sort. For each of these, we plug in the loads and check the flight for feasibility and add the cost (we can be accurate on fuel and flight time because we know all the loads). Then near midnight we do the 0-1 linear programming optimization and determine all the flights. That is, we respond to the unpredictable nature of the loads of each day (airplanes out of service due to maintenance, weather, etc.) by doing some of the calculations once for each flight of the fleet using the latest and quite accurate load data, that is, have a deterministic problem.

But, there can be problems where there is unpredictability, say, over time. In that case, there is some successful research, and broadly one approach is stochastic dynamic programming. So, when a problem is not deterministic, maybe we can still do well.

Or, we are feeding 100,000 chickens. The feed might include ingredients corn, soy beans, fish meal, ..., and for each ingredient we have a "nutritional label" and cost, say, per pound. We also know what nutrition we want for each chicken. So, we run a linear program and find how much of each ingredient to use -- we solve the deterministic problem of giving the chickens the nutrition we want at least total cost. And if the prices or availabilities of some of the ingredients vary, we can change a few numbers and run the linear programming calculation again.

So, net: We can do a lot with optimization.

marcus_holmes• 2 years ago

I disagree: you can optimise for resilience. It means trading off efficiency, for sure, but it's very possible. Have 2 or 3 failover servers, redundant people, backup after every transaction. Mirror your infrastructure to a provider in another hemisphere. It's not cheap, but you can do it.

I think the author means "against efficiency" rather than "against optimisation". And then I'd totally agree.

rcxdude• 2 years ago

Indeed. It is for sure possible to have an inefficient and non-resilient system. But I think the general point that efficiency tends to trade off against resilience is an important one which too few people understand. (one specific example from queuing theory which I wish more people understood was the tradeoff between resource utilization and latency. If you want something or someone to be busy almost all of the time, there must be a long queue which increases the latency of access, in the limit getting to 100% utilization the expected queue length grows without bound).

Against Optimization

Related Stories