Circuit Design and Applied Optimization (Part 1)

https://stefanabikaram.com/writing/circuit-design-and-applied-optimization/

by stefanpie • 5 years ago

95 32 5 years ago

hyperman14• 5 years ago

The formula for the duty cycle makes no sense. My rusty brain thinks it should be D=(R1+R2)/(R1+2R2), but it's multiple decades ago I used it so I might be wrong

stefanpie6• 5 years ago

You are totally correct, this is only a typo in latex which I have just pushed a fix for. Thank for you the note.

hyperman14• 5 years ago

Now it says 404. The correct URL has a -1 after it

Update: Eight o'clock and all is well ;-). Thanks for the interesting article.

neltnerb2• 5 years ago

This reminds me of an undergraduate course on diffusion (atomic scale movement of atoms in a solid).

There are known formal differential equations that let you solve for the exact diffusion profile given geometry, composition, diffusion rate constants, and temperature (basically). So on the homework they asked us to tell them how long it took for the concentration of the dopant to reach 10% at a 10 micron depth.

So of course we all reached for this new math we had learned.

We all got it wrong with the professor commenting that "you should have just used the approximate formula [which was trivial algebra], we only know the diffusion rate to an order of magnitude anyway". This was far more useful as feedback than the 0.2% of our grades we missed credit for.

So I look at this essay and am a bit amused at minimizing calculation error to such an extent in a mathematical model when your resistors have a 5% tolerance and your capacitor is probably at 20% tolerance if you happened to have it in a university EE lab. But I do appreciate the fun of doing it this way too if you're not in a hurry =)

It is crucial for any systems designer to realize that no matter how well you do your math and theory you have to also understand the sensitivity to variability so that you know what math is worth doing and what components we have no choice but to control the tolerances on better.

You can find some fancy software for generating filter networks (similar in concept to this except with more discrete math because component values are discrete...) and ask it to show you sensitivity and you can see exactly how much you'll screw up your perfectly designed 10th order Chebyshev filter if that last resistor is off by 1%...

stefanpie6• 5 years ago

Oh interesting application. I have taken a class in microelectronics physics (doping, electron holes, bandgaps, mosfets, bjts) and it was definitely just focusing on simpler approximations and first and second order effects only.

But you are correct in that after my second or third electronics lab I realized that even standard component values vary to much that this approach is a little overkill and non-practical in a lot of cases. However there is a way to bake in discrete component values into the continuous optimization problems by using a continuous/"soft"/differentiable approximation of the min function to make "pockets" of optimal regions close to component values part of a set (for example e series values). I plan to do more writeups exploring this idea as well as looking at more complex applications such as higher order filters as you mentioned (my op-amp filter labs are what motivate me to look into this as I wasted so many hours in open lab trying to get my component values right).

sidpatil• 5 years ago

However there is a way to bake in discrete component values into the continuous optimization problems by using a continuous/"soft"/differentiable approximation of the min function to make "pockets" of optimal regions close to component values part of a set (for example e series values).

Is this the same concept as uncertainty sets in a robust optimization problem?

neltnerb2• 5 years ago

Yeah, at the least you can just sample from a gaussian distribution around the nominal value; the shape is probably wrong but it's got to be better than assuming no error. Or you could just do error analysis on the formula to find out which terms contribute the most to the final error.

I like that your suggestion is in-line with how we now can use an autoencoder to convert a molecular graph into a continuous latent space which can be used to train a network to predict some property. Now we have a latent space where things that have similar property predictions are near each other in a space we can explore and then decode into new candidate materials/molecules.

Drug discovery, solar panels, I'm looking forward to the field taking off industrially for sure. Applying machine learning and advanced computational techniques to ad-tech is very depressing with problems like these out there.

xondono3• 5 years ago

I had a professor that intentionally baked hard (for the course takers anyway) mathematics into some problems, that were technically solvable during an exam, but that required very good level (they typically included rare/obscure identities). The point was that solving precisely was always a “trap”.

His pet peeve was that engineering wasn’t about finding a precise answer at some point, but about finding good enough answers fast.

I liked that this forced you to stay alert during the problems for simplifications and to understand what you were doing, instead of just droning out method A for solving problem type B.

cinntaile2• 5 years ago

If the course is not a mathematical course that requires this knowledge then adding obscure identities is focusing on the wrong thing entirely. Some people will simply not see these identities because they might never have used them. Irrelevant for the course but relevant for the grade, that's not a good combo imo.

hyperman14• 5 years ago

In feynman's 'Surely you're joking' there's a chapter where he designs some machinery for the army. He gets as guidance to use cogs from the middle of the list of available options, as the smallest or largest parts have all kinds of downsides.

This idea works well in all kinds of situations where you have to select parts. I assume it might do well here, too.

triactual• 5 years ago

This works until Microsoft also picks from the middle to make ten trillion Xbox controllers overnight and you can’t get parts for six months. There are so many competing constraints just in component selection.

hyperman14• 5 years ago

We're talking about E12 resistors here. I'm not going to worry too much.

xondono3• 5 years ago

You might want to look into whatever you are using to render LaTeX, because it doesn’t work on iOS.

I’ve had zero problems with Mathjax so far.

stefanpie6• 5 years ago

I'll take a look at this, thank you for the node. I believe I'm using mathjax currently but I may have set set some options correctly. I also need to go in and verify a11ty as well.

docfort2• 5 years ago

I believe this is supposed to be the first in a series on moving from continuous to discrete optimization, but the EE in me can’t help but point out what I would do in this scenario. It also connects with other interesting aspects of physics.

Looking at the governing equations, you can clearly see that if R1 << R2, then the duty cycle is close to 50%.

With that done, I also would have fixed the cap to something that is available. Ignoring R1 for the moment (because I just need to ensure R2 is bigger), I solve for R2 in the frequency equation. It is approximately 72 kOhms.

I notice that a nonzero value of R1 is really there to tune the frequency. As long as R1 is much smaller than R2, then the frequency equation is more sensitive to changes in R1 than the duty cycle equation. So I can play with different small values of R1 to tune my frequency to get closer.

Finally, since I know that I’m likely using imprecise resistor tolerances, I know that I can pass if I just get close, so I might not need to be picky about R1.

In my opinion, this chain of reasoning (effective modeling, function sensitivity wrt parameters, tolerance specs) is what the lab experiment is actually about. Developing circuits that are tolerant to parameter variation is the key to real hardware products. That gives you so much flexibility in price and manufacturer and requires the designer to keep this kind of reasoning frontmost.

kurthr• 5 years ago

Exactly, and if you know you're not going to get better than 1% metal film resistors, why search more than the discrete space. Between 10^2 and 10^6 there's only 6-96 steps per decade (12 for the standard 10% E12 series). There's less than 400x400 possible resistor values to check.

stefanpie6• 5 years ago

I actually ended up doing some variation of this for scripts I wore for my labs. The space is actually quite small and it's not hard to brute force this problem. Once you get to more open ended designs and larger parameter spaces (like filters) it becomes a bit more challenging to brute force.

hermitdev• 5 years ago

One of my degrees is in EE. This post surely took me back to some of the frustrating design labs and more unknowns than equations to solve them... I also definitely went down the route of just fixing the cap, because well, we only had like 2 or 3 of them in our parts kit, anyway. And I appreciate you mentioning the resistor tolerances because, no, you don't really have a 1 kOhm resistor. You have a 987 or 1009 or something else near 1000 Ohm resistor.

ncmncm• 5 years ago

The insight that the only serious constraint is R1 << R2 is clearly the point of the lab exercise, and it might be missed entirely in numerically searching the space. But adding a flip-flop to the output to divide frequency by two yields an exactly 50% duty cycle, regardless of R1/R2, so you may then optimize for some other quality. The right question to the TA would have been about which of the specs mattered more: duty cycle? power consumption? stability against variation in parts, temperature, supply voltage, noise? absolute frequency? parts count? parts cost? Each different answer yields a different design.

Nowadays a microcontroller might be best, e.g. if it meant you could get away without the resistors and capacitors.

docfort2• 5 years ago

In my experience, knowing what parts of a schematic can be ignored for understanding the core functionality is super important as a designer. Then I’m better able to compare two implementations that purportedly do the same thing on many different axes. Especially if I’m doing integrated circuit design as opposed to discretes.

It also helps to build up conceptual complexity. For example, the post mentions briefly avoiding giant resistors for stability. Good enough rule to start out with. But eventually understanding how an op-amp does its thing using feedback is critical. Especially when comparing IC-equivalents, since now you also need to consider area as another metric of comparison.

Brute force optimization avoids all of this learning, which is good in the short term (get the lab done) and bad in the long term (how to think about circuits).

jason_s• 5 years ago

or you throw away R1 and connect R2 between C and OUT.

WalterBright• 5 years ago

Haha, this is just like designing a spring. You have a catalog of wire sizes, and wire materials, and an envelope. Too many unknowns, so a lot of trial and error before you start to hone in on an exact (but who knows if it is globally optimal) solution.

mayapugai• 5 years ago

I'm just here to say this article was such a pleasure to read. I've always relied on intuition to pick good-enough values based on experience. So this rigorous analytical approach to viewing the objective space and picking a reasonable value is refreshing. It's ironic and rightfully hilarious that you still had to take a gander and pick a "close-enough" value.

I look forward to part 2 where you incorporate the resistor choices that you had. Perhaps also include the statistics of the tolerance into the mix to find the optimal values that we should all be picking for our future 555 timer hijinks.

Your academic research work is also very interesting. Suffice to say I'll be following you on GitHub :)

Cheers and happy (almost) new year!

kragen3• 5 years ago

Brute-force search, as kurthr suggested doing, takes under 250ms to find a solution that's better on both axes by an order of magnitude, without Scipy, though it still took me 15 minutes to set it up.

With R1 = 100Ω, R2 = 330kΩ, C = 2200pF, the frequency error is 7 Hz rather than 39 Hz (0.7% instead of 4%) and the duty-cycle error is 0.008% instead of 0.03%. Probably your capacitor is not going to be stable to 0.7% or precise to 1%, but you can certainly do better than 4%. (Use an NP0/C0G capacitor, not an X7R or something. In larger capacitance values you'd use a film cap instead, but 2200pF is fine for NP0/C0G.)

You probably ought to measure some capacitors before running the optimization if you're only building one circuit, and if you're really interested in precision you might put a couple of trimpots across the resistors and adjust it while watching the scope (modern digital scopes can continuously display the frequency and duty cycle, so this is quick). But that will only help if most of the resistance and capacitance comes from components that won't drift over time or vary too much with temperature.

Multiplying the loss values instead of adding them avoids having to choose weights for them. If you were going to use Newton's method or gradient descent, you might want to square them instead of taking the absolute value in order to get faster convergence, but of course that takes you back to solving a continuous relaxation of the discrete component selection problem you actually have. For branch-and-bound search a continuous relaxation can still be a useful thing to do, though.

    >>> R1 = R2 = 1000
    >>> import math
    >>> u, n, p = 1e-6, 1e-9, 1e-12
    >>> caps = [a * b * c for a in [1.0, 2.2, 4.7] for b in [1, 10, 100] for c in [u, n, p]]
    >>> e24 = [1.0, 1.1, 1.2, 1.3, 1.5, 1.6, 1.8, 2.0, 2.2, 2.4, 2.7, 3.0, 3.3, 3.6, 3.9, 4.3, 4.7, 5.1, 5.6, 6.2, 6.8, 7.5, 8.2, 9.1]
    >>> C = 47 * u
    >>> 1/(math.log(2) * (R1 + 2*R2) * C) # f
    10.231879722616762
    >>> (R1 + R2) / (R1 + 2*R2) # duty cycle
    0.6666666666666666
    >>> rs = [a * 10**b for a in e24 for b in range(2, 6)]
    >>> import time
    >>> s = time.time(); soln = min(((abs(1000 - f)/f) * (abs(0.5 - D)/D), R1, R2, C, f, D) for R1, R2, C, f, D in ((R1, R2, C, 1/(math.log(2) * (R1 + 2*R2) * C), (R1 + R2) / (R1 + 2*R2)) for R1 in rs for R2 in rs for C in caps)); time.time() - s
    0.2474360466003418
    >>> soln
    (1.0000300746386372e-06, 100.0, 330000.0, 2.2000000000000003e-09, 993.4411045771049, 0.5000757460990759)

It would be fair to argue that the nested generator expression there is pretty hard to read, but it didn't really take that long to type, the Python REPL is pretty shitty at editing multiline functions, and I didn't feel like firing up Jupyter. But this way of writing it is definitely less unreadable and works just as well:

    >>> def search(rs1, rs2, cs):
    ...  for R1 in rs1:
    ...   for R2 in rs2:
    ...    for C in cs:
    ...     f = 1/(math.log(2) * (R1 + 2*R2) * C)
    ...     D = (R1 + R2) / (R1 + 2*R2)
    ...     yield (abs(1000 - f)/f) * (abs(0.5 - D)/D), R1, R2, C, f, D

Circuit Design and Applied Optimization (Part 1)

Related Stories