Don't "optimize" conditional moves in shaders with mix()+step()

https://iquilezles.org/articles/gpuconditionals/

by romes • 1 year ago

409 231 1 year ago

ttoinou2• 1 year ago

Thanks Inigo !

  The second wrong thing with the supposedly optimizer version is that it actually runs much slower than the original version. The reason is that the step() function is actually implemented like this:

  float step( float x, float y )
  {
      return x < y ? 1.0 : 0.0;
  }

How are we supposed to know what OpenGL functions are emulated rather than calling GPU primitives ?

Const-me• 1 year ago

The only way is do what OP did – compile your shader, disassemble, and read the assembly.

I do that quite often with my HLSL shaders, learned a lot about that virtual instruction set. For example, it’s interesting GPUs have instruction sincos, but inverse trigonometry is emulated while compiling.

Waterluvian• 1 year ago

This is a great question that I see everywhere in programming and I think it is core to why you measure first when optimizing.

You generally shouldn’t know or care how a built in is implemented. If do care, you’re probably thinking about optimization. At that point the answer is “measure and find out what works better.”

TeMPOraL• 1 year ago

EDIT: I see my source of confusion must be that "branch" must have a well-understood hardware-specific meaning that goes beyond the meaning I grew up with, which is that a conditional is a branch, because the path control takes (at the machine code level) is chosen at runtime. This makes a conditional jump a branch by definition.

How are we supposed to know what OpenGL functions are emulated rather than calling GPU primitives?

To me the problem was obvious, but then again I'm having trouble with both your and author's statements about it.

The problem I saw was, obviously by going for a step() function, people aren't turning logic into arithmetic, they're just hiding logic in a library function call. Just because step() is a built-in or something you'd find used in mathematical paper doesn't mean anything; the definition of step() in mathematics is literally a conditional too.

Now, the way to optimize it properly to have no conditionals, is you have to take a continuous function that resembles your desired outcome (which in the problem in question isn't step() but the thing it was used for!), and tune its parameters to get as close as it can to your target. I.e. typically you'd pick some polynomial and run the standard iterative approximation on it. Then you'd just have an f(x) that has no branching, just a bunch of extra additions and multiplications and some "weirdly specific" constants.

Where I don't get the author is in insisting that conditional move isn't "branching". I don't see how that would be except in some special cases, where lack of branching is well-known but very special implementation detail - like where the author says:

also note that the abs() call does not become a GPU instruction and instead becomes an instruction modifier, which is free.

That's because we standardized on two's complement representation for ints, which has the convenient quality of isolating sign as the most significant bit, and for floats the representation (IEEE-754) was just straight up designed to achieve the same. So in both cases, abs() boils down to unconditionally setting the most significant bit to 0 - or, equivalently, masking it off for the instruction that's reading it.

step() isn't like that, nor any other arbitrary ternary operation construct, and nor is - as far as I know - a conditional move instruction.

As for where I don't get 'ttoinou:

How are we supposed to know what OpenGL functions are emulated rather than calling GPU primitives

The basics like abs() and sqrt() and basic trigonometry are standard knowledge, the rest... does it even matter? step() obviously has to branch somewhere; whether you do it yourself, let a library do it, or let the hardware do it, shouldn't change the fundamental nature.

ttoinou2• 1 year ago

It kinda does when you’re wondering what’s going on in backstage and working with shaders on multiple OS, drivers and hardware.

   Now, the way to optimize it properly to have no conditionals, is you have to take a continuous

I suspect that we shaders authors really like Clean Math and that’s also why we like to think such “optimizations” with the step function is a nice modification :-)

mymoomin• 1 year ago

A "branch" here is a conditional jump. This has the issues the article mentions, which branchless programming avoids:

    Again, there is no branching - the instruction pointer isn't manipulated, there's no branch prediction involved, no instruction cache to invalidate, no nothing.

This has nothing to do with whether the behaviour of some instruction depends on its arguments. Looking at the Microsoft compiler output from the article, the iadd (signed add) instruction will get different results depending on its arguments, and the movc (conditional move) will store different values depending on its arguments, but after each the instruction pointer will just move onto the next instruction, so there are no branches.

burch45• 1 year ago

Branching is different instruction paths, so it requires reading the instructions from different memory that causes a delay jumping to those new instructions rather than plowing ahead on the current stream of instructions. So a conditional jump is a branch but a conditional move is just an instruction that moves one of two values into a register but doesn’t affect what code is executed next.

dahart11• 1 year ago

the meaning I grew up with, which is that a conditional is a branch

A conditional jump is a branch. But a branch has always had a different meaning than a generic “conditional”. There are conditional instructions that don’t jump, e.g. CMP, and the distinction is very important. Branch or conditional jump means the PC can be set to something other than ‘next instruction’. A conditional, such a conditional select or conditional move, one that doesn’t change the PC, is not a branch.

take a continuous function […] Then you’d just have an f(x) that has no branching

One can easily implement conditional functions without branching. You can use a compare instruction followed by a Heaviside function on the result, evaluate both sides of the result, and sum it up with a 2D dot product (against the compare result and its negation). That is occasionally (but certainly not always) faster on a GPU than using if/else, but only if the compiler is otherwise going to produce real branch instructions.

chrisjj23• 1 year ago

a conditional is a branch, because the path control takes (at the machine code level) is chosen at runtime. This makes a conditional jump a branch by definition.

s/conditional is a/conditional jump is a/

Problem solved.

Non-jump conditionals have been a thing for decades.

SideQuark2• 1 year ago

I’ve never seen a GPU with special primitives for any functions than you’d see in pc style assembly. Every time I’ve looked at a decompiled shader, it’s always been pretty much what you think of in C.

Aldo specs like OpenGL specify many intrinsic behavior, which is then implemented as the spec, using standard assembly instructions.

Find an online site that decompiles to various architectures.

account4215• 1 year ago

Why are you supposed to know?

Because you care about performance? step being implemented as a libray function on top of a conditional doesn't really say anything about its performance vs being a dedicated instruction. Don't worry about the implementation.

Because you are curious about GPU architectures? Look at disassembly, (open source) driver code (including LLVM) and/or ISA documentation.

doctorhandshake• 1 year ago

I don’t know enough about these implementations to know if this can be interpreted as a blanket ‘conditionals are fine’ or, rather, ‘ternary operations which select between two themselves non-branching expressions are fine’.

Like does this apply if one of the two branches of a conditional is computationally much more expensive? My (very shallow) understanding was that having, eg, a return statement on one branch and a bunch of work on the other would hamstring the GPU’s ability to optimize execution.

TinkersW3• 1 year ago

A real branch is useful if you can realistically skip a bunch of work, but this requires all the lanes to agree, on a GPU that means 32 to 64 lanes need to all agree, also for something basic like a few arithmetic ops there is no point.

dahart11• 1 year ago

A GPU/SIMT branch works by running both sides, unless all threads in the thread group (warp/wavefront) make the same branch decision. As long as both paths have at least one thread, the GPU will run both paths sequentially and simply set the active mask of threads for each side of the branch. In other words, the threads that don’t take a given branch sit idle while the active threads do their work. (Note “sit idle” might involve doing all the work and throwing away the result.)

If you have two branches, and one is trivial while the other is expensive, and if the compiler doesn’t optimize away the branch already, it may be better for performance to write the code to take both branches unconditionally, and use a conditional assignment at the end.

It’s worth knowing that often there are clever techniques to completely avoid branching. Sometimes these techniques are simple, and sometimes they’re invasive and difficult to implement. It’s easy (for me, anyway) to get stuck thinking in a single-threaded CPU way and not see how to avoid branching until you’ve bumped into and seen some of the ways smart people solve these problems.

toredo1729_22• 1 year ago

Unrelated, but somehow similar: I really hate it that it's not possible to force gcc to transform things like this into a conditional move:

x > c ? y : 0.;

It annoyed me many times and it still does.

fweimer• 1 year ago

What do you mean? Do you want to annotate the condition as unpredictable, so that the compiler always assumes that a conditional move is beneficial?

(Compilers obviously do this transformation, including GCC, but it is not always beneficial, especially on x86-64.)

IshKebab3• 1 year ago

And it's not always possible! E.g. most RISC-V CPUs don't support it yet.

toredo1729_22• 1 year ago

Yes, that would be great. It's not always benefical, but in some (rare, but for me important) cases it's better. Currently, the only way to ensure a conditional move is used, is to use inline assembly. This is not portable and also less maintainable than a "proper" solution.

tjalfi• 1 year ago

clang has the __builtin_unpredictable() intrinsic[0] for this purpose.

[0] https://clang.llvm.org/docs/LanguageExtensions.html#builtin-...

flohofwoe3• 1 year ago

Seems to work just fine on gcc and clang?

https://www.godbolt.org/z/ffEvvjhz8

PS: and it also doesn't matter whether a ternary is used or a traditional if (as one would expect):

https://www.godbolt.org/z/zjb4KdqvK

(the float version also appears to not use branches: https://www.godbolt.org/z/98bdheKK4)

For such simple expression I would expect the compiler to pick the right output pattern based on the target CPU though...

dzaima10• 1 year ago

Not always - https://www.godbolt.org/z/zYxeahf3T. And for any modern (as in, made in the last two decades) x86 processor the branchless version will be hilariously better if the condition is unpredictable (which is a thing the compiler can't know by itself, hence wanting to have an explicit way to request a conditional move instr) and the per-branch code takes less than like multiple dozens of cycles.

ryao11• 1 year ago

Do shader compilers have optimization passes to undo this mistake and if not, could they be added?

DRAGONERO3• 1 year ago

I’d expect most vendors do, at least in their closed source drivers. You could also check in the mesa project if this is implemented but it’s definitely possible to do

ryao11• 1 year ago

Shader compilers tend to be very latency sensitive, so “it takes too long to run” would be a valid reason why it is not done if it is not done.

mirsadm• 1 year ago

I've been caught by this. Even Claude/ChatGPT will suggest it as an optimisation. Every time I've measured a performance drop doing this. Sometimes significant.

WJW• 1 year ago

Is that weird? LLMs will just repeat what is in their training corpus. If most of the internet is recommending something wrong (like this conditional move "optimization") then that is what they will recommend too.

xbar• 1 year ago

Not weird but important to note.

diath• 1 year ago

Even Claude/ChatGPT will suggest it as an optimisation.

LLMs just repeat what people on the internet say, and people are often wrong.

londons_explore• 1 year ago

So why isn't the compiler smart enough to see that the 'optimised' version is the same?

Surely it understands "step()" and can optimize the "step()=0.0" and "step()==1.0" cases separately?

This is presumably always worth it, because you would at least remove one multiplication (usually turning it into a conditional load/store/something else)

NohatCoder3• 1 year ago

It may very well be, it is the type of optimisation where it is quite possible that some compilers may do it some of the time, but it is definitely also possible to write a version that the compiler can't grok.

mbel• 1 year ago

Yup, they most likely do. After all everything is LLVM based nowadays.

Cieric15• 1 year ago

The other part of the optimization issue is that you can't take to long to try anything and everything. Most of the optimizations happen on the driver side, and anything that takes to long will show up as shader compilation stutter. I can't say currently if this is or isn't done, it's just always something you have to think about.

magicalhippo2• 1 year ago

Processors change, compilers change. If you care about such details, best to ship multiple variants and pick the fastest one at runtime.

As I've mentioned here several times before, I've made code significantly faster by removing the hand-rolled assembly and replacing it with plain C or similar. While the assembly might have been faster a decade or two ago, things have changed...

dist-epoch• 1 year ago

Funnily enough, this is sort of what the NVIDIA drivers do: they intercept game shaders and replace them by custom ones optimized by NVIDIA. Which is why you see stuff like this in NVIDIA drivers changelog: "optimized game X, runs 40% faster"

esperent• 1 year ago

I'd love to read more about this, what kind of changes they make and how many games they do it for. Do they ever release technical articles about it?

snicker7• 1 year ago

Imagine being the dev competing game Y and seeing the changelog.

crazygringo2• 1 year ago

Wow, how did they pick which games to optimize?

Does the studio pay them to do it? Because Nvidia wouldn't care otherwise?

Does Nvidia do it unasked, for competitive reasons? To maximize how much faster their GPU's perform than competitors' on the same games? And therefore decide purely by game popularity?

Or is it some kinda of alliance thing between Nvidia and studios, in exchange for something like the studios optimizing for Nvidia in the first place, to further benefit Nvidia's competitive lead?

Cieric15• 1 year ago

I don't work on the nvidia side of things but it's likely to be the same. Shader replacement is only one of a whole host of things we can do to make games run faster. It's actually kind of rare for use to do them since it boats the size of the driver so much. A lot of our options do change how shaders work though, like forcing a shader to use double precision floats instead of the single it was compiled with.

chrisjj23• 1 year ago

"optimized game X, runs 40% faster"

... and looks 4O% crappier? E.g. stuttery, because the driver does not get to see the code ahead of time.

Amadiro• 1 year ago

I think figuring out the fastest version of a shader at runtime is very non-trivial, I'm not aware of any game or engine that can do this.

I think it'd be possible in principle, because most APIs (D3D, GL, Vulkan etc) expose performance counters (which may or may not be reliable depending on the vendor), and you could in principle construct a representative test scene that you replay a couple times to measure different optimizations. But a lot of games are quite dynamic, having dynamically generated scenes and also dynamically generated shaders, so the number of combinations you might have to test seems like an obstacle. Also you might have to ask the user to spend time waiting on the benchmark to finish.

You could probably just do this ahead of time with a bunch of different GPU generations from each vendor if you have the hardware, and then hard-code the most important decision. So not saying it'd be impossible, but yeah I'm not aware of any existing infrastructure for this.

hansvm• 1 year ago

The last time I did anything like this (it was for CPU linear algebra code designed to run in very heterogeneous clusters), I first came up with a parameterization that approximated how I'd expect an algorithm to perform. Then, once for each hardware combination, you sweep through the possible parameterization space. I used log-scaled quantization to make it cheap to index into an array of function pointers based on input specifics.

The important thing to note is that you can do that computation just once, like when you install the game, and it isn't that slow. Your parameterization won't be perfect, but it's not bad to create routines that are much faster than any one implementation on nearly every architecture.

ijustlovemath• 1 year ago

you'd only have to test worst/median case scenes, which you could find with a bit of profiling!

alexvitkov• 1 year ago

This would be acceptable if it meant adding one more shader, but with "modern" graphics APIs forcing us to sometimes have thousands of permutations for the same shader, every variant you add multiplies that count by 2x.

We also don't have an infinite amount of time to work on each shader. You profile on the hardware you care about, and if the choice you've made is slower on some imaginary future processor, so be it - hopefully that processor is faster enough that this doesn't matter.

account4215• 1 year ago

Graphics APIs don't force your to have thousands of shaders. The abstraction in your engine might.

quuxplusone• 1 year ago

I'm sure TFA's conclusion is right; but its argument would be strengthened by providing the codegen for both versions, instead of just the better version. Quote:

"The second wrong thing with the supposedly optimizer [sic] version is that it actually runs much slower than the original version [...] wasting two multiplications and one or two additions. [...] But don't take my word for it, let's look at the generated machine code for the relevant part of the shader"

—then proceeds to show only one codegen: the one containing no multiplications or additions. That proves the good version is fine; it doesn't yet prove the bad version is worse.

azeemba• 1 year ago

The main point is that the conditional didn't actually introduce a branch.

Showing the other generated version would only show that it's longer. It is not expected to have a branch either. So I don't think it would have added much value

idunnoman1222• 1 year ago

Unless you’re writing an essay on why you’re right…

TheRealPomax• 1 year ago

Correct: it would show proof instead of leaving it up to the reader to believe them.

comex• 1 year ago

But it's possible that the compiler is smart enough to optimize the step() version down to the same code as the conditional version. If true, that still wouldn't justify using step(), but it would mean that the step() version isn't "wasting two multiplications and one or two additions" as the post says.

(I don't know enough about GPU compilers to say whether they implement such an optimization, but if step() abuse is as popular as the post says, then they probably should.)

Lockal2• 1 year ago

You missed the second part where article says that "it actually runs much slower than the original version", "wasting two multiplications and one or two additions", based on idea that compiler is unable to do a very basic optimization, implying that compiler compiler will actually multiply by one. No benchmarks, no checking assembly, just straightforward misinformation.

stevemk14ebr• 1 year ago

There are 10 types of people in this work. Those who can extrapolate from missing data, and

account4215• 1 year ago

Making assumptions about performance when you can measure is generally not a good idea.

robertlagrant• 1 year ago

and what? AND WHAT?

creata• 1 year ago

Generated code for RDNA 1:

https://shader-playground.timjones.io/5d3ece620f45091678dcee...

TinkersW3• 1 year ago

It is weird how long misinformation like this sticks around, the conditional move/select approach has been superior for decades on both CPU & GPU, but somehow some people still write the other approach as an "optimization".

Lockal2• 1 year ago

"Conditional move is superior on CPU" is an oversimplification, in reality it was explained in 2007 by Linus[1] and nothing changed since then. Or in fact, branch predictors are constantly improving[2], while cmov data dependency problem can't be solved. Yes, cmov is better in unpredictable branches, but the universal truth is "programmers are notoriously bad at predicting how their programs actually perform"[3]

[1] https://yarchive.net/comp/linux/cmov.html

[2] https://chipsandcheese.com/p/zen-5s-2-ahead-branch-predictor...

[3] https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html#index...

account4215• 1 year ago

cmov data dependency problem can't be solved

Can't is a pretty strong word.

TinkersW3• 1 year ago

I really only care about performance in a SIMD sense, as not using SIMD means you aren't targeting performance to begin with(GPU == SIMD, CPU SSE/AVX == SIMD)

Branches fall off vs cmov/select as your lane count increases, they are still useful but only when you are certain the probability all lanes agreeing is reasonably high.

mahkoh3• 1 year ago

    So, if you ever see somebody proposing this

    float a = mix( b, c, step( y, x ) );

The author seems unaware of

    float a = mix( b, c, y > x );

which encodes the desired behavior and also works for vectors:

    The variants of mix where a is genBType select which vector each returned component comes from. For a component of a that is false, the corresponding component of x is returned. For a component of a that is true, the corresponding component of y is returned.

Thorrez2• 1 year ago

The author doesn't seem to say that mix should be avoided. Just that you shouldn't replace a ternary with step+mix. In your quote, you left out the 2nd half of the sentence: "as an optimization to [ternary]".

mahkoh3• 1 year ago

The author frames his post to be about education:

    please correct them for me. The misinformation has been around for 20 years

But his education will fail as soon as you're operating on more than scalars. It might in fact do more harm than good since it leads the uneducated to believe that mix is not the right tool to choose between two values.

dahart11• 1 year ago

IQ is most definitely aware of that. Are you unaware of who IQ is?

sva_• 1 year ago

I'm going to go on a limb and claim the author is aware:

https://www.shadertoy.com/user/iq

nxpnsv• 1 year ago

He also is one of the main authors of the whole site... legend.

jesse__• 1 year ago

Hah, yeah, this is a brilliant "do you even know who you're talking to?" moment.

petermcneeley• 1 year ago

WGSL has select https://www.w3.org/TR/WGSL/#select-builtin

alkonaut• 1 year ago

I wish there was a good way of knowing when an if forces an actual branch rather than when it doesn't. The reason people do potentially more expensive mix/lerps is because while it might cost a tiny overhead, they are scared of making it a branch.

I do like that the most obvious v = x > y ? a : b; actually works, but it's also concerning that we have syntax where an if is some times a branch and some times not. In a context where you really can't branch, you'd almost like branch-if and non-branching-if to be different keywords. The non-branching one would fail compilation if the compiler couldn't do it without branching. The branching one would warn if it could be done with branching.

ajross3• 1 year ago

it's also concerning that we have syntax where an if is some times a branch and some times not.

That's true on scalar CPUs too though. The CMOV instruction arrived with the P6 core in 1995, for example. Branches are expensive everywhere, even in scalar architectures, and compilers do their best to figure out when they should use an alternative strategy. And sometimes get it wrong, but not very often.

masklinn• 1 year ago

For scalar CPUs, historically CMOV used to be relatively slow on x86, and notably for reliable branching patterns (>75% reliable) branches could be a lot faster.

cmov also has dependencies on all three inputs, so if there's a high level of bias towards the unlikely input having a much higher latency than the likely one a cmov can cost a fair amount of waiting.

Finally cmov were absolutely terrible on P4 (10-ish cycles), and it's likely that a lot of their lore dates back to that.

chrisjj23• 1 year ago

The good way is to inspect the code :)

it's also concerning that we have syntax where an if is some times a branch and some times not.

It would be more concerning if we didn't. We might get a branch on one GPU and none on another.

phkahler• 1 year ago

> The good way is to inspect the code :)

The best way is to profile the code. Time is what we are after, so measure that.

plagiarist• 1 year ago

I think that capability in the shader language would be interesting to have. One might even want it to two-color all functions in the code. Anything annotated nonbranching must have if statements compile down to conditional moves and must only call nonbranching functions.

catlifeonmars• 1 year ago

This is also very relevant for cryptography use cases, where branching is a potential side channel for leaking secret information.

nice_byte2• 1 year ago

godbolt has rga compiler now, you can always paste in hlsl and look at the actual rdna instructions that are generated (what GPU actually runs, not spirv)

mpreda• 1 year ago

Exactly. Consider this example:

  a = f(z);
  b = g(z);
  v = x > y ? a : b;

Assuming computing the two function calls f() and g() is relativelly expensive, it becomes a trade-off whether to emit conditional code or to compute both followed by a select. So it's not a simple choice, and the decision is made by the compiler.

dragontamer• 1 year ago

This is a GPU focused article.

The GPU will almost always execute f and g due to GPU differences vs CPU.

You can avoid the f vs g if you can ensure a scalar Boolean / if statement that is consistent across the warp. So it's not 'always' but requires incredibly specific coding patterns to 'force' the optimizer + GPU compiler into making the branch.

pandaman5• 1 year ago

The reason people do potentially more expensive mix/lerps is because while it might cost a tiny overhead, they are scared of making it a branch.

And the reason for that is the confusing documentation from NVidia and its cg/CUDA compilers. I believe they did not want to scare programmers at first and hid the execution model, talking about "threads" and then they kept using that abstraction to hype up their GPUs ("it has 100500 CUDA threads!"). The result is people coding for GPUs with some bizarre superstitions though.

You actually want branches in the the code. Those are quick. The problem is that you cannot have a branch off a SIMD way so, instead of a branch the compiler will emit code for both branches and the results will be masked out based on the branch's condition.

So, to answer your question - any computation based on shader inputs (vertices, computer shader indices and what not) cannot and won't branch. It will all be executed sequentially with masking. Even in the TFA example, both values of ? operator are computed, the same happens with any conditional on an SIMD value. There can be shortcut branches emitted by the compiler to quickly bypass computations when all ways are the same value but in general case everything will be computed for every condition being true as well as being false.

Only conditionals based on scalar registers (shader constants/unform values) will generate branches and those are super quick.

ryao11• 1 year ago

You can always have the compiler dump the assembly output so you can examine it. I suspect few do that.

account4215• 1 year ago

So, to answer your question - any computation based on shader inputs (vertices, computer shader indices and what not) cannot and won't branch.

It can do an actual branch if the condition ends up the same for the entire workgroup - or to be even more pedantic, for the part of the workgroup that is still alive.

You can also check that explicitly to e.g. take a faster special case branch if possible for the entire workgroup and otherwise a slower general case branch but also for the entire workgroup instead of doing both and then selecting.

torginus6• 1 year ago

I'll comment this here as I got downvoted when I made the point in a standalone comment - this is mostly an academic issue, since you don't want to use step of pixel-level if statements in your shader code, as it will lead to ugly aliasing artifacts as the pixel color transitions from a to b.

What you want is to use smoothstep which blends a bit between these two values and for that you need to compute both paths anyway.

ribit• 1 year ago

Execution with masking is pretty much how broaching works on GPUs. What’s more relevant however is that conditional statements add overhead on terms of additional instructions and execution state management. Eliminating small branches using conditional moves or manual masking can be a performance win.

mwkaufma• 1 year ago

One can do the precisely how it's done in the article -- inspect the assembly.

NohatCoder3• 1 year ago

But you don't generally need to care if the shader code contains a few branches, modern GPUs handles those reasonably well, and the compiler will probably make a reasonable guess about what is fastest.

account4215• 1 year ago

You do need to care about large non-uniform branches as in the general case the GPU will have to execute both sides.

account4215• 1 year ago

You got this the wrong way around: For GPUs conditional moves are the default and real branches are a performance optimization possible only if the branch is uniform (=same side taken for the entire workgroup).

grg03• 1 year ago

The good way of knowing is to look at the assembly generated by the compiler. Maybe not a completely satisfying answer given that the result is heavily vendor-dependent, but unless the high-level language exposes some way of explicitly controlling it, then assembly is what you got.

The reason people do potentially more expensive mix/lerps is because while it might cost a tiny overhead, they are scared of making it a branch.

This is a problem, though. People shouldn't do things potentially, they should look at the actual code that is generated and executed.

DrNosferatu2• 1 year ago

This should be quantified and generalized for a full set of cases - that way the argument would stand far more clearly.

DrNosferatu2• 1 year ago

Something like this:

https://doliveira4.github.io/gpuconditionals/

(no warranty)

ajross3• 1 year ago

For the record, of course real branches do happen in GPU code

Well, for some definition of "real". There are hardware features (on some architectures) that implement semantics that evaluate the same way that "branched" scalar code would. There is no branching at the instruction level, and can't be on SIMD (because the other parallel shaders being evaluated by the same instructions might not have taken the same branch!)

account4215• 1 year ago

There is real branching on the hardware level, but yes it needs to take the same branch for the whole workgroup and anything else needs to be "faked" in some form.

ajross3• 1 year ago

Yeah, exactly: I argue that a global backwards-only branch used to implement loops by checking all lanes for an "end" state is not actually a "real" branch.

It's a semantic argument, but IMHO an important one. Way, way too many users of GPUs don't understand how the code generation actually works, leading to articles like this one. Ambiguous use of terms like "branch" are the problem.

cjbgkagh• 1 year ago

I think the core problem is that when writing code like this you need experience be sure that it won’t have a conditional branch. How many operations past the conditional cause a branch? Which operations can the compiler elide to bring the total below this count? I’m all for writing direct code and relying on smart compilers but it’s often hard to know if and where I’m going to get bitten. Do I always have to inspect the assembly? Do I need a performance testing suit to check for accidental regressions? I find it much easier if I can give the compiler a hint on what I expect it to do, this would be similar to a @tailcall annotation. That way I can explore the design space without worry that I’ll accidentally overstep a some hard to reason about boundary that will tank the performance.

layer8• 1 year ago

This article is also relevant: https://medium.com/@jasonbooth_86226/branching-on-a-gpu-18bf...

“If you consult the internet about writing a branch of a GPU, you might think they open the gates of hell and let demons in. They will say you should avoid them at all costs, and that you can avoid them by using the ternary operator or step() and other silly math tricks. Most of this advice is outdated at best, or just plain wrong.

Let’s correct that.”

grumpy_coder• 1 year ago

I believe the conclusion is correct in 2025, but the article in a way just perpetuates the 'misinformation', making it seem like finding if your code will compile to a dynamic branch or not is easier than it is.

The unfortunate truth with shaders is that they are compiled by the users machine at the point of use. So compiling it on just your machine isn't nearly good enough. NVIDIA pricing means large numbers of customers are running 10 year old hardware. Depending on target market you might even want the code to run on 10 year old integrated graphics.

Does 10 year old integrated graphics across the range of drivers people actually have running prefer conditional moves over more arithmetic ops.. probably, but I would want to keep both versions around and test on real user hardware if this shader was used a lot.

CountHackulus• 1 year ago

I love seeing the codegen output, makes it easy to understand the issue, but claiming that it's faster or slower without actual benchmarks is a bit disappointing.

leeoniya• 1 year ago

this.

why waste brain cells on theory when you should simply bench both versions and validate without buying into any kind of micro-optimization advice at face value.

aappleby• 1 year ago

These sort of avoid-branches optimizations were effective once upon a time as I profiled them on the XBox 360 and some ancient Intel iGPUs, but yeah - don't do this anymore.

Same story for bit extraction and other integer ops - we used to emulate them with float math because it was faster, but now every GPU has fast integer ops.

Agentlien3• 1 year ago

now every GPU has fast integer ops.

Is that true and to what extent? Looking at the ISA for RDNA2[0] for instance - which is the architecture of both PS5 and Xbox Series S|X - all I can find is 32-bit scalar instructions for integers.

[0] https://www.amd.com/content/dam/amd/en/documents/radeon-tech...

LegionMammal9783• 1 year ago

You're likely going to have a rough time with 64-bit arithmetic in any GPU. (At least on Nvidia GPUs, the instruction set doesn't give you anything but a 32-bit add-with-carry to help.) But my understanding is that a lot of the arithmetic hardware used for 53-bit double-precision ops can also be used for 32-bit integer ops, which hasn't always been the case.

qwery2• 1 year ago

It's always been less of a big deal than it used to be -- at least on "big" GPUs -- but the article isn't really about avoiding branches. The code presented is already branchless. The people giving out the advice seem to think they are avoiding branches as optimisation but their understanding of what branching code is is apparently based on if they can see some sort of conditional construct.

mgaunard• 1 year ago

"of course real branches happen in GPU code"

My understanding was that they don't. All executions inside a "branch" always get executed, they're simply predicated to do nothing if the condition to enter is not true.

ack_complete• 1 year ago

That's only if execution is incoherent. If all threads in a warp follow the branch the same way, then all of the instructions in the not taken branch are skipped.

arbitrandomuser• 1 year ago

What is the AMD and Microsoft cshader compiler , how do I generate and inpect these intermediate codes on my computer?

blackle• 1 year ago

For AMD you can use the Radeon GPU Analyzer: https://gpuopen.com/rga/

qwery2• 1 year ago

Some of the mistakes/confusion being pointed out in the article is being replicated here, it seems.

The article is not claiming that conditional branches are free. In fact, the article is not making any point about the performance cost of branching code, as far as I can tell.

The article is pointing out that conditional logic in the form presented does not get compiled into conditionally branching code. And that people should not continue to propagate the harmful advice to cover up every conditional thing in sight[0].

Finally, on actually branching code: that branching code is more complicated to execute is self-evident. There are no free branches. Avoiding branches is likely (within reason) to make any code run faster. Luckily[1], the original code was already branchless. As always, there is no universal metric to say whether optimisation is worthwhile.

[0] the "in sight" is important -- there's no interest in the generated code, just in the source code not appearing to include conditional anythings.

[1] No luck involved, of course ... (I assume people wrote to IQ to suggest apparently glaringly obvious (and wrong) improvements to their shader code, lol)

cwillu• 1 year ago

Hmm, godbolt is showing branches in the vulkan output:

        return x>0.923880?vec2(s.x,0.0):
               x>0.382683?s*sqrt(0.5):
                          vec2(0.0,s.y);

turns into

         %24 = OpLoad %float %x
         %27 = OpFOrdGreaterThan %bool %24 %float_0_923879981
               OpSelectionMerge %30 None
               OpBranchConditional %27 %29 %35
         %29 = OpLabel
         %31 = OpAccessChain %_ptr_Function_float %s %uint_0
         %32 = OpLoad %float %31
         %34 = OpCompositeConstruct %v2float %32 %float_0
               OpStore %28 %34
               OpBranch %30
         %35 = OpLabel
         %36 = OpLoad %float %x
         %38 = OpFOrdGreaterThan %bool %36 %float_0_382683009
               OpSelectionMerge %41 None
               OpBranchConditional %38 %40 %45
         %40 = OpLabel
         %42 = OpLoad %v2float %s
         %44 = OpVectorTimesScalar %v2float %42 %float_0_707106769
               OpStore %39 %44
               OpBranch %41
         %45 = OpLabel
         %47 = OpAccessChain %_ptr_Function_float %s %uint_1
         %48 = OpLoad %float %47
         %49 = OpCompositeConstruct %v2float %float_0 %48
               OpStore %39 %49
               OpBranch %41
         %41 = OpLabel
         %50 = OpLoad %v2float %39
               OpStore %28 %50
               OpBranch %30
         %30 = OpLabel
         %51 = OpLoad %v2float %28
               OpReturnValue %51

https://godbolt.org/z/aqob7YfWq

SideQuark2• 1 year ago

Vulcan opcode shader lang is not executed. It’s a platform neutral intermediate language, so won’t have the special purpose optional instructions most GPUs do since GPUs aren’t required to.

It likely compiles down on the relevant platforms as the original article did.

nosferalatu123• 1 year ago

A lot of the myth that "branches are slow on GPUs" is because, way back on the PlayStation 3, they were quite slow. NVIDIA's RSX GPU was on the PS3; it was documented that it was six cycles IIRC, but it always measured slower than that to me. That was for even a completely coherent branch, where all threads in the warp took the same path. Incoherent branches were slower because the IFEH instruction took six cycles, and the GPU would have to execute both sides of the branch. I believe that was the origin of the "branches are slow on GPUs" myth that continues to this day. Nowadays GPU branching is quite cheap especially coherent branches.

nice_byte2• 1 year ago

coherent branches are "free" but the extra instructions increase register pressure. that's the main reason why dynamic branches are avoided, not that they are inherently "slow".

dahart11• 1 year ago

If someone says branching without qualification, I have to assume it’s incoherent. The branching mechanics might have lower overhead today, but the basic physics of the situation is that throughput on each side of the branch is reduced to the percentage of active threads. If both sides of a branch are taken, and both sides are the same instruction length, the average perf over both sides is at least cut in half. This is why the belief that branches are slow on GPUs is both persistent and true. And this is why it’s worth trying harder to reformulate the problem without branching, if possible.

torginus6• 1 year ago

I'm not going to second guess IQ, who is one of the greatest modern authorities on shaders, but I do have some counterarguments.

- Due to how SIMD works, it's quite likely both paths of the conditional statement get executed, so its a wash

- Most importantly, if statements look nasty on the screen. Having an if statement means a discontinuity in visuals, which means jagged and ugly pixels on the output. Of course having a step function doesnt change this, but that means the code is already in the correct form to replace it with smoothstep, which means you can interpolate between the two variations, which does look good.

torginus6• 1 year ago

Why does this keep getting downvoted? This is fundamentally true, and good advice borne of experience. At least somebody would care to weight in as to why they disagree?

grg03• 1 year ago

I am not 100% sure nor did I downvote, but it doesn't look like a counter-argument at all.

Due to how SIMD works, it's quite likely both paths of the conditional statement get executed, so its a wash

It's not just quite likely, it's what IQ is showing in the disassembly. For a ternary op like this one with trivial expressions on each side, the GPU evals both and then masks the result given the result of the condition.

a step function doesnt change this, but that means the code is already in the correct form to replace it with smoothstep, which means you can interpolate between the two variations, which does look good.

A smoothstep does smooth interpolation of two values. It seems unrelated to the issue in the post. step() relates to the ternary op in the sense that both can be used to express conditionals. The post explains why you wouldn't necessarily want to use step() vs ternary op. smoothstep is related to step in some sense, but not in a way that relates to the article? i.e., going from step() to smoothstep() will entirely change the semantics of the program precisely because of the 'smooth' part.

tsylba• 1 year ago

It's funny because I rarely seen this (wrong approach) done anywhere else but I pick it up by myself (like a lot did I presume) and still am the first to do it everytime I see the occasion, not so for optimizations (while I admit I thought it wouldn't hurt) but for the flow and natural look of it. It feels somehow more right to me to compose effect by signals interpolations rather than clear ternary branch instructions.

Now I'll have to change my ways in fear of being rejected socially for this newly approved bad practice.

At least in WebGPU's WGSL we have the `select` instruction that does that ternary operation hidden as a method, so there is that.

lerp-io• 1 year ago

I've been doing this from day 1 becausee I just assumed you are not supposed to have loops or if else blocks in your shader code. Now I know better, thanks iq ur g.

leguminous• 1 year ago

Using `mix()` isn't necessarily bad. Using boolean operations like `lessThan()` is probably better than `step()`. I just tested two ways of converting linear RGB to sRGB. On AMD, they compile to the same assembly.

Method 1.

    float linear_to_srgb(float v) {
        return v < 0.0031308 ? v * 12.92 : 1.055 * pow(v, 1.0 / 2.4) - 0.055;
    }
    
    vec3 linear_to_srgb(vec3 rgb) {
        return vec3(linear_to_srgb(rgb.r), linear_to_srgb(rgb.g), linear_to_srgb(rgb.b));
    }

Method 2:

    vec3 linear_to_srgb(vec3 rgb) {
        bvec3 cutoff = lessThan(rgb, vec3(0.0031308));
        vec3 upper = vec3(1.055) * pow(rgb, vec3(1.0 / 2.4)) - vec3(0.055);
        vec3 lower = rgb * vec3(12.92);
        return mix(upper, lower, cutoff);
    }

Don't "optimize" conditional moves in shaders with mix()+step()

Related Stories