Advanced Performance Extensions (APX)

https://www.intel.com/content/www/us/en/developer/articles/technical/advanced-performance-extensions-apx.html

by gautamcgoel • 3 years ago

152 97 3 years ago

intel.com

serhack_• 3 years ago

Intel® APX doubles the number of general-purpose registers (GPRs) from 16 to 32.

_chris_• 3 years ago

And adds non-destructive instructions.

"In addition, legacy integer instructions now can also use EVEX to encode a dedicated destination register operand – turning them into three-operand instructions and reducing the need for extra register move instructions."

Overall, APX is providing 10% fewer instructions, 10% fewer loads and more than 20% fewer stores.

Also adding pop2/push2 instructions for moving state faster.

And adding more powerful conditional instructions (loads/stores/compares) and flag-suppression.

jeffbee• 3 years ago

10% fewer instructions but average instruction is longer, so code density is the same, they claim. This still leaves their ISA with the worst code density of any non-obsolete ISA.

FullyFunctional4• 3 years ago

Oh missed your comment and posted essentially the same. These are all interesting changes, predication certainly, but the thing that actually got me the most excited was the press release comment about:

"The processor tracks these new instructions internally and fast-forwards register data between matching PUSH2 and POP2 instructions without going through memory."

I wonder if this implies that pushes don't have to commit to memory if they are popped soon enough? It has always bothered me that we have these huge physical register files but force all the spill and restore to go through memory because of silly anachronistic processor semantics. With a more flexible PUSH/POP semantics we could essentially get the register windows for free.

FullyFunctional4• 3 years ago

"legacy integer instructions now can also use EVEX to encode a dedicated destination register operand – turning them into three-operand instructions"

x86 ISA is growing more RISC-like. Definitely saving on stack spilling is a Good Thing™

dfox• 3 years ago

Apparently the Intel's marketing had forgotten that they already had a product called iAPX (standing for “Advanced Performance arCHitecture”) and it did not go exactly well :)

JohnFen• 3 years ago

I genuinely thought that's what the article was about when I read the title.

kjs33• 3 years ago

You mean the 8086, 80186, 80286 and 80386 and associated support chips?[1] Perhaps you and I have different definitions of 'go well'.[2]

[1] https://en.wikipedia.org/wiki/IAPX [2] I know you're thinking about the iAPX432 and forgot the rest

dmitrygr• 3 years ago

So, in about 30 years when the majority of the CPUs have this, we can use it. Assuming intel does not gate this just to XEON for no reason whatsoever, like they did to AVX512?

brucethemoose27• 3 years ago

"Tiered" x86 packages and executables seem like the inevitable direction for linux distros.

CachyOS and Clear Linux already do this.

Base Arch Linux and openSUSE are working on it. Maybe Fedora too, but I can't remeber

And it wouldn't be totally insane for Windows to do this either.

fooyc2• 3 years ago

How does that work? The binary format embeds variants of the same program?

nwallin• 3 years ago

This is Gentoo's whole shtick. It's the core feature. It's been supported since day 0, over 20 years ago.

I'm honestly a little amazed that more people don't either use Gentoo or adopt its model, given the smorgasbord of mutually incompatible instruction set extensions. It seems very strange that people will buy a CPU that has 32 64-byte ZMM registers with 3 operand instructions, and then use that CPU to run code that operates on 8 16-byte XMM registers with 2 operand instructions.

fooyc2• 3 years ago

Maybe JIT compilers can take profit of this immediately, since they target a single machine?

mike_hearn3• 3 years ago

Yup. It's one of their theoretical advantages that's about to become a lot less theoretical. Historically it hasn't made much difference because optional instructions were hard for JIT compilers for most languages to use (in particular high level JITd languages tend not to support vector instructions very well). But a doubling of registers is the sort of extension that any kind of code can immediately profit from.

Arguably it will be only JITd languages that benefit from this for quite a while. These sorts of fundamental changes are basically a new ISA and the infrastructure isn't really geared up to make doing that easy. Everyone would have to provide two versions of every app and shared library to get the most benefit, maybe even you get combinatorial complexity if people want to upgrade the inter-library calling conventions too. For native AOT compiled code it's going to just be a mess.

pjmlp4• 3 years ago

Modern compilers already allow for conditional code execution depending on CPU sets, and at least in what concerns JVM implementations and the CLR, their JITs are clever enough to already use parts of AVX512, while they are not perfect, it is better than not using them at all.

kmeisthax• 3 years ago

Performance-intensive code (hot loops) could be compiled multiple times for each architecture extension and switched with CPUID. That's how all the various SSE and AVX extensions were rolled out in multimedia code.

AFAIK there was AVX512 in higher-end desktop SKUs, but not the latest E-core designs in Intel's client chips. The main problems are that:

- Operations at 512-bit register widths have significant power draw. Intel chips that support AVX512 have to downclock themselves on AVX512 workloads until their voltage regulators have boosted up to a higher voltage.

- The AVX512 register file is too big to physically fit in the E-core[0] footprint.

Incidentally I do remember Linus Torvalds specifically complaining that AVX512 was being used to implement memcpy in gcc, because it meant running certain programs would lower system performance. So these new architectures tend to be used a lot sooner than the time it takes for it to be safe to make them your minimum compile target.

[0] The BIOS on my Framework laptop refers to these as "Atom cores" - no clue if the current E-core design is derived from Atom or if this is a miscommunication or nickname AMI picked.

adrian_b8• 3 years ago

No, operations at 512-bit register widths do not have significant power draw, as demonstrated by AMD Zen 4.

What has significant power draw is the use of double 512-bit floating-point multipliers, as implemented in the Intel server CPUs (though one core with such multipliers draws significantly less power than two cores having the same throughput).

AMD uses only double 256-bit floating-point multipliers and in general it uses exactly the same execution units for both 256-bit and 512-bit operations, so the AVX-512 operations do not increase the power draw even when they use 512-bit registers.

Also the AVX512 register file is not too big to physically fit in the E-core. Even if the AVX512 register file is 4 times greater than the AVX register file, the E-cores have much a much larger register file used to rename the architecturally visible registers.

Despite these facts, Intel still believes that implementing the full AVX-512 ISA in the E-cores is too expensive, so they have created this new specification of AVX10/256, which is just a subset of AVX-512 including the instructions with an operand size up to 256 bits, and which will be implemented in all future E-cores after some date, perhaps starting in 2025.

wmf4• 3 years ago

Yes, E-cores are Atom cores. You can tell because they have -mont in the codename.

KerrAvon• 3 years ago

So it sounds like (among other things) they're adding 3-address integer instructions to an instruction encoding only used for vector instructions today.

I was not familiar with the AVX vector instructions at this level of detail.

https://en.wikipedia.org/wiki/EVEX_prefix

peterfirefly4• 3 years ago

Have you noticed that some of the bits in the EVEX prefix (after the 62h) byte are inverted?

That's because it hides inside the BOUND instruction in a way that allows it to be used outside of 64-bit mode code. The BOUND instruction never existed in 64-bit mode, so Intel was free to do whatever they wanted with the 62h opcode, but they thought it would be enable EVEX in 16/32-bit code too.

The BOUND instruction must take a memory operand -- so the MOD bits can never be 11b (which would specify a register operand). The MOD bits are the upper two bits of the modrm byte (the byte right after the 62h opcode). So, if we don't allow the "upper" registers in 32-bit mode and only the "lower" 8 registers AND if we invert bit 3 of their register numbers and put those extra register specifier bits in bit 7/6/5 of the byte after the 62h opcode THEN we can fit EVEX into 32-bit mode, because those bits will always be 111b in 32-bit mode!

So outside of 64-bit mode, if we have a 62h opcode with a modrm byte with MOD!=11b: it's a BOUND instruction. If MOD=11b: it's an EVEX prefix.

brucethemoose27• 3 years ago

Intel® APX demonstrates the advantage of the variable-length instruction encodings of x86 – new features enhancing the entire instruction set can be defined with only incremental changes to the instruction-decode hardware. This flexibility has allowed Intel® architecture to adapt and flourish over four decades of rapid advances in computing – and it enables the innovations that will keep it thriving into the future.

pavlov• 3 years ago

Interesting. More registers and separate destination on instructions is about 40 years overdue, but better late than never.

I realized I’ve completely lost track of Intel’s architecture extensions reading this:

“They do not change the size and layout of the XSAVE area as they take up the space left behind by the deprecated Intel® MPX registers.”

Apparently MPX was Memory Protection Extensions and the design was so flawed, it was removed entirely soon after introduction.

theandrewbailey• 3 years ago

More registers is about 40 years overdue, but better late than never.

SSE, AMD64, AVX, and AVX-512 say hi.

kjs33• 3 years ago

And MMX from 1997...so almost 35 of those 40 years ago.

FullyFunctional4• 3 years ago

The "wall of text" TL;DR:

- +16 registers (thus 32) and optionally separate destination (looking very RISC like now)

- PUSH2/POP2 with full forwarding

- Much expanded predication, including predicated loads and stores

This is pretty interesting. Especially the latter can make a big difference for highly unpredictable memory intensive code, like compression.

_old_dude_2• 3 years ago

Also AVX-10, E-cores and P-cores having 512 bits vector registers (see the last references).

jcranmer5• 3 years ago

If I read the note correctly, P-cores won't have 512-bit vector registers, but they will have the other fancy stuff added by AVX-512 (namely, vector predication stuff, static rounding mode instructions, new vector instructions like complex multiply or half-precision float, and 32 vector registers), just only for 128-bit and 256-bit vectors. Which, to be fair, is arguably the more useful parts of AVX-512 anyways; the maximum vector length being upped isn't all that interesting.

adrian_b8• 3 years ago

No, the E-cores will implement only a 256-bit subset of AVX-512, which halves the size of the vector registers to 256-bit and the size of the mask registers to 32-bit. The same subset will be implemented on the P-cores combined with E-cores.

This subset AVX10/256, is the reason for this new specification. It is the Intel response to AMD Zen 4.

When their competitor supports AVX-512 on all products, Intel had to do something to remain competitive. Because they believe that supporting the full AVX-512 on their E-cores is too expensive, they have created a subset of AVX-512, including only the instructions with an operand size up to 256 bits.

peterfirefly4• 3 years ago

+ Two-byte REX2 prefix that replaces the REX prefix (and 0F prefix).

REX2 is D5 + a byte that is a lot like the lower nibble of a REX prefix, only twice as big.

It has two extra bits for the up to three registers that can be named in normal x86 instructions: either two registers or a register operand and a memory operand that can use two registers + a displacement for the memory address.

It also contains the W bit like REX does (64-bit).

It also has a bit called M0 that indicates whether the instruction is in opcode map 0 (primary opcode map) or opcode map 1 (0F opcode map).

That means that a REX2 instruction from opcode map 1 takes the same number of bytes as a REX instruction from opcode map 1. Instructions from opcode map 0 are one byte longer with REX2 than with REX.

Some of the normal ALU instructions also get an EVEX encoding (in map 4). That allows for a different data destination than before (separate from source the operand(s)). It also allows for ALU instructions that don't change the flags, which must be really nice for the out of order/data forwarding circuitry.

drudru• 3 years ago

0xD5 was the old BCD instruction: AAD

user20230724• 3 years ago

Wow, perfect timing

mike_hearn3• 3 years ago

So .... when will it ship? No mention of physical products anywhere.

I wonder why this long delays are still necessary. In the old days yes as there were so many parties to coordinate but nowadays, in theory, Intel could release hardware, the new ISA and compiler/OS patches and binaries on the same day.

wmf4• 3 years ago

They don't want it to leak so they at least need to announce it before engineering samples go out.

TheCondor• 3 years ago

They floated a "white paper" a few months back about removing legacy instructions. https://www.intel.com/content/www/us/en/developer/articles/t...

Taking a little time with these sorts of things isn't bad. This will essentially be a new epic where AMD and Intel may end up being incompatible, at least for a while, there are tooling changes to make, OS support, all sorts of things. If they're serious about dropping 16bit and 32bit from the architecture, doing it during a change like this might make some sense. Do it wrong and they could Osborn themselves, but an Apple like migration strategy would be nice and I'd appreciate it.

mschuster91• 3 years ago

They announce it in time so that llvm, gcc, MSVC, Java, .NET and the browser JS engine vendors can update their stacks to be ready when the silicon ships.

After all why spend a shitload of money on developing a feature you'll want to market when there won't be any reason for customers to buy your new hardware?

mike_hearn3• 3 years ago

Intel employee people who work on Java and LLVM at least, so they could release their own versions of those. I guess V8 wouldn't be hard to add.

The main reason to do a coordinated release would be competitive advantage, I guess, plus to ensure that there are actually compilers that support their new instructions available right from the start. Intel already release their own Linux distro optimized for their chips because other vendors were targeting the lowest common denominator

jcranmer5• 3 years ago

I wonder why this long delays are still necessary. In the old days yes as there were so many parties to coordinate but nowadays, in theory, Intel could release hardware, the new ISA and compiler/OS patches and binaries on the same day.

Most new ISA support requires at least minimal support from the OS to use correctly (e.g., something like saving state on context switching, or reporting hardware support bits correctly). Releasing patches on the same day you release hardware means the new features are literally unusable for all of your customers, and it generally takes several months to go from a patch to a usable OS release. You could in theory release the patches on the same day as the new ISA documentation, but in practice, it's likely to take some time because the people writing the patches aren't the people writing the new ISA whitepapers and approving their publication and it takes time to go from "oh, I can talk about this now" to actually doing so.

adrian_b8• 3 years ago

No earlier than Q1 2025.

None of the products announced for 2024 (e.g. Sierra Forrest, Granite Rapids, Arrow Lake, Arrow Lake S, Lunar Lake) support this.

userbinator• 3 years ago

I wonder why this long delays are still necessary

Making sure the thing actually works? At least I hope they're still doing that... and it's ironic to see this comment at the same time a rather horrible bug was discovered in AMD's CPUs:

https://news.ycombinator.com/item?id=36848680

If anything, Intel should probably slow down and take more care: https://news.ycombinator.com/item?id=16058920

jcranmer5• 3 years ago

High-level overview of what's changing here:

* A new REX-like prefix that extends the number of addressable GPRs to 32 from 16. This only supports instructions that have one-byte opcodes or the 0f prefix, so recent GPR instructions like ADCX or BLSR aren't supported with this format, except.

* The EVEX prefix (used for AVX-512) is also extended to be usable for GPR instructions instead of just vector instructions. This allows three-address instructions to be defined.

* The EVEX prefix for GPR also has a dedicated bit for "do you want to set flags as a result of this instruction."

* New instructions that push/pop 2 GPRs at once

* New instructions that let you conditionally set flags (basically you can do OR/AND in the hardware flags, this sounds useful for compilers).

* New instructions for predicated loads.

* New 64-bit absolute jump instruction

* Also, implementation of the predicated stuff in AVX-512, but for 256-bit vectors. With this note:

A “converged” version of Intel AVX10 with maximum vector lengths of 256 bits and 32-bit opmask registers will be supported across all Intel processors, while 512-bit vector registers and 64-bit opmasks will continue to be supported on some P-core processors.

andromeduck• 3 years ago

It's CISC-V

muricula• 3 years ago

It seems like most of these new instructions and registers correspond to the original armv8 base isa. I'm going to go out on a limb here and suppose that's not an accident. Does anyone know why Intel thinks x86 needs them?

Is the goal here to increase the decode bandwidth of Intel CPUs?

Is the goal to reduce demands on load-store units by increasing the number of registers?

Are they hoping to make it easier to port or JIT armv8 asm to Intel CPUs?

adrian_b8• 3 years ago

Most new instructions are not inspired by Armv8, they just implement the traditional 3-address format for 32 registers, which predates Armv8 by a few decades.

Nevertheless, there are a few instructions inspired by Armv8, mainly PUSH2 and POP2, which correspond to the load register pair and store register pair of Aarch64.

comex• 3 years ago

The new CCMP conditional-compare instruction is also equivalent to ARMv8’s instruction of the same name. If a condition passes, compare two registers; if not, set the condition bits to an arbitrary value.

On ARM that instruction is a pain in the ass when reading disassemblies, because the on-fail condition bits are just specified as a number from 0 to 15; the disassembler doesn’t bother to label which bits are specified, let alone what conditions they correspond to. Unfortunately it seems like Intel is doing the same thing in their assembly syntax, at least if I’m reading the document correctly.

adgjlsfhk12• 3 years ago

well given the difficulty of making wider x86 decoders it does seem pretty reasonable to try reducing the number of load/stores needed

Bulat_Ziganshin3• 3 years ago

yeah, exactly. instruction fusing can turn mov+op2 into 3-reg operation, or push+push into push2. but adding new instructions allows to increase the frontend throughput too.

jamesy0ung• 3 years ago

I thought it was going to be related to the iAPX (Intel Advanced Performance Architecture)

https://en.wikipedia.org/wiki/IAPX

soulbadguy2• 3 years ago

extends the number of addressable GPRs to 32 from 16

I have always been curious as to why the number of GPRs were limited for so long on X86 given that the instruction set is already variable length, and the CPU have typically a very large number of internal arch-register that could be cheaply addressed.

Having looked at the pain of developing a good register allocate in LLVM, and how critical memory access can me in hot/tight loops i would have loved to have even more register something closer to 64 or 128, and let the cpu manage the spilling internally.

jcranmer5• 3 years ago

I have always been curious as to why the number of GPRs were limited for so long on X86 given that the instruction set is already variable length, and the CPU have typically a very large number of internal arch-register that could be cheaply addressed.

Because it's hard-ish to wrench in the encoding bits. x86 registers are addressed via the ModR/M byte, which gives you 4 addressing modes × 2 register operands (if you have 8 registers). Note that non-register-register addressing modes include a second SIB byte which means you have up to three registers encoded in an instruction. x86-64 extended the number to 16 by adding a prefix that has 4 bits, which encoded a 32-bit/64-bit selector and 1 bit for each of the three possible registers (R, X, B). Note that to keep the prefix to only a 1-byte length, they had to reclaim 16 possible opcodes, whose space is pretty limited.

Encoding large register numbers gets pretty chonky in instruction space (3 register ids of 5 bits each is 15 bits of your instruction for operands, and that's before you start considering possible opcodes). It also increases the size of context switches (especially thread contexts), since you have to spill all of those registers even if they're not filled with useful data.

8 registers is definitely too few; it's not clear to me if the extra working set size afforded by 32 registers is worth it over the fatter instructions.

duped• 3 years ago

They're not really limited since the instruction decoder is actually doing a bunch of work to assign to physical registers. The 16 GPRs are an abstraction, iirc.

Even though it's only one extra bit to index 16 registers vs 8, it takes an entire extra byte in the encoding (you have to add the REX prefix with the R, X, and B bits set appropriately for the MSB of the registers you're modifying in the MODR/M byte). And you only get 4 bits to use out of that byte because the leading four are a way to disambiguate it (also iirc, this stuff is terribly documented).

And paying an extra byte per GRP instruction is actually kind of expensive. If you look at your compiler output you'll see a bunch of Exx instructions even in 64 bit long mode when optimizing for instruction decode/size.

It turns out a variable length encoding is really hard to extend without getting weird with it.

peterfirefly4• 3 years ago

And you only get 4 bits to use out of that byte because the leading four are a way to disambiguate it (also iirc, this stuff is terribly documented).

No, it has been extremely well documented since 2000 (three years before AMD had the first 64-bit chip ready).

gpderetta• 3 years ago

Well, originally there where 7 (well, 6 really as frame pointers were pretty much required) general purpose registers. AMD64 bumped that to 14-15. That's a huge change with clear performance benefits. going to 32 is already into diminishing results for most use cases and has non-trivial encoding, function calling and context-switching implications.

peterfirefly4• 3 years ago

Shared libs on the 386 stole a register more, because IP-relative addressing wasn't a thing until AMD64 (except for jmp/call). Shared libs need to refer to their own data + they may need to form pointers to code within them.

(Shared libraries need to have a pointer to their code and data. This is done by creating a small table of entry code in front of the "real" code that sets a register to point to the library code/data memory before jumping to the real code. This code will look different for each process that has loaded the shared library so it gets its own little memory block. The rest of the code/data will be the same in all processes so it can be shared between them -- unless/until some of the data is written to, of course.)

So code in shared libs really only had 5 registers unless the frame pointer was fomitted. That's why it was necessary to tell the compiler whether to build normal code or shared library code.

The AMD64 not only had more registers, it also has an RIP-relative addressing mode so it doesn't need to steal that register for shared libraries.

variadix• 3 years ago

Adding more registers makes context switching take longer and require more memory. It’s a trade off.

kccqzy• 3 years ago

These registers are caller-saved, so no need to save them for system calls. Thread switching might be performance neutral if the OS uses the xsave / xrstor instructions.

soulbadguy2• 3 years ago

couldn't find the info anywhere, but any ETA on this ? Or least what is the first Arch supporting this? Redwood Cove or someting later ?

johnklos• 3 years ago

"Intel® APX demonstrates the advantage of the variable-length instruction encodings of x86 – new features enhancing the entire instruction set can be defined with only incremental changes to the instruction-decode hardware."

In other words, their initial sloppiness was a Feature™: our initial mess was so bad that changes like these don't make it any worse!

ribit• 3 years ago

Do this basically copies ARM’s Aarch64, but with a really awkward instruction encoding? Interesting move, Intel.

voidbert• 3 years ago

I don't know much about CPUs, but isn't this going to increase decoder complexity and binary size? The reduction in memory accesses seems great, but can anybody tell me if there are better ways of achieving this? x86 gets more complex each day.

vgatherps• 3 years ago

These changes reduce the amount of instructions you need in the first place - less stack shuffling with more registers and push2/pop2, less register shuffling with nondestructive operations

ksec• 3 years ago

It is clear this is at least 2 - 3 years in the making. It doesn't seems any of the 2024 products on Intel Roadmap will have APX ( This could be wrong ). So I assume the earliest being 2025.

The question is when will AMD adopt it. Zen 5 is done and Zen 6 may be too late for these changes. Zen 6 is already looking at 2026. If they waited til Zen 7 it will be at least 2028.

Intel is still 35% behind Apple in terms of Pref / Clock on Geekbench.

Advanced Performance Extensions (APX)

Related Stories