This distinction is less a gotcha and more actually immediately obvious in the Windows task manager. It shows cached memory separately and in a different colour. The "in use" part is real in-use excluding the page cache.
On my current Windows 11 install i'm using 7 GB with just this Firefox tab open, and another 18GB of "Cached" RAM.
Same as how `free` on Linux shows it in a different column.