Addressing The Outmoded Swapping And Paging Strategy in OSX

http://dropsafe.crypticide.com/article/3848

by lallysingh • 16 years ago

37 22 16 years ago

dropsafe.crypticide.com

macrael• 16 years ago

Where did the article go? I can't resolve the host.

lallysingh• 16 years ago

I didn't have any problems reloading right now.

http://downforeveryoneorjustme.com/http://dropsafe.crypticid...

It's just you. Maybe try again in a few?

jamesk22• 16 years ago

This article is an example of why Apple was smart for going Unix. It attracts really smart geeks that can help shed light on performance issues in a way normal consumers just can't.

blasdel3• 16 years ago

Unfortunately this is not the smart geek that they are looking for. He's worse than a normal consumer: he's a power user. Note how he's not done any benchmarks at all, he's just insisting that this must be the cause because the handholding Linux distros he's familiar with all use swap partitions, so OS X must just be retarded or something.

Paging is fucked in OS X, but this isn't why.

He focuses entirely on its use of swap files, which is not a problem at all, especially since the middle of the 10.5 betas. Before that, the system would only allocate new swap files and never do compaction or reclamation, so if your filesystem was normally within 10G of being full, it could page its way into filling it, and grind into a performance hell orders of magnitude worse than what he's complaining about (for years I had to reboot every 100 days or so to reclaim swap). Insisting on using fixed swap partitions made sense twenty years ago, but it sure as hell doesn't today as long as you do GC.

Paging on OS X is much higher latency than any other modern unix, but the fault lies with how VM works in the guts of Mach, not some userland configuration trivialities. Chapter 8 of Amit Singh's Mac OS X Internals has details of the implementation but doesn't go too far into why it sucks.

jamesk22• 16 years ago

blasdel, would you reading Mac OS X Internals if Apple wasn't Unix?

Smart Geeks intelligently exploring and debating things like this is a win for any OS.

sid0• 16 years ago

> Insisting on using fixed swap partitions made sense twenty years ago, but it sure as hell doesn't today as long as you do GC.

The main advantage of using fixed swap partitions is that you don't have to go through the file system to access swap. This is why swap files used to be slower on Linux with the 2.4 series of kernels -- with 2.6 and swap files, the kernel builds a map to the LBA at startup for direct physical access, so there's no performance difference any more. Shame on Linux distros that still use swap partitions in 2010.

Windows does the same, which is why Windows doesn't need (and has never supported) swap partitions, either.

However, this sort of map implies that while increasing swap size is easy, reducing the size of swap files is really difficult while the system is online, which is why Windows (and I believe Linux) don't attempt to solve this problem.

swombat• 16 years ago

Would this also make a difference on a Macbook Pro? It seems that it should... When I look in my /var/vm I also have a collection of incremental swapfiles in there.

I also get lots of slow-downs when closing apps - exacerbated by the fact that I swapped my MBP's hard drive for a 5'400rpm, bigger one, which means disk IO is even slower. Anything that reduces disk IO would be welcome.

blasdel3• 16 years ago

If you force-quit / SIGKILL the app, it won't bother to page as much of its bullshit back into memory as it's terminating it. I'm kind of baffled by the continued presence of graceful shutdown procedures in an age when everyone does recovery on startup anyway.

The problem isn't that the 5400rpm drive is 'slower', but that the average latency is higher (though only by about 1.5x) -- it likely has higher streaming bandwidth simply by virtue of greater areal density.

labria• 16 years ago

By the way, Apple does just that in Snow Leopard. The processes now have a "Sudden Termination" flag, and if a program has no unsaved data and is ready to quit, it sets this flag on itself. When shutting down such a process, the system sends it SIGKILL instead of a usual SIGTERM.

godDLL2• 16 years ago

There is also the "throw money at it" fix for this – an SSD. I've been enjoying one, of the Intel X25-M G2 variety, it solves VM paging/thrashing slowness very nicely and is hella fast to boot.

ihartley• 16 years ago

I have the same drive. I currently have 6GB of VM, but I've had it up as far as 14GB with no issues.

godDLL2• 16 years ago

I've actually had Photoshop run out of the 20 GB that are free, by allocating all of it to manipulate a huge image. It takes about two seconds for it to allocate all of that space.

sparky• 16 years ago

no pun intended, right? :P

kentosi• 16 years ago

very well done.

i wonder what the effect would be if you simply increase the paging size, as he describes, without creating the separate partition. this solution might be more appropriate for those of us who simply want to avoid the beachball-of-death, and aren't to hung up on speed.

stcredzero• 16 years ago

These sorts of mistakes seem to be common: forgetting that Moore's law means that memory settings need to be adjusted at least every other year. For some reason, one of the Smalltalk distributions still uses default GC settings from 1994! That's only about two orders of magnitude off! Everyone just knows to go and tweak those.

jrockway2• 16 years ago

Clearly the solution is to disallow multitasking.

But seriously, I find it strange that so many people in the comments section are complaining about how much swap Firefox uses. Don't they have machines with more than 128M of RAM!? (My Firefox is using 131M of RAM right now.)

ori_b3• 16 years ago

My firefox memory usage has regularly gone over 1 gigabyte of RAM. Currently, it's sitting at 531 megabytes of resident pages, with 39 of them shared. It's also at 1.4 gigabytes of address space used.

jrockway2• 16 years ago

Ah. I use conkeror (an xulrunner app) and Adblock Plus, and that's it. If you use plain firefox with a lot of extensions, I guess it uses more memory.

ori_b3• 16 years ago

The reason that firefox shutdown is so slow is because it's paging everything back in to run the destructors for the objects that got paged out, not because the OS is trying to give the swapfile back.

gtani• 16 years ago

revisited:

http://dropsafe.crypticide.com/article/3915

Addressing The Outmoded Swapping And Paging Strategy in OSX

Related Stories