PHP's ob_start() pre-allocates 40k memory per call

http://ilia.ws/archives/244-PHPs-Output-Buffering.html

by vlucas • 15 years ago

36 29 15 years ago

nknight• 15 years ago

The more usual solution for growable buffers is to start small and just double each time you hit the limit, not grow it by a static amount. People have tried a lot of different strategies, but there don't seem to be any that work much better for general purpose use.

Someone• 15 years ago

This is not merely a matter of trying and picking what suits you best. If you grow a buffer by fixed amounts, you end up copying O(n^2) items. If you grow it by a fixed factor, you end up copying O(n) items. The difference can be staggering.

jemfinch2• 15 years ago

You only end up copying O(N^2) elements if you copy elements at all. You need not copy: you just keep a vector of pointers to your fixed buffers, a la GCC's deque implementation. That vector of pointers will be copied (and thus ought to grow by doubles), but each fixed buffer remains the same as the deque grows.

I haven't looked at PHP's implementation, and I've long since learned not to expect intelligent, rational implementation decisions from that language community, but it's certainly possible to grow by a fixed amount and remain O(N) by never copying elements at all.

jemfinch2• 15 years ago

Depending on the number of objects you're allocating, it's often better (for reduced fragmentation and smooth performance) to use an allocation strategy like GCC's std::deque: allocate fixed-size buffers, and use an exponentially growing array to hold pointers to those buffers. Once you start getting into millions of elements in your std::vector, you'll see long program pauses (as the vector grows and all the elements are copied to the new vector) and sometimes have difficulty finding a large enough contiguous buffer.

This is one reason why the C++ cognoscenti recommend using std::deque as your default array-like container rather than std::vector.

leif• 15 years ago

Why on earth are you trying to buffer 1-2K?

CaveTech• 15 years ago

Because in PHP the return response begins as soon as you start non-buffered output, after which you can no longer modify headers, cookies, etc.

wvenable2• 15 years ago

Of course, there's no reason to output everything all the time. It is possible to use string concatenation and output the result at the end of processing rather than 16 layers of ob_start(). That seems a bit excessive.

ars3• 15 years ago

That explains one ob_start() but not 16 of them.

bkrausz• 15 years ago

If I'm reading the PHP source correctly, in 5.4 this is changed to 16kb (this entire code block seems to have been rewritten):

http://svn.php.net/viewvc/php/php-src/trunk/main/output.c?vi...

http://svn.php.net/viewvc/php/php-src/trunk/main/php_output....

   #define PHP_OUTPUT_HANDLER_DEFAULT_SIZE     0x4000

ck2• 15 years ago

I always treated output buffering/capturing like eval

- if you have to use it, you need to question yourself carefully as to "why" and if there is a better way.

forgotusername• 15 years ago

As any European user of HN knows too well, when you send tiny chunks separated by a few ms a time up to TCP, the majority of the planet will get a jittery half-rendered experience of your home page lasting seconds for a handful of actual kb downloaded.

The better question might be "if you want to turn it off, the question is why", or something.

ars3• 15 years ago

Using one ob_start() could make sense for certain cases. But 16 of them? I can't think of any reason to nest ob_start 16 levels deep.

ecaron• 15 years ago

I feel like most people use this as a stop-gap for doing gzipping content to reduce bandwidth, but hopefully more people relying on nginx/apache to do the compression delivery will reduce the circa-2004 PHP mentality that started the practice.

Oh Steve Souders, your quest for speed will doom us all:)

carldall• 15 years ago

It would surely be great if the PHP Docs would include such information right where the function is described.

joelhaasnoot• 15 years ago

It's probably hidden somewhere in the comments from the last 10 years, next to the awful comments written in foreign languages.

PHP's ob_start() pre-allocates 40k memory per call

Related Stories