kernel: pid 1214 (httpd), uid 80: exited on signal 11

# tail /var/log/messages
May 29 22:17:03 web020 kernel: pid 1214 (httpd), uid 80: exited on signal 4
May 29 22:17:03 web020 kernel: pid 1214 (httpd), uid 80: exited on signal 11

- Disabling css&js preprocess fixes it.
- I had the 500 Error here too when enabling css compression

Performance: Optimize CSS option causes php cgi to segfault in pcre function "match"

raulgigea - April 24, 2009 - 18:32

Project:

Drupal

Version:

6.14

Component:

base system

Category:

bug report

Priority:

critical

Assigned:

Unassigned

Status:

active

Jump to:

Most recent comment

Most recent attachment

Description

OS: FreeBSD 7.1-RELEASE-p4

Webserver: lighttpd-1.4.22

PHP 5.2.9 with Suhosin-Patch 0.9.7

PCRE version: pcre-7.9 *!

When using a zen-based theme, the webserver returns an "HTTP Error 500 - Internal server error", because the php cgi process ( both fastcgi and normal cgi versions tested ) segfaults.

I traced the segfault in the php-cgi.core dump file with gdb at the match routine in the pcre library. I also tried increasing and decreasing the pcre.backtrack_limit and pcre.recursion_limit, but it didn't help. ( also: memory_limit = 128M )

I traced the drupal php code which triggers this segfault to this preg_replace call in the drupal_load_stylesheet function:

Lines 1066-1073:

    if ($_optimize) {

      // Perform some safe CSS optimizations.

      $contents = preg_replace('x', '\1', $contents);

    }

I assume the problem lies at the regex and this version of pcre, because the exact same configuration of (drupal,theme,database) doesn't cause the crash on another system ( debian-lenny libpcre3 (7.6-2.1) ) .

The crash only happens on a certain pattern in the css ( here in $contents ), and that's what i'm trying to find out now. I'll post it when I pinpoint it.

I found the same issue here, but it's slightly different, since only one comment has the exact same issue: http://drupal.org/node/331915

Excuse my bad english.

Greetz,

Raul

» Login or register to post comments

#1

asb - April 25, 2009 - 01:22

In the forum is a general overview about this issue, including some more information about the server environment:

D6 on FreeBSD 7.0 & Lighttpd - 500 Internal Server Error

(it's the same D6 site, and we could really need some help with this!)

Greetings, -asb

Login or register to post comments

#2

raulgigea - April 25, 2009 - 14:51

Hi,

first, i want to make a little correction:

The Php Code which triggers the segfault is located at lines 1966-1973, in the file includes/common.inc .

second, i pinpointed the php code which always reproduces the bug. It's:

$contents = "

/* aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

*/

";

   $contents = preg_replace('x', '\1', $contents);

Just put that code in a separate php file, run it and it should crash. ( Eventually increase the number of aaa's, feel free to insert malicious machine code, etc. ;) )

This DoS only seems to affect php version 5.2.9 built against pcre 7.9, since i tested it on following machines:

FreeBSD 7.1-RELEASE-p4, PHP 5.2.9 with Suhosin-Patch 0.9.7 , pcre-7.9. Result: segmentation fault

FreeBSD 7.1-RELEASE-p4, PHP 5.2.8 with Suhosin-Patch 0.9.6.3, pcre-7.8. Result works

FreeBSD 7.1-RELEASE-p4, PHP 5.2.8 with Suhosin-Patch 0.9.6.3, pcre-7.8. Result works

FreeBSD 8.0-CURRENT, PHP 5.2.6 with Suhosin-Patch 0.9.6.2, pcre-7.8. Result works

MacOS 10.5.6, PHP 5.2.6, pcre 7.6. Result works

So, a Drupal Theme must only include a css file with that comment lines somewhere in it, and than drupal/php/webserver crashes ( given it has pcre 7.9 & php 5.2.9 ). It Seems the problem is when there are too many characters inside a multiline comment.

Login or register to post comments

#3

raulgigea - April 25, 2009 - 16:56

Seems the standalone pcre version i gave was irrelevant, since another version is bundeled with every php release.

Also, i managed to reduce the php code to the core of the problem:

   $contents = 'd' . str_repeat('a', 1900) . 'b';

   $contents = preg_replace('/d(a)+b/', '\1', $contents);

And, I observerd, the segfault doesn't occur if I disable mhash.so extension of php from extensions.ini.

Login or register to post comments

#4

laurentchardin - April 29, 2009 - 08:39

I have the same exact symptom on another totally different system: vista, apache 2.0.x and latest version of php (5.2.9-2)

Debugging the dieing apache process shows a bunch of 'match' commands before the stack overflow occurs.

(actually i have 64Mo memory limit)

Here is a trace.

Disabling css&js preprocess fixes it.

Attachment

Size

CrashHang_Report__PID_5020__PID_5908__04242009205858726.zip

37.36 KB

Login or register to post comments

#5

Damien Tournoud - April 29, 2009 - 10:00

@raulgigea: nice work on investigating the issue, but this is doesn't seems to be a Drupal bug. Why not opening a bug on the PHP bug tracker?

Login or register to post comments

#6

raulgigea - May 5, 2009 - 15:39

Hi, well because I wasn't really sure if the development (trunk/head) version of php still has this bug, in which case i should report the bug to the FreeBSD package maintainers, and still not here.

On the other hand, this post might help some users which upgrade to php 5.2.9, which i presume will happend soon, because 5.2.9 ( as a new version) is reaching the unix distros anytime now. Unless i can stop it with a bug report ;).

But i'm compiling the development version of php as i type, and will either open a php bug or post here the result, if it works.

Login or register to post comments

#7

raulgigea - May 5, 2009 - 16:29

Okay, it actually does crash with the cvs snapshot version of php. So here's the new php bug: http://bugs.php.net/bug.php?id=48153 .

Login or register to post comments

#8

raulgigea - May 15, 2009 - 15:12

Hi,

as you might have read, the php guys didn't accept it as a bug aswell.

For a big enough pattern, the preg_match function simply takes too much stack memory. It's not a bug, its the same as coding a recursive function that goes too deep into recursion.

It is the programmers responsability to check that his code doesn't eat up all the stack and choke.

In this case, Lines 1966-1973, in the file includes/common.inc works in most configurations of php, but some systems don't have big enough stack size, or other php modules eat up more stackmemory so that the left one goes out faster than usual.

In any case, if we construct unusual test cases - for example:

- insert >60000 bytes into a multiline Comment in an css file, than it almost surely segfaults on any configuration ( os, libs & php ).

The problem is where the os & php & libs are poorly configured, this case occurs with much less bytes and it can happend that a usual longer multiline comment is able to trigger this stack overflow.

For those cases i provided a patch, that is equivalent to that regex, but withouth using PCRE functions thus without the stack overflow.

Greetz,

Raul

Attachment

Size

strip_comments_without_pcre.patch

1.4 KB

Login or register to post comments

#9

scoorch - July 25, 2009 - 20:45

Raul, great work. I had the 500 Error here too when enabling css compression. Your patch fixed the problem. Thanks so much!

Thomas

Login or register to post comments

#10

asb - July 26, 2009 - 18:44

Is there any chance to get this patch into core, or will we have to maintain this bloody patch for every Drupal release?

Thanks -asb

Login or register to post comments

#11

poshat - August 17, 2009 - 22:11

Hi!

I have same problem. I tried turn on optimization CSS and JS and site working ok.

Login or register to post comments

#12

asb - September 17, 2009 - 12:26

Version:

6.10

» 6.14

Priority:

normal

» critical

This bug is still killing Drupal everytime core is updated (and thus our patch is being overwritten).

Rauls patch from #8 again has to be modified, this time we're getting:

Parse error: syntax error, unexpected '*' in /usr/local/www/drupal/includes/common.inc on line 1927

This is extremely annoying and an absolute showstopper to run Drupal on FreeBSD. Updating the applicable version of Drupal core to 6.14 and setting the priority to "critical" since it forces us to not install security updates.

Greetings, -asb