tracing web applications under FreeBSD

tracing web applications under FreeBSD

Web applications can be traced or debugged on many layers. The highest layer would be what some frameworks like symfony provide in their development consoles. Another layer would be to look at output from php-xdebug in kcachegrind to do profiling. But the lowest possible level is to look at actual system calls used by a running application.

FreeBSD has a base system utility called ktrace. It allows administrator to attach to a running process and log all system calls used by the process. How to use it to trace a web application running under apache?

First apache has to be started in debug mode, with only one worked running. This will make finding the apache process running our application easier to find.

This is how to start apache in debug mode:

# httpd -X &
[1] 34702

Running this in the background will return the ID of the process: 34702. Now, all we have to do is to attach ktrace to this process:

# ktrace -dip 34702

option “-d” means that all descendants (current child processes) of the process will also be traced and option “-i” means that all process spawned by our process will also be traced. “-p” option is used to give the PID of the traced process.

at this point we can run our application from the browser and after doing so take a look at system calls used. Ktrace saves all system calls to a ‘ktrace.out” file in the current directory. kdump utility is used to display contents of this file:

# kdump -R

“-R” option will display time taken between entries so we can estimate how long a syscall took to finish.

we can detach ktrace and stop tracing the process by:

# ktrace -C