grep searchs a string in filename wildcards recursively

To search all files under /dir for "blah", run the following:

# grep -r blah /dir

You can not use wildards directly in this manner, for example, the following will not search all text files:

# grep -r blah *.txt

This doesn't work because the wildcard is expanded by the shell before grep is called. Instead, search the current directory (or whichever one you want) and pass the --include option:

# grep -r blah . --include "*.txt"

Reference:
http://mindspill.net/computing/linux-notes/recursive-grep-and-filename-wildcards/

Debugging load library path issues

Debugging load library path issues
If you get a problem with a library that can't find another, you should check the library's dependencies and make sure that the directory in which the missing library resides is part of the load library path.

One example of this problem, that I experienced recently, occurred when the Magick.so library was unable to open the libMagick.so.10 library. The error message was as follows:

Can't load '/usr/lib/perl5/site_perl/5.8.7/i686-linux/auto/Image/Magick/Magick.so' for module Image::Magick: libMagick.so.10: cannot open shared object file: No such file or directory at /usr/lib/perl5/5.8.7/i686-linux/DynaLoader.pm line 230. at (eval 113) line 1 Compilation failed in require at (eval 113) line 1. BEGIN failed--compilation aborted at (eval 113) line 1.
Find out library dependencies with ldd

The ldd command will check the dependencies of a library.

ldd /the/library/path/libWhatever.so
Using the above example problem:

username@localhost [~]# ldd /usr/lib/perl5/site_perl/5.8.7/i686-linux/auto/Image/Magick/Magick.so
libMagick.so.10 => not found
libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0x0099a000)
libz.so.1 => /usr/lib/libz.so.1 (0x00111000)
libtiff.so.3 => /usr/lib/libtiff.so.3 (0x00121000)
libjpeg.so.62 => /usr/lib/libjpeg.so.62 (0x00d54000)
libXext.so.6 => /usr/X11R6/lib/libXext.so.6 (0x001f4000)
libSM.so.6 => /usr/X11R6/lib/libSM.so.6 (0x0016e000)
libICE.so.6 => /usr/X11R6/lib/libICE.so.6 (0x00910000)
libX11.so.6 => /usr/X11R6/lib/libX11.so.6 (0x0039d000)
libXt.so.6 => /usr/X11R6/lib/libXt.so.6 (0x005bc000)
libpthread.so.0 => /lib/tls/libpthread.so.0 (0x004cc000)
libm.so.6 => /lib/tls/libm.so.6 (0x00216000)
libc.so.6 => /lib/tls/libc.so.6 (0x00bda000)
libdl.so.2 => /lib/libdl.so.2 (0x00177000)
/lib/ld-linux.so.2 (0x002c2000)
You can see in that the libMagick.so.10 library was not found.

Locate the library with find or locate or slocate or by any other means

Find the location of the library that couldn't be found by ldd. The chances are that the library will be in a subdirectory of /usr. Continuing with the above example...

username@localhost [~]# find /usr -name "libMagick.so.10"
/usr/local/lib/libMagick.so.10
Add the library to the load library path with ldconfig

In the words of the man page, ldconfig will configure dynamic linker run time bindings. Add your missing library to the bindings with the following command:

ldconfig /path/to/add
For our libMagick example, this would be:

username@localhost [~]# ldconfig /usr/local/lib
You can see a list of librarys that are being used with the -p flag.

username@localhost [~]# ldconfig -p
907 libs found in cache `/etc/ld.so.cache'
libzvt.so.2 (libc6) => /usr/lib/libzvt.so.2
libzvt.so (libc6) => /usr/lib/libzvt.so
libzip.so (libc6, hwcap: 0x0001000000000000) => /opt/blackdown-jdk-1.4.2.03/jre/lib/i386/libzip.so
libz.so.1 (libc6) => /lib/libz.so.1
libz.so (libc6) => /lib/libz.so
libxslt.so.1 (libc6) => /usr/lib/libxslt.so.1
libxslt.so (libc6) => /usr/lib/libxslt.so
[...cut...]
If you want to make make the changes more permanent, so that you don't have to remember to specify the directory next time you run the command, you can add it to the /etc/ld.so.conf file, though different linux distributions manage this file in different ways.

Reference:
http://mindspill.net/computing/linux-notes/debugging-load-library-path-issues/

FreeBSD - Backup and restore FreeBSD using Fixit CD

Since FreeSBIE have not been updated since year 2007, its kernel recognizing new hardware starts to worry me as it is based on FreeBSD 6.2 and it is going to reach EoL (end of life) by end of this year, 2010. Is time to experiment on new way restoring backup.

The FreeBSD installation process does mentioned about "Fixit" CD booting. After meddling around with it, it's actually referring to another bootable CD which its label name consits of "livefs" (Live File System). It's also known as "disk 2" prior to FreeBSD 7.
e.g.
Installation disc -- > 8.0-RELEASE-i386-disc1.iso
Live File systems disc --> 8.0-RELEASE-i386-livefs.iso

Do take note that the dump/restore instruction from the previous post "Freebsd – Backup & restore for disaster recovery" is still valid. This post is served as a "update" as it won't be using FreeSBIE, rather, it will using the livefs to start the restoration of the partition(s).

Without further ado, here's the instruction :

A quick note, after experimenting with a few backup compress method, below are the summary of my findings :
dump and compress using bzip2 (not so good as bzip is the slowest)
dump -0auLf - /dev/ad0s1a | bzip2 > root_partition.bzip2
dump and compress using gzip (good choice as gzip perform faster than bzip2 and the compression is fairly good)
dump -0auLf - /dev/ad0s1a | gzip > root_partition.gzip
plain dump without compression (fast if target disk can write fast enough)
dump -0auLf root_partition.dump /dev/ad0s1a

My preferred compress method will be using gzip as it's compression ratio is good with fair compressing time taken. In this case, the backup of the partition(s) is done with the command :
dump -0auLf - /dev/ad0s1a | gzip > root_partition.gzip


This is the updated version of how to use Live File System CD to start the restoration process :
Partition & Label the disks
Boot the Installation disc (disc 1)
At the "sysinstall Main Menu", go to "Configure --> Fdisk" and perform the following :
Partition the disk as desire.
Next, type "w" to write changes to disk.
BEWARE:this step will wipe all the contents of the hard disk.
Choose "Label" to label the partitions created in the previous step. Perform the following (similar to previous steps) :
Label the partition as desire.
After labeling the partition, note down the partition label. e.g.
ad0s1a --> "/" or root partition

ad0s2b --> swap partition
Move the up/down keys to highlight the "Disk:" and NOT the partition ("ad0s1a").
Next, type "w" to write changes to disk.
Start the Fixit CD
At the main menu, go to "Fixit --> CDROM/DVD"
At the message "Please insert a FreeBSD live filesystem CD/DVD and press return" message, change the installation disc in the drive to Livefs disc (aka disc2).
Then press enter to continue.
Restore the backup
Use the command "mount" to confirm the partitions are mounted accordingly. "/" or root partition should be mounted as /mnt
Create the external mount point & temporary directory. In this scenario, the backup file is stored in a FAT32 formatted USB external hard disk. The temporary directory is for later use with gzip command.
mkdir /tmp/usb /mnt/writable_tmp
Mount the external USB hard disk :
mount_msdosfs /dev/da0s1 /tmp/usb

*** using the usual command to mount USB hard disk "mount -t msdosfs /dev/da0s1 /tmp/usb" will get an error. See below caveats.
The temporary directory environment variable originally points to a read only directory. This result in "restore" command complaining about having not enough disk space to restore.
Re-point it to a writable disk, use the below command :
export TMPDIR=/tmp/writable_tmp/
Start the restoration process :
cd /mnt
gzcat /tmp/usb/root_partition.gzip | restore -rvf -
Repeat the previous step for the rest of the partitions until all partitions are restored.

CAVEATS :
When executing the command :
mount -t msdosfs /dev/da0s1 /tmp/usb

An error return :
mount: exec mount_msdosfs not found in /sbin:/usr/sbin: No such file or directory

For some reason, mount_msdosfs is in /mnt2/sbin/ but "mount" cannot find it.
Solution 1 :
Replace the "mount" command with equivalent "mount_msdosfs". Execute it without the need to specify parameter "-t msdosfs"
e.g.
mount_msdosfs /dev/da0s1 /tmp/usb

Solution 2
Soft link the command "mount_msdosfs" to the same directory where mount resides :
cd /sbin;ln -s /mnt2/sbin/mount_msdosfs mount_msdosfs

then mount it again with the usual command.

Nemaste !!!

Reference:
http://scratching.psybermonkey.net/2010/01/freebsd-backup-and-restore-freebsd.html

Setup a Pure-FTPd FTP server with virtual users

Setup a Pure-FTPd FTP server with virtual users

Pure-FTPd is a free (BSD), secure, production-quality and standard-conformant FTP server.

This guide provides instructions for using the virtual user system to manage and control users. By using virtual users, FTP accounts can be administrated without affecting system accounts.

Let's initiate Pure-FTPd's installation by entering the following commands:

% su
# portsnap update
# cd /usr/ports/ftp/pure-ftpd
# make config-recursive

A menu containing Pure-FTPd options will pop-up. In my case, I've opted to leave these options at their defaults.

# cat /var/db/ports/pure-ftpd/options
WITH_UTF8=true
WITH_LARGEFILE=true

# make install clean distclean
# rehash

Having finished the installation process we now move into the configuration stage. We'll start by copying the sample configuration file and set the configuration options:

# cd /usr/local/etc
# cp pure-ftpd.conf.sample pure-ftpd.conf
# chmod 444 pure-ftpd.conf

The chmod command was run to be able to edit the file (default permissions are set to -r--r--r--).

# vim pure-ftpd.conf
# Port range for passive connections replies. - for firewalling.
PassivePortRange 30000 30900

# IP address/port to listen to (default=all IP and port 21).
Bind 192.168.100.1,21

# If you want to log all client commands, set this to "yes".
# This directive can be duplicated to also log server responses.
VerboseLog yes

# PureDB user database (see README.Virtual-Users)
PureDB /usr/local/etc/pureftpd.pdb

# Create an additional log file with transfers logged in the standard W3C
# format (compatible with most commercial log analyzers)
AltLog w3c:/var/log/pureftpd.log

# Automatically create home directories if they are missing
CreateHomeDir yes

# Disallow anonymous connections. Only allow authenticated users.
NoAnonymous yes

# Disallow anonymous users to upload new files (no = upload is allowed)
AnonymousCantUpload yes

The CreateHomeDir option makes adding virtual users more easy by creating a user's home directory upon login (if it doesn't already exist).

We can either import users with system-level accounts (defined in /etc/master.passwd) at once or create new users manually. To import users that already exist on your system into the virtual user database, enter these commands:

# pure-pwconvert >> /usr/local/etc/pureftpd.passwd
# chmod 600 /usr/local/etc/pureftpd.passwd
# pure-pw mkdb

It should be noted that pure-pwconvert only imports accounts that have shell access. Accounts with the shell set to nologin have to be added manually.

To add users to the Pure-FTPd virtual user database manually, we need to create a system-level account that will be associated with virtual users. Create a new user named ftp like this:

# pw useradd -n ftp -s /sbin/nologin -w no -d /home/ftp -c "FTP user" -m

Note: When you install pureftp, an ftp group is created, but no ftp user; this results in the error "mail pure-ftpd:(?:?) [ERROR] Unable to find the 'ftp' account". So we need to manually create the ftp user.

Having done this we can now add users to the virtual users database using the commands below:

# pure-pw useradd ftp_test -u ftp -g ftp -d /home/ftp/ftp_test
# pure-pw mkdb

Replace user with the desired username. With -d flag, the user will be chrooted. If you want to give user access to the whole filesystem, use -D instead of -d.

If you want to add additional users, just repeat the commands above with a different user.

Allow clients' IP addresse:
# pure-pw usermod user_name -r 209.17.11.12
# pure-pw mkdb

To reset a user's password:
# pure-pw passwd user_name
# pure-pw mkdb

To view info about one user:
# pure-pw show user_name

To view a list of users:
# pure-pw list

To remove a user:
# pure-pw userdel user_name
# pure-pw mkdb

Now to start Pure-FTPd:

# /usr/local/etc/rc.d/pure-ftpd onestart

Initiate a FTP connection to test the server:

% ftp localhost

    Trying 127.0.0.1...
    Connected to localhost.
    220---------- Welcome to Pure-FTPd [TLS] ----------
    220-You are user number 2 of 50 allowed.
    220-Local time is now 13:39. Server port: 21.
    220-IPv6 connections are also welcome on this server.
    220 You will be disconnected after 15 minutes of inactivity.
    Name (localhost:username):

Now log in with a user account created as explained above. Commands such as ls, cp, pwd and less work just like in tcsh and bash shells. To quit the FTP session type exit.

To configure Pure-FTPd to start at boot time:

# echo 'pureftpd_enable="YES"' >> /etc/rc.conf

To restart Pure-FTPd and determine if it is running:

# /usr/local/etc/rc.d/pure-ftpd restart
# /usr/local/etc/rc.d/pure-ftpd status

# sockstat | grep :21

Rotate pure-ftpd log file:
# vim /etc/newsyslog.conf
/var/log/pureftpd.log 600 7 100000 * JC /var/run/pure-ftpd.pid

# /etc/rc.d/newsyslog restart

Pure-FTPd provides useful features for personal users as well as hosting providers. I've only touched the tip of the iceberg so do take a look at the project's website for the excellent documentation that is available.

PF supports FTP servers and FTP clients behind NAT

WAN0 IP: 11.11.11.11
WAN1 IP: 22.22.22.22

FTP0 server (behind NAT) IP: 192.168.100.1
FTP1 server (behind NAT) IP: 192.168.100.2

# vim /etc/rc.local
### ftp-proxy for outside world FTP clients accessing internal FTP server behand NAT.
/usr/sbin/setfib 0 /usr/sbin/ftp-proxy -R 192.168.100.1 -p 21 -b 11.11.11.11
/usr/sbin/setfib 1 /usr/sbin/ftp-proxy -R 192.168.100.2 -p 21 -b 22.22.22.22

### Note:
### -b // bind to the external IP.
### -R // redirect to internal FTP server IP.

### ftp-proxy for internal FTP clients behand NAT accessing outside world FTP servers.
/usr/sbin/ftp-proxy -p 8021 -b 127.0.0.1

# vim /etc/pf.conf
### [FTP] RDR DMZ to Outside (through ftp-proxy)
### Note: make sure you have this line "ftpproxy_enable="YES"" in your /etc/rc.conf.
nat-anchor "ftp-proxy/*"
rdr-anchor "ftp-proxy/*"
rdr pass proto tcp from $dmz_if/24 to any port 21 -> 127.0.0.1 port 8021

### enabloe log for debugging purpose.
block log all

### [FTP] from ROUTER to ANY
pass out quick log on $wan_if0 proto tcp from $wan_if0 to any port 21 rtable 0
pass out quick log on $wan_if0 proto tcp from $wan_if0 to any port > 1023 rtable 0

pass out quick log on $wan_if1 proto tcp from $wan_if1 to any port 21 rtable 1
pass out quick log on $wan_if1 proto tcp from $wan_if1 to any port > 1023 rtable 1

### [FTP] from ANY to $dmz_ftp0_ip0 $dmz_ftp1_ip0
pass in quick log on $wan_if0 proto tcp from any to $wan_if0 port 21 rtable 0
pass in quick log on $wan_if0 proto tcp from any to $dmz_ftp0_ip0 port 30000:30900 rtable 0
pass in quick log on $wan_if1 proto tcp from any to $wan_if1 port 21 rtable 1
pass in quick log on $wan_if1 proto tcp from any to $dmz_ftp1_ip0 port 30000:30900 rtable 1

pass out quick log on $dmz_if proto tcp from $dmz_if to $dmz_ftp0_ip0 port 21 rtable 0
pass out quick log on $dmz_if proto tcp from $dmz_if to $dmz_ftp0_ip0 port 30000:30900 rtable 0
pass out quick log on $dmz_if proto tcp from $dmz_if to $dmz_ftp1_ip0 port 21 rtable 1
pass out quick log on $dmz_if proto tcp from $dmz_if to $dmz_ftp1_ip0 port 30000:30900 rtable 1

### [FTP] We need to have an anchor for ftp-proxy.
anchor "ftp-proxy/*"

### [FTP] from DMZ to ANY
pass in quick log on $dmz_if proto tcp from $dmz_if/24 to any port 21 rtable 0

Flush the pf.conf setting:
# pfctl -f /etc/pf.conf

Try to connect to ftp:
# ftp ftp.freebsd.org
Name: anonymous
Password: anonymous

Reference:
Setup a Pure-FTPd FTP server with virtual users
http://gala4th.blogspot.com/2012/01/setup-pure-ftpd-ftp-server-with-virtual.html

http://gala4th.blogspot.com/2012/01/pf-supports-ftp-servers-and-ftp-clients.html
http://forums.freebsd.org/showthread.php?t=591

Resolutions for 2012

[] C

[] gcc, gdb, automake, autoconf

[] LLVM, clang, LLDB, OpenCL, KLEE

[] Git

[] Python

[] PostgreSQL

[] Dtrace

[] ZFS

[] RAID

[] English

[] NFS

remove old svn revisons history

Say you have 1000 revisions and you want to shrink to only have the revisions from r950-r1000. You can do the following:

svnadmin dump /path/to/current/repo -r950:1000 > small_svn.dump
svnadmin create /path/to/new/repo
svnadmin load /path/to/new/repo < small_svn.dump Reference:
http://stackoverflow.com/questions/4350145/how-to-remove-old-svn-revisons

disable PHP script execution in upload attachment directory

disable PHP script execution in upload attachment directory

// Apache

<Directory /website/attachments>
  php_flag engine off 
</Directory>

// Nginx

location /sites/default/files/ { 
  location ~ .*\.(php)?$
  { 
    deny all; 
  }
}