Monitor systems using Munin Part II

Monitor systems using Munin Part II
by GKONTOS on FEBRUARY 20, 2011

We saw earlier how easy it is to set up munin in FreeBSD. Now, what if a system that you wish to monitor is located somewhere beyond your firewall(s) perimeter ? You could install munin-node and let the whole world to grab your system statistics or become a victim of a future exploit! You could on the other hand tunnel the traffic via ssh.

Openssh has the ability to create a tunnel to encapsulate another protocol in an encrypted session. Which means that you can pretty much pass any traffic you want, even bypass firewall restrictions. Lets try it out. First you have to set up munin-node on the target host.

#cd/usr/ports/sysutils/munin-node/ && make install clean
Edit /usr/local/etc/munin/munin-node.conf and change the bind address to host 127.0.0.1. Start munin-node

#echo 'munin_node="YES"' > /etc/rc.conf && /usr/local/etc/rc.d/munin-node start
Make sure that the daemon is listening

host# telnet localhost 4949
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
# munin node at host.my.domain
Now lets try to set up the tunnel. From the munin master initiate a ssh tunnel

ssh -2 -N -L 5000:host.my.domain:4949 user@host.my.domain
What we just did ? We created a tunnel to host.my.domain as user and from tcp port 5000 to tcp port 4949. Try connecting to the localhost from munin-master.

host# telnet localhost 5000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
# munin node at host.my.domain
Beautiful! Now lets add this node to the monitor servers. Edit /usr/local/etc/munin/munin.conf in the munin-master, and add the new host.

[this host]
address 127.0.0.1
use_node_name yes
[host.my.domain]
address 127.0.0.1
port 5000
use_node_name yes
There a couple of things though that need improvement. First, you have to type a password and second what happens if the ssh session is terminated.

Setting up ssh with key exchange
Setting up ssh authentication with key exchange is not only easier, it is also more secure. Log on to the master-node with the account you wish to create the key and issue the following command:

gkontos@hp>ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/gkontos/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/gkontos/.ssh/id_rsa.
Your public key has been saved in /home/gkontos/.ssh/id_rsa.pub.
The key fingerprint is:
ed:3e:d7:62:48:7e:2b:f1:d5:94:e3:13:ee:7a:fa:aa gkontos@mydomain.loc
The key's randomart image is:
+--[ RSA 2048]----+
Good, now you should 2 files, one is id_rsa.pub and it contain your public key, the other one is id_rsa and it has the private key to encrypt data to the remote server.
To enable auto ssh login without being prompt for a password, create the ./~ssh/authorized_keys2 on the munin-node and copy the public key into it. Try logging again

ssh -2 -N -L 5000:host.my.domain:4949 user@host.my.domain
You should now be able to log in without typing any password.

Maintain the ssh tunnel
Autossh is a nice program that monitors and restart ssh sessions and tunnels. It is also very easy to install.

#cd /usr/ports/security/autossh && make install clean
Now lets try to connect with auto ssh

autossh -2 -fN -M 20000 -L 5000:localhost:4949 user@host.my.domain
That should do it. If you check you will see that the tunnel is up. Autossh will monitor the connection and will attempt to connect again if it is lost. So, unless you reboot the machine your tunnel will be up for ever.
I was thinking of writing a startup script because on occasions I do reboot my machines. But because I m too lazy I use a crontab entry like this

gkontos@hp>crontab -l
@reboot /usr/local/bin/autossh -2 -fN -M 20000 -L 5000:localhost:4949 gkontos@my.server
So, each time my machine reboots a tunnel is started automatically!

Reference:
http://www.aisecure.net/?p=60