Friday, May 31, 2013

What you should know about Volume Shadow Copy/System Restore in Windows 7 & Vista (FAQ)

What you should know about Volume Shadow Copy/System Restore in Windows 7 & Vista (FAQ)

What is volume shadow copy?

Volume Shadow Copy is a service that creates and maintains snapshots (“shadow copies”) of disk volumes in Windows 7 and Vista. It is the back-end of the System Restore feature, which enables you to restore your system files to a previous state in case of a system failure (e.g. after a failed driver or software installation).

Does volume shadow copy protect only my system files?

No. Volume Shadow Copy maintains snapshots of entire volumes. By default, it is turned on for your system volume (C:) and protects all the data on that volume, including all the system files, program files, user settings, documents, etc.

How is this different from what’s in Windows XP?

In Windows XP, System Restore does not use the Volume Shadow Copy service. Instead, it uses a much simpler mechanism: the moment a program attempts to overwrite a system file, Windows XP makes a copy of it and saves it in a separate folder. In Windows XP, System Restore does not affect your documents – it only protects files with certain extensions (such as DLL or EXE), the registry, and a few other things (details). It specifically excludes all files in the user profile and the My Documents folder (regardless of file extension).

When are the shadow copies created?

Volume shadow copies (restore points) are created before the installation of device drivers, system components (e.g. DirectX), Windows updates, and some applications.
In addition, Windows automatically creates restore points at hard-to-predict intervals. The first thing to understand here is that the System Restore task on Vista and 7 will only execute if your computer is idle for at least 10 minutes and is running on AC power. Since the definition of “idle” is “0% CPU usage and 0% disk input for 90% of the last 15 minutes, plus no keyboard/mouse activity” (source), it could take days for your machine to be idle, especially if you have a lot of programs running in the background.
As you see, the frequency with which automatic restore points are created is hard to estimate, but if you use your machine every day on AC power and nothing prevents it from entering an idle state, you can expect automatic restore points to be created every 1-2 days on Windows Vista and every 7-8 days on Windows 7. Of course, the actual frequency will be higher if you count in the restore points created manually by you and those created before software installations.
Here’s a more precise description: By default, the System Restore task is scheduled to run every time you start your computer and every day at midnight, as long as your computer is idle and on AC power. The task will wait for the right conditions for up to 23 hours. These rules are specified in Scheduled Tasks and can be changed by the user. If the task is executed successfully, Windows will create a restore point, but only if enough time has passed since the last restore point (automatic or not) was created. On Windows Vista the minimum interval is 24 hours; on Windows 7 it is 7 days. As far as I know, this interval cannot be changed.

What cool things can I do with Volume Shadow Copy?

  • If your system malfunctions after installing a new video card driver or firewall software, you can launch System Restore and roll back to a working system state from before the installation. If you can’t get your system to boot, you can also do this from the Windows Setup DVD. This process is reversible, i.e. your current state will be automatically saved as a restore point, to which you can later go back. (Note: System Restore will not roll back your documents and settings, just the system files.)
  • previous_versionsIf you accidentally delete 10 pages of your dissertation, you can right-click the document, choose Restore previous versions, and access a previous version of it. You can open it (in read-only mode) or copy it to a new location.
  • If you accidentally delete a file or folder, you can right-click the containing folder, choose Restore previous versions, and open the folder as it appeared at the time a shadow copy was made (see screenshot below). All the files and folders that you deleted will be there!

previous_folder

Note: While the Volume Shadow Copy service and System Restore are included in all versions of Windows Vista, the Previous versions user interface is only available in Vista Business, Enterprise and Ultimate. On other Vista versions, the previous versions of your files are still there; you just cannot access them easily. The Previous versions UI is available in all versions of Windows 7. It is not available in any version of Windows 8.

Is Volume Shadow Copy a replacement for versioning?

No. A versioning system lets you access all versions of a document; every time you save a document, a new version is created. Volume Shadow Copy only allows you to go back to the moment when a restore point was made, which could be several days ago. So if you do screw up your dissertation, you might have to roll back to a very old version.

Is Volume Shadow Copy a replacement for backups?

No, for the following reasons:
  • Shadow copies are not true snapshots. When you create a restore point, you’re not making a new copy of the drive in question — you’re just telling Windows: start tracking the changes to this drive; if something changes, back up the original version so I can go back to it. Unchanged data will not be backed up. If the data on your drive gets changed (corrupted) for some low-level reason like a hardware error, VSC will not know that these changes happened and will not back up your data. (see below for a more detailed description of how VSC works)
  • The shadow copies are stored on the same volume as the original data, so when that volume dies, you lose everything.
  • With the default settings, there is no guarantee that shadow copies will be created regularly. In particular, Windows 7 will only create an automatic restore point if the most recent restore point is more than 7 days old. On Windows Vista, the minimum interval is 24 hours, but remember that the System Restore task will only run if your computer is on AC power and idle for at least 10 minutes, so it could take days before the conditions are right, especially if you run a lot of background processes or do not use your computer frequently.
  • There is no guarantee that a suitable shadow copy will be there when you need it. Windows deletes old shadow copies without a warning as soon as it runs out of shadow storage. With a lot of disk activity, it may even run out of space for a single shadow copy. In that case, you will wind up with no shadow copies at all; and again, there will be no message to warn you about it.

How much disk space do Volume Shadow Copies take up?

By default, the maximum amount of storage available for shadow copies is 5% (on Windows 7) or 15% (on Vista), though only some of this space may be actually allocated at a given moment.
You can change the maximum amount of space available for shadow copies in Control Panel | System | System protection | Configure.

How efficient is Volume Shadow Copy?

It’s quite efficient. The 5% of disk space that it gets by default is usually enough to store several snapshots of the disk in question. How is this possible?
The first thing to understand is that volume shadow copies are not true snapshots. When a restore point is created, Volume Shadow Copy does not create a full image of the volume. If it did, it would be impossible to store several shadow copies of a volume using only 5% of that volume’s capacity.
Here’s what really happens when a restore point is created: VSC starts tracking the changes made to all the blocks on the volume. Whenever anyone writes data to a block, VSC makes a copy of that block and saves it on a hidden volume. So blocks are “backed up” only when they are about to get overwritten. The benefit of this approach is that no backup space is wasted on blocks that haven’t changed at all since the last restore point was created.
Notice that VSC operates on the block level, that is below the file system level. It sees the disk as a long series of blocks. (Still, it has some awareness of files, as you can tell it to exclude certain files and folders.)
The second important fact is that shadow copies are incremental. Suppose it’s Wednesday and your system has two shadow copies, created on Monday and Tuesday. Now, when you overwrite a block, a backup copy of the block is saved in the Tuesday shadow copy, but not in the Monday shadow copy. The Monday copy only contains the differences between Monday and Tuesday. More recent changes are only tracked in the Tuesday copy.
In other words, if we were to roll back an entire volume to Monday, we would take the volume as it is now, “undo” the changes made since Tuesday (using the blocks saved in the Tuesday shadow copy), and finally “undo” the changes made between Monday and Tuesday. So the oldest shadow copy is dependent on all the more recent shadow copies.

When I delete a 700 MB file, does VSC add 700 MB of data to the shadow copy?

No. When you delete a file, all that Windows does is remove the corresponding entry (file name, path, properties) from the Master File Table. The blocks (units of disk space) that contained the file’s contents are marked as unused, but they are not actually deleted. So all the data that was in the file is still there in the same blocks, until the blocks get overwritten (e.g. when you copy another file to the same volume).
Therefore, if you delete a 700 MB movie file, Volume Shadow Copy does not have to back up 700 MB of data. Because it operates on the block level, it does not have to back up anything, as the blocks occupied by the file are unchanged! The only thing it has to back up is the blocks occupied by the Master File Table, which has changed.
If you then start copying other files to the same disk, some of the blocks formerly occupied by the 700 MB file will get overwritten. VSC will make backups of these blocks as they get overwritten.

If VSS is constantly backing up blocks of data that get overwritten, what actually happens when a restore point is created if data is automatically being backed up anyway?

Not much — VSS simply starts backing up the data to a new place, while leaving the “old place” there (at least until it runs out of space). Now you have two places to which you can restore your system, each representing a different point in time. When you create a restore point, you’re simply telling VSS: “I want to be able to go back to this point in time”.
Note that it’s a mistake to think that VSS is backing up every change you make! It only backs up enough to enable you to go to a specific point in time. Here’s an example scenario to clear things up:
  1. You create a file (version #1)
  2. You create a restore point
  3. You change the file (resulting in version #2) — VSS backs up version #1
  4. A week later, you change the file again (resulting in version #3) — VSS doesn’t back anything up, because it already has version #1 backed up. As a result, you can no longer go back to version #2. You can only go back to version #1 — the one that existed when the restore point was created.
(Note that actually VSS doesn’t operate on files but on blocks, but the principle is the same.)

What are the security implications of Volume Shadow Copy?

Suppose you decide to protect one of your documents from prying eyes. First, you create an encrypted copy using an encryption application. Then, you “wipe” (or “secure-delete”) the original document, which consists of overwriting it several times and deleting it. (This is necessary, because if you just deleted the document without overwriting it, all the data that was in the file would physically remain on the disk until it got overwritten by other data. See question above for an explanation of how file deletion works.)
Ordinarily, this would render the original, unencrypted document irretrievable. However, if the original file was stored on a volume protected by the Volume Shadow Copy service and it was there when a restore point was created, the original file will be retrievable using Previous versions. All you need to do is right-click the containing folder, click Restore previous versions, open a snapshot, and, lo and behold, you’ll see the original file that you tried so hard to delete!
The reason wiping the file doesn’t help, of course, is that before the file’s blocks get overwritten, VSC will save them to the shadow copy. It doesn’t matter how many times you overwrite the file, the shadow copy will still be there, safely stored on a hidden volume.

Is there a way to securely delete a file on a volume protected by VSC?

No. Shadow copies are read-only, so there is no way to delete a file from all the shadow copies.
A partial solution is to delete all the shadow copies (by choosing Control Panel | System | System protection | Configure | Delete) before you wipe the file. This prevents VSC from making a copy of the file right before you overwrite it. However, it is quite possible that one of the shadow copies you just deleted already contained a copy of the file (for example, because it had recently been modified). Since deleting the shadow copies does not wipe the disk space that was occupied by them, the contents of the shadowed file will still be there on the disk.
So, if you really wanted to be secure, you would also have to wipe the blocks that used to contain the shadow copies. This would be very hard to do, as there is no direct access to that area of the disk.
Some other solutions to consider:
  • You could make sure you never save any sensitive data on a volume that’s protected by VSC. Of course, you would need a separate VSC-free volume for such data.
  • system_protectionYou could disable VSC altogether. (After disabling VSC, you may want to wipe the free space on your drive to overwrite the blocks previously occupied by VSC, which could contain shadow copies of your sensitive data.) However, if you disable VSC, you also lose System Restore functionality.  Curiously, Windows offers no option to enable VSC only for system files. If you want to protect your system, you also have to enable Previous versions (see screenshot to the right).
  • The most secure approach is to use an encrypted system volume. That way, no matter what temporary files, shadow copies, etc. Windows creates, it will all be encrypted.
Notice that VSC only VSC only lets you recover files that existed when a restore point was created. So if the sequence of events is as follows:
create file → create restore point → make encrypted copy → overwrite original file
the original file will be recoverable. But if the sequence is:
create restore point → create file → make encrypted copy → overwrite original file
you are safe. If you make sure to encrypt and wipe files as soon as you create them, so that no restore point gets created after they are saved on disk in unencrypted form, there will be no way to recover them with VSC. However, it is not easy to control when Windows creates a restore point; for example, it can do it at any time, just because your computer happens to be idle.

Can I prevent VSC from keeping snapshots of certain files and folders?

Yes, but you have to edit the registry to do that. Here are detailed instructions from MSDN.

What happens when VSC runs out of space?

Most of the time, most of the data on your disk stays unchanged. However, suppose you uninstall a 5 GB game and then install another 5 GB game in its place. This means that 5 GB worth of blocks got overwritten and had to be backed up by VSC.
In such “high-churn” scenarios, VSC can run out of space pretty quickly. What happens then? VSC deletes as many previous shadow copies as necessary, starting from the oldest, until it has enough space for the latest copy. In the rare event that there isn’t enough space even for the one most recent copy, all the shadow copies will be deleted. There are no partial copies.
Thanks to Adi Oltean, who was one of the engineers of Volume Shadow Copy at Microsoft, for answering my questions on the subject.

Reference:
http://blog.szynalski.com/2009/11/23/volume-shadow-copy-system-restore/

Tuesday, May 28, 2013

Network traffic & bandwidth monitoring with darkstat on FreeBSD

Darstat captures network traffic, calculates statistics about usage, and serves reports over HTTP.

Darstat provides the following features:

  • Traffic graphs, reports per host, shows ports for each host.
  • Embedded web-server with deflate compression.
  • Asynchronous reverse DNS resolution using a child process.
  • Small. Portable. Single-threaded. Efficient. Uncomplicated.
Install darkstat:
# cd /usr/ports/net-mgmt/darkstat
# make config-recursive
# make install clean
# rehash

To enable darkstat at boot time add the following lines to /etc/rc.conf:
# vi /etc/rc.conf
darkstat_enable="YES"
darkstat_interface="em0"

Note: change the darkstat_interface to reflect your network interface (mine is em0).

Bellow you'll find a set of optional configurations flags that can be added to /etc/rc.conf:

darkstat_dir="/var/run/darkstat"
darkstat_pidname="darkstat.pid"
darkstat_dropuser="nobody"
darkstat_flags=""

Start darkstat:
# /usr/local/etc/rc.d/darkstat start

To check darkstat daemon status:
# /usr/local/etc/rc.d/darkstat status

By default darkstat serves graphs to http://localhost:667, so fire up your browser and point to the location. If you are planning (like me) on accessing to the graphs for another location add port 667 to your router Port Forwarding settings.

Hope you enjoy darkstat. I find it extremely useful and refreshing.

Reference:
http://linux-bsd-sharing.blogspot.ca/2008/10/howto-network-traffic-bandwidth.html

Monitor network traffic with vnStat on FreeBSD

vnStat is a console-based network traffic monitor. It keeps a log of hourly, daily and monthly network traffic for the selected interface(s).

In short, vnStat is a console-based network traffic monitor that uses the network interface statistics provided by the kernel as information source. This means that vnStat won't actually be sniffing any traffic and also ensures light use of system resources. Optional image output is available starting from version 1.7.

Install vnStat:
# cd /usr/ports/net/vnstat
# make config-recursive
# make install clean
# rehash

Installation Document can be found here:
# less /usr/local/share/doc/vnstat/INSTALL_BSD

Create a directory to store the vnStat database:
# mkdir -p /home/srv/bsd-mon/vnstat

Create the configuration file:
# cp /usr/local/etc/vnstat.conf.sample /usr/local/etc/vnstat.conf

Edit the configuration file:
# vi /usr/local/etc/vnstat.conf
### default interface
Interface "em0"

### location of the database directory
DatabaseDir "/home/srv/bsd-mon/vnstat"

Find out the active network interface:
# ifconfig -a
em0: flags=8843 metric 0 mtu 1500

Specify an interface and update its database for the first time:
# vnstat -u -i em0

Note: it is normal to see following message:

Error: Unable to read database "/home/srv/bsd-mon/vnstat/em0".
Info: -> A new database has been created.

Verify the vnStat database has been created:
# ls /home/srv/bsd-mon/vnstat
em0

Add a new crontab job:
# crontab -e
### run vnstat update every 5 minutes if installed
*/5 * * * * root if [ -x /usr/local/bin/vnstat ] && [ `ls -l /home/srv/bsd-mon/vnstat/ | wc -l` -ge 1 ]; then /usr/local/bin/vnstat -u; fi

Note: the default database dir is /var/db/vnstat/. Make sure you do change it.
Note: crontab example can be found at /usr/local/share/doc/vnstat/vnstat-cron.

Make sure vnstat is running properly every 5 minutes by looking at the modified time:
# ls -l /home/srv/bsd-mon/vnstat

To view network traffic:
# vnstat

To view daily traffic:
# vnstat -d

To view monthly traffic:
# vnstat -m

To display traffic for the last 24 hours:
# vnstat -h

To display all time top10 traffic days:
# vnstat -t

To display traffic for 7 days, current and previous week
# vnstat -w

To display current transfer rate for the selected interface in real time until interrupted. Statistics will be shown after interruption if runtime was more than 10 seconds:
# vnstat -l

To display current transfer rate for the selected interface in real time until interrupted:
# vnstat -l -i em0

Reference:
http://www.cyberciti.biz/faq/freebsd-install-vnstat-network-traffic-monitor-software/

Monday, May 27, 2013

Install and Enable SNMP Service in Windows XP, 7 and 2008

SNMP (Simple Network Management Protocol) is an internet protocol used in network management systems to monitor network-attached devices such as computers, servers, routers, switches, gateways, wireless access points, VoIP phones, and etc. for conditions that warrant administrative attention. SNMP provides management data in the form of variables on the managed systems, which describe the system configuration parameter or current status value.

How to Install and Enable the SNMP Service

1. In Windows XP and Windows 2008, click Start button, then go to Control Panel and run Add or Remove Programs applet. On Add or Remove Programs dialog, click Add/Remove Windows Components to open Windows Components wizard.In Windows Vista, click Start button, then go to Control Panel. Click on Programs link and then click on Turn Windows features on or off. If you’re prompted with User Account Control dialog, click “Continue”.

2. In Components of Windows XP and 2008, click on the Management and Monitoring Tools (make sure that you do not select or clear, tick or untick its check box to change the existing selection), and then click Details.In Windows Features of 7, locate SNMP feature.

3. Select and tick the check box of Simple Network Management Protocolor SNMP feature.

4. Click OK. Also click Next if you’re in Windows XP or 2008. SNMP service will be installed on the system. You may require to insert the Windows setup CD/DVD disc into optical drive.
  • SNMP Service which is the main engine with agents that monitor the activity in the network devices and report the information to the monitoring console workstation.
  • SNMP Trap Service which receives trap messages generated by local or remote SNMP agents and forwards the messages to SNMP management programs running on this computer.
5. SNMP will start automatically after installation. But it’s recommended to verify the service status from Services in Control Panel, and if it’s stopped, you can start the SNMP service from there.Two new services will be created:

Windows doesn’t assign any community string to the SNMP service by default, and also only allow access from localhost or local devices. Further configuration is needed to add in desired community string, which act as the password to grant reply to any SNMP request from remote system.

How to Configure SNMP Service (Add “public” community string)

1. Click on Start button, then go to Control Panel.
2. In Windows Vista, click on System and Maintenance link.
3. Open Adminstrative Tools.
4. Run Services applet.
5. Locate and right click on SNMP Service, then select Properties.
6. In SNMP Service Properties window, click on Traps tab.
7. In the “Community name” text box, enter public or any other case-sensitive SNMP community name to which this computer will send trap messages.
8. Click on Add to list button.

How to Configure Security for SNMP Service for a Community

1. Continue from above steps, click on Security tab. If you already close SNMP Service Properties window, re-open it.
2. Under “Accepted community names” section, click Add button.
3. Select the appropriate permission level for the community string in the “Community Rights” drop down list to specify how the host processes SNMP requests from the selected community. Normally READ ONLY is recommended.
4. In the “Community Name” box, type public or any case-sensitive community name that you want.
5. Click on Add button.

In order for the SNMP service to accept and receive SNMP request packets from any host on the network, including external remote host regardless of identity, click Accept SNMP packets from any host.

To limit the acceptance of SNMP packets, click Accept SNMP packets from these hosts, and then click Add, and then type the appropriate host name, IP or IPX address in the Host name, IP or IPX address box. You can restrict the access to local host or limited servers only by using this setting. Finish off by clicking Add button again.

Click OK when done. Note that you may need to reboot for the settings to take effect.

Reference:
http://www.stoufis.gr/blog/topics/496

HTTPS on Nginx with SSL Certificate

A directory to keep a copy of certificates as a backup:
# cd /root
# mkdir nginx_ssl

Only root have the access:
# chmod 700 nginx_ssl

Create Certificate

Now, you need to understand that one server can hold multiple certificates, but only one per listening IP address. So, if your server is listening on one IP address, you can only have one certificate for the server. Follow me so far? All of your virtual domains can share the same certificate, but clients will get warning prompts when they connect to a secure site where the certificate does not match the domain name. If your server is listening on multiple IP addresses, your virtual hosts have to be IP-based -- not name-based. This is something to consider when creating your certificate. :-)

# cd /root/nginx_ssl
# openssl genrsa -des3 -out server.key 1024

Note: You will now be prompted to enter in a password. Write this down as you will need it later.

We need to make a Certificate Signing Request (CSR):
# openssl req -new -key server.key -out server.csr

Note: Enter your password when it asks for it. Make sure you enter your FQDN for the "Common Name" portion.

Self-signing your Certificate

You could always pay money to Verisign or Thawte for this but it costs $$$. Here is the way to do it:

# openssl x509 -req -days 365 -in /root/nginx_ssl/server.csr -signkey /root/nginx_ssl/server.key -out /root/nginx_ssl/server.crt
# chmod 400 *

Now your cert is good for 365 days. If you want to make it longer, go right ahead and do so :-)

If you would like more information about SSL Certs, go to http://httpd.apache.org/docs-2.0/ssl/ssl_faq.html#aboutcerts

Now we need to copy the certs to the right place:
# mkdir /usr/local/etc/nginx/nginx_ssl
# chmod 500 /usr/local/etc/nginx/nginx_ssl
# ls -ld /usr/local/etc/nginx/nginx_ssl
# cp /root/nginx_ssl/server.key /usr/local/etc/nginx/nginx_ssl
# cp /root/nginx_ssl/server.crt /usr/local/etc/nginx/nginx_ssl
# chmod 400 /usr/local/etc/nginx/nginx_ssl/server.key
# chmod 400 /usr/local/etc/nginx/nginx_ssl/server.crt

Note: you don't need to copy the server.csr file.

Add these lines to nginx.conf:
# vi /usr/local/etc/nginx/nginx.conf
ssl                  on;
ssl_certificate      nginx_ssl/server.crt;
ssl_certificate_key  nginx_ssl/server.key;

ssl_session_timeout  5m;

ssl_protocols        SSLv2 SSLv3 TLSv1;
ssl_ciphers          HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers   on;

Add this line to the FastCGI block of nginx.conf file if you want to see print_r($_SERVER['HTTPS']) in PHP:
fastcgi_param HTTPS on;

To this block in nginx.conf:
location ~ \.php$ {
                    fastcgi_split_path_info ^(.+\.php)(/.+)$;
                    #NOTE: You should have "cgi.fix_pathinfo = 0" in php.ini
                    include fastcgi_params;
                    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                    fastcgi_param HTTPS on;
                    fastcgi_intercept_errors on;
                    #fastcgi_pass 127.0.0.1:9000;  #pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
                    fastcgi_pass unix:/tmp/php-fpm.sock;
            }

Add following line if you want Nginx to support wildcard domains / hostname:
fastcgi_param SERVER_NAME $http_host;

Reference:
http://freebsdrocks.net/index.php?option=com_content&task=view&id=17&Itemid=25

empty $_SERVER['SERVER_NAME'] php variable. Let Nginx support any wildcard domains hostname

empty $_SERVER['SERVER_NAME'] php variable. Let Nginx support any wildcard domains hostname

# vim /usr/local/etc/nginx/nginx.conf
### make sure server_name variable is commented out.
#server_name localhost;

### add this line
fastcgi_param SERVER_NAME $http_host;

Drupal Base URL HTTPS support

Posted by codycraven on September 21, 2010 at 7:00pm

Base URL will begin with https:// if the current page is being accessed through the HTTPS protocol. This depends on the availability of $_SERVER['HTTPS'] which may not be available if a reverse proxy or load balancer is in use.

To make PHP $_SERVER['HTTPS'] available in Nginx, see:
http://blog.ijun.org/2013/05/getting-php-https-detection-working-in.html

Reference:
http://api.drupal.org/api/drupal/developer!globals.php/global/base_url/6

Getting PHP HTTPS-Detection Working in Nginx

when I print_r($_SERVER) variable, I did not see the $_SERVER['HTTPS'] key. You will need to tell Nginx to set the HTTPS parameter when it hands the request off to the FastCGI wrapper.

Add this line:
fastcgi_param HTTPS on;

To this block in /usr/local/etc/nginx/nginx.conf:
location ~ \.php$ {
                    fastcgi_split_path_info ^(.+\.php)(/.+)$;
                    #NOTE: You should have "cgi.fix_pathinfo = 0" in php.ini
                    include fastcgi_params;
                    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
                    fastcgi_param HTTPS on;
                    fastcgi_intercept_errors on;
                    #fastcgi_pass 127.0.0.1:9000;  #pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
                    fastcgi_pass unix:/tmp/php-fpm.sock;
            }

Reference:
http://blog.chrismeller.com/getting-php-https-detection-working-in-nginx

SSL certificate providers

VeriSign
Thawte
GlobalSign

Friday, May 17, 2013

to get a list of bash key bindings

# bind -p
or
# bind -P

Thursday, May 16, 2013

things to do after installing Linux

The first thing to do is make sure you have a 256-color terminfo file for your terminal:
# find /lib/terminfo /usr/share/terminfo -name "*256*"

Configure Bash shell
# vi ~/.bashrc
alias ls='ls --color=auto'
alias ll='ls -alF'

export PS1="\e[0;32m\u@\h \w #\e[m "

#######
# Note: on Ubuntu, xterm-256color may be in different place, try this:
# find /lib/terminfo /usr/share/terminfo -name "*256*"
#######
if [ -e /usr/share/terminfo/x/xterm-256color ]; then
        export TERM='xterm-256color'
else
        export TERM='xterm-color'
fi

Reload the bash setting:
# source ~/.bashrc

# type source
source is a shell builtin

Make sure it's using xterm-256color:
# echo $TERM
xterm-256color

# tput colors
256

Bash tab auto-completion completion:
# vi ~/.inputrc
### enable filename auto-completion
set show-all-if-ambiguous on
set show-all-if-unmodified on

Note: http://blog.ijun.org/2011/01/mac-os-bash-tab-auto-completion.html

Install GNU Screen
# yum update
# yum install screen

Configure GNU Sceen
# vi ~/.screenrc
# to configure how screen looks like.
hardstatus alwayslastline " %{= W} %-Lw %{= KY}%n%f %t%{-}%+Lw %=| "
# to avoid ncurses menu messed up when you are trying to run make install command.
#cjkwidth off
# Always start screen with utf8 enabled. (Alternative method is to run screen with -U parameter)
defutf8 on
#utf8 on on

Reference:
outputs a grid of 256 colors for checking your terminal in console

bash - enable single tab hit once auto completion

bash - enable single tab hit once auto completion

If it's the former I have no clue, though ctrl-R kind of does that. If the latter, you can create a file called .inputrc in your home directory with the following lines:

# vi ~/.inputrc
### tab auto-completion
set show-all-if-ambiguous on
set show-all-if-unmodified on

### if you don't want case-sensitivity
set completion-ignore-case on

### bash history completion to complete what's already on the line
"\e[A": history-search-backward
"\e[B": history-search-forward

Note: .inputrc will get read whenever you start a new bash process.

Note: After you modify ~/.inputrc, restart your shell or use Ctrl+X, Ctrl+R to tell it to re-read ~/.inputrc.

Checking your Ubuntu Version

# lsb_release -a

Show the List of Installed Packages on Ubuntu or Debian

Show a list of all the currently installed packages:
# dpkg --get-selections

Find the locations of the files within a package from the list by using the dpkg -L command, such as:
# dpkg -L tmux

Wednesday, May 15, 2013

how to start ssh server service

start: Unknown job: ssh
Before installing SSH server, it's a good idea to update apt to ensure we have the latest packages.

# sudo apt-get update
# sudo apt-get install openssh-server
# rehash

Note: Bash doesn't have the rehash builtin command. Use "hash -r" instead.

Start SSH service:
# sudo /etc/init.d/ssh start

OR

# sudo service ssh start

If you are using the latest version of Ubuntu such as 12.04 LTS or 13.04+, you need to use upstart job based commands:
# sudo stop ssh
# sudo start ssh
# sudo restart ssh
# sudo status ssh

To prevent login password prompt slowly:
// add following line on SSH server
# vi /etc/ssh/sshd_config
UseDNS no

// add following line on SSH client
# vi /etc/ssh/ssh_config
UseDNS no

List all services with the status for each of them:
# initctl list
# initctl help

Note initctl list shell command lists the contents of /etc/init rather than the suggested dbus-send command.

You can list all of the upstart jobs with by querying upstart over dbus:
# dbus-send --print-reply --system --dest=com.ubuntu.Upstart \
/com/ubuntu/Upstart com.ubuntu.Upstart0_6.GetAllJobs

You may have to change 0_6 to reflect the version of upstart you have; this command works on my lucid install.

Reference:
http://askubuntu.com/questions/218/command-to-list-services-that-start-on-startup

ssh login password prompt slowly on Linux

If you're using host names (e.g., "ssh host.example.com") then you must have proper forward and reverse DNS resolution at both ends. There are ways to avoid this:

1) Add "UseDNS no" to /etc/ssh/sshd_config on the server and /etc/ssh/ssh_config on the client; or

2) Set up a DNS server with proper forward and reverse ("in-addr.arpa") records for both the client and server; or

3) Create entries in the hosts file on both the client and server with the appropriate names and addresses; or

4) Use IP addresses rather than names.

Most of the time when you have a long delay in connecting via ssh to a system it is DNS related. This is talked about in the OpenSSH FAQ item 3.3: http://www.openssh.org/faq.html#3.3

By default sshd will do a reverse look-up on the incoming IP of a connection and this can cause delays if your systems IP address (IPv4 or IPv6) is not DNS resolvable.

The usual work around is to add "UseDNS no" to the sshd_config.

Beyond the reverse dns lookup issue mentioned, another setting that some people find helps is "GSSAPIAuthentication no" in /etc/ssh/ssh_config on the client.

Reference:
http://ubuntuforums.org/showthread.php?t=2004298

outputs a grid of 256 colors for checking your terminal in console

outputs a grid of 256 colors for checking your terminal in console

It should look like this:


256colors2.pl from http://www.frexx.de/xterm-256-notes/data/256colors2.pl
#!/usr/bin/perl
# Author: Todd Larason <jtl@molehill.org>
# $XFree86: xc/programs/xterm/vttests/256colors2.pl,v 1.2 2002/03/26 01:46:43 dickey Exp $

# use the resources for colors 0-15 - usually more-or-less a
# reproduction of the standard ANSI colors, but possibly more
# pleasing shades

# colors 16-231 are a 6x6x6 color cube
for ($red = 0; $red < 6; $red++) {
    for ($green = 0; $green < 6; $green++) {
 for ($blue = 0; $blue < 6; $blue++) {
     printf("\x1b]4;%d;rgb:%2.2x/%2.2x/%2.2x\x1b\\",
     16 + ($red * 36) + ($green * 6) + $blue,
     ($red ? ($red * 40 + 55) : 0),
     ($green ? ($green * 40 + 55) : 0),
     ($blue ? ($blue * 40 + 55) : 0));
 }
    }
}

# colors 232-255 are a grayscale ramp, intentionally leaving out
# black and white
for ($gray = 0; $gray < 24; $gray++) {
    $level = ($gray * 10) + 8;
    printf("\x1b]4;%d;rgb:%2.2x/%2.2x/%2.2x\x1b\\",
    232 + $gray, $level, $level, $level);
}


# display the colors

# first the system ones:
print "System colors:\n";
for ($color = 0; $color < 8; $color++) {
    print "\x1b[48;5;${color}m  ";
}
print "\x1b[0m\n";
for ($color = 8; $color < 16; $color++) {
    print "\x1b[48;5;${color}m  ";
}
print "\x1b[0m\n\n";

# now the color cube
print "Color cube, 6x6x6:\n";
for ($green = 0; $green < 6; $green++) {
    for ($red = 0; $red < 6; $red++) {
 for ($blue = 0; $blue < 6; $blue++) {
     $color = 16 + ($red * 36) + ($green * 6) + $blue;
     print "\x1b[48;5;${color}m  ";
 }
 print "\x1b[0m ";
    }
    print "\n";
}


# now the grayscale ramp
print "Grayscale ramp:\n";
for ($color = 232; $color < 256; $color++) {
    print "\x1b[48;5;${color}m  ";
}
print "\x1b[0m\n";

How to Install and Use tmux on Ubuntu 13.04

Before installing tmux, it's a good idea to update apt to ensure we have the latest packages.
# sudo apt-get update

Then install tmux:
# sudo apt-get install tmux

Run tmux:
# tmux

Run existing tmux:
# tmux a -d

tmux.conf configuration file:
As per dpkg -L tmux which shows you what files the package installed, there is no default tmux.conf included in the package. /etc/tmux.conf is just a location that you may use (only makes sense with multiple users using tmux) that will be evaluated before ~/.tmux.conf. You have to create your own .conf file

# dpkg -L
# ls /usr/share/doc/tmux/examples

# vi ~/.tmux.conf

### Note: key meaning
### C- means ctrl-, so C-x is ctrl-x.
### M- means meta (generally left-alt or escape)-, so M-x is left-alt-x.

### Set the prefix to ^A.
### Note: the default prefix is Ctrl-b.
unbind C-b
set -g prefix ^A
bind a send-prefix

### Set the maximum number of lines held in window history.
set -g history-limit 5000

### terminal 256 colors.
### As the tmux manual suggests: for tmux to work correctly, this must be set to "screen" or a derivative of it.
### Note: modify following line and set it to the terminal supported by your system (run cat /etc/termcap | egrep 'screen-256color|xterm-256color' to verify), and uncomment it.
###set -g default-terminal screen-256color
set -g default-terminal xterm-256color

### Instructs tmux to expect UTF-8 sequences to appear in this window.
setw -g utf8 on

### Instruct tmux to treat top-bit-set characters in the status-left and status-right strings as UTF-8;
set -g status-utf8 on

### date format: hostname weekday month day, hour:minute
set -g status-right '#h %a %b %d, %H:%M'

### Set status line background colour.
set -g status-bg black

### Set status line foreground colour.
set -g status-fg yellow

### Set status line background colour for the currently active window.
setw -g window-status-current-bg magenta

### Set status line foreground colour for the currently active window.
setw -g window-status-current-fg white

### Reload tmux.conf configuration file
### Note: alternative way to reload the configuration file:
### Method 1: Run from command line: tmux source-file ~/.tmux.conf
### Method 2: In any tmux sessions: [prefix Ctrl-b] : source-file /usr/local/etc/tmux.conf
### Method 3: bind the source-file command with a key like following line, then you type: ctrl-b-r
bind r source-file /usr/local/etc/tmux.conf

### [START] vi- and vim-like bindings
### split windows like vim
### vim's definition of a horizontal/vertical split is reversed from tmux's
bind s split-window -v
bind v split-window -h

### move around panes with hjkl, as one would in vim after pressing ctrl-w
bind h select-pane -L
bind j select-pane -D
bind k select-pane -U
bind l select-pane -R

### resize panes like vim
### feel free to change the "5" to however many lines you want to resize by, only
### one at a time can be slow
bind < resize-pane -L 5
bind > resize-pane -R 5
bind - resize-pane -D 5
bind + resize-pane -U 5

### bind : to command-prompt like vim
### this is the default in tmux already
bind : command-prompt

### vi-style controls for copy mode
setw -g mode-keys vi
### [END] vi- and vim-like bindings

Reference:
http://askubuntu.com/questions/192401/where-is-the-default-tmux-conf-file-located

how to change the hostname

# sudo vi /etc/hostname

How do I set the root password so I can use su instead of sudo on Ubuntu

Warning: Directly logging in as root is like playing with fire, because one little typo is enough to lose critical data or make your system unbootable. Note that desktop environments will also function incorrectly if you login to them as root.

See these questions for the reasons behind why sudo is preferred and why root-login is disabled by default:


# sudo -i
# passwd

How to find out all processes traffic bandwidth use with (nethogs) top like utility

How to find out all processes traffic bandwidth use with (nethogs) top like utility

  • nethogs
  • iftop
  • nettop
  • ntop
  • ettercap
  • darkstat
  • htop
  • iotop
  • Glances

find a missing library

# yum provides "*/(file)"

Is used to find out what package provides a certain file. This is especially useful when you are compiling some code and get an error like "error: libbluetooth.so.2 not found, exiting."

Example: yum provides "*/libbluetooth.so.2"

Monday, May 13, 2013

lint - statically checking C programs

lint - a tool for statically checking C programs for security vulnerabilities and coding mistakes.

http://en.wikipedia.org/wiki/Lint_%28software%29
http://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis

Friday, May 3, 2013

How are hex sequence translated to assembly without ambiguity

8B EC 56 8B F4 68 00 70 40 00 FF 15 BC 82 40 
 
A senquence like above can be segmented in various ways,each segment can be translated to corresponding assembly instruction, but each binary executable has its only DEFINITE assembly ,what's the mathematical principle that avoids ambiguity?

UPDATE
The answer with most votes doesn't actually answer my question at all.
 ===
I updated the most popular answer to hopefully make it a bit more clear. – Gabe Oct 13 '10 at 16:00
===
A binary doesn't have a definate set, it has a set of definate starting points(for example, you could skip up to four bytes of prefixes and still get almost the same output). Imo, you'd do well looking at the info on sandpile.org and have a look at the ollydbg & bea mini disassem engine – Necrolis Oct 13 '10 at 16:46
===
You don't have to accept the most upvoted answer if you don't like it. – Vilx- Oct 13 '10 at 21:16
===
I'm pretty sure I've heard tales of getting very small demos which rely on running the same bytes as different code depending where you start; certainly anything on the x86 with has an optional lock prefix on its first instruction can be two functions depending which instruction you jump to. – Pete Kirkham Oct 13 '10 at 22:05
 ===
First of all you have to distinguish between RISC and CISC architectures.

In a RISC architecture you usually have instructions of the same size, so ambiguity cannot be presented. Your CPU will fetch for example 4 bytes for every instruction, and since it will have to start from somewhere (your CPU doesn't have just a sequence like the one you presented, it will have a starting point for sure) once that it has the right alignment no problem can occur.

What happens with a CISC instruction set is essentially the same: starting from the entry point of the program it will fetch instructions accordingly to your opcodes. It doesn't need to know how to matematically distinguish ambiguities since it won't happen that it just doesn't know how long is the next instruction or where the last one finished.

So asking how to separate every instruction is like asking how to separate every word in
thepenisonthetable
There's not mathematical proof but you know which letters are correct together and which ones are not meaningful. The previous sentence contains "son" but you know that it is obtained from "is on". You wouldn't be able to say so without having a meaningful phrase, but your CPU only executes meaningful programs so what's the point?

So if the CPU could work on the previous sentence it will find the first senseful instruction "the", then "pen", "is", "on" and the "son" couldn't never be recognized anyway.

EDIT:
To be cleared, in CISC architectures, the only contraint you have to be sure not to have ambiguities is to avoid having an instruction that is a prefix of another. Let's assume a finite alphabet composed by letters a-z instead that hex numbers (just for practical purposes).

If the program counter points to
abbcbcaabdeffabd
 
you can have that abb is a whole instruction. In that case ab wouldn't be a valid instruction, otherwise the CPU couldn't know where to stop, at the same time abbc can't be an instruction too or it may create problems. Keeping it on you can have for example that ca is the next instruction, c couldn't and cbc neither.

You can extend this argumentation to the whole string. You will see that, if the CPU finds itself in a state in which the next byte of the binary points to the FIRST byte of an instruction, and there are no instruction that are prefixes of other instruction, then in the next state the program counter will point to the FIRST byte of the next, correct, instruction.
===
 I would argue that your example is not even 100% parse-able unless you assert that it is a complete sentence in English. It could be 'the pen is on the table,' or 'the penis on the table.' – Dave Oct 13 '10 at 13:51
 ===
Can you elaborate about the opcodes of CISC ? – ollydbg Oct 13 '10 at 15:24
 ===
It's like the example I provided with the sentence. Let's change it to remove the hidden ambiguity of "pen is". Assume "the cat is on the table". As long as every word (or instruction) is not a prefix of another existing word (that is not true for natural language, but it is for CPU instructions) a CPU that knows where to start will always be able to split up instructions with no ambiguity. – Jack Oct 13 '10 at 15:33
===
Knowing your starting point.

In other words, given a specific starting byte of an instruction, it is unambiguous where the instruction ends, thus giving you the starting byte of the next instruction and allowing you to continue. Given an arbitrary block of memory it is impossible to break it up into individual instructions without knowing where the first instruction begins.

From a more mathematical perspective, there is no valid instruction whose bytes are a prefix of another valid instruction. So if ab is valid, then you know that ab cd cannot be valid so ab must be one instruction and cd is the start of the next instruction.
 ===
Nice :-) However, given certain sequences it seems like it would be possible to determine the "starting point" based upon the detection of invalid instructions (or the detection of sequences which seem valid). If the above is true, would you give a particular "name" to such an operation? – user166390 Oct 12 '10 at 16:57
===
As flippant as this is, it's true. There are a number of exploits involving referencing assembly code at the wrong offset to produce unexpected behavior. – Andres Oct 12 '10 at 16:59
===
What if there're these 4 instructions: 8B EC,,, 56 8B F4 68 ,,,8B EC 56 8B, ,,F4 68 ,how does CPU know whether 8B EC 56 8B F4 68 should be interpreted as 8B EC + 56 8B F4 68 , or 8B EC 56 8B + F4 68 ? – justnobody Oct 13 '10 at 8:26
===
@justnobody: There are no such cases. If 8B EC is an instruction, then 8B EC 56 8B is not one. That's what the ISA specification of the architecture defines uniquely. – Bahbar Oct 13 '10 at 15:55
===
The sequence you listed shows exactly 1 number. In binary, it's 100010111110110001010110100010111111010001101000000000000111000001000000000000001111111100010101101111001000001001000000
 
In decimal, it's 726522768938664460674442126658667072. These are all just different ways of writing exactly the same value. A particular architecture's ISA will divide the bits into fields and assign them meaning. Most processors have easy to get manuals that describe the meaning assigned to each of the bits in those fields.
===
But the bits can be divided in various ways,how does computer figures the final segmentation? – ollydbg Oct 12 '10 at 17:03
 ===
@ollydbg, Do you want to know how the hardware physically does it, or how the programmer knows what the hardware will do with their bits? – nmichaels Oct 12 '10 at 17:06
===
Dont confuse linearly trying to disassemble with execution order of code. The binary is decoded in execution order, starting with a known location. Other than intentional hacks for various reasons there is no ambiguity.

Try writing a disassembler for a variable word length instruction set. At the end of the day it has to be done in execution order, and even there you can only disassemble some of the program as some branches can be based on addresses computed at run time. Modern compiler generated code is much better than older hand assembled code. In an old standup arcade game for example there are conditional branches preceeded by an instruction that guarantees that only one of the conditions is met (why was that in there? we will never know) and the data that follows the conditional branch resembles an opcode in such a way that you run into a collision with other opcodes.

Old dos programs trying to defeat disassemblers would have self modifying code, one instruction would modify another instruction one or two instructions ahead, if single stepped that modification would happen, but if run at full speed the instruction was already fetched in the pipeline, and the modified/broken one in memory was not used. pretty neat trick.

Anyway, the answer to your question is do not look at the bytes in linear order, look at them in execution order starting at the addresses defined by the reset and other vectors in the vector table.
 ===
 on fixed length instruction sets you can examine the data linearly from 0 to N, so long as you remember that not all of those bytes are instructions (and you examine them using the correct alignment). – dwelch Oct 12 '10 at 
===
It sounds like the answer to your question is the somewhat flippant "Know your starting point", but maybe you want something a little more verbose. Given your string:

8B EC 56 8B F4 68 00 70 40 00 FF 15 BC 82 40
 
AND a starting point (Let's say the 8B is your starting point) there is only one possible interpretation of the bytes.

So let's say one operation is 8B EC 56 8B (Depending on your operation length, etc), then the NEXT operation is F4 68... In this case, it's impossible for the machine to try to interpret an operation 56 8B F4 68 because no operation ended at just that point.

Now, if your start point was the 56, then you can get that group but cannot get either of the ones mentioned previously.

The layout of your memory is very specific and start points/jump points are exact and unforgiving--they are required as surely as the code itself.
 ===
If I understand your question correctly, you're trying to understand why

8B EC 56 8B F4 68 00 70 40 00 FF 15 BC 82 40

Could be split e.g. as

8BEC 568BF4 68007040 00FF 15BC 8240

Rathern than say,

8B EC568B F4 68007040 00FF 15BC 8240

This is entirely specified by the ISA of your architecture. That document describes exactly how instructions are uniquely constructed from a series of bytes.

For the ISA to be well formed, a single series of bytes can correspond to at most a single series of decoded instructions (might be less, if there are invalid instructions).
To get a bit more concrete, lets take the x86 example: If you want to know what each byte corresponds to, have a look here.

You'll see that, e.g. an instruction starting with 00 is an add (additional parameters are in the next byte, with a specific encoding).

You'll also see that some values are actually prefixes that modify the following instruction (0F - prefix to extend the opcode space, 26, 2E, 36, 3E, 64, 65, 66, 67, F0, F2, F3), and that some of them take different meaning based on the exact following instruction. Those are not opcodes, but they can alter the encoding of the arguments of the opcode, or introduce a completely new opcode space (e.g. SSE uses 0F).

Overall, the x86 encoding is very complex, thanks for disassemblers.
 ===
There might also be some clues elsewhere about what is a valid starting address. There is always a reset vector address, and there are usually interrupt vector addresses, which all must be valid start points for blocks of code. More usefully, if you come across a jump or call instruction elsewhere which references an address in the block you are trying to decode, then that gives you another start address.

I think I see your worry, and as far as I know its correct - if the program counter gets upset by one and that causes invalid instructions or unintended instructions to be executed, the program probably crashes. True, and also if you run into a data block and try to execute that. At least the latter can be avoided by using a Harvard architecture, where code and data are in seperate memory spaces and may be different bit widths.
 ===
If you open a binary in a hex editor, copy a portion of data and paste in a disassembler, you will very probably not copy a complete instruction. Let's use your example.. in a Windows XP 32bits SP3 English, if I assembly 8B EC 56 8B F4 68 00 70 40 00 FF 15 BC 82 40 I'll get:

Hex dump          Command
8BEC              mov ebp,esp
56                push esi
8BF4              mov esi,esp
68 00704000       push 407000
FF15 BC824000     call dword ptr ds:[4082bc]
 
As you can see it assembled completely different then the answer of the other guys below...

Now let's pretend that instead of assembling 8B EC 56 8B F4 68 00 70 40 00 FF 15 BC 82 40
you added C0 opcode at the beginning C0 8B EC 56 8B F4 68 00 70 40 00 FF 15 BC 82 40

Now check below what a single byte did in our instructions:
 
Hex dump             Command
C08B EC568BF4 68     ror byte ptr ds:[ebx+f48b56ec],68
0070 40              add byte ptr ds:[eax+40],dh
00FF                 add bh,bh
15 BC824000          adc eax,4082bc

As you can see it was completely modified!

The processor know where to start and what arguments to the instruction to take by the opcode instruction. In the first example our first opcode was 8B so it knows that it can be followed by another byte. If this byte is EC so the instruction ends here, and it means mov ebp, esp.

In our second example it start with C0 and it can be followed by another byte, meaning another instruction. Then C08B is the instruction, and EC568BF4 68 is the argument.

Imagine that inside the processor has a huge (but nano) circuit, that behave like a chain of ifs (conditions), that depending on the value of the hexcode (or opcode) it knows "where to go" or "how to behave".
 ===
Maybe you find it interesting to think about the other direction: How would you have to design your code to be easy to segment for others? You could require the most significant bit of the byte starting a sequence to be zero, and those in the middle of a sequence to be one, like UTF-8 does it. Then if you start from a random position – assuming you know where the bytes are – it is easy to find the next sequence. Going one step further, how would you code a pure bit stream such that the start of a sequence is easy to find. How many bits were wasted by such a coding?

Since you asked about the maths, I think the relevant topics are “Coding Theory”, “Variable-length codes” or “Prefix codes”.

How do you find a gene in a sequence of base pairs?
 
Reference:
http://stackoverflow.com/questions/3917098/how-are-hex-sequence-translated-to-assembly-without-ambiguity
 
 
 
 
 
 
 
 
 

Configure the network card on CentOS on VirtualBox

Show the name network interface:# ifconfig -a

Modify eth0's configuration file:
# vi /etc/sysconfig/network-scripts/ifcfg-eth0
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.1.212
NETMASK=255.255.255.0

Note: if you want to use DHCP, change BOOTPROTO=dhcp

Define default gateway (router IP) and hostname:
# vi /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=centos64.local
GATEWAY=192.168.1.1

Make sure you have correct DNS server defined:
# vi /etc/resolv.conf
nameserver 8.8.8.8

Restart networking:
# /etc/init.d/network restart

Install tmux on FreeBSD Tutorial Step by step

Install tmux on FreeBSD Tutorial Step by step

Install tmux
# cd /usr/ports/sysutils/tmux/ ; make install clean ; rehash

tmux options
# make showconfig
===> The following configuration options are available for tmux-1.5:
LIBEVENT2=on "Use libevent version 2"
LIBEVENT_STATIC=off "Build with static libevent"
BACKSPACE=on "Build with tty/keys patch"
===> Use 'make config' to modify these settings

Configuration Examples:
# ls /usr/local/share/examples/tmux

By default, tmux loads the system configuration file from /usr/local/etc/tmux.conf, if present, then looks for a user configuration file at ~/.tmux.conf.

Add following lines to your tmux.conf:
# vim /usr/local/etc/tmux.conf
or
# vim ~/.tmux.conf

### Note: key meaning
### C- means ctrl-, so C-x is ctrl-x.
### M- means meta (generally left-alt or escape)-, so M-x is left-alt-x.

### Set the prefix to ^A.
### Note: the default prefix is Ctrl-b.
unbind C-b
set -g prefix ^A
bind a send-prefix

### Set the maximum number of lines held in window history.
set -g history-limit 5000

### terminal 256 colors.
### As the tmux manual suggests: for tmux to work correctly, this must be set to "screen" or a derivative of it.
### Note: modify following line and set it to the terminal supported by your system (run cat /etc/termcap | egrep 'screen-256color|xterm-256color' to verify), and uncomment it.
###set -g default-terminal screen-256color
###set -g default-terminal xterm-256color

### Instructs tmux to expect UTF-8 sequences to appear in this window.
setw -g utf8 on

### Instruct tmux to treat top-bit-set characters in the status-left and status-right strings as UTF-8;
set -g status-utf8 on

### date format: hostname weekday month day, hour:minute
set -g status-right '#H %a %b %d, %H:%M'

### Set status line background colour.
set -g status-bg black

### Set status line foreground colour.
set -g status-fg yellow

### Set status line background colour for the currently active window.
setw -g window-status-current-bg magenta

### Set status line foreground colour for the currently active window.
setw -g window-status-current-fg white

### Reload tmux.conf configuration file
### Note: alternative way to reload the configuration file:
### Method 1: Run from command line: tmux source-file ~/.tmux.conf
### Method 2: In any tmux sessions: [prefix Ctrl-b] : source-file /usr/local/etc/tmux.conf
### Method 3: bind the source-file command with a key like following line, then you type: ctrl-b-r
bind r source-file /usr/local/etc/tmux.conf

### [START] vi- and vim-like bindings
### split windows like vim
### vim's definition of a horizontal/vertical split is reversed from tmux's
bind s split-window -v
bind v split-window -h

### move around panes with hjkl, as one would in vim after pressing ctrl-w
bind h select-pane -L
bind j select-pane -D
bind k select-pane -U
bind l select-pane -R

### resize panes like vim
### feel free to change the "5" to however many lines you want to resize by, only
### one at a time can be slow
bind < resize-pane -L 5
bind > resize-pane -R 5
bind - resize-pane -D 5
bind + resize-pane -U 5

### bind : to command-prompt like vim
### this is the default in tmux already
bind : command-prompt

### vi-style controls for copy mode
setw -g mode-keys vi
### [END] vi- and vim-like bindings

Add following line to your ~/.cshrc:
# vim ~/.cshrc
### Note: modify following line and set it to the terminal supported by your system (run cat /etc/termcap | egrep 'screen-256color|xterm-256color' to verify), and uncomment it.
#setenv TERM screen-256color
#setenv TERM xterm-256color

Add following lines to your ~/.vimrc:
# vim ~/.vimrc
" To enable 256 colors in vim, put this your .vimrc before setting the colorscheme.
set t_Co=256

Use following script to test verify whether your terminal supports 256 colors:

a small perl script that outputs a grid of 256 colors in your console.

It should look like this:

To see a list of TERM values supported by the system:
# cat /etc/termcap

# cat /etc/termcap | egrep 'screen-256color|xterm-256color'

Determine the terminal capability interface:
# tput Co
8

# tput colors
115

# cap_mkdb

# setenv | grep TERM

# echo $TERM

# tmux

# tmux ls

# tmux a -d

# tmux attach -d -t 1

Move window, reorder window

The 'swap-window' command is closest to what you want. "Prefix :" (that is "Ctrl-B :" by default) brings you to the tmux-command prompt. There you enter 'swap-window -s 3 -t 1' to let window number 3 and window number 1 swap their positions.

To move the current window to the top, do 'swap-window -t 0' (if base-index is 0, as it is by default).

You can bind that command to a key (T for "top" for example) by adding

bind-key T swap-window -t 0

to your ~/.tmux.conf.


Note: if you see following error messages after run the tmux command:
csh: Cannot open /etc/termcap.
csh: using dumb terminal settings.

Make sure you set following line to the terminal supported by your system (run cat /etc/termcap | egrep 'screen-256color|xterm-256color' to verify) in your /usr/local/etc/tmux.conf:
set -g default-terminal screen-256color
or
set -g default-terminal xterm-256color


Other useful tool:
tmuxinator - Manage complex tmux sessions easily

screen and tmux

A comparison of the features (or more-so just a table of notes for accessing some of those features) for GNU screen and BSD-licensed tmux.
This document released for use under the PPL license available at http://code.dayid.org/ppl/ppl.txt
Want more information about tmux and screen? Check out This page also

The formatting here is simple enough to understand (I would hope). ^ means ctrl+, so ^x is ctrl+x. M- means meta (generally left-alt or escape)+, so M-x is left-alt+x
Action tmux screen
start a new session tmux OR
tmux new OR
tmux new-session
screen
re-attach a detached session tmux attach OR
tmux attach-session
screen -r
re-attach an attached session (detaching it from elsewhere) tmux attach -d OR
tmux attach-session -d OR
tmux a -d
screen -dr OR
screen -RD
re-attach an attached session (keeping it attached elsewhere) tmux attach OR
tmux attach-session
screen -x
detach from currently attached session ^b d OR
^b :detach
^a ^d OR
^a :detach
rename-window to newname ^b , <newname> OR
^b :rename-window <newname>
^a A <newname>
list windows ^b w ^a w
list windows in chooseable menu ^a "
go to window # ^b # ^a #
go to last-active window ^b l ^a l
go to next window ^b n ^a n
go to previous window ^b p ^a p
see keybindings ^b ? ^a ?
list sessions ^b s OR
tmux ls OR
tmux list-sessions
screen -ls
toggle visual bell ^a ^g
create another shell ^b c ^a c
exit current shell ^d ^d
split pane horizontally ^b "
split pane vertically ^b %
switch to another pane ^b o
kill the current pane ^b x OR (logout/^D)
close other panes except the current one ^b !
swap location of panes ^b ^o
show time ^b t
show numeric values of panes^b q


Reference:
http://gala4th.blogspot.com/2011/09/install-tmux-on-freebsd-tutorial-step.html
http://www.dayid.org/os/notes/tm.html

x86 Assembly: How do Disassemblers know how to break up instructions?

How does a x86 disassembler know where to break up the instructions?

I am looking at the 8088 instruction set. For example the move instruction has 7 variations that range from 2 to 4 bytes. The instructions themselves seem to follow no particular order. Another reason for Why is x86 ugly?.

For example:
                        76543210  76543210  76543210  76543210
reg/mem to/from reg     100010dw  ||regr/m  
imm to reg/mem          1100011w  ||000r/m  dat       dat w=1
imm to reg              1011wreg  data      dat w=1
imm to accum            1010000w  addr-low  addrhigh
accum to mem            1010001w  addr-low  addrhigh
reg/mem to seg          10001100  ||0ssr/m
seg to reg/mem          10001100  ||0ssr/m

Legend:
||=mod {NO-DISP=0,DISP-LOW,DISP-HIGH,REG}
ss=seg enum{es=0,cs,ss,ds}
reg=enum{ax=0,bx,cd,dx,bx,sp,bp,si,di (if w=1)} enum{al,bl...} (if w=0)
r/m=reg or mem (mod=3 then REG, else mem)

many instruction can overlap in the first byte:
                        76543210  76543210  76543210  76543210
push                    11111111  ||110r/m
inc                     1111111w  ||000r/m

The bitmasks appears to have arbitrary assignment. How does a disassembler break apart the instructions?

This question is a sub set of How to write a disassembler.
 ===
Looking at my 8086/8088 Users Manual Programmers reference (ISBN 1-55512-010-5), likely decades out of print...Appendix A shows the instruction decoding in opcode order 0b00000000 thru 0b11111111. Does not appear to be chaotic at all. Add, sub, and, xor, cmp, etc are all grouped in such a way that a mux can use the opcode bits directly to route the inputs and outputs, and other bits select the operation the alu performs on those bits.

For writing a disassembler you want to use this kind of table or an opcode chart for the top level sorting of instructions.

In your particular example, notice how whenever you see the first opcode as 0xFF there are three bits in the middle of the second byte that tell you the rest of the story as to which instruction is which. All 8 of those combinations (one is undefined) are represented and easily decoded from those 3 bits.
Yes, the x86 instruction set is crazy. Interesting and fun features, but considerably better instruction sets have been invented since. The only reason x86 has not gone the way of the 6502 for example is momentum, not quality.

You should look at this one too:
http://stackoverflow.com/questions/3917098/how-are-hex-sequence-translated-to-assembly-without-ambiguity

How to disassemble this and any other variable word length instruction set is by doing it in execution order. You will fail if you try to do it linearly in address order. Start with the vector table to get the entry addresses then follow those instructions in address order, making a note of and following all branches until you hit an unconditional branch or return or other instruction that terminates that string of instructions. Repeat this for every branch destination. That wont cover all of the instructions possible as the code may compute addresses while executing (not much you can do about disassembling that).

If any of this code was hand written intentionally or accidentally to trip up a disassembler you can expect to have collisions where the second or third byte of one opcode based on one execution path appears to be the first opcode of an instruction based on a different execution path. For example a clear a flag instruction followed by a conditional branch if flag is clear, followed by a byte of data, followed by a real instruction that is a branch destination. Yep, I have come across this. And it should be trapped by your disassembler, you need to put checks in to stop disassembling one or both of those execution paths when they collide. For complete disassembly expect to have to support some sort of user input to exclude addresses as opcodes, as well as for the user to manually add valid opcodes for you to follow the execution path from.

For fixed length instruction sets you can easily disassemble in address or execution order, your choice, address order from 0 to the end of memory is the easiest of course. Dont error out on undefined instructions, just mark them as such and keep going, some of those are data.

x86 is definitely the LAST variable length instruction set I would attempt to disassemble and I have written many disassemblers. No desire to ever attempt that project. Start with some fixed length ones like the pic and arm/thumb. Try the msp430 for variable word length, then maybe the 6502 (asteroids, asteroids deluxe, lunar lander, etc). Maybe a week or two worth of evenings to cover the above and get the feel for it, then attack the x86 if the desire remains. If you limit yourself strictly to the 8088/8086 it is not so bad, need to make sure your tools are generating those instructions and not getting into the 386 on up instructions.

If push vs inc is bothering you, definitely try something else like the msp430 for example first.
===
Thank you very much for the long descriptive answer. Things are starting to make a little more sense. The OP Code map reminds me of EBCDIC mlsite.net/8086/#tbl_oper (8086 Map). It does not appear so random any more. I was trying to envision it as a linear or set bit-mask to flag the instruction type. This site provides disassembler written in Python: mlsite.net/blog/?p=58

Reference: 
http://stackoverflow.com/questions/3983735/x86-assembly-how-do-disassemblers-know-how-to-break-up-instructions?rq=1

How to write a disassembler

I'm interested in writing an x86 dissembler as an educational project.

The only real resource I have found is Spiral Space's, "How to write a disassembler". While this gives a nice high level description of the various components of a disassembler, I'm interested in some more detailed resources. I've also taken a quick look at NASM's source code but this is somewhat of a heavyweight to learn from.

I realize one of the major challenges of this project is the rather large x86 instruction set I'm going to have to handle. I'm also interested in basic structure, basic disassembler links, etc.

Can anyone point me to any detailed resources on writing a x86 disassembler?
 ===
 Not an answer but the answer in stackoverflow.com/questions/82432/… is also a good read for those who are starting. 
===
Start with some small program that has been assembled, and which gives you both the generated code and the instructions. Get yourself a reference with the instruction architecture, and work through some of the generated code with the architecture reference, by hand. You'll find that the instructions have a very stereotypical structure of inst op op op with varying number of operands. All you need to do is translate the hex or octal representation of the code to match the instructions; a little playing around will reveal it.

That process, automated, is the core of a disassembler. Ideally, you're probably going to want to construct a n array of instruction structures internally (or externally, if the program is really large). You can then translate that array into the instructions in assembler format.
===
You need a table of opcodes to load from.

The fundamental lookup datastructure is a trie, however a table will do well enough if you don't care much about speed.

To get the base opcode type, beginswith match on the table.

There are a few stock ways of decoding register arguments; however, there are enough special cases to require implementing most of them individually.
Since this is educational, have a look at ndisasm.
===

I would recommend checking out some open source disassemblers, preferably distorm and especially "disOps (Instructions Sets DataBase)" (ctrl+find it on the page).
The documentation itself is full of juicy information about opcodes and instructions.
Quote from https://code.google.com/p/distorm/wiki/x86_x64_Machine_Code
80x86 Instruction:
A 80x86 instruction is divided to a number of elements:
  1. Instruction prefixes, affects the behaviour of the instruction's operation.
  2. Mandatory prefix used as an opcode byte for SSE instructions.
  3. Opcode bytes, could be one or more bytes (up to 3 whole bytes).
  4. ModR/M byte is optional and sometimes could contain a part of the opcode itself.
  5. SIB byte is optional and represents complex memory indirection forms.
  6. Displacement is optional and it is a value of a varying size of bytes(byte, word, long) and used as an offset.
  7. Immediate is optional and it is used as a general number value built from a varying size of bytes(byte, word, long).
The format looks as follows:
/-------------------------------------------------------------------------------------------------------------------------------------------\
|*Prefixes | *Mandatory Prefix | *REX Prefix | Opcode Bytes | *ModR/M | *SIB | *Displacement (1,2 or 4 bytes) | *Immediate (1,2 or 4 bytes) |
\-------------------------------------------------------------------------------------------------------------------------------------------/
* means the element is optional.
The data structures and decoding phases are explained in https://code.google.com/p/distorm/wiki/diStorm_Internals
Quote:
Decoding Phases
  1. [Prefixes]
  2. [Fetch Opcode]
  3. [Filter Opcode]
  4. [Extract Operand(s)]
  5. [Text Formatting]
  6. [Hex Dump]
  7. [Decoded Instruction]
Each step is explained also.

The original (dead) links are kept for historical reasons:
http://ragestorm.net/distorm/vol1.html and http://ragestorm.net/distorm/vol2.html
===
Take a look at section 17.2 of the 80386 Programmer's Reference Manual. A disassembler is really just a glorified finite-state machine. The steps in disassembly are:
  1. Check if the current byte is an instruction prefix byte (F3, F2, or F0); if so, then you've got a REP/REPE/REPNE/LOCK prefix. Advance to the next byte.
  2. Check to see if the current byte is an address size byte (67). If so, decode addresses in the rest of the instruction in 16-bit mode if currently in 32-bit mode, or decode addresses in 32-bit mode if currently in 16-bit mode
  3. Check to see if the current byte is an operand size byte (66). If so, decode immediate operands in 16-bit mode if currently in 32-bit mode, or decode immediate operands in 32-bit mode if currently in 16-bit mode
  4. Check to see if the current byte is a segment override byte (2E, 36, 3E, 26, 64, or 65). If so, use the corresponding segment register for decoding addresses instead of the default segment register.
  5. The next byte is the opcode. If the opcode is 0F, then it is an extended opcode, and read the next byte as the extended opcode.
  6. Depending on the particular opcode, read in and decode a Mod R/M byte, a Scale Index Base (SIB) byte, a displacement (0, 1, 2, or 4 bytes), and/or an immediate value (0, 1, 2, or 4 bytes). The sizes of these fields depend on the opcode , address size override, and operand size overrides previously decoded.
The opcode tells you the operation being performed. The arguments of the opcode can be decoded form the values of the Mod R/M, SIB, displacement, and immediate value. There are a lot of possibilities and a lot of special cases, due to the complex nature of x86. See the links above for a more thorough explanation.
 
Intel manual says Groups 1 through 4 may be placed in any order relative to each other, so steps 1-4 may be not in that order
===
Checkout objdump sources - it's a great tool, it contains many opcode tables and it's sources can provide a nice base for making your own disassembler.
 
Reference: 
http://stackoverflow.com/questions/924303/how-to-write-a-disassembler?rq=1