Wednesday, July 19, 2017

Install MySQL, Apache, PHP on Ubuntu 16.04

Install MySQL, Apache, PHP on Ubuntu 16.04

Install VMware tools:

VM > Guest > Install/Upgrade VMware Tools

# su -
# df -h
# cd /media/jun/VMware\ Tools/
# ls -la
# tar zxvf VMwareTools-9.4.0-1280544.tar.gz -C /tmp/
# cd /tmp
# ls
# cd vmware-tools-distrib/
# ls
# ./vmware-install.pl -d
# reboot

Note: For more info https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1022525

Move Ubuntu launcher to the bottom:

# gsettings set com.canonical.Unity.Launcher launcher-position Bottom

Update the package repository:

# apt-get update

Install SSH server:

# apt-get install openssh-server
# systemctl status sshd.service
# systemctl restart sshd.service

Compile and install the latest Git 2.13.1:

# apt-get install dh-autoreconf libcurl4-gnutls-dev libexpat1-dev gettext libz-dev libssl-dev
# apt-get install curl
# cd /usr/local/src/
# curl -L https://github.com/git/git/archive/v2.13.1.tar.gz -o git.tar.gz
# tar zxvf git.tar.gz
# cd git-2.13.1/
# make configure
# ./configure --prefix=/usr
# make all
# make install

Compile and install the latest Vim 8:

# apt-get install libncurses5-dev python-dev ruby-dev libperl-dev ruby-dev liblua5.3-dev exuberant-ctags cscope

// Fix liblua paths
# ln -s /usr/include/lua5.3 /usr/include/lua
# ln -s /usr/lib/x86_64-linux-gnu/liblua5.3.so /usr/local/lib/liblua.so

# cd /usr/local/src
# git clone https://github.com/vim/vim.git

# cd vim
# ./configure --prefix=/usr --with-features=huge --enable-multibyte --enable-pythoninterp --enable-rubyinterp --enable-perlinterp --enable-luainterp --enable-cscope
# make
# make install

# hash -r
# vim --version

Install MTA mail server:

# apt-get install postfix

Note: select "Internet site".

Note: If you need to reconfigure the postfix setting, run either one of the following:

# dpkg-reconfigure -plow postfix

or

# apt-get purge postfix

For other mail related packages:

# apt-get install mailutils

Install mail client:

# apt-get install bsd-mailx
# echo "test message" | mailx -s 'test subject' myemail@mydomain.com

Install Git:

# add-apt-repository ppa:git-core/ppa
# apt-get update

# apt-cache policy git
# apt-cache madison git

# apt-get install git=1:2.11.0-2~ppa0~ubuntu16.04.1

# git --version

Install MySQL5.7:

# apt-get install mysql-server

Install Apache2.4:

# apt-get install apache2

Enable the following Apache modules if you are connecting to PHP through a unix socket (/run/php/php7.0-fpm.sock):

# a2enmod proxy proxy_fcgi rewrite

Install and enable the following Apache modules if you are connecting to PHP through a TCP socket (127.0.0.1:9000):

# apt-get install libapache2-mod-fastcgi
# a2enmod fastcgi rewrite

Edit apache2.conf:

# vim /etc/apache2/apache2.conf

AllowOverride All

Set up a virtual host:

# cd /etc/apache2/sites-available
# cp 000-default.conf mag2.local.conf
# vim mag2.local

Check the configuration:

# apache2ctl -V
# apache2ctl -t
# apache2ctl -M
# apache2ctl configtest

Enable the site:

# a2ensite mag2.local

Install PHP7.0:

# apt-get install php7.0-fpm php-xdebug
# apt-get install php7.0-xml php7.0-curl php7.0-zip php7.0-gd php7.0-bcmath php7.0-intl php7.0-mbstring php7.0-mcrypt php7.0-mysql
# apt-get install php7.0-json php7.0-opcache

# vim /etc/php/7.0/fpm/php.ini

date.timezone = America/Vancouver
display_errors = On
display_startup_errors = On
error_reporting = E_ALL
error_log = /var/log/php_errors.log

# a2enconf php7.0-fpm.conf

Start MySQL, PHP, and Apache:

# systemctl restart mysql.service
# systemctl restart php7.0-fpm.service
# systemctl restart apache2.service

# ps auxww | grep -i mysql
# ps auxww | grep -i php-fpm
# ps auxww | grep -i apache2

List all the installed PHP packages:

# dpkg -l | grep php| awk '{print $2}' |tr "\n" " "

Show the available package version:

# apt-cache search php
# apt-cache policy php

Install the specific package version:

# apt-get install php7=7.0+35ubuntu6

Note: You can look up old versions of packages at their site http://www.debian.org/distrib/packages

Install older version of PHP (PHP5.6):

# add-apt-repository ppa:ondrej/php

# apt-get update

# apt-get install php5.6-fpm

# apt-get install php5.6-gd php5.6-intl php5.6-json php5.6-mbstring php5.6-mcrypt php5.6-mysql php5.6-opcache php5.6-xml

# a2disconf php7.0-fpm.conf
# a2enconf php5.6-fpm.conf

# systemctl restart apache2.service

Install PHPStorm:

# cd ~jun/Downloads/
# tar xf PhpStorm-*.tar.gz -C /opt/
# cd /opt/PhpStorm-163.10504.2/
# ./bin/phpstorm.sh

Generate a self-signed SSL certificate:

# openssl req -x509 -nodes -days 365 -newkey rsa:2048 -subj "/C=CA/ST=British Columbia/L=Vancouver/O=My Company Name/CN=erp.local" -keyout /etc/ssl/private/test.local.key -out /etc/ssl/certs/test.local.crt

CentOS and Red Hat 7: Install Linux, Apache MPM, MariaDB, PHP (LAMP) Stack

CentOS and Red Hat 7: Install Linux, Apache MPM, MariaDB, PHP (LAMP) Stack

Quick Start

Install IUS repository, utilities, Git, Go:

# curl 'https://setup.ius.io/' -o setup-ius.sh \
&& bash setup-ius.sh \
&& rm -f setup-ius.sh \
&& yum -y update \
&& yum -y install ntp wget epel-release tmux tree git2u \
&& curl 'https://storage.googleapis.com/golang/go1.8.3.linux-amd64.tar.gz' -o go.tar.gz \
&& tar -zxvf go.tar.gz -C /usr/local \
&& rm -f go.tar.gz \
&& systemctl enable ntpd.service && systemctl restart ntpd.service

Install the environmental configuration:

# cd \
&& git clone https://github.com/junhsieh/config_centos_v2.git \
&& sh config_centos_v2/script/setHomeConfig.sh \
&& source ~/.bashrc \
&& tmux

Note: git remote set-url origin git@github.com:junhsieh/config_centos_v2.git

Install Vim:

# yum -y install gcc make ncurses ncurses-devel \
ruby ruby-devel lua lua-devel luajit \
luajit-devel ctags python python-devel \
python3 python3-devel tcl-devel \
perl perl-devel perl-ExtUtils-ParseXS \
perl-ExtUtils-XSpp perl-ExtUtils-CBuilder \
perl-ExtUtils-Embed \
ctags cscope

# cd /usr/local/src \
&& git clone https://github.com/vim/vim.git \
&& cd vim \
&& ./configure --prefix=/usr --with-features=huge --enable-multibyte --enable-rubyinterp --enable-pythoninterp --enable-perlinterp --enable-luainterp --enable-cscope \
&& make \
&& make install \
&& hash -r \
&& vim --version | grep VIM

Install Docker:

# yum -y install yum-utils \
&& yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo \
&& yum makecache fast \
&& yum -y install docker-ce \
&& systemctl enable docker \
&& systemctl start docker \
&& usermod -aG docker jun \
&& docker version

Note: Log out and log back in so that your Docker group membership is re-evaluated.

# ps auxww | grep -i docker

Enable firewall ports on Docker Manager node:

# firewall-cmd --permanent --zone=public \
--add-port=2377/tcp \
--add-port=7946/tcp \
--add-port=7946/udp \
--add-port=4789/udp \
&& firewall-cmd --reload \
&& systemctl restart docker.service \
&& firewall-cmd --zone=public --list-all

Note: Add TCP port 2376 if you need to use Docker Machine to work. Docker Machine is used to orchestrate Docker hosts.

Enable firewall ports on Docker Worker nodes:

# firewall-cmd --permanent --zone=public \
--add-port=7946/tcp \
--add-port=7946/udp \
--add-port=4789/udp \
&& firewall-cmd --reload \
&& systemctl restart docker.service \
&& firewall-cmd --zone=public --list-all

Note: Add TCP port 2376 if you need to use Docker Machine to work. Docker Machine is used to orchestrate Docker hosts.

Install Apache 2.4:

# yum -y httpd24u httpd24u-mod_ssl

Install PHP 7.1:

# yum -y php71u-common php71u-cli php71u-fpm php71u-fpm-httpd php71u-opcache php71u-xml php71u-json php71u-pdo php71u-mysqlnd php71u-intl php71u-mbstring php71u-mcrypt php71u-gd php71u-process

Install sysdig:

# curl -s https://s3.amazonaws.com/download.draios.com/stable/install-sysdig | sudo bash

// the ncurses user interface for sysdig
# csysdig

// the definitive system and process troubleshooting tool
# sysdig

Check CentOS version:

# cat /etc/redhat-release

CentOS Linux release 7.1.1503 (Core)

Set up Network:

# ip addr

2: eno16777736: <broadcast> mtu 1500 qdisc pfifo_fast state UP qlen 1000

# vi /etc/sysconfig/network-scripts/ifcfg-eno16777736

BOOTPROTO=none

IPADDR=192.168.6.9
PREFIX=24
GATEWAY=192.168.6.1
DNS1=8.8.8.8
DNS2=8.8.4.4

PEERDNS=yes
ONBOOT=yes
USERCTL=no

Note: USERCTL=no // Non-root users are not allowed to control this device.

# systemctl restart network

# ip addr

# /bin/ipcalc --netmask 192.168.6.9/24

NETMASK=255.255.255.0

# cat /etc/resolv.conf

Change hostname:

# hostnamectl status

# hostnamectl set-hostname cent-dev.local

# hostnamectl status

# cat /etc/hostname

If you are using Amazon's AWS EC2 instance, append the following string at the bottom of the file to ensure that the hostname is preserved between restarts/reboots:

# vim /etc/cloud/cloud.cfg

preserve_hostname: true

More info: https://aws.amazon.com/premiumsupport/knowledge-center/linux-static-hostname-rhel7-centos7/

Set up timezone:

# timedatectl
# timedatectl list-timezones
# timedatectl set-timezone America/Vancouver
# timedatectl

Set up date and time:

# date +%Y%m%d -s "20081128"
# date +%T -s "10:13:13"

To sync date and time automatically:

# yum -y update

# yum -y install ntp

# ntpdate 0.us.pool.ntp.org

Note: ntpdate is deprecated as of September 2012

You can change which ntp server to use:

# vi /etc/ntp.conf

Set ntpd to start up on boot time:

# systemctl enable ntpd.service
# systemctl restart ntpd.service

# ps auxww|grep -i ntpd
# ntpstat
# timedatectl

Set the Hardware Clock to the current System Time:

# hwclock --systohc

Note: hwclock is a utility for accessing the hardware clock. Hardware clock is independent of the operation system you use and works even when the machine is shut down. This program is used to find out the time from the hardware clock and set the system time at boot time.

Update the ~/.bashrc configuration:

# vi ~/.bashrc

### alias
alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'

alias gs='git status'
alias gc='git commit -a -m "up"'
alias gp='git push'
alias gpp='git pull'

alias ls='ls --color=auto'
alias ll='ls -la'
alias h='history'

### ls with color (try "ls --color=auto").
#export CLICOLOR=1 # Use colors (if possible)
#export LSCOLORS="ExGxFxdxCxDxDxBxBxExEx"

### set up a clean UTF-8 environment
### run: locale command
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8

### display history command with date and time
export HISTTIMEFORMAT="%m/%d/%y %T "

### Prompt
PS1='\[\e[0;32m\]\u@\h \w \$\[\e[0m\] '

#######
# Note: on Ubuntu, xterm-256color may be in different place, try this:
# find /lib/terminfo /usr/share/terminfo -name "*256*"
# Note: tmux respects screen-256color
#######
if [ -e /usr/share/terminfo/x/xterm-256color ]; then
  export TERM='xterm-256color'
else
  export TERM='xterm-color'
fi

### Make bash check its window size after a process completes
shopt -s checkwinsize

# Source global definitions
if [ -f /etc/bashrc ]; then
  . /etc/bashrc
fi

# source ~/.bashrc

Update the ~/.inputrc configuration:

# vi ~/.inputrc

### enable filename tab auto-completion
set show-all-if-ambiguous on
set show-all-if-unmodified on

### if you don't want case-sensitivity
#set completion-ignore-case on

### bash history completion to complete what's already on the line
### arrow up
"\e[A": history-search-backward
### arrow down
"\e[B": history-search-forward

Make sure there is a swap space on your system:

# cat /proc/meminfo | grep -i swap

SwapCached:           36 kB
SwapTotal:       2097148 kB
SwapFree:        2095160 kB

Note: if your system does not have the swap space, please refer to http://blog.ijun.org/2015/04/add-swap-to-amazon-ec2-instance-ebs.html for more information.

Install EPEL and additional repositories on CentOS and Red Hat:

# yum -y install wget
# yum -y install epel-release

http://blog.ijun.org/2014/11/install-epel-and-additional.html
https://fedoraproject.org/wiki/EPEL

Subscribing to the IUS Community Project Repository:

# curl 'https://setup.ius.io/' -o setup-ius.sh
# bash setup-ius.sh

# ls -l /etc/yum.repos.d/ius*

-rw-r--r--. 1 root root 1150 Apr 16  2015 /etc/yum.repos.d/ius-archive.repo
-rw-r--r--. 1 root root 1131 Apr 16  2015 /etc/yum.repos.d/ius-dev.repo
-rw-r--r--. 1 root root 1073 Apr 16  2015 /etc/yum.repos.d/ius.repo
-rw-r--r--. 1 root root 1150 Apr 16  2015 /etc/yum.repos.d/ius-testing.repo

To find which package provides the ifconfig command:

# yum provides ifconfig

To get the ifconfig command into our system:

# yum -y install net-tools

# ifconfig | awk '/inet /{print $2}'

To install DNS tools:

# yum install bind-utils

Install tmux:
# yum -y install tmux

# vi ~/.tmux.conf

# tmux
or
# tmux a -d

Install vim-enhanced:

It is actually very easy to compile Vim by yourself. Please refer to the following article if you are interested in:

http://blog.ijun.org/2016/07/compile-latest-vim-74-on-centos7.html

# vim --version
-syntax
-python

# yum -y install vim-enhanced

# vim --version
+syntax
+python

# yum list installed | grep -i vim
# yum info vim-enhanced

Install tree:

# yum -y install tree

Install fortune game:

# yum -y install fortune-mod.x86_64

Install Glances:

# yum -y install glances

Install Git:

# yum install git

Or from IUS repo (preferred):

# yum install git2u

Install NetCat:

# yum -y install nmap-ncat
# nc localhost 8080

Install firewalld:

# yum -y install firewalld

(optional)
# yum install firewall-config

Note: If you don't mind using a GUI you could use firewall-config instead. If you need something for the console you will have to use firewall-cmd instead.

# systemctl enable firewalld

# systemctl restart firewalld

Install MariaDB:

# yum install mariadb-server

Setting UTF8 defaults for MySQL:

We recommend against MySQL's utf8 character set, since it does not support 4-byte unicode characters, and strings containing them will be truncated. This is fixed by the newer utf8mb4 character set.

# vim /etc/my.cnf

[mysqld]
# Version 5.5.3 introduced "utf8mb4", which is recommended
collation-server = utf8mb4_general_ci # Replaces utf8_general_ci
character-set-server = utf8mb4 # Replaces utf8

default-storage-engine = InnoDB

max_allowed_packet = 16M

# This option makes InnoDB to store each created table into its own .ibd file.
innodb_file_per_table

# Don't resolve hostnames. All hostnames are IP's or 'localhost'.
skip-name-resolve

#The number of simultaneous clients allowed.
max_connections = 200

# uncomment to disable the InnoDB storage engine
#skip-innodb

innodb_buffer_pool_size = 4G
innodb_additional_mem_pool_size = 16M
innodb_log_file_size = 256M
innodb_log_buffer_size = 8M

# systemctl enable mariadb.service

# systemctl restart mariadb.service

# systemctl is-active mariadb.service

# /usr/bin/mysql_secure_installation

# mysql -u root -p

mysql> GRANT ALL PRIVILEGES ON *.* TO 'test'@'192.168.0.%' IDENTIFIED BY '123456';
mysql> FLUSH PRIVILEGES;

mysql> CREATE DATABASE mydb DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

Set up .my.cnf configuration file:

# touch ~/.my.cnf
# chmod 600 ~/.my.cnf
# vim ~/.my.cnf

[client]
host = localhost
port = 3306
user = root
password = MyPassword

Add New Rule to firewalld to allow access to MySQL:

# firewall-cmd --permanent --zone=trusted --add-source=192.168.0.1/32
# firewall-cmd --permanent --zone=trusted --add-port=3306/tcp
# firewall-cmd --reload
# firewall-cmd --zone=trusted --list-all

or

# systemctl restart firewalld.service

Check out the zone file to inspect the XML configuration:

# cat /etc/firewalld/zones/public.xml
# cat /etc/firewalld/zones/trusted.xml

Install Apache:

# yum install httpd mod_ssl

Or from IUS repo (preferred):

# yum install httpd24u httpd24u-mod_ssl

# systemctl status httpd

● httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled)

Note: it will tell you if the service starts on boot.

# systemctl enable httpd.service

# systemctl restart httpd.service

# systemctl reload httpd.service

# systemctl -l status httpd.service

# journalctl -xn

# systemctl is-active httpd.service

# apachectl configtest

# httpd -V

# apachectl graceful

httpd service default configuration files:

  • Default config file: /etc/httpd/conf/httpd.conf
  • Configuration files which load modules : /etc/httpd/conf.modules.d/ directory (e.g. PHP)
  • Select MPMs (Processing Model) as loadable modules [worker, prefork (default)] and event: /etc/httpd/conf.modules.d/00-mpm.conf
  • Default ports: 80 and 443 (SSL)
  • Default log files: /var/log/httpd/{access_log,error_log}

Set up a symbolic link:

# cd / ; ln -s var/www/html www

Install PHP:

# yum install php php-mysqlnd php-fpm php-gd php-mbstring php-pdo php-xml php-soap
# yum install php-pear php-devel pcre-devel gcc gcc-c++ make

# systemctl restart httpd.service

Note: you need php-devel pcre-devel gcc make for PHP APC. Try to add httpd-devel if failed.

You need the php-mcrypt to run Magento:

# yum install php-mcrypt

http://blog.ijun.org/2014/11/how-to-install-php-mcrypt-on-centos-7.html

Open port 80 firewall access:

# firewall-cmd --zone=public --add-port=http/tcp
# firewall-cmd --zone=public --add-port=https/tcp

# firewall-cmd --permanent --zone=public --add-port=http/tcp
# firewall-cmd --permanent --zone=public --add-port=https/tcp

# firewall-cmd --reload
or
# systemctl restart firewalld.service

# firewall-cmd --zone=public --list-all

Allow a IP address with a specific port:

# firewall-cmd --zone=public --add-rich-rule='rule family="ipv4" source address="1.2.3.4" port protocol="tcp" port="3306" accept'

To remove the rule:

# firewall-cmd --zone=public --remove-rich-rule='rule family="ipv4" source address="1.2.3.4" port protocol="tcp" port="3306" accept'

Block a IP address:

# firewall-cmd --zone=public --add-rich-rule='rule family="ipv4" source address="1.2.3.4" reject'

To remove the rule:

# firewall-cmd --zone=public --remove-rich-rule='rule family="ipv4" source address="1.2.3.4" reject'

If you get 403 forbidden error, then you probably have problem with SELinux, to deal with Security-Enhanced Linux (SELinux):

# namei -l /var/www/html/magento19

# ls -dZ /var/www/html
drwxr-xr-x. root root system_u:object_r:httpd_sys_content_t:s0 /var/www/html

# chcon -R --reference=/var/www/html /var/www/html/magento19
Or
# chcon -R --type=httpd_sys_content_t /var/www/html/magento19
Or for read and write permission:
# chcon -R -t httpd_sys_rw_content_t /var/www/html/magento19/app/etc

# ps auxwwZ | grep httpd
# ls -dZ /var/www/html/magento19/app/etc
# tail /var/log/audit/audit.log
# tail /var/log/messages

To turn off Security-Enhanced Linux (SELinux):

# setenforce 0

To turn on Security-Enhanced Linux (SELinux):

# setenforce 1

To get the status of a system running SELinux:

# sestatus

Set up Apache MPM and PHP-FPM:

With Apache 2.4, the official module to use is mod_proxy_fcgi instead of the ancient mod_fastcgi. That module, as well as mod_fcgid, were third party modules.

Note: mod_proxy_fcgi now supports network sockets since Apache 2.4.9 ( Unix socket support for mod_proxy_fcgi )

List built-in or shared modules:

# httpd -M | grep -iE 'proxy_module|proxy_fcgi_module|rewrite_module|mpm_event_module|deflate_module|vhost_alias_module|ssl_module'

 deflate_module (shared)
 rewrite_module (shared)
 vhost_alias_module (shared)
 mpm_event_module (shared)
 proxy_module (shared)
 proxy_fcgi_module (shared)
 ssl_module (shared)

List loaded modules:

# httpd -t -D DUMP_MODULES | grep -iE 'proxy_module|proxy_fcgi_module|rewrite_module|mpm_event_module|deflate_module|vhost_alias_module|ssl_module'

 deflate_module (shared)
 rewrite_module (shared)
 vhost_alias_module (shared)
 mpm_event_module (shared)
 proxy_module (shared)
 proxy_fcgi_module (shared)
 ssl_module (shared)

Edit mpm.conf:

# vim /etc/httpd/conf.modules.d/00-mpm.conf

Comment out the following line:

LoadModule mpm_prefork_module modules/mod_mpm_prefork.so

Uncomment the following line:

LoadModule mpm_event_module modules/mod_mpm_event.so

Make sure the following two lines exist:

# grep -E 'mod_proxy.so|mod_proxy_fcgi' /etc/httpd/conf.modules.d/00-proxy.conf

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so

Add the "if checking" surround the following three lines:

# vim /etc/httpd/conf.d/php.conf

<IfModule mod_php5.c>
  <FilesMatch \.php$>
    SetHandler application/x-httpd-php
  </FilesMatch>
</IfModule>

Add the "if checking" surround the following two lines:

# vim /etc/httpd/conf.d/php.conf

<IfModule mod_php5.c>
  php_value session.save_handler "files"
  php_value session.save_path    "/var/lib/php/session"
</IfModule>

Change the following line:

# vim /etc/httpd/conf/httpd.conf

<Directory "/var/www/html">
  AllowOverride All

  # New directive needed in Apache 2.4.3: 
  Require all granted
</Directory>

Create and Edit the vhosts file:

# vim /etc/httpd/conf.d/httpd-vhosts.conf

#
# Use name-based virtual hosting.
#
NameVirtualHost *:80

#
# VirtualHost example:
# Almost any Apache directive may go into a VirtualHost container.
# The first VirtualHost section is used for all requests that do not
# match a ServerName or ServerAlias in any <VirtualHost> block.
#
<VirtualHost *:80>
    ServerAdmin webmaster@dummy-host.example.com
    DocumentRoot "/var/www/html/host_not_found"
    ErrorLog "/var/log/httpd/host_not_found-error_log"
    CustomLog "/var/log/httpd/host_not_found-access_log" common
</VirtualHost>

<VirtualHost *:80>
    ServerAdmin webmaster@dummy-host.example.com
    DocumentRoot "/var/www/html/magento1.9.1"
    ServerName us.centos.local

    ### Disable PHP script execution for this directory. We don't want to reverse-proxy this subdirectory.
    <Location "/var">
      ProxyPass !
    </Location>

    ### Disable PHP script execution for this directory. We don't want to reverse-proxy this subdirectory.
    <Location "/media">
      ProxyPass !
    </Location>

    ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://127.0.0.1:9000/var/www/html/magento1.9.1/$1

    SetEnv MAGE_RUN_CODE "us_centos_local"
    SetEnv MAGE_RUN_TYPE "website"

    ### a request for / will need to be mapped to a resource on the fcgi backend. Failure to address this may cause a blank response, commonly known as a WSOD (White Screen of Death), especially if only a request URI containing the php extension is proxied, such as this example. The processing chain will first map a request for / to /index.php, then proxy to the PHP-FPM backend correctly.
    DirectoryIndex /index.php index.php index.html index.htm

    ErrorLog "/var/log/httpd/us.centos.local-error_log"
    CustomLog "/var/log/httpd/us.centos.local-access_log" common
</VirtualHost>

Setting up an SSL secured Web server:

<VirtualHost *:443>
    SSLEngine on
    SSLCertificateFile /etc/pki/tls/certs/mydomain.com.crt
    SSLCertificateKeyFile /etc/pki/tls/private/mydomain.com.key
    SSLCertificateChainFile /etc/pki/tls/certs/gd_bundle-mydomain.com.crt

    ServerAdmin webmaster@dummy-host.example.com
    DocumentRoot "/var/www/html/magento1.9.1"
    ServerName us.centos.local

    ### we don't want to reverse-proxy this subdirectory.
    <Location "/var">
      ProxyPass !
    </Location>

    ### we don't want to reverse-proxy this subdirectory.
    <Location "/media">
      ProxyPass !
    </Location>

    ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://127.0.0.1:9000/var/www/html/magento1.9.1/$1

    SetEnv MAGE_RUN_CODE "us_centos_local"
    SetEnv MAGE_RUN_TYPE "website"

    ErrorLog "/var/log/httpd/us.centos.local443-error_log"
    CustomLog "/var/log/httpd/us.centos.local443-access_log" common
</VirtualHost>

Make sure the security context of the certification files are cert_t:

# cd /etc/pki/tls/certs/mydomain.com
# chcon -t cert_t *

If you do not see the correct security context, you will see the following message when restarting Apache:

Mar 16 23:58:00 ip-172-31-21-55 httpd[22680]: AH00526: Syntax error on line 19 of /etc/httpd/conf.d/httpd-vhosts.conf:
Mar 16 23:58:00 ip-172-31-21-55 httpd[22680]: SSLCertificateFile: file '/etc/pki/tls/certs/mydomain.com/b1210d10x4d812c5.crt' does not exist or is empty
Mar 16 23:58:00 ip-172-31-21-55 systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE
Mar 16 23:58:00 ip-172-31-21-55 systemd[1]: Failed to start The Apache HTTP Server.
Mar 16 23:58:00 ip-172-31-21-55 systemd[1]: Unit httpd.service entered failed state.
Mar 16 23:58:00 ip-172-31-21-55 systemd[1]: httpd.service failed.

Restart Apache:

# systemctl reload httpd.service

# httpd -t -D DUMP_VHOSTS

To enable gzip compression:

# egrep 'deflate|header' /etc/httpd/conf.modules.d/00-base.conf

LoadModule deflate_module modules/mod_deflate.so
LoadModule headers_module modules/mod_headers.so

# httpd -t -D DUMP_MODULES | grep deflate

deflate_module (shared)

# vim /etc/httpd/conf.d/mod_deflate.conf

<IfModule mod_deflate.c>
  AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css text/js application/javascript application/x-javascript application/x-httpd-php
</IfModule>

# systemctl restart httpd.service

Install Xdebug for debugging PHP:

# pecl install Xdebug

# vim /etc/php.d/xdebug.ini

zend_extension="/usr/lib64/php/modules/xdebug.so"

; When this setting is set to on, the tracing of function calls will be enabled just before the script is run. This makes it possible to trace code in the auto_prepend_file.
xdebug.auto_trace = 1
xdebug.trace_output_dir = "/tmp"
xdebug.collect_params = 4

; Enables Xdebug's profiler which creates files in the profile output directory. Those files can be read by KCacheGrind to visualize your data.
xdebug.profiler_enable = 1
xdebug.profiler_output_dir = "/tmp"

; Controls the protection mechanism for infinite recursion protection. The value of this setting is the maximum level of nested functions that are allowed before the script will be aborted.
xdebug.max_nesting_level = 100

; shows a human readable / computer readable trace file.
xdebug.trace_format = 0

; This setting tells Xdebug to gather information about which variables are used in a certain scope. This analysis can be quite slow as Xdebug has to reverse engineer PHP's opcode arrays. This setting will not record which values the different variables have, for that use xdebug.collect_params. This setting needs to be enabled only if you wish to use xdebug_get_declared_vars().
xdebug.collect_vars = 0

; When set to '1' the trace files will be appended to, instead of being overwritten in subsequent requests.
; Note: this option can be useful if you could not find your function calls anywhere.
xdebug.trace_options = 1

To search all other php modules:

# yum search php

Edit php.ini:

# vim /etc/php.ini

cgi.fix_pathinfo = 0

The default value is 1, which is an extremely insecure setting because it tells PHP to attempt to execute the closest file it can find if a PHP file does not match exactly. This basically would allow users to craft PHP requests in a way that would allow them to execute scripts that they shouldn't be allowed to execute.

Note: if I set it to "cgi.fix_pathinfo = 0", I would get "Access denied (403)" (see security.limit_extensions) or no input file specified error when setting up Magento. You can either:

1. commented out the cgi.fix_pathinfo = 0 line.
2. set cgi.fix_pathinfo = 1
3. try to set "security.limit_extensions = " in the /etc/php-fpm.d/www.conf file.

http://stackoverflow.com/questions/23390531/access-denied-403-for-php-files-with-nginx-php-fpm
http://serverfault.com/questions/627903/is-the-php-option-cgi-fix-pathinfo-really-dangerous-with-nginx-php-fpm

display_errors = On
log_errors = On
error_log = /var/log/php_errors.log

Note: make sure you do:

# touch /var/log/php_errors.log
# chmod 660 /var/log/php_errors.log
# chown root:apache /var/log/php_errors.log

# chcon -t httpd_log_t /var/log/php_errors.log
Or
# chcon -u system_u -t httpd_log_t /var/log/php_errors.log

Install XCache:

XCache is a fast, stable ​PHP opcode cacher that has been proven and is now running on production servers under high load. It is tested (on linux) and supported on all of the latest ​PHP release branches such as PHP_5_1 PHP_5_2 PHP_5_3 PHP_5_4 PHP_5_5. It is more stable than APC.

Warning: APC would cause segfault segmentation fault. Use XCache instead.

# yum install php-xcache xcache-admin

# systemctl restart httpd
# systemctl restart php-fpm
# php -v

PHP 5.4.16 (cli) (built: Oct 31 2014 12:59:36)
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2013 Zend Technologies
    with XCache v3.1.1, Copyright (c) 2005-2014, by mOo
    with XCache Optimizer v3.1.1, Copyright (c) 2005-2014, by mOo
    with XCache Cacher v3.1.1, Copyright (c) 2005-2014, by mOo
    with XCache Coverager v3.1.1, Copyright (c) 2005-2014, by mOo

# echo -n "Your Password" | md5sum

5afd8756ebeda48acf7eb645503dae60

# vim /etc/php.d/xcache.ini

[xcache]

xcache.admin.user = "admin name"
xcache.admin.pass = "5afd8756ebeda48acf7eb645503dae60"

xcache.size = 60M

; This number divides total cache size into threads in order to increase the efficiency. If you have 128M xcache.size and you set the count as 4, that means each thread will manage 32M size of cache.
xcache.count = 1

xcache.cacher = On
xcache.stat = On
xcache.optimizer = On

# cp -r /usr/share/xcache/ /var/www/html/

# systemctl restart php-fpm

http://localhost/xcache/

Install APC:

Warning: APC would cause segfault segmentation fault. Use XCache instead.

# pecl install apc

# vim /etc/php.d/apc.ini

; Enable APC for PHP
extension=apc.so
apc.enabled=1

; The number of seconds a cache entry is allowed to idle in a slot before APC dumps the cache
apc.ttl=72000
apc.user_ttl=72000
apc.gc_ttl=3600

; Size of memory for apc ( 1024 M)
apc.shm_size=1024M

; Enable apc stats.
apc.stat=1

; Enable APC for command line php operations.
apc.enable_cli=1

; Allow 2 seconds after a file is created before it is cached. This will prevent premature PHP pages to get cached.
apc.file_update_protection=2

; Maximum size of single file that apc can store.
apc.max_file_size=1M

; Maximum number of files APC can store ( rotation).
apc.num_files_hint=200000

; Maximum number of users data entries that APC can store.
apc.user_entries_hint=20000

Copy the apc.php file:

# cp /usr/share/pear/apc.php /var/www/html

Set up php-fpm:

Apach 2.4.8 mod_proxy: Added support for unix domain sockets as the backend server endpoint.

# vim /etc/php-fpm.d/www.conf

If you are using Apach 2.4.8 or above, please change the following line from:

listen = 127.0.0.1:9000

To:

listen = /var/run/php-fpm/php-fpm.sock

Now, there are different ways to actually forward requests for .php files to this module, ranging from everything (using ProxyPass) to very specific or rewritten files or patterns (using mod_rewrite with the [P] flag).

The method I chose (using ProxyPassMatch) lies somewhere in between these in complexity and flexibility, since it allows you to set one rule for all PHP content of a specific vhost, but will only proxy .php files (or URLs that contain the text .php somewhere in the request).

TCP socket (IP and port) approach

Edit the configuration for a vhost of your choice, and add the following line to it:

# vim /etc/httpd/conf.d/httpd-vhosts.conf

ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://127.0.0.1:9000/path/to/your/documentroot/$1

### a request for / will need to be mapped to a resource on the fcgi backend. Failure to address this may cause a blank response, commonly known as a WSOD (White Screen of Death), especially if only a request URI containing the php extension is proxied, such as this example. The processing chain will first map a request for / to /index.php, then proxy to the PHP-FPM backend correctly.
    DirectoryIndex /index.php index.php index.html index.htm

Note: please do change /path/to/your/documentroot to for example /var/www/html/drupal8

Look confusing ? Let's run through it:

ProxyPassMatch

only proxy content that matches the specified regex pattern; in this case:

^/(.*\.php(/.*)?)$

from the documentroot onwards, match everything ending in .php (with the dot escaped), optionally followed by a slash and any continued path you like (some applications use this so-called PathInfo to pass arguments to the php script.)

The ^ (caret) and $ (dollar) signs are used to anchor both the absolute start and end of the URL, to make sure no characters from the request escape our pattern match.

The nested parentheses enable us to refer to the entire request-URI (minus the leading slash) as $1, while still keeping the trailing pathinfo optional.

fcgi://127.0.0.1:9000

forward via mod_proxy_fcgi, using the fastCGI protocol, to the port our php-fpm daemon is listening on.

This determines which fastcgi pool will serve requests proxied by this rule.

/path/to/your/documentroot/

IMPORTANT! This must exactly match the real filesystem location of your php files, because that is where the php-fpm daemon will look for them.

php-fpm just interprets the php files passed to it; it is not a web server, nor does it understand your web servers' namespace, virtualhost layout, or aliases.

IMPORTANT! Read the above again

$1

expands to the entire request-URI from the original request, minus the leading slash (because we already added that above.)

DirectoryIndex /index.php index.php index.html index.htm

Note: a request for / will need to be mapped to a resource on the fcgi backend. Failure to address this may cause a blank response, commonly known as a WSOD (White Screen of Death), especially if only a request URI containing the php extension is proxied, such as this example. The processing chain will first map a request for / to /index.php, then proxy to the PHP-FPM backend correctly.

unix domain socket (UDS) approach

Edit the configuration for a vhost of your choice, and add the following line to it:

# vim /etc/httpd/conf.d/httpd-vhosts.conf

ProxyPassMatch ^/(.*\.php(/.*)?)$ unix:/path/to/socket.sock|fcgi://127.0.0.1:9000/path/to/your/documentroot/

unix:/path/to/socket.sock

the path to your fpm socket

Note that with this approach, the captured request URI ($1) is not passed after the path

Enable php-fpm to start on boot:

# systemctl enable php-fpm.service

Start our PHP processor:

# systemctl restart php-fpm.service
# systemctl restart httpd.service

Test PHP:

# php -r "echo 'hi';";

Check the PHP-FPM setting:

<?php
echo phpinfo();
?>

You should see the message: Server API: FPM/FastCGI

# httpd -V

Server MPM:     event

To create a locked user account:

# useradd dev -m -c 'git user'

Unlock the account by issuing the passwd command to assign a password and set password aging guidelines:

# passwd dev

Add a new group called web:

# groupadd web

Add the dev user to the web group:

# usermod -a -G web dev

Show The Groups a User Is In:

# groups dev
# id -Gn dev

Find out the primary group of a user:

# getent group dev

To allow Apache to connect network and sendmail send email:

# setsebool -P httpd_can_network_connect 1
# setsebool -P httpd_can_sendmail 1

Note: -P in the above command means Persistent (across reboots)

# getsebool -a | grep -i httpd_can

httpd_can_network_connect --> on
httpd_can_sendmail --> on

# sestatus -b | grep httpd_can

httpd_can_network_connect                   on
httpd_can_sendmail                          on

Install Memcached:

# yum install memcached
# yum install php-pecl-memcached

# systemctl enable memcached.service
# systemctl restart memcached.service

# memcached-tool localhost:11211 display
# memcached-tool localhost:11211 stats
# memcached-tool localhost:11211 dump

Install redis server:

# yum install redis
# yum install php-pecl-redis

Two important redis server configuration files:

# less /etc/redis.conf
# less /etc/redis-sentinel.conf

Start the Redis server:

# systemctl start redis.service

Check the running status of Redis server:

# systemctl status redis.service

To test the installation of Redis:

# redis-cli ping

PONG

To enable Redis server at system's booting time:

# systemctl enable redis.service

To get the listening port 6379 of Redis server:

# ss -nlp | grep redis

To install Redis PHP extension:

# pecl install redis

Add the following line to /etc/php.d/redis.ini:

# echo 'extension=redis.so' >> /etc/php.d/redis.ini

Check to see if Redis PHP extension is installed:

# pecl list | grep redis

redis   2.2.7   stable

Restart Apache and PHP-FPM:

# systemctl restart httpd.service
# systemctl restart php-fpm.service

To see if Redis extension is being loaded by PHP:

# php -m | grep redis

To allow Apache to connect to the Redis server:

# setsebool -P httpd_can_network_connect 1

Note: If you have turned on Security-Enhanced Linux (SELinux), httpd scripts by default are not allowed to connect out to the network.

To list all Redis Databases:

# redis-cli info keyspace

To clear remove delete all data from a particular Redis database:

# redis-cli

127.0.0.1:6379> info keyspace
127.0.0.1:6379> select 0
127.0.0.1:6379> keys *
127.0.0.1:6379> flushdb
127.0.0.1:6379> keys *

To clear remove delete all data from all Redis database:

# redis-cli flushall

Dumping all key/value pairs in a Redis db:

# redis-cli -n 0 keys \*

# redis-cli -n 0 keys \* | xargs -n 1 redis-cli dump

Note: the 0 is the database number.

To store PHP sessions in Redis:

Storing PHP session files in RAM can be much more efficient than storing on disk and can also save some IO. To configure this, you should modify the main php.ini file and change session.save_handler to redis.

# vim /etc/php.ini

session.save_handler = redis

session.save_path = "tcp://127.0.0.1:6379"

# systemctl restart php-fpm

# php -r 'echo phpinfo();' | grep redis

Registered save handlers => files user redis
session.save_handler => redis => redis

# vim test.php

<?php
session_start();

$_SESSION['favcolor'] = 'green';

echo '<pre>' . print_r($_SESSION, TRUE) . '</pre>';
?>

# php test.php

# redis-cli info keyspace
# Keyspace
db0:keys=68,expires=39,avg_ttl=2977110
db2:keys=427,expires=427,avg_ttl=1856886

# redis-cli -n 0 keys \*| grep -i session

PHPREDIS_SESSION:vhauaf8qpdj146kirsbivrh4i7

Note: the 0 is the database number.

Redis setup hints

  • We suggest deploying Redis using the Linux operating system. Redis is also tested heavily on osx, and tested from time to time on FreeBSD and OpenBSD systems. However Linux is where we do all the major stress testing, and where most production deployments are working.
  • Make sure to set the Linux kernel overcommit memory setting to 1. Add vm.overcommit_memory = 1 to /etc/sysctl.conf and then reboot or run the command sysctl vm.overcommit_memory=1 for this to take effect immediately.
  • Make sure to disable Linux kernel feature transparent huge pages, it will affect greatly both memory usage and latency in a negative way. This is accomplished with the following command: echo never > sys/kernel/mm/transparent_hugepage/enabled.
  • Make sure to setup some swap in your system (we suggest as much as swap as memory). If Linux does not have swap and your Redis instance accidentally consumes too much memory, either Redis will crash for out of memory or the Linux kernel OOM killer will kill the Redis process.
  • Set an explicit maxmemory option limit in your instance in order to make sure that the instance will report errors instead of failing when the system memory limit is near to be reached.
  • If you are using Redis in a very write-heavy application, while saving an RDB file on disk or rewriting the AOF log Redis may use up to 2 times the memory normally used. The additional memory used is proportional to the number of memory pages modified by writes during the saving process, so it is often proportional to the number of keys (or aggregate types items) touched during this time. Make sure to size your memory accordingly.
  • Use daemonize no when run under daemontools.
  • Even if you have persistence disabled, Redis will need to perform RDB saves if you use replication, unless you use the new diskless replication feature, which is currently experimental.
  • If you are using replication, make sure that either your master has persistence enabled, or that it does not automatically restarts on crashes: slaves will try to be an exact copy of the master, so if a master restarts with an empty data set, slaves will be wiped as well.

Running Redis on EC2

  • Use HVM based instances, not PV based instances.
  • Don't use old instances families, for example: use m3.medium with HVM instead of m1.medium with PV.
  • The use of Redis persistence with EC2 EBS volumes needs to be handled with care since sometimes EBS volumes have high latency characteristics.
  • You may want to try the new diskless replication (currently experimetnal) if you have issues when slaves are synchronizing with the master.

Reference:

http://blog.ijun.org/2014_04_01_archive.html

http://blog.ijun.org/2014/11/configuring-magento-to-use-redis.html

http://blog.ijun.org/2014/12/install-apache-24-php-56-and-mysql-56.html

http://redis.io/topics/admin

http://www.cyberciti.biz/faq/howto-install-linux-apache-mariadb-php-lamp-stack-on-centos7-rhel7/

http://serverfault.com/questions/629937/centos-7-apache2-httpd-mod-fastcgi-installation-impossible

http://blog.famillecollet.com/post/2014/08/01/Apache-httpd-server-2.4.10-and-PHP-FPM-5.6-in-Fedora-21

https://wiki.apache.org/httpd/PHP-FPM

http://sharadchhetri.com/2014/10/04/install-redis-server-centos-7-rhel-7/

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/s1-networkscripts-interfaces.html

http://technovergence-en.blogspot.ca/2012/03/mysql-from-utf8-to-utf8mb4.html

Monday, July 10, 2017

Build OpenJDK 8 on Ubuntu 14.04

# apt-get update \
&& apt-get install build-essential mercurial zip openjdk-7-jdk libX11-dev libxext-dev libxrender-dev libxtst-dev libxt-dev libcups2-dev libfreetype6-dev libasound2-dev ccache

# cd /usr/local/src \
&& hg clone http://hg.openjdk.java.net/jdk8u/jdk8u

# cd jdk8u/ \
&& bash ./get_source.sh

# bash ./configure -with-freetype-include=/usr/include/freetype2 -with-freetype-lib=/usr/lib/x86_64-linux-gnu/

# make all

# make install

Thursday, July 6, 2017

PHP http query

<?php

$header = "Content-type: application/json\r\n"
. "Content-Length: " . strlen($data) . "\r\n"
;


$context = stream_context_create([
        'http' => [
                'method'  => 'POST',
                'ignore_errors' => true,
                'header'=> $header,
                'content' => $data,
        ],
        'ssl' => [
                // set some SSL/TLS specific options
                'verify_peer' => false,
                'verify_peer_name' => false,
                'allow_self_signed' => true
        ],
]);

$result = file_get_contents('http://example.com/', false, $context);

Sunday, July 2, 2017

cannot assign to struct field in map error

cannot assign to struct field in map error

data["p1"] isn't quite a regular addressable value: hashmaps can grow at runtime, and then their values get moved around in memory, and the old locations become outdated. If values in maps were treated as regular addressable values, those internals of the map implementation would get exposed.

So, instead, data["p1"] is a slightly different thing called a "map index expression" in the spec; if you search the spec for the phrase "index expression" you'll see you can do certain things with them, like read them, assign to them, and use them in increment/decrement expressions (for numeric types). But you can't do everything. They could have chosen to implement more special cases than they did, but I'm guessing they didn't just to keep things simple.

Issue code:

package main

import (
        "fmt"
)

type Person struct {
        Name string
}

func main() {
        data := map[string]Person{
                "p1": Person{},
        }

        data["p1"].Name = "Jun"

        fmt.Printf("%v\n", data)
}

Solution:

The solution is to make the map value a regular old pointer.

package main

import (
        "encoding/json"
        "fmt"
)

type Person struct {
        Name string
}

func main() {
        data := map[string]*Person{
                "p1": &Person{},
        }

        (*data["p1"]).Name = "Jun" // Or simply data["p1"].Name = "Jun"

        dataJSON, _ := json.MarshalIndent(data, "", "  ")
        fmt.Printf("%s\n", dataJSON)
}

Reference:

https://stackoverflow.com/questions/32751537/why-do-i-get-a-cannot-assign-error-when-setting-value-to-a-struct-as-a-value-i

Sunday, June 18, 2017

Install and Configure Elasticsearch on Ubuntu 16.04

Install and Configure Elasticsearch on Ubuntu 16.04

Installing the Oracle JDK:

# add-apt-repository ppa:webupd8team/java
# apt-get update
# apt-get install oracle-java9-installer

Multiple Java installations can be installed on one server. You can use the following command to configure which version is the default for use:

# update-alternatives --config java

Install Elasticsearch:

# wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
# apt-get install apt-transport-https
# echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list

# apt-get update && apt-get install elasticsearch

# systemctl daemon-reload
# systemctl enable elasticsearch.service
# systemctl start elasticsearch.service

# vim ~/.bashrc

export JAVA_HOME="/usr/lib/jvm/java-9-oracle"

# echo $JAVA_HOME

/usr/lib/jvm/java-9-oracle

To tail the journal:

# journalctl -f

To list journal entries for the elasticsearch service:

# journalctl --unit elasticsearch

To list journal entries for the elasticsearch service starting from a given time:

# journalctl --unit elasticsearch --since "2017-06-17 15:09:11"

To test Elasticsearch is running:

# curl http://localhost:9200/

Configure Elasticsearch:

# cd /etc/elasticsearch

To check the cluster health:

# curl -X GET 'localhost:9200/_cat/health?v&pretty'

Get a list of nodes in the cluster:

# curl -X GET 'localhost:9200/_cat/nodes?v'

Create the index named "customer":

# curl -X PUT 'localhost:9200/customer?pretty'

List all indices:

# curl -X GET 'localhost:9200/_cat/indices?v'

Add a customer document into the customer index, "external type, with an ID of 1:

# curl -X PUT 'localhost:9200/customer/external/1?pretty&pretty' -H 'Content-Type: application/json' -d'
{
"name": "John Doe"
}
'

Retrieve the document we just added:

# curl -X GET 'localhost:9200/customer/external/1?pretty&pretty'

Replace the document:

# curl -X PUT 'localhost:9200/customer/external/1?pretty&pretty' -H 'Content-Type: application/json' -d'
{
"name": "Jane Doe"
}
'

Update the document:

# curl -X POST 'localhost:9200/customer/external/1/_update?pretty&pretty' -H 'Content-Type: application/json' -d'
{
"doc": { "name": "Jane Doe", "age": 20 }
}
'

Updates can also be performed by using simple scripts:

# curl -X POST 'localhost:9200/customer/external/1/_update?pretty&pretty' -H 'Content-Type: application/json' -d'
{
"script" : "ctx._source.age += 5"
}
'

Note: ctx._source refers to the current source document that is about to be updated.

Bulk insert the multiple documents:

# curl -X POST 'localhost:9200/customer/external/_bulk?pretty&pretty' -H 'Content-Type: application/json' -d'
{"index":{"_id":"1"}}
{"name": "John Doe 3" }
{"index":{"_id":"2"}}
{"name": "Jane Doe 3" }
'

Note: the existing documents will be replaced instead of updated.

Update the first document and delete the second document in one bulk operation:

# curl -X POST 'localhost:9200/customer/external/_bulk?pretty&pretty' -H 'Content-Type: application/json' -d'
{"update":{"_id":"1"}}
{"doc": { "name": "John Doe becomes Jane Doe" } }
{"delete":{"_id":"2"}}
'

Delete a document:

# curl -X DELETE 'localhost:9200/customer/external/1?pretty&pretty'

Delete the customer index:

# curl -X DELETE 'localhost:9200/customer?pretty&pretty'

Load data into the cluster:

# curl -H "Content-Type: application/json" -XPOST 'localhost:9200/bank/account/_bulk?pretty&refresh' --data-binary "@accounts.json"
# curl 'localhost:9200/_cat/indices?v'

Note: the sample data can be generated from http://www.json-generator.com/

Query all documents in the index:

# curl -X GET 'localhost:9200/bank/_search?q=*&sort=account_number:asc&pretty&pretty'

Alternative query method:

# curl -X GET 'localhost:9200/bank/_search?pretty' -H 'Content-Type: application/json' -d'
{
"query": { "match_all": {} },
"sort": [
{ "account_number": "asc" }
]
}
'

Returns all accounts containing the term "mill" or "lane" in the address:

# curl -X GET 'localhost:9200/bank/_search?pretty' -H 'Content-Type: application/json' -d'
{
"query": { "match": { "address": "mill lane" } }
}
'

Returns all accounts containing the phrase "mill lane" in the address:

# curl -X GET 'localhost:9200/bank/_search?pretty' -H 'Content-Type: application/json' -d'
{
"query": { "match_phrase": { "address": "mill lane" } }
}
'

Returns all accounts containing "mill" and "lane" in the address:

# curl -X GET 'localhost:9200/bank/_search?pretty' -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must": [
{ "match": { "address": "mill" } },
{ "match": { "address": "lane" } }
]
}
}
}
'

Returns all accounts containing "mill" or "lane" in the address:

# curl -X GET 'localhost:9200/bank/_search?pretty' -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"should": [
{ "match": { "address": "mill" } },
{ "match": { "address": "lane" } }
]
}
}
}
'

Returns all accounts that contain neither "mill" nor "lane" in the address:

# curl -X GET 'localhost:9200/bank/_search?pretty' -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must_not": [
{ "match": { "address": "mill" } },
{ "match": { "address": "lane" } }
]
}
}
}
'

Reference:

https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/_delete_an_index.html

https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-elasticsearch-on-ubuntu-16-04

Thursday, May 25, 2017

Redirect both stdout and stderr to a file:

Redirect both stdout and stderr to a file:
# ls &> filename

Note: This operator is now functional, as of Bash 4, final release.

Redirect stderr to stdout (&1), and then redirect stdout to a file:
# ls > filename 2>&1

Note: 2>&1 redirects file descriptor 2 (stderr) to file descriptor 1 (stdout).

Reference:

https://stackoverflow.com/questions/7526971/how-to-redirect-both-stdout-and-stderr-to-a-file

Wednesday, May 17, 2017

gpg: agent_genkey failed: Permission denied

$ gpg2 --gen-key

// On Ubuntu
gpg: agent_genkey failed: Permission denied
Key generation failed: Permission denied

// On CentOS
gpg: cancelled by user
gpg: Key generation canceled.

Solution:

$ ls -la $(tty)

crw--w----. 1 someone tty 136, 9 May 17 20:47 /dev/pts/9

$ sudo chown MyUserName /dev/pts/9

$ gpg2 --gen-key

Monday, May 15, 2017

sign_and_send_pubkey: signing failed: agent refused operation

sign_and_send_pubkey: signing failed: agent refused operation

Try to add the private key identities to the authentication agent:

# ssh-add

To see a list of fingerprints of all identities:

# ssh-add -l

Reference:

https://askubuntu.com/questions/762541/ubuntu-16-04-ssh-sign-and-send-pubkey-signing-failed-agent-refused-operation

Friday, May 12, 2017

Use rpmbuild to build a custom RPM package on CentOS 7

Use rpmbuild to build a custom RPM package on CentOS 7

Install the necessary tools:

# yum install rpm-build rpmdevtools rpmlint

Create the necessary directories:

# rpmdev-setuptree

// or create them manually
# mkdir -p ~/rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS,tmp}

# ls -l ~/rpmbuild

Set up the temporary directory:

# vim ~/.rpmmacros

%_tmppath %(echo $HOME)/rpmbuild/tmp

Prepare some program/files for the RPM package:

# cd ~/rpmbuild
# mkdir -p ~/rpmbuild/SOURCES/test-repo-1.0.0
# echo -e '#!/bin/sh\necho "Hello World"' > ~/rpmbuild/SOURCES/test-repo-1.0.0/hello
# chmod +x ~/rpmbuild/SOURCES/test-repo-1.0.0/hello
# tar zcvf ~/rpmbuild/SOURCES/test-repo-1.0.0.tar.gz -C ~/rpmbuild/SOURCES/ test-repo-1.0.0/

Edit the spec file:

# vim ~/rpmbuild/SPECS/test-repo.spec

%define NAME test-repo
%define VERSION 1.0.0
%define INSTALL_DIR /usr/local
%define OWNER root

# Disable creating debuginfo RPM
%define debug_package %{nil}

# Strip debug symbols (possibly making the program not funtion properly)
#%define __strip /bin/true

Name: %NAME
Version: %VERSION

# Refer to:
# https://fedoraproject.org/wiki/How_to_create_an_RPM_package
# https://fedoraproject.org/wiki/Packaging:DistTag?rd=Packaging/DistTag
Release: 1%{?dist}

Summary: Package foo summary
Source: %NAME-%VERSION.tar.gz
License: MIT

# Refer to:
# cat /usr/share/doc/rpm-4.11.3/GROUPS
Group: Development/Tools

%description
Package foo description.

%prep

# Start uncompressing
%setup -q

%build

%install

# Turn this on to find out the available environment variables.
#env

install -m 0755 -d ${RPM_BUILD_ROOT}%INSTALL_DIR/%NAME
#mkdir -p ${RPM_BUILD_ROOT}%INSTALL_DIR/%NAME
cp -r * ${RPM_BUILD_ROOT}%INSTALL_DIR/%NAME/

#make install DESTDIR=$RPM_BUILD_ROOT

%clean
rm -rf ${RPM_BUILD_DIR}/*
rm -rf ${RPM_BUILD_ROOT}
rm -rf %_tmppath/*

%post
echo . .
echo .Wring some descripton here to show after package installation!.

%files
%INSTALL_DIR
%defattr(-,%OWNER,%OWNER)

%changelog

Validate the spec:

# cd ~/rpmbuild
# rpmlint SPECS/test-repo.spec

SPECS/test-repo.spec:10: W: macro-in-comment %define
SPECS/test-repo.spec:42: W: macro-in-comment %INSTALL_DIR
SPECS/test-repo.spec:42: W: macro-in-comment %NAME
SPECS/test-repo.spec: W: invalid-url Source0: test-repo-1.0.0.tar.gz
0 packages and 1 specfiles checked; 0 errors, 4 warnings.

Note: Do not worry if you see the warnings.

Build RPM without the source:

# cd ~/rpmbuild
# rpmbuild -v -bb SPECS/test-repo.spec

Build RPM with the source:

# rpmbuild -v -ba SPECS/test-repo.spec

Install the RPM:

# rpm -ivh RPMS/x86_64/test-repo-1.0.0-1.el7.centos.x86_64.rpm

Check the RPM installation:

# ls -la /usr/local/test-repo/

List the installed RPM:

# rpm -qa | grep test-repo

test-repo-1.0.0-1.el7.centos.x86_64

List the files in the installed RPM:

# rpm -q --filesbypkg test-repo

test-repo                 /usr/local
test-repo                 /usr/local/test-repo
test-repo                 /usr/local/test-repo/hello

To find out which RPM a file belongs to:

# rpm -qf /usr/local/test-repo/hello

test-repo-1.0.0-1.el7.centos.x86_64

Remove the RPM:

# rpm -e test-repo

Reference:

https://fedoraproject.org/wiki/How_to_create_an_RPM_package

Wednesday, May 10, 2017

gpg --gen-key hangs at gaining enough entropy on CentOS 7

gpg --gen-key hangs at gaining enough entropy on CentOS 7

Solution 1:

Install random number generator:

# yum install rng-tools

# systemctl enable rngd

# systemctl restart rngd

Solution 2:

# dd if=/dev/sda of=/dev/zero

Reference:

https://serverfault.com/questions/471412/gpg-gen-key-hangs-at-gaining-enough-entropy-on-centos-6#

Sunday, April 30, 2017

Changing Ubuntu full Screen Resolution in a Hyper-V VM

Changing Ubuntu full Screen Resolution in a Hyper-V VM

# vi /etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash video=hyperv_fb:1920×1080"

# update-grub
# sync; reboot

Reference:

https://blogs.msdn.microsoft.com/virtual_pc_guy/2014/09/19/changing-ubuntu-screen-resolution-in-a-hyper-v-vm/

Friday, April 14, 2017

Save the selected lines to a file

To save the selected lines to a file, select a region using visual mode and then enter:

:w /tmp/filename

To insert/paste text from a file:

:r /tmp/filename

Set up Xdebug

Set up Xdebug

https://github.com/joonty/vdebug

SELinux does not allow httpd to connect to other network resources by default. Turn it on:

# setsebool -P httpd_can_network_connect 1
# getsebool -a | grep httpd_can
or
# chcon -v -t httpd_sys_content_t /usr/lib64/php/modules/xdebug.so

Note: http://stackoverflow.com/questions/2207489/apache-not-loading-xdebug-but-does-when-started-from-the-command-line

Edit ~/.vimrc:

# vim ~/.vimrc

let g:vdebug_options = {}
let g:vdebug_options["port"] = 9009

Edit xdebug configuration:

# vim /etc/php.d/15-xdebug.ini

; Enable xdebug extension module
zend_extension=xdebug.so
;zend_extension=/usr/lib64/php/modules/xdebug.so

xdebug.default_enable=1
xdebug.remote_enable=1
xdebug.remote_handler=dbgp
xdebug.remote_host=localhost
xdebug.remote_port=9009
xdebug.remote_log=/tmp/xdebug.log
xdebug.remote_connect_back=0
xdebug.remote_autostart=0
xdebug.remote_mode=req

xdebug.max_nesting_level=1000

Sample script:

# vim test.php

<?php
here();

function here() {
    xdebug_break();
    echo name('Jun');
}

function name($name) {
    $name .= '1';
    return 'Hello ' . $name . ' ' . date('Y-m-d H:i:s');
}

Once in debugging mode, the following default mappings are available:

<F5>: start/run (to next breakpoint/end of script)
<F2>: step over
<F3>: step into
<F4>: step out
<F6>: stop debugging (kills script)
<F7>: detach script from debugger
<F9>: run to cursor
<F10>: toggle line breakpoint
<F11>: show context variables (e.g. after "eval")
<F12>: evaluate variable under cursor
:Breakpoint <type> <args>: set a breakpoint of any type (see :help VdebugBreakpoints)
:VdebugEval <code>: evaluate some code and display the result
<Leader>e: evaluate the expression under visual highlight and display the result

To stop debugging, press <F6>. Press it again to close the debugger interface.

If you can't get a connection, then chances are you need to spend a bit of time setting up your environment. Type :help Vdebug for more information.

Browser:

http://dru8.local/hello?XDEBUG_SESSION_START=1

Note: Append XDEBUG_SESSION_START=1 to the end of the URL.

Tuesday, April 11, 2017

Compile the latest Vim 8.0 on CentOS 7

Compile the latest Vim 8.0 on CentOS 7

Remove the existing vim if you have already installed it:

# yum list installed | grep -i vim

vim-common.x86_64                    2:7.4.160-1.el7                @base
vim-enhanced.x86_64                  2:7.4.160-1.el7                @base
vim-filesystem.x86_64                2:7.4.160-1.el7                @base
vim-minimal.x86_64                   2:7.4.160-1.el7                @anaconda

# yum remove vim-enhanced vim-common vim-filesystem

Note: You do not need to remove vim-minimal because sudo depends on it.

For Red Hat/CentOS users:

# yum install gcc make ncurses ncurses-devel
# yum git
# yum install ruby ruby-devel lua lua-devel luajit \
luajit-devel ctags python python-devel \
python3 python3-devel tcl-devel \
perl perl-devel perl-ExtUtils-ParseXS \
perl-ExtUtils-XSpp perl-ExtUtils-CBuilder \
perl-ExtUtils-Embed

or

# yum clean all
# yum grouplist
# yum groupinfo "Development Tools"
# yum groupinstall "Development tools"
# yum install ncurses ncurses-devel

For debian/Ubuntu users:

# apt-get remove vim vim-runtime vim-tiny vim-common

# apt-get install libncurses5-dev libgnome2-dev libgnomeui-dev \
libgtk2.0-dev libatk1.0-dev libbonoboui2-dev \
libcairo2-dev libx11-dev libxpm-dev libxt-dev python-dev \
python3-dev ruby-dev git

# apt-get install libncurses5-dev python-dev libperl-dev ruby-dev liblua5.2-dev

// Fix liblua paths
# ln -s /usr/include/lua5.2 /usr/include/lua
# ln -s /usr/lib/x86_64-linux-gnu/liblua5.2.so /usr/local/lib/liblua.so

If you want to install the optional packages With yum groupinstall command:

# vim /etc/yum.conf

group_package_types=default, mandatory, optional

Note: the default setting is default, mandatory.

Install ctags and cscope:

# yum install ctags cscope

Build vim:

# cd /usr/local/src

Download vim source (it is better to get it from GitHub because you can get all the latest patches from there):

# git clone https://github.com/vim/vim.git
# cd vim
or
# wget ftp://ftp.vim.org/pub/vim/unix/vim-7.4.tar.bz2
# tar -xjf vim-7.4.tar.bz2
# cd vim74

Show the configuration options:

# ./configure --help

Configure:

# ./configure --prefix=/usr --with-features=huge --enable-multibyte --enable-rubyinterp --enable-pythoninterp --enable-perlinterp --enable-luainterp --enable-cscope

Build:

# make
or
# make VIMRUNTIMEDIR=/usr/share/vim/vim74

# make install

re-hash the environment:

# hash -r

Check vim version:

# vim --version | less

VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Jul 24 2016 14:27:26)
Included patches: 1-2102
...
+lua +multi_byte +perl +python +ruby

Check vim patches:

# vim

:echo has("patch-7.4-2102")

1

Reference:

https://github.com/Valloric/YouCompleteMe/wiki/Building-Vim-from-source

https://gist.github.com/holguinj/11064609

http://www.fullybaked.co.uk/articles/installing-latest-vim-on-centos-from-source

http://www.vim.org/git.php

Sunday, April 9, 2017

AutoHotkey

example.ahk:

; reload the autohotkey script.
^!r::Reload

; input date and time
#d::
  FormatTime, mydatetime,, M/d/yyyy HH:mm:ss
  SendInput, %mydatetime%
return

; input date
#f::
  FormatTime, mydate,, M/d/yyyy
  SendInput, %mydate%
return

; input date
#y::
  FormatTime, mydate,, yyyy-MM-dd
  SendInput, %mydate%
return

Wednesday, April 5, 2017

PHPUnit did not output the result in color

PHPUnit did not output the result in color when colors=true in phpunit.xml. It's because it was missing the posix extension, which is provided by the php-process package.

# yum install php71u-process

Tuesday, April 4, 2017

ReflectionException: Class PHPUnit_Framework_Error does not exist

ReflectionException: Class PHPUnit_Framework_Error does not exist

    /**
     * @expectedException PHPUnit_Framework_Error
     */
    public function testFailingInclude()
    {
        include 'not_existing_file.php';
    }

Solution 1:

/**
 * @expectedException \PHPUnit\Framework\Error\Warning
 */
public function testFailingInclude()
{
    include 'not_existing_file.php';
}

Solution 2:

public function testFailingInclude()
{
    $this->expectException(\PHPUnit\Framework\Error\Error::class);

    include 'not_existing_file.php';
}

Thursday, March 16, 2017

Redis: (error) CROSSSLOT Keys in request don't hash to the same slot

Redis: (error) CROSSSLOT Keys in request don't hash to the same slot

# redis-cli -c -p 30001

127.0.0.1:30001> SADD mycolor1 R G B
-> Redirected to slot [12383] located at 127.0.0.1:30003
(integer) 3
127.0.0.1:30003> SADD mycolor2 G B Y
-> Redirected to slot [60] located at 127.0.0.1:30001
(integer) 3
127.0.0.1:30001> SUNION mycolor1 mycolor2
(error) CROSSSLOT Keys in request don't hash to the same slot

In a cluster topology, the keyspace is divided into hash slots. Different nodes will hold a subset of hash slots.

Multiple keys operations, transactions, or Lua scripts involving multiple keys are allowed only if all the keys involved are in hash slots belonging to the same node.

Redis Cluster implements all the single key commands available in the non-distributed version of Redis. Commands performing complex multi-key operations like Set type unions or intersections are implemented as well as long as the keys all belong to the same node.

You can force the keys to belong to the same node by using Hash Tags:

> SADD {colorlib}.mycolor1 R G B
> SADD {colorlib}.mycolor2 G B Y
> SUNION {colorlib}.mycolor1 {colorlib}.mycolor2

Reference:

http://stackoverflow.com/questions/38042629/redis-cross-slot-error

http://redis.io/topics/cluster-spec#keys-hash-tag

Assign multiple IP addresses to one single network interface

Assign multiple IP addresses to one single network interface

Method 1:

# cd /etc/sysconfig/network-scripts/
# vim ifcfg-eno16777736

IPADDR0="192.168.6.60"
PREFIX0="24"

IPADDR1="192.168.6.61"
PREFIX1="24"

IPADDR2="10.0.0.2"
PREFIX2="24"
GATEWAY2="10.0.0.1"

Note: I am using a different subnet 10.0.0.0/24 here, too.

Note: If you run ifconfig command, these IP aliases would not show up. Because ifconfig is essentially deprecated. The replacement is the ip command.

# systemctl restart network.service
# ip addr

2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:40:a1:f2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.6.10/24 brd 192.168.6.255 scope global eno16777736
       valid_lft forever preferred_lft forever
    inet 10.0.0.2/24 brd 10.0.0.255 scope global eno16777736
       valid_lft forever preferred_lft forever
    inet 192.168.6.60/24 brd 192.168.6.255 scope global secondary eno16777736
       valid_lft forever preferred_lft forever
    inet 192.168.6.61/24 brd 192.168.6.255 scope global secondary eno16777736
       valid_lft forever preferred_lft forever

Method 2:

# cd /etc/sysconfig/network-scripts/
# cp ifcfg-eno16777736 ifcfg-eno16777736:0
# cp ifcfg-eno16777736 ifcfg-eno16777736:1

# vim ifcfg-eno16777736:0

DEVICE="eno16777736:0"
IPADDR="192.168.6.60"

# vim ifcfg-eno16777736:1

DEVICE="eno16777736:1"
IPADDR="192.168.6.61"

# systemctl restart network.service

Method 3:

If you would like to create a range of multiple IP addresses to a particular network interface:

# cd /etc/sysconfig/network-scripts/
# vim ifcfg-eno16777736

NM_CONTROLLED=NO

Note: This setting is required on Redhat/CentOS 7.x for enabling the range files, which allows us to utilize the range files by having the interface no longer be controlled by the Network Manager system.

# touch ifcfg-eno16777736-range0
# vim ifcfg-eno16777736-range0

IPADDR_START="192.168.6.63"
IPADDR_END="192.168.6.68"
PREFIX="24"
CLONENUM_START="0"

Note: CLONENUM_START is the number of virtual device the first IP address will be assigned to. If you have more than one range file, then you need to make sure that this number is set to the next available number.

# systemctl restart network.service

Reference:

http://www.tecmint.com/create-multiple-ip-addresses-to-one-single-network-interface/

https://www.ubiquityhosting.com/blog/configure-ip-ranges-on-centos-7-redhat-7/

http://askubuntu.com/questions/227457/ifconfig-not-showing-all-ips-bound-to-the-machine

Wednesday, March 15, 2017

What's the difference between Unix socket and TCP/IP socket when setting up PHP-FPM?

What's the difference between Unix socket and TCP/IP socket when setting up PHP-FPM?

A UNIX socket is an inter-process communication mechanism that allows bidirectional data exchange between processes running on the same machine.

IP sockets (especially TCP/IP sockets) are a mechanism allowing communication between processes over the network. In some cases, you can use TCP/IP sockets to talk with processes running on the same computer (by using the loopback interface).

UNIX domain sockets know that they’re executing on the same system, so they can avoid some checks and operations (like routing); which makes them faster and lighter than IP sockets. So if you plan to communicate with processes on the same host, this is a better option than IP sockets.

Edit: As per Nils Toedtmann's comment: UNIX domain sockets are subject to file system permissions, while TCP sockets can be controlled only on the packet filter level.
===
When you are using TCP, you are also using the whole network stack. Even if you are on the same machine, this implies that packets are encapsulated and decapsulated to use the network stack and the related protocols.

If you use unix domain sockets, you will not be forced to go through all the network protocols that are required otherwise. The sockets are identified solely by the inodes on your hard drive.
===
I believe that UNIX domain sockets in theory give better throughput than TCP sockets on the loopback interface, but in practice the difference is probably negligible.

Data carried over UNIX domain sockets don't have to go up and down through the IP stack layers.

re: Alexander's answer. AFAIK you shouldn't get any more than one context switch or data copy in each direction (i.e. for each read() or write()), hence why I believe the difference will be negligble. The IP stack doesn't need to copy the packet as it moves between layers, but it does have to manipulate internal data structures to add and remove higher-layer packet headers.

unix domain socket (UDS) work like system pipes and it send ONLY data, not send checksum and other additional info, not use three-way handshake as TCP sockets.
===
Unix sockets can have owners - users and groups, TCP sockets cannot. Therefore, Unix sockets are more secure - but you cannot separate your webserver (eg, NginX) from your PHP application server (eg. PHP5-FPM) across the network.
===

Reference:

http://serverfault.com/questions/124517/whats-the-difference-between-unix-socket-and-tcp-ip-socket

http://stackoverflow.com/questions/257433/postgresql-unix-domain-sockets-vs-tcp-sockets/257479#257479

http://unix.stackexchange.com/questions/91774/performance-of-unix-sockets-vs-tcp-ports

https://www.reddit.com/r/webhosting/comments/2mgyzg/unix_domain_socket_vs_tcp_can_someone_explain_and/

Wednesday, March 1, 2017

PHPStorm + XDebug Setup Walkthrough

# vim /etc/php/7.0/fpm/conf.d/20-xdebug.ini

zend_extension=xdebug.so

xdebug.remote_enable=1
xdebug.remote_port=9000
xdebug.profiler_enable=1
xdebug.profiler_output_dir="/tmp/xdebug"
xdebug.idekey="PHPSTORM"
xdebug.remote_autostart=1
xdebug.remote_host=localhost
xdebug.remote_mode=req
xdebug.remote_connect_back=1
xdebug.max_nesting_level=200
xdebug.var_display_max_depth=1000
xdebug.var_display_max_children=256
xdebug.var_display_max_data=4096

3. setup the IDE settings

preference > languages and framework > PHP >

3.1. set the language level to the correct PHP version of this project

3.2. set an interpreter (set the parent directory of where the bin directory of PHP executable is loaded)

3.2.1. click the … button > click the + button > other local > set PHP Excitable path,

to find the path type in the terminal: $ which php

example: /usr/local/Cellar/php56/5.6.5/bin/php

4. restart phpstorm

5. now let's make it work

5.1. run > edit configuration > click the green + button on the left > select b. php web application

5.2. name: anything example ur {application name - debugger}

5.3. server: localhost (browse > + > name: whatever | host: localhost or 127.0.0.1)

5.4. click ok

5.5. start url: the link of ur project homepage: http://127.0.0.1:80/SomethingNew/

5.6. click ok

Reference:

http://stackoverflow.com/questions/9183179/phpstorm-xdebug-setup-walkthrough

Sunday, February 26, 2017

How to do a regular expression replace string in MySQL?

How to do a regular expression replace string in MySQL?

Install a MySQL user defined function called lib_mysqludf_preg from:

http://www.mysqludf.org/

https://github.com/mysqludf/lib_mysqludf_preg

For RedHat / CentOS:

# yum -y install pcre-devel gcc make automake mysql-devel

For Debian / Ubuntu:

# apt-get update
# apt-get install build-essential libmysqld-dev libpcre3-dev

Install lib_mysqludf_preg:

# git clone https://github.com/mysqludf/lib_mysqludf_preg.git
# ./configure
# make
# make install
# make installdb
# make test

Example:

SELECT CONVERT( PREG_REPLACE( '/fox/i' , 'dog' , 'The brown fox' ) USING UTF8) as replaced;

Saturday, February 25, 2017

To reset admin password:

Method 1 - reset admin password directly in database:

mysql> SET @salt = MD5(UNIX_TIMESTAMP());
mysql> UPDATE admin_user SET password = CONCAT(SHA2(CONCAT(@salt, 'MyNewPassword'), 256), ':', @salt, ':1') WHERE username = 'admin';

Method 2 - using PHP to generate the password. Then, reset it in database:

# php -r '$salt = md5(time()); echo hash("sha256", $salt . $argv[1]).":$salt:1\n";' MyNewPassword

66bdd4e5008cad465a6cd23eb6ac3aa6ef4c65d07a179157bab11935f9f4d62f:a4506164831ba6f12474a3ffe57602d0:1

mysql> UPDATE admin_user SET password = '<code above>' WHERE username='admin';

Method 3 - Generating the password. Then, reset it in database:

Add the following line at the last line and loot at the footer of any page.

# vim pub/index.php

echo \Magento\Framework\App\ObjectManager::getInstance()->get("\Magento\Framework\Encryption\Encryptor")->getHash("MyNewPassword");

Method 4 - Create a new admin user. Then, reset the previous admin password:

# php bin/magento admin:user:create --admin-user=admin2 --admin-password=MyNewPassword2 --admin-email=admin@example.com --admin-firstname=admin --admin-lastname=admin

Reference:

http://magento.stackexchange.com/questions/90922/how-to-reset-lost-admin-password-in-magento-2/161792#161792

Wednesday, February 22, 2017

To dump access database to MySQL sql file

To dump access database to MySQL sql file

1. Use Navicat

https://www.navicat.com/

2. Use Access To MySQL by bullzip

http://www.bullzip.com/products/a2m/info.php

Wednesday, February 15, 2017

View Google Chrome downloaded cached images and videos

View Google Chrome downloaded cached images and videos

Type: chrome://cache/

Print SQL query for the collection for debugging in Magento 2

Print SQL query for the collection for debugging in Magento 2

$productCollection->getSelect()->assemble();

Change Session timeout in Magento 2

Change Session timeout in Magento 2

System > Configuration > Advanced > Admin > Security > Session Lifetime (Seconds)

# vim php.ini

session.gc_maxlifetime = 36000

Tuesday, February 14, 2017

Get product tier pricing programmatically in Magento 2

Get product tier pricing programmatically in Magento 2

<?php
use \Magento\Framework\App\Bootstrap;

#require __DIR__ . '/../app/bootstrap.php';
require '/www/mag2.local/app/bootstrap.php';

$bootstrap = Bootstrap::create(BP, $_SERVER);
$objectManager = $bootstrap->getObjectManager();

### Setting area code
### NOTE: for more info http://devdocs.magento.com/guides/v2.1/architecture/archi_perspectives/components/modules/mod_and_areas.html
$state = $objectManager->get('\Magento\Framework\App\State');
#$state->setAreaCode('base');

$productId = 1;
$objectManager = \Magento\Framework\App\ObjectManager::getInstance();
$product_obj = $objectManager->create('\Magento\Catalog\Model\Product')->load($productId);

getDefaultGroup($product_obj);
getAnyGroup($product_obj);

function getDefaultGroup($product_obj) {
 $tier_price = $product_obj->getTierPrice();

 if(count($tier_price) > 0){
  echo "price_id\tprice_qty\tprice\twebsite_price\n";

  foreach($tier_price as $price){
   echo $price['price_id'];
   echo "\t";
   echo $price['price_qty'];
   echo "\t";
   echo $price['price'];
   echo "\t";
   echo $price['website_price'];
   echo "\n";
  }
 } else {
  echo 'There is no tiering price for the default group.' . PHP_EOL;
 }
}

function getAnyGroup($product_obj) {
 $tier_price = $product_obj->getTierPrices();

 if(count($tier_price) > 0){
  echo "price_qty\tprice\tCustomerGroupId\n";

  foreach($tier_price as $price){
   echo $price->getQty();
   echo "\t";
   echo $price->getValue();
   echo "\t";
   echo $price->getCustomerGroupId();
   echo "\t";
   echo "\n";
   print_r($price->getData());
   echo "\t";
   echo "\n";
  }
 }
}

#print_r($tier_price);
#print_r(get_class_methods($price));

Get customer information programmatically in Magento 2

Get customer information programmatically in Magento 2

<?php
use \Magento\Framework\App\Bootstrap;

#require __DIR__ . '/../app/bootstrap.php';
require '/www/mag2.local/app/bootstrap.php';

$bootstrap = Bootstrap::create(BP, $_SERVER);

$objectManager = $bootstrap->getObjectManager();

### Setting area code
### NOTE: for more info http://devdocs.magento.com/guides/v2.1/architecture/archi_perspectives/components/modules/mod_and_areas.html
$state = $objectManager->get('\Magento\Framework\App\State');
#$state->setAreaCode('base');

$storeManager = $objectManager->get('\Magento\Store\Model\StoreManagerInterface');
$storeId = $storeManager->getStore()->getId();

#$websiteId = $storeManager->getWebsite()->getWebsiteId();
$websiteId = $storeManager->getStore($storeId)->getWebsiteId();

// Customer Factory to Create Customer
$customerFactory = $objectManager->get('\Magento\Customer\Model\CustomerFactory');
$customer = $customerFactory->create();
$customer->setWebsiteId($websiteId);
$customer->loadByEmail('test@example.com');  

$data = $customer->getData(); 
print_r($data);

$defaultBilling = $customer->getDefaultBillingAddress();
$defaultShipping = $customer->getDefaultShippingAddress();

foreach ($customer->getAddresses() as $address) {
 echo 'IsDefaultBillingAddress: ' . ($defaultBilling && $defaultBilling->getId() == $address->getId() ? 'Yes' : 'No') . PHP_EOL;
 echo 'IsDefaultShippingAddress: ' . ($defaultShipping && $defaultShipping->getId() == $address->getId() ? 'Yes' : 'No') . PHP_EOL;

 echo 'ID: ' . $address->getId() . PHP_EOL;
 echo 'First Name: ' . $address->getFirstname() . PHP_EOL;
 echo 'Last Name: ' . $address->getLastname() . PHP_EOL;
 echo 'Street: ' . implode("\n", $address->getStreet()) . PHP_EOL;
 echo 'City: ' . $address->getCity() . PHP_EOL;
 echo 'Country: ' . $address->getCountry() . PHP_EOL;
 echo 'Region: ' . $address->getRegion() . PHP_EOL;
 echo 'Postal Code: ' . $address->getPostcode() . PHP_EOL;
 echo 'Phone: ' . $address->getTelephone() . PHP_EOL;
 echo PHP_EOL;

 #print_r(get_class_methods($address));
 #break;
}

Get product information programmatically in Magento 2

Get product information programmatically in Magento 2

<?php
use \Magento\Framework\App\Bootstrap;

#require __DIR__ . '/../app/bootstrap.php';
require '/www/mag2.local/app/bootstrap.php';

$bootstrap = Bootstrap::create(BP, $_SERVER);
$objectManager = $bootstrap->getObjectManager();

### Setting area code
### NOTE: for more info http://devdocs.magento.com/guides/v2.1/architecture/archi_perspectives/components/modules/mod_and_areas.html
$state = $objectManager->get('\Magento\Framework\App\State');
#$state->setAreaCode('base');

$storeManager = $objectManager->get('\Magento\Store\Model\StoreManagerInterface');
$storeId = $storeManager->getStore()->getId();

$productCollectionFactory = $objectManager->get('\Magento\Catalog\Model\ResourceModel\Product\CollectionFactory');
$productCollection = $productCollectionFactory->create();
$productCollection->addAttributeToSelect('*');

foreach ($productCollection as $product) {
 echo 'Id: ' . $product->getId() . PHP_EOL;
 echo 'Sku: ' . $product->getSku() . PHP_EOL;
 echo 'Price: ' . $product->getPrice() . PHP_EOL;
 echo 'Weight: ' . $product->getWeight() . PHP_EOL;
 print_r($product->getData());
 echo PHP_EOL;
}

Get store information programmatically in Magento 2

Get store information programmatically in Magento 2

<?php
use \Magento\Framework\App\Bootstrap;

#require __DIR__ . '/../app/bootstrap.php';
require '/www/mag2.local/app/bootstrap.php';

$bootstrap = Bootstrap::create(BP, $_SERVER);
$objectManager = $bootstrap->getObjectManager();

### Setting area code
### NOTE: for more info http://devdocs.magento.com/guides/v2.1/architecture/archi_perspectives/components/modules/mod_and_areas.html
$state = $objectManager->get('\Magento\Framework\App\State');
#$state->setAreaCode('base');

$storeManager = $objectManager->get('\Magento\Store\Model\StoreManagerInterface');
$storeId = $storeManager->getStore()->getId();

$baseURL = $storeManager->getStore($storeId)->getBaseUrl();
$mediaBaseURL = $storeManager->getStore($storeId)->getBaseUrl(\Magento\Framework\UrlInterface::URL_TYPE_MEDIA);
$linkBaseURL = $storeManager->getStore($storeId)->getBaseUrl(\Magento\Framework\UrlInterface::URL_TYPE_LINK);

$websiteId = $storeManager->getStore($storeId)->getWebsiteId();
$storeCode = $storeManager->getStore($storeId)->getCode();
$storeName = $storeManager->getStore($storeId)->getName();
$currentUrl = $storeManager->getStore($storeId)->getCurrentUrl();
$isActive = $storeManager->getStore($storeId)->isActive();
$isFrontUrlSecure = $storeManager->getStore($storeId)->isFrontUrlSecure();
$isCurrentlySecure = $storeManager->getStore($storeId)->isCurrentlySecure();

echo 'baseURL: ' . $baseURL . PHP_EOL;
echo 'mediaBaseURL: ' . $mediaBaseURL . PHP_EOL;
echo 'linkBaseURL: ' . $linkBaseURL . PHP_EOL;
echo 'websiteId: ' . $websiteId . PHP_EOL;
echo 'storeCode: ' . $storeCode . PHP_EOL;
echo 'storeName: ' . $storeName . PHP_EOL;
echo 'currentUrl: ' . $currentUrl . PHP_EOL;
echo 'isActive: ' . $isActive . PHP_EOL;
echo 'isFrontUrlSecure: ' . var_export($isFrontUrlSecure, true) . PHP_EOL;
echo 'isCurrentlySecure: ' . var_export($isCurrentlySecure, true) . PHP_EOL;