Simple life, Complicated mind

Saturday, November 18, 2017

Jet Profiler for MySQL

Jet Profiler for MySQL

is a real-time query performance and diagnostics tool for the MySQL database server. It's core features:

- Query, table and user performance
- Graphical visualisation
- Low overhead
- User friendly

https://www.jetprofiler.com/

Friday, November 17, 2017

Install MySQL 5.7, Apache 2.4, PHP 7.1 on Ubuntu 16.04

Install MySQL 5.7, Apache 2.4, PHP 7.1 on Ubuntu 16.04

Install VMware tools:

VM > Guest > Install/Upgrade VMware Tools

# su -
# df -h
# cd /media/jun/VMware\ Tools/
# ls -la
# tar zxvf VMwareTools-9.4.0-1280544.tar.gz -C /tmp/
# cd /tmp
# ls
# cd vmware-tools-distrib/
# ls
# ./vmware-install.pl -d
# reboot

Note: For more info https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1022525

Move Ubuntu launcher to the bottom:

# gsettings set com.canonical.Unity.Launcher launcher-position Bottom

Update the package repository:

# apt-get update

Upgrades packages with auto-handling of dependencies:

# apt-get dist-upgrade

or

# apt full-upgrade

Install SSH server:

# apt-get install openssh-server
# systemctl status sshd.service
# systemctl restart sshd.service

Compile and install the latest Git 2.14.2 from source code:

# apt-get install dh-autoreconf libcurl4-gnutls-dev libexpat1-dev gettext libz-dev libssl-dev
# apt-get install curl
# cd /usr/local/src/
# curl -L https://github.com/git/git/archive/v2.14.2.tar.gz -o git.tar.gz
# tar zxvf git.tar.gz
# cd git-2.14.2/
# make configure
# ./configure --prefix=/usr
# make all
# make install

# git --version

git version 2.14.2

Install Git from ppa:

# add-apt-repository ppa:git-core/ppa
# apt-get update

# apt-cache policy git
# apt-cache madison git

# apt-get install git=1:2.11.0-2~ppa0~ubuntu16.04.1

# git --version

Compile and install the latest Vim 8:

# apt-get install libncurses5-dev python-dev ruby-dev libperl-dev ruby-dev liblua5.3-dev exuberant-ctags cscope

// Fix liblua paths
# ln -s /usr/include/lua5.3 /usr/include/lua
# ln -s /usr/lib/x86_64-linux-gnu/liblua5.3.so /usr/local/lib/liblua.so

# cd /usr/local/src
# git clone https://github.com/vim/vim.git

# cd vim
# ./configure --prefix=/usr --with-features=huge --enable-multibyte --enable-pythoninterp --enable-rubyinterp --enable-perlinterp --enable-luainterp --enable-cscope
# make
# make install

# hash -r
# vim --version | head

Install MTA mail server:

# apt-get install postfix

Note: select "Internet site".

Note: If you need to reconfigure the postfix setting, run either one of the following:

# dpkg-reconfigure -plow postfix

or

# apt-get purge postfix

For other mail related packages:

# apt-get install mailutils

Install mail client:

# apt-get install bsd-mailx
# echo "test message" | mailx -s 'test subject' myemail@mydomain.com

For hexdump command:

# apt-get install bsdmainutils

# hexdump -c test.log

Install MySQL5.7:

# apt-cache policy mysql-server
# apt-cache search mysql-server
# apt-cache show mysql-server | less
# apt show mysql-server

# apt-get install mysql-server

# vim /etc/mysql/mysql.conf.d/mysqld.cnf
bind-address = 0.0.0.0

# vim ~/.my.cnf
[client]
host = localhost
port = 3306
user = root
password = MyPassword

# chmod 400 ~/.my.cnf

# mysql -e "SHOW variables WHERE variable_name REGEXP 'open_files_limit|table_open_cache|max_connections';"
+----------------------------+-------+
| Variable_name              | Value |
+----------------------------+-------+
| max_connections            | 151   |
| open_files_limit           | 1024  |
| table_open_cache           | 431   |
| table_open_cache_instances | 16    |
+----------------------------+-------+

Note: You will see the following error message in the error.log file if you did not change the open files limit:
[Warning] Changed limits: max_open_files: 1024 (requested 5000)
[Warning] Changed limits: table_open_cache: 431 (requested 2000)

# mkdir /etc/systemd/system/mysql.service.d
# vim /etc/systemd/system/mysql.service.d/override.conf

[Service]
#LimitNOFILE=infinity
LimitNOFILE=5000

#LimitMEMLOCK=infinity

# systemctl daemon-reload
# systemctl restart mysql

# mysql -e "SHOW variables WHERE variable_name REGEXP 'open_files_limit|table_open_cache|max_connections';"
+----------------------------+-------+
| Variable_name              | Value |
+----------------------------+-------+
| max_connections            | 151   |
| open_files_limit           | 5000  |
| table_open_cache           | 2000  |
| table_open_cache_instances | 16    |
+----------------------------+-------+

To check MySQL process's limit:

# cat /proc/$(pgrep mysqld$)/limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             15614                15614                processes
Max open files            5000                 5000                 files
Max locked memory         65536                65536                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       15614                15614                signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us

Note: https://dev.mysql.com/doc/refman/5.7/en/using-systemd.html

Note: https://stackoverflow.com/questions/30901041/can-not-increase-max-open-files-for-mysql-max-connections-in-ubuntu-15

Note: https://serverfault.com/questions/821695/mysqld-service-for-systemd-failed-to-parse-resource-value-ignoring-40000-l

To move a MySQL data directory to another directory:

# mysql -e "SELECT @@datadir;"
+-----------------+
| @@datadir       |
+-----------------+
| /var/lib/mysql/ |
+-----------------+

# systemctl stop mysql
# systemctl status mysql

# vim /etc/mysql/mysql.conf.d/mysqld.cnf
datadir         = /home/mysql

# vim /etc/apparmor.d/tunables/alias
alias /var/lib/mysql/ -> /home/mysql/,

Note: We need to tell AppArmor to let MySQL write to the new directory by creating an alias between the default directory and the new location.

Note: If you skipped the AppArmor configuration step, you would see the following error message:

Job for mysql.service failed because the control process 
exited with error code. See "systemctl status mysql.service" 
and "journalctl -xe" for details.

# systemctl restart apparmor
# systemctl restart mysql

To move the existing to MySQL directory to /home:

# rsync -av /var/lib/mysql /home

Or, you can run the following commands to initialize the MySQL data directory:

# mkdir /home/mysql \
&& chown mysql:mysql /home/mysql \
&& chmod 700 /home/mysql \
&& mysqld --initialize-insecure

Note: This option is used to initialize a MySQL installation by creating the data directory and populating the tables in the mysql system database.

Note: If you use --initialize, the random initial password is stored at: tail -n 1 /var/log/mysql/error.log.

Note: You can also start mysqld with --skip-grant-tables to access the database and change the password.

# systemctl start mysql && systemctl status mysql

Login MySQL with the above commands if you initialized MySQL data directory with --initialize-insecure option.
# mysql -u root --skip-password
mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY 'new_password';

Check the current MySQL data directory:

# mysql -e "SELECT @@datadir;"
+--------------+
| @@datadir    |
+--------------+
| /home/mysql/ |
+--------------+

To change the root password if you did not know the current root password:

# vim /root/tmp/mysql-init.txt
GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY 'YourPassWordHere' WITH GRANT OPTION;
GRANT SUPER ON *.* TO 'root'@'localhost';
FLUSH PRIVILEGES;

# mysqld --init-file=/root/tmp/mysql-init.txt

Use RAM-DISK for tmpdir:

# mysql -e "SHOW GLOBAL STATUS LIKE 'Created_tmp%tables';"
# mysql -e "SHOW GLOBAL VARIABLES LIKE '%table_size';"

# mkdir -p /mnt/ramdisk
# mount -t tmpfs -o size=512M tmpfs /mnt/ramdisk
# chown mysql:mysql /mnt/ramdisk

# id mysql

uid=123(mysql) gid=130(mysql) groups=130(mysql)

# vim /etc/fstab

tmpfs           /mnt/ramdisk     tmpfs   rw,uid=123,gid=130,mode=1770,size=512M    0       0

Note: You need to change the uid and gid of MySQL.

# mysql -e "SHOW GLOBAL VARIABLES LIKE 'tmpdir';"

# vim /etc/apparmor.d/local/usr.sbin.mysqld

/mnt/ramdisk rw,
owner /mnt/ramdisk/** rwkl,

Note: The first line gives read and write access to the directory, the second line gives read, write, lock(k) and link(l) access to all the files and the directories inside the directory owned by the mysql user.

# vim /etc/mysql/mysql.conf.d/mysqld.cnf

[mysqld]
tmpdir      = /mnt/ramdisk

# systemctl restart apparmor.service
# systemctl restart mysql.service

Install PHP7.0:

# command -v add-apt-repository >/dev/null 2>&1 \
|| { echo >&2 "add-apt-repository is not installed. I will install it for you"; apt-get install python-software-properties; }

# add-apt-repository -y ppa:ondrej/php
# apt-get update

# apt-cache policy php7.1

# apt-get install php7.1-fpm
# apt-get install php7.1-xml php7.1-curl php7.1-zip php7.1-gd php7.1-bcmath php7.1-intl php7.1-mbstring php7.1-mcrypt php7.1-mysql
# apt-get install php7.1-json php7.1-opcache
# apt-get install php-xdebug

# php -v
PHP 7.1.10-1+ubuntu16.04.1+deb.sury.org+1 (cli) (built: Sep 29 2017 17:04:25) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies
    with Zend OPcache v7.1.10-1+ubuntu16.04.1+deb.sury.org+1, Copyright (c) 1999-2017, by Zend Technologies
    with Xdebug v2.5.5, Copyright (c) 2002-2017, by Derick Rethans

# vim /etc/php/7.1/fpm/pool.d/www.conf

;listen = /run/php/php7.1-fpm.sock
listen = 127.0.0.1:9000

Note: You can choose to use either a Unix socket (for local access only) or TCP socket (for the other server on the network to access).

# systemctl restart php7.1-fpm.service && systemctl status php7.1-fpm.service

# ss -an | grep :9000
tcp    LISTEN     0      128    127.0.0.1:9000                  *:*

# vim /etc/php/7.1/fpm/php.ini

date.timezone = America/Vancouver
display_errors = On
display_startup_errors = On
error_reporting = E_ALL
error_log = /var/log/php_errors.log

List all the installed PHP packages:

# dpkg -l | grep php| awk '{print $2}' |tr "\n" " "

Show the available package version:

# apt-cache search php
# apt-cache policy php

Install the specific package version:

# apt-get install php7=7.0+35ubuntu6

Note: You can look up old versions of packages at their site http://www.debian.org/distrib/packages

Install older version of PHP (PHP5.6):

# add-apt-repository ppa:ondrej/php

# apt-get update

# apt-get install php5.6-fpm

# apt-get install php5.6-gd php5.6-intl php5.6-json php5.6-mbstring php5.6-mcrypt php5.6-mysql php5.6-opcache php5.6-xml

# a2disconf php7.0-fpm.conf
# a2enconf php5.6-fpm.conf

# systemctl restart apache2.service

Install Apache2.4:

# command -v add-apt-repository >/dev/null 2>&1 \
|| { echo >&2 "add-apt-repository is not installed. I will install it for you"; apt-get install python-software-properties; }

# add-apt-repository -y ppa:ondrej/apache2
# apt-get update

# apt-cache policy apache2

# apt-get install apache2

Enable the following modules to talk to PHP:

# cat /etc/apache2/conf-available/php7.1-fpm.conf

# a2enmod proxy proxy_fcgi rewrite setenvif ssl
# a2enconf php7.1-fpm.conf

If your apache is talking to PHP through a TCP socket (127.0.0.1:9000) instead of a Unix socket (/run/php/php7.1-fpm.sock), you will need to modify the following line:

# vim /etc/apache2/conf-available/php7.1-fpm.conf
    <FilesMatch ".+\.ph(ar|p|tml)$">
        #SetHandler "proxy:unix:/run/php/php7.1-fpm.sock|fcgi://localhost"
        SetHandler "proxy:fcgi://127.0.0.1:9000"
    </FilesMatch>

# apache2ctl configtest
# systemctl restart apache2 && systemctl status apache2

Install and enable the following Apache modules if you are connecting to PHP through a TCP socket (127.0.0.1:9000):

# apt-get install libapache2-mod-fastcgi
# a2enmod fastcgi rewrite setenvif


Edit apache2.conf:

# vim /etc/apache2/apache2.conf

AllowOverride All

Set up a virtual host:

# cd /etc/apache2/sites-available
# cp 000-default.conf mag2.local.conf
# vim mag2.local

Check the configuration:

# apache2ctl -V
# apache2ctl -t
# apache2ctl -M
# apache2ctl configtest

Enable the site:

# a2ensite mag2.local

Start MySQL, PHP, and Apache:

# systemctl restart mysql.service
# systemctl restart php7.0-fpm.service
# systemctl restart apache2.service

# ps auxww | grep -i mysql
# ps auxww | grep -i php-fpm
# ps auxww | grep -i apache2

Install PHPStorm:

# cd ~jun/Downloads/
# tar xf PhpStorm-*.tar.gz -C /opt/
# cd /opt/PhpStorm-163.10504.2/
# ./bin/phpstorm.sh

Generate a self-signed SSL certificate:

# openssl req -x509 -nodes -days 365 -newkey rsa:2048 -subj "/C=CA/ST=British Columbia/L=Vancouver/O=My Company Name/CN=erp.local" -keyout /etc/ssl/private/test.local.key -out /etc/ssl/certs/test.local.crt

Reference:

https://www.digitalocean.com/community/tutorials/how-to-move-a-mysql-data-directory-to-a-new-location-on-ubuntu-16-04

http://www.fromdual.com/mysql-tmpdir-on-ram-disk

http://www.victordodon.com/changing-mysql-tmpdir-in-ubuntu/

Saturday, November 11, 2017

Theme the node edit form of the custom content type in Drupal 8

Theme the node edit form of the custom content type in Drupal 8

/**
* Implements hook_theme_suggestions_alter().
*/
function MyModuleName_theme_suggestions_alter(array &$suggestions, array $variables, $hook) {
  if ($hook == 'node_edit_form') {
    if ($node = \Drupal::routeMatch()->getParameter('node')) {
      $content_type = $node->bundle();
    } else {
      $current_path = \Drupal::service('path.current')->getPath();
      $path_args = explode('/', $current_path);
      $content_type = $path_args[3];
    }
    $suggestions[] = 'node_edit_form__' . $content_type; // Note: You can also specify a custom theme ID here. See hook_theme() below.
  }
}

The following code (hook_theme) is optional:

/**
 * Implements  hook_theme($existing, $type, $theme, $path)
 */
function mydemo_theme() {
  return [
    'my_custom_theme_id' => [
      'render element' => 'form',
      #'path' => '/var/www/html/drupal8/web/modules/custom/mydemo/templates',
      'template' => 'asdf222',
    ],
  ];
}

Next, create twig templates in your theme's template directory in the form of node-edit-form--NODE-TYPE-SEPARATED-WITH-DASHES.html.twig:


<b>Test</b>

{{ form.form_id }}
{{ form.form_token }}

{{ form }}

Reference:

https://www.drupal.org/forum/support/post-installation/2015-10-31/drupal-8-node-edit-template

https://api.drupal.org/api/drupal/core%21lib%21Drupal%21Core%21Render%21theme.api.php/function/hook_theme_suggestions_HOOK_alter/8.2.x

https://drupal.stackexchange.com/questions/200602/override-theme-template-from-module-without-implementing-a-theme

Sunday, October 15, 2017

Saturday, October 14, 2017

MySQL resolve Host name to IP Address or Vice Versa

MySQL resolve Host name to IP Address or Vice Versa

# resolveip 192.168.5.11

# resolveip mysql-slave

Reference:

https://dev.mysql.com/doc/refman/5.7/en/resolveip.html