tag:blogger.com,1999:blog-23618009749995100682024-03-13T14:29:56.101-07:00Simple life, Complicated mindJun Hsiehhttp://www.blogger.com/profile/00640061359079216681noreply@blogger.comBlogger2678125tag:blogger.com,1999:blog-2361800974999510068.post-91968330770266406992019-06-03T21:59:00.000-07:002019-06-03T21:59:16.693-07:00Setup A Centralized Log Server Using Rsyslog on Ubuntu 16.04 LTSSetup A Centralized Log Server Using Rsyslog on Ubuntu 16.04 LTS<br />
<br />
<b>Rsyslog Server:</b><br />
<br />
# vim /etc/rsyslog.conf<br />
<br />
<pre><code class="html"># provides UDP syslog reception
module(load="imudp")
input(type="imudp" port="514")
# provides TCP syslog reception
module(load="imtcp")
input(type="imtcp" port="514")</code></pre><br />
# vim /etc/rsyslog.d/tmpl.conf<br />
<br />
<pre><code class="html">$template TmplAuth, "/var/log/client_logs/%HOSTNAME%/%PROGRAMNAME%.log"
$template TmplMsg, "/var/log/client_logs/%HOSTNAME%/%PROGRAMNAME%.log"
authpriv.* ?TmplAuth
*.info;mail.none;authpriv.none;cron.none ?TmplMsg</code></pre><br />
# systemctl restart rsyslog<br />
<br />
<b>Rsyslog Client:</b><br />
<br />
# vim /etc/rsyslog.conf<br />
<br />
<pre><code class="html">##RULES##
*.* @192.168.1.200:514</code></pre><br />
<b>Note:</b> The @ symbol before the IP address tells rsyslog to use UDP to send the messages. Change this to @@ to use TCP.<br />
<br />
# systemctl restart rsyslog<br />
<br />
# logger -s " This is my Rsyslog client "<br />
<br />
# logger --server 127.0.0.1 --port 9000 --udp --rfc3164 "my testing msg"<br />
<br />
# tree /var/log/client_logs/<br />
<br />
<b>The following is a list of RFCs that define the Syslog protocol:</b><br />
<br />
<a href="http://tools.ietf.org/html/rfc3195">RFC 3195</a> Reliable Delivery for syslog<br />
<a href="http://tools.ietf.org/html/rfc5424">RFC 5424</a> The Syslog Protocol<br />
<a href="http://tools.ietf.org/html/rfc5425">RFC 5425</a> TLS Transport Mapping for Syslog<br />
<a href="http://tools.ietf.org/html/rfc5426">RFC 5426</a> Transmission of Syslog Messages over UDP<br />
<a href="http://tools.ietf.org/html/rfc5427">RFC 5427</a> Textual Conventions for Syslog Management<br />
<a href="http://tools.ietf.org/html/rfc5848">RFC 5848</a> Signed Syslog Messages<br />
<a href="http://tools.ietf.org/html/rfc6012">RFC 6012</a> Datagram Transport Layer Security (DTLS) Transport Mapping for Syslog<br />
<br />
<b>Reference:</b> <br />
<br />
<a href="http://yallalabs.com/linux/how-to-setup-a-centralized-log-server-using-rsyslog-on-ubuntu-16-04-lts/" target="_blank">http://yallalabs.com/linux/how-to-setup-a-centralized-log-server-using-rsyslog-on-ubuntu-16-04-lts/</a><br />
<br />
<a href="https://success.trendmicro.com/solution/TP000086250-What-are-Syslog-Facilities-and-Levels" target="_blank">https://success.trendmicro.com/solution/TP000086250-What-are-Syslog-Facilities-and-Levels</a><br />
<br />
<a href="https://en.wikipedia.org/wiki/Syslog" target="_blank">https://en.wikipedia.org/wiki/Syslog</a><br />
<br />
<a href="https://www.elastic.co/blog/how-to-centralize-logs-with-rsyslog-logstash-and-elasticsearch-on-ubuntu-14-04" target="_blank">https://www.elastic.co/blog/how-to-centralize-logs-with-rsyslog-logstash-and-elasticsearch-on-ubuntu-14-04</a><br />
<br />
<a href="https://www.elastic.co/guide/en/beats/filebeat/master/filebeat-input-syslog.html" target="_blank">https://www.elastic.co/guide/en/beats/filebeat/master/filebeat-input-syslog.html</a>Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-63160027787190607232019-04-09T20:00:00.001-07:002019-04-09T20:00:07.053-07:00Web Application Vulnerability Scanning ToolsWeb Application Vulnerability Scanning Tools<br />
<br />
Nessus<br />
<a href="http://www.tenable.com/products/nessus" target="_blank">http://www.tenable.com/products/nessus</a><br />
<br />
PortSwigger<br />
<a href="https://portswigger.net/" target="_blank">https://portswigger.net/</a><br />
<br />
qualys<br />
<a href="https://www.qualys.com/" target="_blank">https://www.qualys.com/</a><br />
<br />
Scan Titan<br />
<a href="https://www.scantitan.com/" target="_blank">https://www.scantitan.com/</a><br />
<br />
Nikto<br />
<a href="https://cirt.net/Nikto2" target="_blank">https://cirt.net/Nikto2</a><br />
<br />
CIS (Center for Internet Security) Security benchmark<br />
<a href="https://www.cisecurity.org/cis-benchmarks/" target="_blank">https://www.cisecurity.org/cis-benchmarks/</a><br />
<br />
Vulnerability Scanning Tools<br />
<a href="https://www.owasp.org/index.php/Category:Vulnerability_Scanning_Tools" target="_blank">https://www.owasp.org/index.php/Category:Vulnerability_Scanning_Tools</a>Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-55128473588742104282019-03-18T21:55:00.000-07:002019-03-18T21:55:17.065-07:00Use Mosh instead of SSHMosh is a replacement for SSH. It's more robust and responsive, especially over Wi-Fi WiFi, cellular, and long-distance links.<br />
<br />
<b>On Ubuntu:</b><br />
<br />
# add-apt-repository ppa:keithw/mosh<br />
# apt-get update && apt-get install mosh<br />
<br />
<b>On CentOS:</b><br />
<br />
# yum -y update<br />
<br />
# yum install epel-release<br />
<br />
# yum install mosh<br />
<br />
# firewall-cmd --permanent --zone=public --add-port=60000-60020/udp<br />
<br />
<b>Note:</b> If you are using any other program to manage your firewall, then you will need to manually ensure to open the UDP ports from 60000 to 61000. However, if you only expect to have a small number of concurrent connections, then a smaller range of ports can be opened provided it begins at port 60000 (e.g 60000:60020).<br />
<br />
<b>Note:</b> If you are using Amazon EC2, you will need to open the ports in Amazon's security group (firewall).<br />
<br />
# firewall-cmd --reload<br />
<br />
# firewall-cmd --list-all<br />
<br />
<pre><code class="html">public (default)
interfaces:
sources:
services: dhcpv6-client ssh
ports: 80/tcp 60000-60020/udp
masquerade: no
forward-ports:
icmp-blocks:
rich rules:</code></pre><br />
In most use cases, Mosh is a drop-in replacement for SSH, meaning many SSH commands need only a simple alteration. For example:<br />
<br />
client # mosh user@example.com<br />
<br />
However, if you use any other arguments with SSH (such as -p), then a slightly different syntax is needed:<br />
<br />
client # mosh --ssh="ssh -i myserver.pem -p 22" centos@1.2.3.4<br />
<br />
<b>Note:</b> if you see the "It is required that your private key files are NOT accessible by others." error message, run chmod 400 myserver.pem.<br />
<br />
<b>Note:</b> you do not need to start mosh-server manually, mosh-server will start automatically once it's connected.<br />
<br />
Use nc command to test the udp connection:<br />
<br />
client # nc -u 1.2.3.4 60000-60020<br />
<br />
<b>Note:</b> to install nc, run yum install nmap-ncat.<br />
<br />
server # ps auxww|grep -i mosh<br />
<br />
<pre><code class="html">centos 21000 1.0 0.0 171784 5748 ? S 05:15 0:00 mosh-server new -s -c 256 -l LANG=en_US.UTF-8 -l LANGUAGE=en_US.UTF-8 -l LC_ALL=en_US.UTF-8</code></pre><br />
server # ss -lnu | grep :6000<br />
<br />
<pre><code class="html">tcp UNCONN 0 0 172.31.28.16:60001 *:*</code></pre><br />
<b>Reference:</b> <br />
<br />
<a href="https://www.digitalocean.com/community/tutorials/how-to-install-and-use-mosh-on-a-vps" target="_blank">https://www.digitalocean.com/community/tutorials/how-to-install-and-use-mosh-on-a-vps</a><br />
<a href="https://mosh.mit.edu/" target="_blank">https://mosh.mit.edu/</a>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-52331993820110524872019-03-17T11:48:00.000-07:002019-03-17T11:48:47.474-07:00Install MySQL 5.7, Apache 2.4, PHP 7.1 on Ubuntu 16.04Install MySQL 5.7, Apache 2.4, PHP 7.1 on Ubuntu 16.04<br />
<br />
<b>Install VMware tools:</b><br />
<br />
VM > Guest > Install/Upgrade VMware Tools<br />
<br />
# su -<br />
# df -h<br />
# cd /media/jun/VMware\ Tools/<br />
# ls -la<br />
# tar zxvf VMwareTools-9.4.0-1280544.tar.gz -C /tmp/<br />
# cd /tmp<br />
# ls<br />
# cd vmware-tools-distrib/<br />
# ls<br />
# ./vmware-install.pl -d<br />
# reboot<br />
<br />
<b>Note:</b> For more info <a href="https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1022525" target="_blank">https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1022525</a><br />
<br />
<b>Move Ubuntu launcher to the bottom:</b><br />
<br />
# gsettings set com.canonical.Unity.Launcher launcher-position Bottom<br />
<br />
<b>Update the package repository:</b><br />
<br />
# apt-get update<br />
<br />
<b>Upgrades packages with auto-handling of dependencies:</b><br />
<br />
# apt-get dist-upgrade<br />
<br />
or<br />
<br />
# apt full-upgrade<br />
<br />
<b>Install SSH server:</b><br />
<br />
# apt-get install openssh-server<br />
# systemctl status sshd.service<br />
# systemctl restart sshd.service<br />
<br />
<b>Compile and install the latest Git 2.20.1 from source code:</b><br />
<br />
# apt-get install make gcc dh-autoreconf libcurl4-gnutls-dev libexpat1-dev gettext zlib1g-dev libssl-dev \<br />
&& apt-get install curl \<br />
&& cd /usr/local/src/ \<br />
&& curl -L https://github.com/git/git/archive/v2.20.1.tar.gz -o git.tar.gz \<br />
&& tar zxvf git.tar.gz \<br />
&& cd git-2.20.1/ \<br />
&& make configure \<br />
&& ./configure --prefix=/usr \<br />
&& make all \<br />
&& make install<br />
<br />
# git --version<br />
<br />
<pre><code class="html">git version 2.20.1</code></pre><br />
<b>Install Git from ppa:</b><br />
<br />
# add-apt-repository ppa:git-core/ppa<br />
# apt-get update<br />
<br />
# apt-cache policy git<br />
# apt-cache madison git<br />
<br />
# apt-get install git=1:2.11.0-2~ppa0~ubuntu16.04.1<br />
<br />
# git --version<br />
<br />
<b>Compile and install the latest Vim 8:</b><br />
<br />
# apt-get install libncurses5-dev python-dev python3-dev ruby-dev libperl-dev ruby-dev liblua5.3-dev exuberant-ctags cscope<br />
<br />
// Fix liblua paths<br />
# ln -s /usr/include/lua5.3 /usr/include/lua \<br />
&& ln -s /usr/lib/x86_64-linux-gnu/liblua5.3.so /usr/local/lib/liblua.so<br />
<br />
# cd /usr/local/src \<br />
&& git clone https://github.com/vim/vim.git --depth 1 \<br />
&& cd vim \<br />
&& ./configure \<br />
--prefix=/usr \<br />
--with-features=huge \<br />
--enable-multibyte \<br />
--enable-pythoninterp \<br />
--enable-python3interp \<br />
--enable-rubyinterp \<br />
--enable-perlinterp \<br />
--enable-luainterp \<br />
--enable-cscope \<br />
&& make \<br />
&& make install \<br />
&& hash -r \<br />
&& vim --version | head<br />
<br />
<b>Install MTA mail server:</b><br />
<br />
# apt-get install postfix<br />
<br />
<b>Note:</b> select "Internet site".<br />
<br />
<b>Note:</b> If you need to reconfigure the postfix setting, run either one of the following:<br />
<br />
# dpkg-reconfigure -plow postfix<br />
<br />
or<br />
<br />
# apt-get purge postfix<br />
<br />
<b>For other mail related packages:</b><br />
<br />
# apt-get install mailutils<br />
<br />
<b>Install mail client:</b><br />
<br />
# apt-get install bsd-mailx<br />
# echo "test message" | mailx -s 'test subject' myemail@mydomain.com<br />
<br />
<b>For hexdump command:</b><br />
<br />
# apt-get install bsdmainutils<br />
<br />
# hexdump -c test.log<br />
<br />
<b>Install MySQL5.7:</b><br />
<br />
# apt-cache policy mysql-server<br />
# apt-cache search mysql-server<br />
# apt-cache show mysql-server | less<br />
# apt show mysql-server<br />
<br />
# apt-get install mysql-server<br />
<br />
# vim /etc/mysql/mysql.conf.d/mysqld.cnf<br />
<pre><code class="html">bind-address = 0.0.0.0</code></pre><br />
# vim ~/.my.cnf<br />
<pre><code class="html">[client]
host = localhost
port = 3306
user = root
password = MyPassword</code></pre><br />
# chmod 400 ~/.my.cnf<br />
<br />
# mysql -e "SHOW variables WHERE variable_name REGEXP 'open_files_limit|table_open_cache|max_connections';"<br />
<pre><code class="html">+----------------------------+-------+
| Variable_name | Value |
+----------------------------+-------+
| max_connections | 151 |
| open_files_limit | 1024 |
| table_open_cache | 431 |
| table_open_cache_instances | 16 |
+----------------------------+-------+</code></pre><br />
<b>Note:</b> You will see the following error message in the error.log file if you did not change the open files limit:<br />
<pre><code class="html">[Warning] Changed limits: max_open_files: 1024 (requested 5000)
[Warning] Changed limits: table_open_cache: 431 (requested 2000)</code></pre><br />
# mkdir /etc/systemd/system/mysql.service.d<br />
# vim /etc/systemd/system/mysql.service.d/override.conf<br />
<br />
<pre><code class="html">[Service]
#LimitNOFILE=infinity
LimitNOFILE=5000
#LimitMEMLOCK=infinity</code></pre><br />
# systemctl daemon-reload<br />
# systemctl restart mysql<br />
<br />
# mysql -e "SHOW variables WHERE variable_name REGEXP 'open_files_limit|table_open_cache|max_connections';"<br />
<pre><code class="html">+----------------------------+-------+
| Variable_name | Value |
+----------------------------+-------+
| max_connections | 151 |
| open_files_limit | 5000 |
| table_open_cache | 2000 |
| table_open_cache_instances | 16 |
+----------------------------+-------+</code></pre><br />
<b>To check MySQL process's limit:</b><br />
<br />
# cat /proc/$(pgrep mysqld$)/limits<br />
<pre><code class="html">Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 15614 15614 processes
Max open files 5000 5000 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 15614 15614 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us</code></pre><br />
<b>Note:</b> <a href="https://dev.mysql.com/doc/refman/5.7/en/using-systemd.html" target="_blank">https://dev.mysql.com/doc/refman/5.7/en/using-systemd.html</a><br />
<br />
<b>Note:</b> <a href="https://stackoverflow.com/questions/30901041/can-not-increase-max-open-files-for-mysql-max-connections-in-ubuntu-15" target="_blank">https://stackoverflow.com/questions/30901041/can-not-increase-max-open-files-for-mysql-max-connections-in-ubuntu-15</a><br />
<br />
<b>Note:</b> <a href="https://serverfault.com/questions/821695/mysqld-service-for-systemd-failed-to-parse-resource-value-ignoring-40000-l" target="_blank">https://serverfault.com/questions/821695/mysqld-service-for-systemd-failed-to-parse-resource-value-ignoring-40000-l</a><br />
<br />
<b>To move a MySQL data directory to another directory:</b><br />
<br />
# mysql -e "SELECT @@datadir;"<br />
<pre><code class="html">+-----------------+
| @@datadir |
+-----------------+
| /var/lib/mysql/ |
+-----------------+</code></pre><br />
# systemctl stop mysql<br />
# systemctl status mysql<br />
<br />
# vim /etc/mysql/mysql.conf.d/mysqld.cnf<br />
<pre><code class="html">datadir = /home/mysql</code></pre><br />
# vim /etc/apparmor.d/tunables/alias<br />
<pre><code class="html">alias /var/lib/mysql/ -> /home/mysql/,</code></pre><br />
<b>Note:</b> We need to tell AppArmor to let MySQL write to the new directory by creating an alias between the default directory and the new location.<br />
<br />
<b>Note:</b> If you skipped the AppArmor configuration step, you would see the following error message:<br />
<br />
<pre><code class="html">Job for mysql.service failed because the control process
exited with error code. See "systemctl status mysql.service"
and "journalctl -xe" for details.</code></pre><br />
# systemctl restart apparmor<br />
# systemctl restart mysql<br />
<br />
<b>To move the existing to MySQL directory to /home:</b><br />
<br />
# rsync -av /var/lib/mysql /home<br />
<br />
Or, you can run the following commands to initialize the MySQL data directory:<br />
<br />
# mkdir /home/mysql \<br />
&& chown mysql:mysql /home/mysql \<br />
&& chmod 700 /home/mysql \<br />
&& mysqld --initialize-insecure<br />
<br />
<b>Note:</b> This option is used to initialize a MySQL installation by creating the data directory and populating the tables in the mysql system database.<br />
<br />
<b>Note:</b> If you use --initialize, the random initial password is stored at: tail -n 1 /var/log/mysql/error.log.<br />
<br />
<b>Note:</b> You can also start mysqld with --skip-grant-tables to access the database and change the password.<br />
<br />
# systemctl start mysql && systemctl status mysql<br />
<br />
Login MySQL with the above commands if you initialized MySQL data directory with --initialize-insecure option.<br />
# mysql -u root --skip-password<br />
mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY 'new_password';<br />
<br />
<b>Check the current MySQL data directory:</b><br />
<br />
# mysql -e "SELECT @@datadir;"<br />
<pre><code class="html">+--------------+
| @@datadir |
+--------------+
| /home/mysql/ |
+--------------+</code></pre><br />
<b>To change the root password if you did not know the current root password:</b><br />
<br />
# vim /root/tmp/mysql-init.txt<br />
<pre><code class="html">GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' IDENTIFIED BY 'YourPassWordHere' WITH GRANT OPTION;
GRANT SUPER ON *.* TO 'root'@'localhost';
FLUSH PRIVILEGES;</code></pre><br />
# mysqld --init-file=/root/tmp/mysql-init.txt<br />
<br />
<b>Use RAM-DISK for tmpdir:</b><br />
<br />
# mysql -e "SHOW GLOBAL STATUS LIKE 'Created_tmp%tables';"<br />
# mysql -e "SHOW GLOBAL VARIABLES LIKE '%table_size';"<br />
<br />
# mkdir -p /mnt/ramdisk<br />
# mount -t tmpfs -o size=512M tmpfs /mnt/ramdisk<br />
# chown mysql:mysql /mnt/ramdisk<br />
<br />
# id mysql<br />
<br />
<pre><code class="html">uid=123(mysql) gid=130(mysql) groups=130(mysql)</code></pre><br />
# vim /etc/fstab<br />
<br />
<pre><code class="html">tmpfs /mnt/ramdisk tmpfs rw,uid=123,gid=130,mode=1770,size=512M 0 0</code></pre><br />
<b>Note:</b> You need to change the uid and gid of MySQL.<br />
<br />
# mysql -e "SHOW GLOBAL VARIABLES LIKE 'tmpdir';"<br />
<br />
# vim /etc/apparmor.d/local/usr.sbin.mysqld<br />
<br />
<pre><code class="html">/mnt/ramdisk rw,
owner /mnt/ramdisk/** rwkl,</code></pre><br />
<b>Note:</b> The first line gives read and write access to the directory, the second line gives read, write, lock(k) and link(l) access to all the files and the directories inside the directory owned by the mysql user. <br />
<br />
# vim /etc/mysql/mysql.conf.d/mysqld.cnf<br />
<br />
<pre><code class="html">[mysqld]
tmpdir = /mnt/ramdisk</code></pre><br />
# systemctl restart apparmor.service<br />
# systemctl restart mysql.service<br />
<br />
<b>Install PHP7.1:</b><br />
<br />
# command -v add-apt-repository >/dev/null 2>&1 \<br />
|| { echo >&2 "add-apt-repository is not installed. I will install it for you"; apt-get install python-software-properties; }<br />
<br />
# add-apt-repository -y ppa:ondrej/php<br />
# apt-get update<br />
<br />
# apt-cache policy php7.1<br />
<br />
# apt-get install php7.1-fpm<br />
# apt-get install php7.1-xml php7.1-curl php7.1-zip php7.1-gd php7.1-bcmath php7.1-intl php7.1-mbstring php7.1-mcrypt php7.1-mysql<br />
# apt-get install php7.1-json php7.1-opcache<br />
# apt-get install php-xdebug<br />
<br />
# php -v<br />
<pre><code class="html">PHP 7.1.10-1+ubuntu16.04.1+deb.sury.org+1 (cli) (built: Sep 29 2017 17:04:25) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies
with Zend OPcache v7.1.10-1+ubuntu16.04.1+deb.sury.org+1, Copyright (c) 1999-2017, by Zend Technologies
with Xdebug v2.5.5, Copyright (c) 2002-2017, by Derick Rethans</code></pre><br />
# vim /etc/php/7.1/fpm/pool.d/www.conf<br />
<br />
;listen = /run/php/php7.1-fpm.sock<br />
listen = 127.0.0.1:9000<br />
<br />
<b>Note:</b> You can choose to use either a Unix socket (for local access only) or TCP socket (for the other server on the network to access).<br />
<br />
<b>Set up xdebug:</b><br />
<br />
# vim /etc/php/7.1/fpm/php.ini<br />
<br />
<pre><code class="html">[Xdebug]
xdebug.default_enable=1
xdebug.remote_enable=1
xdebug.remote_handler=dbgp
xdebug.remote_host=localhost
xdebug.remote_port=9009
xdebug.remote_log=/tmp/xdebug.log
xdebug.remote_connect_back=0
xdebug.remote_autostart=1
xdebug.remote_mode=req
xdebug.max_nesting_level=1000
xdebug.var_display_max_depth = 5
xdebug.var_display_max_children = 256
xdebug.var_display_max_data = 1024</code></pre><br />
# systemctl restart php7.1-fpm.service && systemctl status php7.1-fpm.service<br />
<br />
# ss -an | grep :9000<br />
<pre><code class="html">tcp LISTEN 0 128 127.0.0.1:9000 *:*</code></pre><br />
# vim /etc/php/7.1/fpm/php.ini<br />
<br />
<pre><code class="html">date.timezone = America/Vancouver
display_errors = On
display_startup_errors = On
error_reporting = E_ALL
error_log = /var/log/php_errors.log</code></pre><br />
<b>List all the installed PHP packages:</b><br />
<br />
# dpkg -l | grep php| awk '{print $2}' |tr "\n" " "<br />
<br />
<b>Show the available package version:</b><br />
<br />
# apt-cache search php<br />
# apt-cache policy php<br />
<br />
<b>Install the specific package version:</b><br />
<br />
# apt-get install php7=7.0+35ubuntu6<br />
<br />
<b>Note:</b> You can look up old versions of packages at their site http://www.debian.org/distrib/packages<br />
<br />
<b>Install older version of PHP (PHP5.6):</b><br />
<br />
# add-apt-repository ppa:ondrej/php<br />
<br />
# apt-get update<br />
<br />
# apt-get install php5.6-fpm<br />
<br />
# apt-get install php5.6-gd php5.6-intl php5.6-json php5.6-mbstring php5.6-mcrypt php5.6-mysql php5.6-opcache php5.6-xml<br />
<br />
# a2disconf php7.0-fpm.conf<br />
# a2enconf php5.6-fpm.conf<br />
<br />
# systemctl restart apache2.service<br />
<br />
<b>Install Apache2.4:</b><br />
<br />
# command -v add-apt-repository >/dev/null 2>&1 \<br />
|| { echo >&2 "add-apt-repository is not installed. I will install it for you"; apt-get install python-software-properties; }<br />
<br />
# add-apt-repository -y ppa:ondrej/apache2<br />
# apt-get update<br />
<br />
# apt-cache policy apache2<br />
<br />
# apt-get install apache2<br />
<br />
<b>Enable the following modules to talk to PHP:</b><br />
<br />
# cat /etc/apache2/conf-available/php7.1-fpm.conf<br />
<br />
# a2enmod proxy proxy_fcgi rewrite setenvif ssl<br />
# a2enconf php7.1-fpm.conf<br />
<br />
<b>If your apache is talking to PHP through a TCP socket (127.0.0.1:9000) instead of a Unix socket (/run/php/php7.1-fpm.sock), you will need to modify the following line:</b><br />
<br />
# vim /etc/apache2/conf-available/php7.1-fpm.conf<br />
<pre><code class="html"># Define a matching worker.
# The part that is matched to the SetHandler is the part that
# follows the pipe. If you need to distinguish, "localhost; can
# be anything unique.
<Proxy "fcgi://localhost/" enablereuse=on max=10>
</Proxy>
<FilesMatch ".+\.ph(ar|p|tml)$">
#SetHandler "proxy:unix:/run/php/php7.1-fpm.sock|fcgi://localhost"
SetHandler "proxy:fcgi://127.0.0.1:9000"
</FilesMatch></code></pre><br />
# apache2ctl configtest<br />
# systemctl restart apache2 && systemctl status apache2<br />
<br />
<span style="text-decoration: line-through;"><b>Install and enable the following Apache modules if you are connecting to PHP through a TCP socket (127.0.0.1:9000):</b><br />
<br />
# apt-get install libapache2-mod-fastcgi<br />
# a2enmod fastcgi rewrite setenvif</span><br />
<br />
<b>Edit apache2.conf:</b><br />
<br />
# vim /etc/apache2/apache2.conf<br />
<br />
<pre><code class="html">AllowOverride All</code></pre><br />
<b>Set up a virtual host:</b><br />
<br />
# cd /etc/apache2/sites-available<br />
# cp 000-default.conf mag2.local.conf<br />
# vim mag2.local<br />
<br />
<b>Check the configuration:</b><br />
<br />
# apache2ctl -V<br />
# apache2ctl -t<br />
# apache2ctl -M<br />
# apache2ctl configtest<br />
<br />
<b>Enable the site:</b><br />
<br />
# a2ensite mag2.local<br />
<br />
<b>Start MySQL, PHP, and Apache:</b><br />
<br />
# systemctl restart mysql.service<br />
# systemctl restart php7.0-fpm.service<br />
# systemctl restart apache2.service<br />
<br />
# ps auxww | grep -i mysql<br />
# ps auxww | grep -i php-fpm<br />
# ps auxww | grep -i apache2<br />
<br />
<b>Install PHPStorm:</b><br />
<br />
# cd ~jun/Downloads/<br />
# tar xf PhpStorm-*.tar.gz -C /opt/<br />
# cd /opt/PhpStorm-163.10504.2/<br />
# ./bin/phpstorm.sh<br />
<br />
<b>Generate a self-signed SSL certificate:</b><br />
<br />
# openssl req -x509 -nodes -days 365 -newkey rsa:2048 -subj "/C=CA/ST=British Columbia/L=Vancouver/O=My Company Name/CN=erp.local" -keyout /etc/ssl/private/test.local.key -out /etc/ssl/certs/test.local.crt<br />
<br />
<b>Install node:</b><br />
<br />
$ curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.11/install.sh | bash<br />
<br />
$ command -v nvm<br />
<br />
<pre><code class="html">nvm</code></pre><br />
$ nvm ls-remote<br />
$ nvm install 8.9.3<br />
$ nvm use 8.9.3<br />
$ node -v<br />
$ nvm ls<br />
<br />
$ echo '{}' > package.json<br />
$ npm install webpack eslint js-beautify --save-dev<br />
<br />
<b>Reference:</b> <br />
<br />
<a href="https://wiki.apache.org/httpd/PHP-FPM" target="_blank">https://wiki.apache.org/httpd/PHP-FPM</a><br />
<br />
<a href="http://httpd.apache.org/docs/2.4/mod/mod_proxy_fcgi.html" target="_blank">http://httpd.apache.org/docs/2.4/mod/mod_proxy_fcgi.html</a><br />
<br />
<a href="https://www.digitalocean.com/community/tutorials/how-to-move-a-mysql-data-directory-to-a-new-location-on-ubuntu-16-04" target="_blank">https://www.digitalocean.com/community/tutorials/how-to-move-a-mysql-data-directory-to-a-new-location-on-ubuntu-16-04</a><br />
<br />
<a href="http://www.fromdual.com/mysql-tmpdir-on-ram-disk" target="_blank">http://www.fromdual.com/mysql-tmpdir-on-ram-disk</a><br />
<br />
<a href="http://www.victordodon.com/changing-mysql-tmpdir-in-ubuntu/" target="_blank">http://www.victordodon.com/changing-mysql-tmpdir-in-ubuntu/</a><br />
<br />
<a href="https://blog.remirepo.net/post/2014/03/28/PHP-FPM-and-HTTPD-2.4-improvement" target="_blank">https://blog.remirepo.net/post/2014/03/28/PHP-FPM-and-HTTPD-2.4-improvement</a>Jun Hsiehhttp://www.blogger.com/profile/00640061359079216681noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-5080980810662821782019-02-23T17:50:00.002-08:002019-02-23T17:50:20.159-08:00Migrate to stamblerre/gocode in YouCompleteMe for Go module supportMigrate to stamblerre/gocode in YouCompleteMe for Go module support<br />
<br />
apt install build-essential cmake python3-dev<br />
<br />
git clone --depth 1 https://github.com/Valloric/YouCompleteMe.git<br />
<br />
cd /home/jun/.vim/pack/plugins/start/YouCompleteMe<br />
git submodule update --init --recursive<br />
<br />
vim /home/jun/.vim/pack/plugins/start/YouCompleteMe/third_party/ycmd/ycmd/completers/go/go_completer.py<br />
<br />
mdempsky/gocode<br />
stamblerre/gocode<br />
<br />
vim /home/jun/.vim/pack/plugins/start/YouCompleteMe/third_party/ycmd/.gitmodules<br />
<br />
mdempsky/gocode<br />
stamblerre/gocode<br />
<br />
rm -rf third_party/ycmd/third_party/go/src/github.com/mdempsky/<br />
<br />
vim ./third_party/ycmd/build.py<br />
<br />
cd third_party/ycmd/third_party/go/src/github.com<br />
mkdir stamblerre<br />
cd stamblerre<br />
git clone --depth 1 https://github.com/stamblerre/gocode.git<br />
<br />
cd /home/jun/.vim/pack/plugins/start/YouCompleteMe<br />
<br />
./install.py --go-completer<br />
<br />
<b>Reference:</b> <br />
<br />
<a href="https://github.com/Valloric/YouCompleteMe" target="_blank">https://github.com/Valloric/YouCompleteMe</a><br />
<br />
<a href="https://github.com/stamblerre/gocode" target="_blank">https://github.com/stamblerre/gocode</a>Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-13119610033184514842019-02-10T11:33:00.001-08:002019-02-10T11:33:00.856-08:00Apache SAML libraryApache SAML library<br />
<br />
1. mod_shib<br />
<br />
2. mod_auth_mellonJun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-750843407256703392019-02-10T10:54:00.002-08:002019-02-10T10:54:23.730-08:00HTML a tag target="_blank" vulnerability tabnabbingHTML a tag target="_blank" vulnerability tabnabbing<br />
<br />
<pre><code class="html">window.opener.location = 'https://fakewebsite/facebook.com/PHISHING-PAGE.html';</code></pre><br />
<b>Solution 1:</b><br />
<br />
<pre><code class="html"><a target="_blank" rel="noopener noreferrer">demo</a></code></pre><br />
<b>Solution 2:</b><br />
<br />
<pre><code class="js">var newWnd = window.open();
newWnd.opener = null;</code></pre><br />
<b>Reference:</b> <br />
<br />
<a href="https://www.jitbit.com/alexblog/256-targetblank---the-most-underestimated-vulnerability-ever/" target="_blank">https://www.jitbit.com/alexblog/256-targetblank---the-most-underestimated-vulnerability-ever/</a><br />
<br />
<a href="https://medium.com/@ali.dev/how-to-fix-target-blank-a-security-and-performance-issue-in-web-pages-2118eba1ce2f" target="_blank">https://medium.com/@ali.dev/how-to-fix-target-blank-a-security-and-performance-issue-in-web-pages-2118eba1ce2f</a>Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-15217753142406458702019-01-26T09:56:00.000-08:002019-01-26T09:56:33.862-08:00CentOS and Red Hat 7: Install Linux, Apache MPM, MariaDB, PHP (LAMP) StackCentOS and Red Hat 7: Install Linux, Apache MPM, MariaDB, PHP (LAMP) Stack<br />
<br />
<b>Quick Start</b><br />
<br />
<b>Install IUS repository, utilities, Git, Go, php-cs-fixer:</b><br />
<br />
# curl 'https://setup.ius.io/' -o setup-ius.sh \<br />
&& bash setup-ius.sh \<br />
&& rm -f setup-ius.sh \<br />
&& yum -y update \<br />
&& yum -y install ntp wget epel-release tmux tree git2u \<br />
&& rm -rf /usr/local/go \<br />
&& curl -L 'https://dl.google.com/go/go1.11.5.linux-amd64.tar.gz' -o go.tar.gz \<br />
&& tar -zxvf go.tar.gz -C /usr/local \<br />
&& rm -f go.tar.gz \<br />
&& curl -L http://cs.sensiolabs.org/download/php-cs-fixer-v2.phar -o php-cs-fixer \<br />
&& chmod 755 php-cs-fixer \<br />
&& mv php-cs-fixer /usr/local/bin/php-cs-fixer \<br />
&& systemctl enable ntpd.service && systemctl restart ntpd.service<br />
<br />
# echo 'export GOPATH="$HOME/go"' >> ~/.bashrc \<br />
&& echo 'export PATH="$PATH:/usr/local/go/bin:$GOPATH/bin:/usr/local/bin"' >> ~/.bashrc \<br />
&& source ~/.bashrc<br />
<br />
<b>Install the environmental configuration:</b><br />
<br />
# cd \<br />
&& git clone https://github.com/junxie6/config_centos_v2.git \<br />
&& bash config_centos_v2/script/setHomeConfig.sh \<br />
&& source ~/.bashrc \<br />
&& tmux<br />
<br />
<b>Note:</b> git remote set-url origin git@github.com:junxie6/config_centos_v2.git<br />
<br />
<b>Install Vim:</b><br />
<br />
# yum -y install gcc make ncurses ncurses-devel \<br />
ruby ruby-devel lua lua-devel luajit \<br />
luajit-devel ctags python python-devel \<br />
python3 python3-devel tcl-devel \<br />
perl perl-devel perl-ExtUtils-ParseXS \<br />
perl-ExtUtils-XSpp perl-ExtUtils-CBuilder \<br />
perl-ExtUtils-Embed \<br />
ctags cscope<br />
<br />
# cd /usr/local/src \<br />
&& git clone https://github.com/vim/vim.git --depth 1\<br />
&& cd vim \<br />
&& ./configure --prefix=/usr --with-features=huge --enable-multibyte --enable-rubyinterp --enable-pythoninterp --enable-perlinterp --enable-luainterp --enable-cscope \<br />
&& make \<br />
&& make install \<br />
&& hash -r \<br />
&& vim --version | grep VIM<br />
<br />
<b>Install Docker:</b><br />
<br />
# yum -y install yum-utils \<br />
&& yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo \<br />
&& yum makecache fast \<br />
&& yum -y install docker-ce \<br />
&& systemctl enable docker \<br />
&& systemctl start docker \<br />
&& usermod -aG docker jun \<br />
&& docker version<br />
<br />
<b>Note:</b> Log out and log back in so that your Docker group membership is re-evaluated.<br />
<br />
# ps auxww | grep -i docker<br />
<br />
<b>Install docker-compose:</b><br />
<br />
# curl -L https://github.com/docker/compose/releases/download/1.15.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose \<br />
&& chmod 755 /usr/local/bin/docker-compose<br />
<br />
<b>Enable firewall ports on Docker Manager node:</b><br />
<br />
# firewall-cmd --permanent --zone=public \<br />
--add-port=2377/tcp \<br />
--add-port=7946/tcp \<br />
--add-port=7946/udp \<br />
--add-port=4789/udp \<br />
&& firewall-cmd --reload \<br />
&& systemctl restart docker.service \<br />
&& firewall-cmd --zone=public --list-all<br />
<br />
<b>Note:</b> Add TCP port 2376 if you need to use Docker Machine to work. Docker Machine is used to orchestrate Docker hosts.<br />
<br />
<b>Enable firewall ports on Docker Worker nodes:</b><br />
<br />
# firewall-cmd --permanent --zone=public \<br />
--add-port=7946/tcp \<br />
--add-port=7946/udp \<br />
--add-port=4789/udp \<br />
&& firewall-cmd --reload \<br />
&& systemctl restart docker.service \<br />
&& firewall-cmd --zone=public --list-all<br />
<br />
<b>Note:</b> Add TCP port 2376 if you need to use Docker Machine to work. Docker Machine is used to orchestrate Docker hosts.<br />
<br />
<b>Install Apache 2.4:</b><br />
<br />
# yum -y install httpd24u httpd24u-mod_ssl<br />
<br />
<b>Install PHP 7.1:</b><br />
<br />
# yum -y install php71u-common php71u-cli php71u-fpm php71u-fpm-httpd php71u-opcache php71u-xml php71u-json php71u-pdo php71u-mysqlnd php71u-intl php71u-mbstring php71u-mcrypt php71u-gd php71u-process<br />
<br />
<b>Install sysdig:</b><br />
<br />
# curl -s https://s3.amazonaws.com/download.draios.com/stable/install-sysdig | sudo bash<br />
<br />
// the ncurses user interface for sysdig<br />
# csysdig<br />
<br />
// the definitive system and process troubleshooting tool<br />
# sysdig<br />
<br />
<b>Check CentOS version:</b><br />
<br />
# cat /etc/redhat-release<br />
<br />
<pre><code class="html">CentOS Linux release 7.1.1503 (Core)</code></pre><br />
<b>Set up a static IP address for the network:</b><br />
<br />
# ip addr<br />
<br />
<pre><code class="html">2: eno16777736: <broadcast> mtu 1500 qdisc pfifo_fast state UP qlen 1000</code></pre><br />
# vi /etc/sysconfig/network-scripts/ifcfg-eno16777736<br />
<br />
<pre><code class="html">BOOTPROTO=none
IPADDR=192.168.6.9
PREFIX=24
GATEWAY=192.168.6.1
DNS1=8.8.8.8
DNS2=8.8.4.4
PEERDNS=yes
ONBOOT=yes
USERCTL=no</code></pre><br />
<b>Note:</b> USERCTL=no // Non-root users are not allowed to control this device.<br />
<br />
# systemctl restart network<br />
<br />
# ip addr<br />
<br />
# /bin/ipcalc --netmask 192.168.6.9/24<br />
<br />
<pre><code class="html">NETMASK=255.255.255.0</code></pre><br />
# cat /etc/resolv.conf<br />
<br />
<b>Change hostname:</b><br />
<br />
# hostnamectl status<br />
<br />
# hostnamectl set-hostname cent-dev.local<br />
<br />
# hostnamectl status<br />
<br />
# cat /etc/hostname<br />
<br />
<b>If you are using Amazon's AWS EC2 instance, append the following string at the bottom of the file to ensure that the hostname is preserved between restarts/reboots:</b><br />
<br />
# vim /etc/cloud/cloud.cfg<br />
<br />
<pre><code class="html">preserve_hostname: true</code></pre><br />
More info: <a href="https://aws.amazon.com/premiumsupport/knowledge-center/linux-static-hostname-rhel7-centos7/" target="_blank">https://aws.amazon.com/premiumsupport/knowledge-center/linux-static-hostname-rhel7-centos7/</a><br />
<br />
<b>Set up timezone:</b><br />
<br />
# timedatectl<br />
# timedatectl list-timezones<br />
# timedatectl set-timezone America/Vancouver<br />
# timedatectl<br />
<br />
<b>Set up date and time:</b><br />
<br />
# date +%Y%m%d -s "20081128"<br />
# date +%T -s "10:13:13"<br />
<br />
<b>To sync date and time automatically:</b><br />
<br />
# yum -y update<br />
<br />
# yum -y install ntp<br />
<br />
# ntpdate 0.us.pool.ntp.org<br />
<br />
<b>Note:</b> ntpdate is deprecated as of September 2012<br />
<br />
<b>You can change which ntp server to use:</b><br />
<br />
# vi /etc/ntp.conf<br />
<br />
<b>Set ntpd to start up on boot time:</b><br />
<br />
# systemctl enable ntpd.service<br />
# systemctl restart ntpd.service<br />
<br />
# ps auxww|grep -i ntpd<br />
# ntpstat<br />
# timedatectl<br />
<br />
<b>Set the Hardware Clock to the current System Time:</b><br />
<br />
# hwclock --systohc<br />
<br />
<b>Note:</b> hwclock is a utility for accessing the hardware clock. Hardware clock is independent of the operation system you use and works even when the machine is shut down. This program is used to find out the time from the hardware clock and set the system time at boot time.<br />
<br />
<b>Update the ~/.bashrc configuration:</b><br />
<br />
# vi ~/.bashrc<br />
<br />
<pre><code class="sh">### alias
alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'
alias gs='git status'
alias gc='git commit -a -m "up"'
alias gp='git push'
alias gpp='git pull'
alias ls='ls --color=auto'
alias ll='ls -la'
alias h='history'
### ls with color (try "ls --color=auto").
#export CLICOLOR=1 # Use colors (if possible)
#export LSCOLORS="ExGxFxdxCxDxDxBxBxExEx"
### set up a clean UTF-8 environment
### run: locale command
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8
### display history command with date and time
export HISTTIMEFORMAT="%m/%d/%y %T "
### Prompt
PS1='\[\e[0;32m\]\u@\h \w \$\[\e[0m\] '
#######
# Note: on Ubuntu, xterm-256color may be in different place, try this:
# find /lib/terminfo /usr/share/terminfo -name "*256*"
# Note: tmux respects screen-256color
#######
if [ -e /usr/share/terminfo/x/xterm-256color ]; then
export TERM='xterm-256color'
else
export TERM='xterm-color'
fi
### Make bash check its window size after a process completes
shopt -s checkwinsize
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi</code></pre><br />
# source ~/.bashrc<br />
<br />
<b>Update the ~/.inputrc configuration:</b><br />
<br />
# vi ~/.inputrc<br />
<br />
<pre><code class="sh">### enable filename tab auto-completion
set show-all-if-ambiguous on
set show-all-if-unmodified on
### if you don't want case-sensitivity
#set completion-ignore-case on
### bash history completion to complete what's already on the line
### arrow up
"\e[A": history-search-backward
### arrow down
"\e[B": history-search-forward</code></pre><br />
Make sure there is a swap space on your system:<br />
<br />
# cat /proc/meminfo | grep -i swap<br />
<br />
<pre><code class="html">SwapCached: 36 kB
SwapTotal: 2097148 kB
SwapFree: 2095160 kB</code></pre><br />
<b>Note:</b> if your system does not have the swap space, please refer to <a href="http://blog.ijun.org/2015/04/add-swap-to-amazon-ec2-instance-ebs.html" target="_blank">http://blog.ijun.org/2015/04/add-swap-to-amazon-ec2-instance-ebs.html</a> for more information.<br />
<br />
<b>Install EPEL and additional repositories on CentOS and Red Hat:</b><br />
<br />
# yum -y install wget<br />
# yum -y install epel-release<br />
<br />
<a href="http://blog.ijun.org/2014/11/install-epel-and-additional.html">http://blog.ijun.org/2014/11/install-epel-and-additional.html</a><br />
<a href="https://fedoraproject.org/wiki/EPEL">https://fedoraproject.org/wiki/EPEL</a><br />
<br />
<b>Subscribing to the IUS Community Project Repository:</b><br />
<br />
# curl 'https://setup.ius.io/' -o setup-ius.sh<br />
# bash setup-ius.sh<br />
<br />
# ls -l /etc/yum.repos.d/ius*<br />
<br />
<pre><code class="html">-rw-r--r--. 1 root root 1150 Apr 16 2015 /etc/yum.repos.d/ius-archive.repo
-rw-r--r--. 1 root root 1131 Apr 16 2015 /etc/yum.repos.d/ius-dev.repo
-rw-r--r--. 1 root root 1073 Apr 16 2015 /etc/yum.repos.d/ius.repo
-rw-r--r--. 1 root root 1150 Apr 16 2015 /etc/yum.repos.d/ius-testing.repo</code></pre><br />
<b>To find which package provides the ifconfig command:</b><br />
<br />
# yum provides ifconfig<br />
<br />
<b>To get the ifconfig command into our system:</b><br />
<br />
# yum -y install net-tools<br />
<br />
# ifconfig | awk '/inet /{print $2}'<br />
<br />
<b>To get the ss command into our system:</b><br />
<br />
# yum -y install iproute<br />
<br />
<b>To install DNS tools:</b><br />
<br />
# yum install bind-utils<br />
<br />
<b>Install tmux:</b><br />
# yum -y install tmux<br />
<br />
# vi ~/.tmux.conf<br />
<br />
# tmux<br />
or<br />
# tmux a -d<br />
<br />
<b>Install vim-enhanced:</b><br />
<br />
It is actually very easy to compile Vim by yourself. Please refer to the following article if you are interested in:<br />
<br />
<a href="http://blog.ijun.org/2016/07/compile-latest-vim-74-on-centos7.html">http://blog.ijun.org/2016/07/compile-latest-vim-74-on-centos7.html</a><br />
<br />
# vim --version<br />
-syntax<br />
-python<br />
<br />
# yum -y install vim-enhanced<br />
<br />
# vim --version<br />
+syntax<br />
+python<br />
<br />
# yum list installed | grep -i vim<br />
# yum info vim-enhanced<br />
<br />
<b>Install tree:</b><br />
<br />
# yum -y install tree<br />
<br />
<b>Install fortune game:</b><br />
<br />
# yum -y install fortune-mod.x86_64<br />
<br />
<b>Install Glances:</b><br />
<br />
# yum -y install glances<br />
<br />
<b>Install Git:</b><br />
<br />
# yum install git<br />
<br />
Or from IUS repo (preferred):<br />
<br />
# yum install git2u<br />
<br />
<b>Install NetCat:</b><br />
<br />
# yum -y install nmap-ncat<br />
# nc localhost 8080<br />
<br />
<b>Install firewalld:</b><br />
<br />
# yum -y install firewalld<br />
<br />
(optional)<br />
# yum install firewall-config<br />
<br />
<b>Note:</b> If you don't mind using a GUI you could use firewall-config instead. If you need something for the console you will have to use firewall-cmd instead.<br />
<br />
# systemctl enable firewalld<br />
<br />
# systemctl restart firewalld<br />
<br />
<b>Install MariaDB:</b><br />
<br />
# yum install mariadb-server<br />
<br />
<b>Setting UTF8 defaults for MySQL:</b><br />
<br />
We recommend against MySQL's utf8 character set, since it does not support 4-byte unicode characters, and strings containing them will be truncated. This is fixed by the newer utf8mb4 character set.<br />
<br />
# vim /etc/my.cnf<br />
<br />
<pre><code class="html">[mysqld]
# Version 5.5.3 introduced "utf8mb4", which is recommended
collation-server = utf8mb4_general_ci # Replaces utf8_general_ci
character-set-server = utf8mb4 # Replaces utf8
default-storage-engine = InnoDB
max_allowed_packet = 16M
# This option makes InnoDB to store each created table into its own .ibd file.
innodb_file_per_table
# Don't resolve hostnames. All hostnames are IP's or 'localhost'.
skip-name-resolve
#The number of simultaneous clients allowed.
max_connections = 200
# uncomment to disable the InnoDB storage engine
#skip-innodb
innodb_buffer_pool_size = 4G
innodb_additional_mem_pool_size = 16M
innodb_log_file_size = 256M
innodb_log_buffer_size = 8M</code></pre><br />
# systemctl enable mariadb.service<br />
<br />
# systemctl restart mariadb.service<br />
<br />
# systemctl is-active mariadb.service<br />
<br />
# /usr/bin/mysql_secure_installation<br />
<br />
# mysql -u root -p<br />
<br />
mysql> GRANT ALL PRIVILEGES ON *.* TO 'test'@'192.168.0.%' IDENTIFIED BY '123456';<br />
mysql> FLUSH PRIVILEGES;<br />
<br />
mysql> CREATE DATABASE mydb DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;<br />
<br />
<b>Set up .my.cnf configuration file:</b><br />
<br />
# touch ~/.my.cnf<br />
# chmod 600 ~/.my.cnf<br />
# vim ~/.my.cnf<br />
<br />
<pre><code class="html">[client]
host = localhost
port = 3306
user = root
password = MyPassword</code></pre><br />
<b>Add New Rule to firewalld to allow access to MySQL:</b><br />
<br />
# firewall-cmd --permanent --zone=trusted --add-source=192.168.0.1/32<br />
# firewall-cmd --permanent --zone=trusted --add-port=3306/tcp<br />
# firewall-cmd --reload<br />
# firewall-cmd --zone=trusted --list-all<br />
<br />
or<br />
<br />
# systemctl restart firewalld.service<br />
<br />
<b>Check out the zone file to inspect the XML configuration:</b><br />
<br />
# cat /etc/firewalld/zones/public.xml<br />
# cat /etc/firewalld/zones/trusted.xml<br />
<br />
<b>Install Apache:</b><br />
<br />
# yum install httpd mod_ssl<br />
<br />
Or from IUS repo (preferred):<br />
<br />
# yum install httpd24u httpd24u-mod_ssl<br />
<br />
# systemctl status httpd<br />
<br />
<pre><code class="html">● httpd.service - The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled)</code></pre><br />
<b>Note:</b> it will tell you if the service starts on boot.<br />
<br />
# systemctl enable httpd.service<br />
<br />
# systemctl restart httpd.service<br />
<br />
# systemctl reload httpd.service<br />
<br />
# systemctl -l status httpd.service<br />
<br />
# journalctl -xn<br />
<br />
# systemctl is-active httpd.service<br />
<br />
# apachectl configtest<br />
<br />
# httpd -V<br />
<br />
# apachectl graceful<br />
<br />
<b>httpd service default configuration files:</b><br />
<br />
<ul><li>Default config file: /etc/httpd/conf/httpd.conf</li>
<li>Configuration files which load modules : /etc/httpd/conf.modules.d/ directory (e.g. PHP)</li>
<li>Select MPMs (Processing Model) as loadable modules [worker, prefork (default)] and event: /etc/httpd/conf.modules.d/00-mpm.conf</li>
<li>Default ports: 80 and 443 (SSL)</li>
<li>Default log files: /var/log/httpd/{access_log,error_log}</li>
</ul><br />
<b>Set up a symbolic link:</b><br />
<br />
# cd / ; ln -s var/www/html www<br />
<br />
<b>Install PHP:</b><br />
<br />
# yum install php php-mysqlnd php-fpm php-gd php-mbstring php-pdo php-xml php-soap<br />
# yum install php-pear php-devel pcre-devel gcc gcc-c++ make<br />
<br />
# systemctl restart httpd.service<br />
<br />
<b>Note:</b> you need php-devel pcre-devel gcc make for PHP APC. Try to add httpd-devel if failed.<br />
<br />
<b>You need the php-mcrypt to run Magento:</b><br />
<br />
# yum install php-mcrypt<br />
<br />
<a href="http://blog.ijun.org/2014/11/how-to-install-php-mcrypt-on-centos-7.html">http://blog.ijun.org/2014/11/how-to-install-php-mcrypt-on-centos-7.html</a><br />
<br />
<b>Open port 80 firewall access:</b><br />
<br />
# firewall-cmd --zone=public --add-port=http/tcp<br />
# firewall-cmd --zone=public --add-port=https/tcp<br />
<br />
# firewall-cmd --permanent --zone=public --add-port=http/tcp<br />
# firewall-cmd --permanent --zone=public --add-port=https/tcp<br />
<br />
# firewall-cmd --reload<br />
or<br />
# systemctl restart firewalld.service<br />
<br />
# firewall-cmd --zone=public --list-all<br />
<br />
<b>Allow a IP address with a specific port:</b><br />
<br />
# firewall-cmd --zone=public --add-rich-rule='rule family="ipv4" source address="1.2.3.4" port protocol="tcp" port="3306" accept'<br />
<br />
To remove the rule:<br />
<br />
# firewall-cmd --zone=public --remove-rich-rule='rule family="ipv4" source address="1.2.3.4" port protocol="tcp" port="3306" accept'<br />
<br />
<b>Block a IP address:</b><br />
<br />
# firewall-cmd --zone=public --add-rich-rule='rule family="ipv4" source address="1.2.3.4" reject'<br />
<br />
To remove the rule:<br />
<br />
# firewall-cmd --zone=public --remove-rich-rule='rule family="ipv4" source address="1.2.3.4" reject'<br />
<br />
<b>If you get 403 forbidden error, then you probably have problem with SELinux, to deal with Security-Enhanced Linux (SELinux):</b><br />
<br />
# namei -l /var/www/html/magento19<br />
<br />
# ls -dZ /var/www/html<br />
drwxr-xr-x. root root system_u:object_r:httpd_sys_content_t:s0 /var/www/html<br />
<br />
# chcon -R --reference=/var/www/html /var/www/html/magento19<br />
Or<br />
# chcon -R --type=httpd_sys_content_t /var/www/html/magento19<br />
Or for read and write permission:<br />
# chcon -R -t httpd_sys_rw_content_t /var/www/html/magento19/app/etc<br />
<br />
# ps auxwwZ | grep httpd<br />
# ls -dZ /var/www/html/magento19/app/etc<br />
# tail /var/log/audit/audit.log<br />
# tail /var/log/messages<br />
<br />
To turn off Security-Enhanced Linux (SELinux):<br />
<br />
# setenforce 0<br />
<br />
To turn on Security-Enhanced Linux (SELinux):<br />
<br />
# setenforce 1<br />
<br />
To get the status of a system running SELinux:<br />
<br />
# sestatus<br />
<br />
<b>Set up Apache MPM and PHP-FPM:</b><br />
<br />
With Apache 2.4, the official module to use is <b>mod_proxy_fcgi</b> instead of the ancient <b>mod_fastcgi</b>. That module, as well as <b>mod_fcgid</b>, were third party modules.<br />
<br />
<b>Note:</b> mod_proxy_fcgi now supports network sockets since Apache 2.4.9 ( Unix socket support for mod_proxy_fcgi )<br />
<br />
<b>List built-in or shared modules:</b><br />
<br />
# httpd -M | grep -iE 'proxy_module|proxy_fcgi_module|rewrite_module|mpm_event_module|deflate_module|vhost_alias_module|ssl_module'<br />
<br />
<pre><code class="html"> deflate_module (shared)
rewrite_module (shared)
vhost_alias_module (shared)
mpm_event_module (shared)
proxy_module (shared)
proxy_fcgi_module (shared)
ssl_module (shared)</code></pre><br />
<b>List loaded modules:</b><br />
<br />
# httpd -t -D DUMP_MODULES | grep -iE 'proxy_module|proxy_fcgi_module|rewrite_module|mpm_event_module|deflate_module|vhost_alias_module|ssl_module'<br />
<br />
<pre><code class="html"> deflate_module (shared)
rewrite_module (shared)
vhost_alias_module (shared)
mpm_event_module (shared)
proxy_module (shared)
proxy_fcgi_module (shared)
ssl_module (shared)</code></pre><br />
<b>Edit mpm.conf:</b><br />
<br />
# vim /etc/httpd/conf.modules.d/00-mpm.conf<br />
<br />
Comment out the following line:<br />
<br />
<pre><code class="html">LoadModule mpm_prefork_module modules/mod_mpm_prefork.so</code></pre><br />
Uncomment the following line:<br />
<br />
<pre><code class="html">LoadModule mpm_event_module modules/mod_mpm_event.so</code></pre><br />
<b>Make sure the following two lines exist:</b><br />
<br />
# grep -E 'mod_proxy.so|mod_proxy_fcgi' /etc/httpd/conf.modules.d/00-proxy.conf<br />
<br />
<pre><code class="html">LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so</code></pre><br />
<b>Add the "if checking" surround the following three lines:</b><br />
<br />
# vim /etc/httpd/conf.d/php.conf<br />
<br />
<pre><code class="html"><IfModule mod_php5.c>
<FilesMatch \.php$>
SetHandler application/x-httpd-php
</FilesMatch>
</IfModule></code></pre><br />
<b>Add the "if checking" surround the following two lines:</b><br />
<br />
# vim /etc/httpd/conf.d/php.conf<br />
<br />
<pre><code class="html"><IfModule mod_php5.c>
php_value session.save_handler "files"
php_value session.save_path "/var/lib/php/session"
</IfModule></code></pre><br />
<b>Change the following line:</b><br />
<br />
# vim /etc/httpd/conf/httpd.conf<br />
<br />
<pre><code class="html"><Directory "/var/www/html">
AllowOverride All
# New directive needed in Apache 2.4.3:
Require all granted
</Directory></code></pre><br />
<b>Create and Edit the vhosts file:</b><br />
<br />
Apache supports talking to php-fpm through SetHandler directive since Apache 2.4.9. We no longer need to use the ProxyPassMatch directive.<br />
<br />
As the ProxyPassMatch directive is evaluated as the very beginning of each request:<br />
- AddType (for MultiView) or DirectoryIndex directives are not usable<br />
- right management per directory is not available<br />
- each Alias directive needs another proxy rule<br />
<br />
The SetHandler directive, evaluated later, is much more flexible / usable.<br />
<br />
To redirect the PHP scripts to the FPM server:<br />
<br />
<pre><code class="html"><FilesMatch \.php$>
SetHandler "proxy:fcgi://127.0.0.1:9000"
</FilesMatch></code></pre><br />
<b>Note:</b> You will also need to enable the following modules:<br />
<br />
<pre><code class="html">LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so</code></pre><br />
Warning: if you remove or disable mod_php, you also need to remove all the php_value and php-flag directives:<br />
<br />
<pre><code class="html"><IfModule mod_php5.c>
php_value session.save_handler "files"
php_value session.save_path "/var/lib/php/session"
php_value soap.wsdl_cache_dir "/var/lib/php/wsdlcache"
</IfModule></code></pre><br />
<b>Note:</b> For more detail, visit <a href="https://blog.remirepo.net/post/2014/03/28/PHP-FPM-and-HTTPD-2.4-improvement" target="_blank">https://blog.remirepo.net/post/2014/03/28/PHP-FPM-and-HTTPD-2.4-improvement</a><br />
<br />
# vim /etc/httpd/conf.d/httpd-vhosts.conf<br />
<br />
<pre><code class="html">#
# Use name-based virtual hosting.
#
NameVirtualHost *:80
#
# VirtualHost example:
# Almost any Apache directive may go into a VirtualHost container.
# The first VirtualHost section is used for all requests that do not
# match a ServerName or ServerAlias in any <VirtualHost> block.
#
<VirtualHost *:80>
ServerAdmin webmaster@dummy-host.example.com
DocumentRoot "/var/www/html/host_not_found"
ErrorLog "/var/log/httpd/host_not_found-error_log"
CustomLog "/var/log/httpd/host_not_found-access_log" common
</VirtualHost>
<VirtualHost *:80>
ServerAdmin webmaster@dummy-host.example.com
DocumentRoot "/var/www/html/magento1.9.1"
ServerName us.centos.local
### Disable PHP script execution for this directory. We don't want to reverse-proxy this subdirectory.
<Location "/var">
ProxyPass !
</Location>
### Disable PHP script execution for this directory. We don't want to reverse-proxy this subdirectory.
<Location "/media">
ProxyPass !
</Location>
ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://127.0.0.1:9000/var/www/html/magento1.9.1/$1
SetEnv MAGE_RUN_CODE "us_centos_local"
SetEnv MAGE_RUN_TYPE "website"
### a request for / will need to be mapped to a resource on the fcgi backend. Failure to address this may cause a blank response, commonly known as a WSOD (White Screen of Death), especially if only a request URI containing the php extension is proxied, such as this example. The processing chain will first map a request for / to /index.php, then proxy to the PHP-FPM backend correctly.
DirectoryIndex /index.php index.php index.html index.htm
ErrorLog "/var/log/httpd/us.centos.local-error_log"
CustomLog "/var/log/httpd/us.centos.local-access_log" common
</VirtualHost>
</code></pre><br />
<b>Setting up an SSL secured Web server:</b><br />
<br />
<pre><code class="html"><VirtualHost *:443>
SSLEngine on
SSLCertificateFile /etc/pki/tls/certs/mydomain.com.crt
SSLCertificateKeyFile /etc/pki/tls/private/mydomain.com.key
SSLCertificateChainFile /etc/pki/tls/certs/gd_bundle-mydomain.com.crt
ServerAdmin webmaster@dummy-host.example.com
DocumentRoot "/var/www/html/magento1.9.1"
ServerName us.centos.local
### we don't want to reverse-proxy this subdirectory.
<Location "/var">
ProxyPass !
</Location>
### we don't want to reverse-proxy this subdirectory.
<Location "/media">
ProxyPass !
</Location>
ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://127.0.0.1:9000/var/www/html/magento1.9.1/$1
SetEnv MAGE_RUN_CODE "us_centos_local"
SetEnv MAGE_RUN_TYPE "website"
ErrorLog "/var/log/httpd/us.centos.local443-error_log"
CustomLog "/var/log/httpd/us.centos.local443-access_log" common
</VirtualHost></code></pre><br />
<b>Make sure the security context of the certification files are cert_t:</b><br />
<br />
# cd /etc/pki/tls/certs/mydomain.com<br />
# chcon -t cert_t *<br />
<br />
If you do not see the correct security context, you will see the following message when restarting Apache:<br />
<br />
<pre><code class="html">Mar 16 23:58:00 ip-172-31-21-55 httpd[22680]: AH00526: Syntax error on line 19 of /etc/httpd/conf.d/httpd-vhosts.conf:
Mar 16 23:58:00 ip-172-31-21-55 httpd[22680]: SSLCertificateFile: file '/etc/pki/tls/certs/mydomain.com/b1210d10x4d812c5.crt' does not exist or is empty
Mar 16 23:58:00 ip-172-31-21-55 systemd[1]: httpd.service: main process exited, code=exited, status=1/FAILURE
Mar 16 23:58:00 ip-172-31-21-55 systemd[1]: Failed to start The Apache HTTP Server.
Mar 16 23:58:00 ip-172-31-21-55 systemd[1]: Unit httpd.service entered failed state.
Mar 16 23:58:00 ip-172-31-21-55 systemd[1]: httpd.service failed.</code></pre><br />
<b>Restart Apache:</b><br />
<br />
# systemctl reload httpd.service<br />
<br />
# httpd -t -D DUMP_VHOSTS<br />
<br />
<b>To enable gzip compression:</b><br />
<br />
# egrep 'deflate|header' /etc/httpd/conf.modules.d/00-base.conf<br />
<br />
<pre><code class="html">LoadModule deflate_module modules/mod_deflate.so
LoadModule headers_module modules/mod_headers.so</code></pre><br />
# httpd -t -D DUMP_MODULES | grep deflate<br />
<br />
<pre><code class="html">deflate_module (shared)</code></pre><br />
# vim /etc/httpd/conf.d/mod_deflate.conf<br />
<br />
<pre><code class="html"><IfModule mod_deflate.c>
AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css text/js application/javascript application/x-javascript application/x-httpd-php
</IfModule></code></pre><br />
# systemctl restart httpd.service<br />
<br />
<b>Install Xdebug for debugging PHP:</b><br />
<br />
# pecl install Xdebug<br />
<br />
# vim /etc/php.d/xdebug.ini<br />
<br />
<pre><code class="html">; Enable xdebug extension module
zend_extension=xdebug.so
;zend_extension=/usr/lib64/php/modules/xdebug.so
xdebug.default_enable=1
xdebug.remote_enable=1
xdebug.remote_handler=dbgp
xdebug.remote_host=localhost
xdebug.remote_port=9009
xdebug.remote_log=/tmp/xdebug.log
xdebug.remote_connect_back=0
xdebug.remote_autostart=0
xdebug.remote_mode=req
xdebug.max_nesting_level=1000
xdebug.var_display_max_depth = 5
xdebug.var_display_max_children = 256
xdebug.var_display_max_data = 1024</code></pre><br />
or<br />
<br />
<pre><code class="html">zend_extension="/usr/lib64/php/modules/xdebug.so"
; When this setting is set to on, the tracing of function calls will be enabled just before the script is run. This makes it possible to trace code in the auto_prepend_file.
xdebug.auto_trace = 1
xdebug.trace_output_dir = "/tmp"
xdebug.collect_params = 4
; Enables Xdebug's profiler which creates files in the profile output directory. Those files can be read by KCacheGrind to visualize your data.
xdebug.profiler_enable = 1
xdebug.profiler_output_dir = "/tmp"
; Controls the protection mechanism for infinite recursion protection. The value of this setting is the maximum level of nested functions that are allowed before the script will be aborted.
xdebug.max_nesting_level = 100
; shows a human readable / computer readable trace file.
xdebug.trace_format = 0
; This setting tells Xdebug to gather information about which variables are used in a certain scope. This analysis can be quite slow as Xdebug has to reverse engineer PHP's opcode arrays. This setting will not record which values the different variables have, for that use xdebug.collect_params. This setting needs to be enabled only if you wish to use xdebug_get_declared_vars().
xdebug.collect_vars = 0
; When set to '1' the trace files will be appended to, instead of being overwritten in subsequent requests.
; Note: this option can be useful if you could not find your function calls anywhere.
xdebug.trace_options = 1</code></pre><br />
<b>To search all other php modules:</b><br />
<br />
# yum search php<br />
<br />
<b>Edit php.ini:</b><br />
<br />
# vim /etc/php.ini<br />
<br />
<pre><code class="html">cgi.fix_pathinfo = 0</code></pre><br />
The default value is 1, which is an extremely insecure setting because it tells PHP to attempt to execute the closest file it can find if a PHP file does not match exactly. This basically would allow users to craft PHP requests in a way that would allow them to execute scripts that they shouldn't be allowed to execute.<br />
<br />
<b>Note:</b> if I set it to "cgi.fix_pathinfo = 0", I would get "<b>Access denied (403)</b>" (see security.limit_extensions) or <b>no input file specified</b> error when setting up Magento. You can either:<br />
<br />
1. commented out the cgi.fix_pathinfo = 0 line.<br />
2. set cgi.fix_pathinfo = 1<br />
3. try to set "security.limit_extensions = " in the /etc/php-fpm.d/www.conf file.<br />
<br />
<a href="http://stackoverflow.com/questions/23390531/access-denied-403-for-php-files-with-nginx-php-fpm">http://stackoverflow.com/questions/23390531/access-denied-403-for-php-files-with-nginx-php-fpm</a><br />
<a href="http://serverfault.com/questions/627903/is-the-php-option-cgi-fix-pathinfo-really-dangerous-with-nginx-php-fpm">http://serverfault.com/questions/627903/is-the-php-option-cgi-fix-pathinfo-really-dangerous-with-nginx-php-fpm</a><br />
<br />
<pre><code class="html">display_errors = On
log_errors = On
error_log = /var/log/php_errors.log</code></pre><br />
<b>Note:</b> make sure you do:<br />
<br />
# touch /var/log/php_errors.log<br />
# chmod 660 /var/log/php_errors.log<br />
# chown root:apache /var/log/php_errors.log<br />
<br />
# chcon -t httpd_log_t /var/log/php_errors.log<br />
Or<br />
# chcon -u system_u -t httpd_log_t /var/log/php_errors.log<br />
<br />
<b>Install XCache:</b><br />
<br />
XCache is a fast, stable PHP opcode cacher that has been proven and is now running on production servers under high load. It is tested (on linux) and supported on all of the latest PHP release branches such as PHP_5_1 PHP_5_2 PHP_5_3 PHP_5_4 PHP_5_5. It is more stable than APC.<br />
<br />
<b>Warning: APC would cause segfault segmentation fault. Use XCache instead.</b><br />
<br />
# yum install php-xcache xcache-admin<br />
<br />
# systemctl restart httpd<br />
# systemctl restart php-fpm<br />
# php -v<br />
<br />
<pre><code class="html">PHP 5.4.16 (cli) (built: Oct 31 2014 12:59:36)
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2013 Zend Technologies
with XCache v3.1.1, Copyright (c) 2005-2014, by mOo
with XCache Optimizer v3.1.1, Copyright (c) 2005-2014, by mOo
with XCache Cacher v3.1.1, Copyright (c) 2005-2014, by mOo
with XCache Coverager v3.1.1, Copyright (c) 2005-2014, by mOo</code></pre><br />
# echo -n "Your Password" | md5sum<br />
<br />
<pre><code class="html">5afd8756ebeda48acf7eb645503dae60</code></pre><br />
# vim /etc/php.d/xcache.ini<br />
<br />
<pre><code class="html">[xcache]
xcache.admin.user = "admin name"
xcache.admin.pass = "5afd8756ebeda48acf7eb645503dae60"
xcache.size = 60M
; This number divides total cache size into threads in order to increase the efficiency. If you have 128M xcache.size and you set the count as 4, that means each thread will manage 32M size of cache.
xcache.count = 1
xcache.cacher = On
xcache.stat = On
xcache.optimizer = On</code></pre><br />
# cp -r /usr/share/xcache/ /var/www/html/<br />
<br />
# systemctl restart php-fpm<br />
<br />
http://localhost/xcache/<br />
<br />
<b>Install APC:</b><br />
<br />
<b>Warning: APC would cause segfault segmentation fault. Use XCache instead.</b><br />
<br />
# pecl install apc<br />
<br />
# vim /etc/php.d/apc.ini<br />
<br />
<pre><code class="html">; Enable APC for PHP
extension=apc.so
apc.enabled=1
; The number of seconds a cache entry is allowed to idle in a slot before APC dumps the cache
apc.ttl=72000
apc.user_ttl=72000
apc.gc_ttl=3600
; Size of memory for apc ( 1024 M)
apc.shm_size=1024M
; Enable apc stats.
apc.stat=1
; Enable APC for command line php operations.
apc.enable_cli=1
; Allow 2 seconds after a file is created before it is cached. This will prevent premature PHP pages to get cached.
apc.file_update_protection=2
; Maximum size of single file that apc can store.
apc.max_file_size=1M
; Maximum number of files APC can store ( rotation).
apc.num_files_hint=200000
; Maximum number of users data entries that APC can store.
apc.user_entries_hint=20000</code></pre><br />
<b>Copy the apc.php file:</b><br />
<br />
# cp /usr/share/pear/apc.php /var/www/html<br />
<br />
<b>Set up php-fpm:</b><br />
<br />
Apach 2.4.8 mod_proxy: Added support for unix domain sockets as the backend server endpoint.<br />
<br />
# vim /etc/php-fpm.d/www.conf<br />
<br />
If you are using Apach 2.4.8 or above, please change the following line from:<br />
<br />
<pre><code class="html">listen = 127.0.0.1:9000</code></pre><br />
To:<br />
<br />
<pre><code class="html">listen = /var/run/php-fpm/php-fpm.sock</code></pre><br />
Now, there are different ways to actually forward requests for .php files to this module, ranging from everything (using ProxyPass) to very specific or rewritten files or patterns (using mod_rewrite with the [P] flag).<br />
<br />
The method I chose (using <a href="https://wiki.apache.org/httpd/PHP-FPM">ProxyPassMatch</a>) lies somewhere in between these in complexity and flexibility, since it allows you to set one rule for all PHP content of a specific vhost, but will only proxy .php files (or URLs that contain the text .php somewhere in the request).<br />
<br />
<b>TCP socket (IP and port) approach</b><br />
<br />
Edit the configuration for a vhost of your choice, and add the following line to it:<br />
<br />
# vim /etc/httpd/conf.d/httpd-vhosts.conf<br />
<br />
<pre><code class="html">ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://127.0.0.1:9000/path/to/your/documentroot/$1
### a request for / will need to be mapped to a resource on the fcgi backend. Failure to address this may cause a blank response, commonly known as a WSOD (White Screen of Death), especially if only a request URI containing the php extension is proxied, such as this example. The processing chain will first map a request for / to /index.php, then proxy to the PHP-FPM backend correctly.
DirectoryIndex /index.php index.php index.html index.htm</code></pre><br />
<b>Note:</b> please do change <span style="color: blue;">/path/to/your/documentroot</span> to for example <span style="color: blue;">/var/www/html/drupal8</span><br />
<br />
Look confusing ? Let's run through it:<br />
<br />
<b>ProxyPassMatch</b><br />
<br />
only proxy content that matches the specified regex pattern; in this case:<br />
<br />
<b>^/(.*\.php(/.*)?)$</b><br />
<br />
from the documentroot onwards, match everything ending in .php (with the dot escaped), optionally followed by a slash and any continued path you like (some applications use this so-called PathInfo to pass arguments to the php script.)<br />
<br />
The ^ (caret) and $ (dollar) signs are used to anchor both the absolute start and end of the URL, to make sure no characters from the request escape our pattern match.<br />
<br />
The nested parentheses enable us to refer to the entire request-URI (minus the leading slash) as $1, while still keeping the trailing pathinfo optional.<br />
<br />
<b>fcgi://127.0.0.1:9000</b><br />
<br />
forward via mod_proxy_fcgi, using the fastCGI protocol, to the port our php-fpm daemon is listening on.<br />
<br />
This determines which fastcgi pool will serve requests proxied by this rule.<br />
<br />
<b>/path/to/your/documentroot/</b><br />
<br />
IMPORTANT! This must exactly match the real filesystem location of your php files, because that is where the php-fpm daemon will look for them.<br />
<br />
php-fpm just interprets the php files passed to it; it is not a web server, nor does it understand your web servers' namespace, virtualhost layout, or aliases.<br />
<br />
IMPORTANT! Read the above again<br />
<br />
<b>$1</b><br />
<br />
expands to the entire request-URI from the original request, minus the leading slash (because we already added that above.)<br />
<br />
<b>DirectoryIndex /index.php index.php index.html index.htm</b><br />
<br />
<b>Note:</b> a request for / will need to be mapped to a resource on the fcgi backend. Failure to address this may cause a blank response, commonly known as a WSOD (White Screen of Death), especially if only a request URI containing the php extension is proxied, such as this example. The processing chain will first map a request for / to /index.php, then proxy to the PHP-FPM backend correctly.<br />
<br />
<b>unix domain socket (UDS) approach</b><br />
<br />
Edit the configuration for a vhost of your choice, and add the following line to it:<br />
<br />
# vim /etc/httpd/conf.d/httpd-vhosts.conf<br />
<br />
<pre><code class="html">ProxyPassMatch ^/(.*\.php(/.*)?)$ unix:/path/to/socket.sock|fcgi://127.0.0.1:9000/path/to/your/documentroot/</code></pre><br />
<b>unix:/path/to/socket.sock</b><br />
<br />
the path to your fpm socket<br />
<br />
Note that with this approach, the captured request URI ($1) is not passed after the path<br />
<br />
<b>Enable php-fpm to start on boot:</b><br />
<br />
# systemctl enable php-fpm.service<br />
<br />
<b>Start our PHP processor:</b><br />
<br />
# systemctl restart php-fpm.service<br />
# systemctl restart httpd.service<br />
<br />
<b>Test PHP:</b><br />
<br />
# php -r "echo 'hi';";<br />
<br />
<b>Check the PHP-FPM setting:</b><br />
<br />
<pre><code class="php"><?php
echo phpinfo();
?>
</code></pre><br />
You should see the message: Server API: FPM/FastCGI<br />
<br />
# httpd -V<br />
<br />
<pre><code class="html">Server MPM: event</code></pre><br />
<b>To create a locked user account:</b><br />
<br />
# useradd dev -m -c 'git user'<br />
<br />
<b>Unlock the account by issuing the passwd command to assign a password and set password aging guidelines:</b><br />
<br />
# passwd dev<br />
<br />
<b>Add a new group called web:</b><br />
<br />
# groupadd web<br />
<br />
<b>Add the dev user to the web group:</b><br />
<br />
# usermod -a -G web dev<br />
<br />
<b>Show The Groups a User Is In:</b><br />
<br />
# groups dev<br />
# id -Gn dev<br />
<br />
<b>Find out the primary group of a user:</b><br />
<br />
# getent group dev<br />
<br />
<b>To allow Apache to connect network and sendmail send email:</b><br />
<br />
# setsebool -P httpd_can_network_connect 1<br />
# setsebool -P httpd_can_sendmail 1<br />
<br />
<b>Note:</b> -P in the above command means Persistent (across reboots)<br />
<br />
# getsebool -a | grep -i httpd_can<br />
<br />
<pre><code class="html">httpd_can_network_connect --> on
httpd_can_sendmail --> on</code></pre><br />
# sestatus -b | grep httpd_can<br />
<br />
<pre><code class="html">httpd_can_network_connect on
httpd_can_sendmail on</code></pre><br />
<b>Install Memcached:</b><br />
<br />
# yum install memcached<br />
# yum install php-pecl-memcached<br />
<br />
# systemctl enable memcached.service<br />
# systemctl restart memcached.service<br />
<br />
# memcached-tool localhost:11211 display<br />
# memcached-tool localhost:11211 stats<br />
# memcached-tool localhost:11211 dump<br />
<br />
<b>Install redis server:</b><br />
<br />
# yum install redis<br />
# yum install php-pecl-redis<br />
<br />
<b>Two important redis server configuration files:</b><br />
<br />
# less /etc/redis.conf<br />
# less /etc/redis-sentinel.conf<br />
<br />
<b>Start the Redis server:</b><br />
<br />
# systemctl start redis.service<br />
<br />
<b>Check the running status of Redis server:</b><br />
<br />
# systemctl status redis.service<br />
<br />
<b>To test the installation of Redis:</b><br />
<br />
# redis-cli ping<br />
<br />
PONG<br />
<br />
<b>To enable Redis server at system's booting time:</b><br />
<br />
# systemctl enable redis.service<br />
<br />
<b>To get the listening port 6379 of Redis server:</b><br />
<br />
# ss -nlp | grep redis<br />
<br />
<b>To install Redis PHP extension:</b><br />
<br />
# pecl install redis<br />
<br />
<b>Add the following line to /etc/php.d/redis.ini:</b><br />
<br />
# echo 'extension=redis.so' >> /etc/php.d/redis.ini<br />
<br />
<b>Check to see if Redis PHP extension is installed:</b><br />
<br />
# pecl list | grep redis<br />
<br />
<pre><code class="html">redis 2.2.7 stable</code></pre><br />
<b>Restart Apache and PHP-FPM:</b><br />
<br />
# systemctl restart httpd.service<br />
# systemctl restart php-fpm.service<br />
<br />
<b>To see if Redis extension is being loaded by PHP:</b><br />
<br />
# php -m | grep redis<br />
<br />
<b>To allow Apache to connect to the Redis server:</b><br />
<br />
# setsebool -P httpd_can_network_connect 1<br />
<br />
<b>Note:</b> If you have turned on Security-Enhanced Linux (SELinux), httpd scripts by default are not allowed to connect out to the network.<br />
<br />
<b>To list all Redis Databases:</b><br />
<br />
# redis-cli info keyspace<br />
<br />
<b>To clear remove delete all data from a particular Redis database:</b><br />
<br />
# redis-cli<br />
<br />
127.0.0.1:6379> info keyspace<br />
127.0.0.1:6379> select 0<br />
127.0.0.1:6379> keys *<br />
127.0.0.1:6379> flushdb<br />
127.0.0.1:6379> keys *<br />
<br />
<b>To clear remove delete all data from all Redis database:</b><br />
<br />
# redis-cli flushall<br />
<br />
<b>Dumping all key/value pairs in a Redis db:</b><br />
<br />
# redis-cli -n 0 keys \*<br />
<br />
# redis-cli -n 0 keys \* | xargs -n 1 redis-cli dump<br />
<br />
<b>Note:</b> the 0 is the database number.<br />
<br />
<b>To store PHP sessions in Redis:</b><br />
<br />
Storing PHP session files in RAM can be much more efficient than storing on disk and can also save some IO. To configure this, you should modify the main php.ini file and change session.save_handler to redis.<br />
<br />
# vim /etc/php.ini<br />
<br />
<pre><code class="html">session.save_handler = redis
session.save_path = "tcp://127.0.0.1:6379"</code></pre><br />
# systemctl restart php-fpm<br />
<br />
# php -r 'echo phpinfo();' | grep redis<br />
<br />
<pre><code class="html">Registered save handlers => files user redis
session.save_handler => redis => redis</code></pre><br />
# vim test.php<br />
<br />
<pre><code class="php"><?php
session_start();
$_SESSION['favcolor'] = 'green';
echo '<pre>' . print_r($_SESSION, TRUE) . '</pre>';
?></code></pre><br />
# php test.php<br />
<br />
# redis-cli info keyspace<br />
<pre><code class="html"># Keyspace
db0:keys=68,expires=39,avg_ttl=2977110
db2:keys=427,expires=427,avg_ttl=1856886</code></pre><br />
# redis-cli -n 0 keys \*| grep -i session<br />
<br />
<pre><code class="html">PHPREDIS_SESSION:vhauaf8qpdj146kirsbivrh4i7</code></pre><br />
<b>Note:</b> the 0 is the database number.<br />
<br />
<b>Redis setup hints</b><br />
<br />
<ul><li>We suggest deploying Redis using the <b>Linux operating system</b>. Redis is also tested heavily on osx, and tested from time to time on FreeBSD and OpenBSD systems. However Linux is where we do all the major stress testing, and where most production deployments are working.</li>
<li>Make sure to set the Linux kernel <b>overcommit memory setting to 1</b>. Add vm.overcommit_memory = 1 to /etc/sysctl.conf and then reboot or run the command sysctl vm.overcommit_memory=1 for this to take effect immediately.</li>
<li>Make sure to disable Linux kernel feature transparent huge pages, it will affect greatly both <b>memory usage and latency</b> in a negative way. This is accomplished with the following command: <b>echo never > sys/kernel/mm/transparent_hugepage/enabled</b>.</li>
<li>Make sure to <b>setup some swap</b> in your system (we suggest as much as swap as memory). If Linux does not have swap and your Redis instance accidentally consumes too much memory, either Redis will crash for out of memory or the Linux kernel OOM killer will kill the Redis process.</li>
<li>Set an explicit maxmemory option limit in your instance in order to make sure that the instance will report errors instead of failing when the system memory limit is near to be reached.</li>
<li>If you are using Redis in a very write-heavy application, while saving an RDB file on disk or rewriting the AOF log <b>Redis may use up to 2 times the memory normally used</b>. The additional memory used is proportional to the number of memory pages modified by writes during the saving process, so it is often proportional to the number of keys (or aggregate types items) touched during this time. Make sure to size your memory accordingly.</li>
<li>Use daemonize no when run under daemontools.</li>
<li>Even if you have persistence disabled, Redis will need to perform RDB saves if you use replication, unless you use the new diskless replication feature, which is currently experimental.</li>
<li>If you are using replication, make sure that either your master has persistence enabled, or that it does not automatically restarts on crashes: slaves will try to be an exact copy of the master, so if a master restarts with an empty data set, slaves will be wiped as well.</li>
</ul><br />
<b>Running Redis on EC2</b><br />
<br />
<ul><li>Use HVM based instances, not PV based instances.</li>
<li>Don't use old instances families, for example: use m3.medium with HVM instead of m1.medium with PV.</li>
<li>The use of Redis persistence with <b>EC2 EBS volumes</b> needs to be handled with care since sometimes EBS volumes have high latency characteristics.</li>
<li>You may want to try the new <b>diskless replication</b> (currently experimetnal) if you have issues when slaves are synchronizing with the master.</li>
</ul><br />
<b>Reference:</b> <br />
<br />
<a href="http://blog.ijun.org/2014_04_01_archive.html" target="_blank">http://blog.ijun.org/2014_04_01_archive.html</a><br />
<br />
<a href="http://blog.ijun.org/2014/11/configuring-magento-to-use-redis.html" target="_blank">http://blog.ijun.org/2014/11/configuring-magento-to-use-redis.html</a><br />
<br />
<a href="http://blog.ijun.org/2014/12/install-apache-24-php-56-and-mysql-56.html" target="_blank">http://blog.ijun.org/2014/12/install-apache-24-php-56-and-mysql-56.html</a><br />
<br />
<a href="http://redis.io/topics/admin">http://redis.io/topics/admin</a><br />
<br />
<a href="http://www.cyberciti.biz/faq/howto-install-linux-apache-mariadb-php-lamp-stack-on-centos7-rhel7/">http://www.cyberciti.biz/faq/howto-install-linux-apache-mariadb-php-lamp-stack-on-centos7-rhel7/</a><br />
<br />
<a href="http://serverfault.com/questions/629937/centos-7-apache2-httpd-mod-fastcgi-installation-impossible">http://serverfault.com/questions/629937/centos-7-apache2-httpd-mod-fastcgi-installation-impossible</a><br />
<br />
<a href="http://blog.famillecollet.com/post/2014/08/01/Apache-httpd-server-2.4.10-and-PHP-FPM-5.6-in-Fedora-21">http://blog.famillecollet.com/post/2014/08/01/Apache-httpd-server-2.4.10-and-PHP-FPM-5.6-in-Fedora-21</a><br />
<br />
<a href="https://wiki.apache.org/httpd/PHP-FPM">https://wiki.apache.org/httpd/PHP-FPM</a><br />
<br />
<a href="http://sharadchhetri.com/2014/10/04/install-redis-server-centos-7-rhel-7/">http://sharadchhetri.com/2014/10/04/install-redis-server-centos-7-rhel-7/</a><br />
<br />
<a href="https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/s1-networkscripts-interfaces.html">https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/s1-networkscripts-interfaces.html</a><br />
<br />
<a href="http://technovergence-en.blogspot.ca/2012/03/mysql-from-utf8-to-utf8mb4.html" target="_blank">http://technovergence-en.blogspot.ca/2012/03/mysql-from-utf8-to-utf8mb4.html</a>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-53631416125744855862019-01-12T12:50:00.000-08:002019-01-12T12:50:53.554-08:00Install Vue<b>Install NVM (Node Version Manager):</b><br />
<br />
$ curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.34.0/install.sh | bash<br />
<br />
<b>Note:</b> You need to logout and log back in.<br />
<br />
<b>List the available node versions:</b><br />
<br />
$ nvm ls-remote<br />
<br />
<b>Install a node version:</b><br />
<br />
$ nvm install 10.15.0<br />
<br />
<b>Use a certain version of node:</b><br />
<br />
$ nvm use 10.15.0<br />
<br />
<b>Check node version:</b><br />
<br />
$ node -v<br />
<br />
<b>List installed node versions:</b><br />
<br />
$ nvm ls<br />
<br />
<b>Install Vue:</b><br />
<br />
$ npm install -g @vue/cli<br />
<br />
<b>Create a Vue project:</b><br />
<br />
$ vue create my_project<br />
<br />
<b>Switch to the project and add more tools:</b><br />
<br />
$ cd my_project<br />
$ npm install vue-axios axios iview --save<br />
$ npm install js-beautify --save-dev<br />
<br />
<b>Add the following lines under "scripts" key:</b><br />
<br />
$ vim package.json<br />
<br />
<pre><code class="html">"scripts": {
"build": "vue-cli-service build --mode production",
"watch": "vue-cli-service build --mode development --watch",
},</code></pre><br />
<b>Set up some Vue configuration:</b><br />
<br />
$ vim vue.config.js<br />
<br />
<pre><code class="html">module.exports = {
baseUrl: '/dist/',
outputDir: 'dist',
runtimeCompiler: true,
};</code></pre><br />
<b>Install a hot reload tool to run Go application:</b><br />
<br />
$ go get -u github.com/oxequa/realize<br />
$ realize init<br />
<br />
<b>Start realize:</b><br />
<br />
$ realize start<br />
Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-43304984782812504752018-11-30T20:53:00.000-08:002018-11-30T20:53:42.158-08:00To determine Vue current development modeIn main.js:<br />
<br />
<pre><code class="js">windows['VueMode'] = process.env.NODE_ENV;</code></pre><br />
<b>Note:</b> If you would like to use process.env.NODE_ENV in a sub component, put it inside one of the lifecycle hooks. For example, created, or mounted.Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-36888059569694056332018-11-16T20:15:00.002-08:002018-11-16T20:15:47.295-08:00mtail - extract whitebox monitoring data from application logs for collection in a timeseries databasemtail - extract whitebox monitoring data from application logs for collection in a timeseries database<br />
<br />
$ curl -L https://github.com/google/mtail/releases/download/v3.0.0-rc16/mtail_v3.0.0-rc16_linux_amd64 -o mtail<br />
<br />
$ vim linecounter.mtail<br />
<br />
<pre><code class="html"># simple line counter
counter line_count
/$/ {
line_count++
}</code></pre><br />
$ mtail --progs linecounter.mtail --logs debug.log<br />
<br />
$ echo hi1 >> debug.log<br />
$ echo hi2 >> debug.log<br />
<br />
$ curl http://localhost:3903/metrics<br />
<br />
<b>Reference:</b> <br />
<br />
<a href="https://groob.io/posts/prometheus-intro/" target="_blank">https://groob.io/posts/prometheus-intro/</a><br />
<br />
<a href="https://github.com/google/mtail" target="_blank">https://github.com/google/mtail</a>Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-51217329108701135642018-10-29T20:09:00.000-07:002018-10-29T20:09:55.109-07:00Running php-fpm in Docker based on CentOS 7Running php-fpm in Docker based on CentOS 7<br />
<br />
<b>Dockerfile:</b><br />
<br />
<pre><code class="html">FROM centos:centos7
RUN curl 'https://setup.ius.io/' -o setup-ius.sh \
&& bash setup-ius.sh \
&& rm -f setup-ius.sh \
&& yum -y update \
&& yum -y install \
php71u-common \
php71u-cli \
php71u-fpm \
php71u-opcache \
php71u-xml \
php71u-json \
php71u-pdo \
php71u-mysqlnd \
php71u-intl \
php71u-mbstring \
php71u-mcrypt \
php71u-gd \
php71u-soap \
php71u-process \
php71u-pecl-redis \
php71u-pecl-xdebug \
php71u-fpm-httpd
EXPOSE 9000
CMD ["php-fpm", "-F"]</code></pre><br />
<b>Build the docker image:</b><br />
<br />
# docker build -t junhsieh/php7.1-fpm:0.0.0 .<br />
<br />
<b>Some important settings to be changed:</b><br />
<br />
# vim /etc/php-fpm.d/www.conf<br />
<br />
<pre><code class="html">; <b>Change ownership:</b>
user = php-fpm
group = php-fpm
; <b>Note:</b> Ubuntu uses www-data user. Add php-fpm user to www-data group if the other container used it.
; # groupadd -g 33 www-data
; # useradd www-data -m -c 'web user' -u 33 -g 33
; # usermod -a -G www-data php-fpm
; # id php-fpm
; Now, restart this container to ensure php-fpm user is in www-data group.
; <b>Bind port 9000 to the all interfaces:</b>
listen = 9000
;listen = [::]:9000
; <b>Note:</b> PHP-FPM has a listen.client_allowed setting which allows you to set a list of IPs that can connect, or leave blank for any IP to connect. However, even with it being left blank, the issue still persisted. Digging into the official PHP-FPM repo, I discovered that you also need to set listen = [::]:9000 which then began to allow any IP to connect.
; <b>Note:</b> <a href="https://stackoverflow.com/questions/19806945/docker-and-connections-between-containers">https://stackoverflow.com/questions/19806945/docker-and-connections-between-containers</a>
; <b>Comment out the following line:</b>
;listen.allowed_clients = 127.0.0.1
; <b>Note:</b> "listen.allowed_clients = any" will not work.
; <b>Note:</b> "listen.allowed_clients = other-container-name" will not work. IP address only.
; <b>Uncomment the following line to debug the issue:</b>
catch_workers_output = yes
; <b>Note:</b> Comment it out on production.
</code></pre><br />
<b>xdebug setting:</b><br />
<br />
<pre><code class="html">### xdebug setting
###
; Enable xdebug extension module
zend_extension=xdebug.so
;zend_extension=/usr/lib64/php/modules/xdebug.so
xdebug.default_enable=1
xdebug.remote_enable=1
xdebug.remote_handler=dbgp
xdebug.remote_host=localhost
xdebug.remote_port=9009
; Note: php-fpm uses port 9000 as well.
xdebug.remote_log=/tmp/xdebug.log
xdebug.remote_connect_back=0
xdebug.remote_autostart=0
xdebug.remote_mode=req
xdebug.max_nesting_level=1000</code></pre><br />
<b>Reference:</b><br />
<br />
<a href="https://developers.redhat.com/blog/2014/12/29/running-php-fpm-in-docker/" target="_blank">https://developers.redhat.com/blog/2014/12/29/running-php-fpm-in-docker/</a>Jun Hsiehhttp://www.blogger.com/profile/00640061359079216681noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-52692573741017487812018-10-14T20:53:00.002-07:002018-10-14T20:53:49.107-07:00usermod and groupmod not found in Alpine Linux Docker Imageusermod and groupmod not found in Alpine Linux Docker Image<br />
<br />
$ vim Dockerfile<br />
<br />
<pre><code class="html">FROM alpine:latest
RUN apk --no-cache add shadow \
&& usermod -u 2500 elasticsearch \
&& groupmod -g 2500 elasticsearch</code></pre>Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-36576868052402285662018-10-08T18:56:00.000-07:002018-10-08T18:56:03.700-07:00Caused by: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative names matching IP address XXXXX found<pre><code class="html">Caused by: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative names matching IP address XXXXX found
at sun.security.ssl.Alerts.getSSLException(Unknown Source) ~[na:1.8.0_51]
at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source) ~[na:1.8.0_51] </code></pre><br />
# vim LDAPConnTest.java<br />
<br />
<pre><code class="java">Hashtable<String, Object> objEnvironment;
objEnvironment = new Hashtable<String, Object>(11);
objEnvironment.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
objEnvironment.put(Context.PROVIDER_URL, "LDAPS://domain:636");
objEnvironment.put(Context.SECURITY_AUTHENTICATION, "simple");
objEnvironment.put(Context.SECURITY_PRINCIPAL, <username>);
objEnvironment.put(Context.SECURITY_CREDENTIALS, <Password>);
objEnvironment.put("java.naming.ldap.attributes.binary", <attributes>);
System.setProperty("javax.net.ssl.trustStore", "certificates".concat(File.separator).concat("cacerts"));
this.objLDAPContext = new InitialLdapContext(objEnvironment, null);</code></pre><br />
# java main -Dcom.sun.jndi.ldap.object.disableEndpointIdentification=true<br />
<br />
Note: Do not use the -Dcom.sun.jndi.ldap.object.disableEndpointIdentification=true option on production server.<br />
<br />
<pre><code class="html">Improve LDAP support Endpoint identification has been enabled on LDAPS connections.
To improve the robustness of LDAPS (secure LDAP over TLS ) connections, endpoint identification algorithms have been enabled by default.
Note that there may be situations where some applications that were previously able to successfully connect to an LDAPS server may no longer be able to do so. Such applications may, if they deem appropriate, disable endpoint identification using a new system property: com.sun.jndi.ldap.object.disableEndpointIdentification.
Define this system property (or set it to true) to disable endpoint identification algorithms.</code></pre><br />
<b>Reference:</b> <br />
<br />
<a href="https://stackoverflow.com/questions/51622117/issue-with-dns-naming-and-certificates-ldap-context" target="_blank">https://stackoverflow.com/questions/51622117/issue-with-dns-naming-and-certificates-ldap-context</a><br />
<br />
<a href="https://www.oracle.com/technetwork/java/javase/8u181-relnotes-4479407.html" target="_blank">https://www.oracle.com/technetwork/java/javase/8u181-relnotes-4479407.html</a>Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-44321095887677676652018-10-08T18:10:00.000-07:002018-10-08T18:10:20.881-07:00fail to run in alpine docker with error "no such file or directory"fail to run in alpine docker with error "no such file or directory"<br />
<br />
<b>Edit main.go:</b><br />
<br />
# vim main.go<br />
<br />
<pre><code class="go">package main
import (
"fmt"
"log"
"net/http"
)
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Printf("Hello World\n")
fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:])
}
func main() {
http.HandleFunc("/", handler)
log.Fatal(http.ListenAndServe(":8080", nil))
}</code></pre><br />
<b>Build main.go:</b><br />
<br />
# CGO_ENABLED=0 go build main.go<br />
<br />
<b>Note:</b> Static build: GOOS=linux GOARCH=amd64 go build main.go<br />
<br />
<b>Dockerfile:</b><br />
<br />
<pre><code class="html">#
FROM alpine:latest
RUN apk --no-cache add ca-certificates
COPY main /usr/local/bin
CMD /usr/local/bin/main</code></pre><br />
<b>Build Docker image:</b><br />
<br />
# docker build -t exp/main:0.0.0 .<br />
Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-17545993775824582982018-09-23T17:03:00.000-07:002018-09-23T17:03:21.084-07:00Golang, mysql: Error 1040: Too many connections too many open filesGolang, mysql: Error 1040: Too many connections too many open files<br />
<br />
<ul><li>Opening and closing databases can cause exhaustion of resources.</li>
<li>Failing to read all rows or use <span style="background-color: yellow;">rows.Close()</span> reserves connections from the pool.</li>
<li>Using <span style="background-color: yellow;">Query()</span> for a statement that doesn’t return rows will reserve a connection from the pool.</li>
<li>Failing to be aware of how <a href="http://go-database-sql.org/prepared.html">prepared statements</a> work can lead to a lot of extra database activity.</li>
</ul><br />
<b>Check MySQL connections:</b><br />
<br />
<pre><code class="sql">mysql> SHOW PROCESSLIST;</code></pre><br />
<pre><code class="html">mysql> SHOW STATUS LIKE '%connections%';</code></pre><br />
<pre><code class="html">mysql> SHOW STATUS LIKE '%threads%';</code></pre><br />
<pre><code class="html">Check network connection files:</code></pre><br />
# lsof -i<br />
<br />
<b>Set some connection limits in Go code:</b><br />
<br />
<pre><code class="go">db.SetMaxOpenConns(1000)
db.SetMaxIdleConns(300)
db.SetConnMaxLifetime(0)</code></pre><br />
<b>Release the resource after used:</b><br />
<br />
<pre><code class="go">rows, err := db.Query("select name from beehives")
if err != nil {
panic(err)
}
defer rows.Close()</code></pre><br />
<b>Reference:</b> <br />
<br />
<a href="https://stackoverflow.com/questions/28135580/golang-mysql-error-1040-too-many-connections" target="_blank">https://stackoverflow.com/questions/28135580/golang-mysql-error-1040-too-many-connections</a><br />
<br />
<a href="http://go-database-sql.org/surprises.html" target="_blank">http://go-database-sql.org/surprises.html</a><br />
<br />
<a href="http://go-database-sql.org/prepared.html" target="_blank">http://go-database-sql.org/prepared.html</a><br />
<br />
<a href="https://access.redhat.com/solutions/1160343" target="_blank">https://access.redhat.com/solutions/1160343</a>Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-21748997322445276012018-09-22T15:42:00.000-07:002018-09-22T15:42:53.204-07:00Add Swap to a Amazon EC2 instance with an EBS (Elastic Block Store) volumeThe Amazon EC2 instance does not come with the swap partition by default. You will need to add the swap or paging space manually.<br />
<br />
Swap space are useful for systems having less memory (RAM). If your system facing problem of lack of memory continuously and you don’t want to increase memory on server, Then it can be helpful to enable swap in your system. Swap is comparatively much slower than physical memory but operating system uses swap space in case system goes out of memory. To know more about working of swap visit <a href="http://en.wikipedia.org/wiki/Paging">here</a>.<br />
<br />
<b>Creating a swap file in current file system:</b><br />
<br />
# dd if=/dev/zero of=/myswap bs=1M count=4096<br />
<br />
<b>Note:</b> if - input file.<br />
<b>Note:</b> of - output file.<br />
<b>Note:</b> bs - block size.<br />
<br />
# mkswap /myswap<br />
# chown root:root /myswap<br />
# chmod 0600 /myswap<br />
<br />
# swapon /myswap<br />
<br />
# free -h<br />
<br />
<pre><code class="html"> total used free shared buff/cache available
Mem: 3.8G 940M 115M 5.6M 2.8G 2.6G
Swap: 4.0G 0B 4.0G</code></pre><br />
# swapon -s<br />
<br />
<pre><code class="html">Filename Type Size Used Priority
/myswap file 4194300 0 -1</code></pre><br />
<b>To make the swap enable on system boot, run the following command:</b><br />
<br />
# sh -c "echo /myswap swap swap defaults 0 0 >> /etc/fstab"<br />
<br />
Or edit the /etc/fstab:<br />
<br />
# vim /etc/fstab<br />
<br />
<pre><code class="html">/myswap swap swap defaults 0 0</code></pre><br />
<b>To verify the swap:</b><br />
<br />
# cat /etc/fstab | grep -i swap<br />
<br />
<pre><code class="html">/myswap swap swap defaults 0 0</code></pre><br />
# cat /proc/meminfo | grep -i swap<br />
<br />
<pre><code class="html">SwapCached: 0 kB
SwapTotal: 4194300 kB
SwapFree: 4194300 kB</code></pre><br />
<b>To check the current system's swappiness:</b><br />
<br />
Swappiness is a ratio of how often the system will write to the swapfile: if set to zero, the system will only swap to avoid running out of memory (the error above); if set to 100, the system will attempt to swap all the time. The default is set at 60. Since we want to utilize the swap only when necessary.<br />
<br />
The Linux kernel provides a tweakable setting that controls how often the swap file is used, called swappiness.<br />
<br />
A swappiness setting of zero means that the disk will be avoided unless absolutely necessary (you run out of memory), while a swappiness setting of 100 means that programs will be swapped to disk almost instantly.<br />
<br />
Ubuntu system comes with a default of 60, meaning that the swap file will be used fairly often if the memory usage is around half of my RAM.<br />
<br />
# cat /proc/sys/vm/swappiness<br />
<br />
<pre><code class="html">30</code></pre><br />
<b>To configure swappiness:</b><br />
<br />
# sh -c "echo vm.swappiness = 0 >> /etc/sysctl.conf && sysctl -p"<br />
<br />
<b>Add the swap space to a second disk instead of the current disk:</b><br />
<br />
If you would like to add the swap space to a second disk, first we need to add extra disk in our system first. In my case new disk mounted as /dev/xvdd (It may change in your case). Then, run the following commands:<br />
<br />
# mkswap -f /dev/xvdd<br />
# swapon /dev/xvdd<br />
<br />
# vim /etc/fstab<br />
<br />
<pre><code class="html">dev/xvdd swap swap defaults 0 0</code></pre><br />
If at all possible, I'd advise not to use swap on EC2 unless you're 99% certain you won't have to use it (I.E. it's only there for emergency). When we disabled swap on some of our EC2 instances our monthly EBS IO costs probably halved.<br />
<br />
You are right, the Ubuntu EC2 EBS images don't come with swap space configured (for 11.04 at least). The "regular" instance-type images do have a swap partition, albeit only 896 MB on the one I tested.<br />
<br />
If some process blows up and you don't have swap space, your server could come to a crawling halt for a good while before the OOM killer kicks in, whereas with swap, it merely gets slow. For that reason, I always like to have swap space around, even with enough RAM. Here's your options:<br />
<br />
Create an EBS volume (2-4 times the size of your RAM), attach it to your instance (I like calling it /dev/xvdm for "memory"), sudo mkswap /dev/xvdm, add it to fstab, sudo swapon -a, and you're good to go. I have done this before and it works fine, and it is probably a bit faster than using a swap file, but for a server that doesn't normally depend on swap performance, I personally think the minor performance improvement is not worth the added complexity of having to attach a volume. (Update: It's probably not faster than a swap file on instance storage, since EBS has become known for lousy and unpredictable performance.)<br />
<br />
Or you might be able to repartition your disk to add a swap partition, though this might require creating a new AMI. I have not been able to do this in a running instance, because I cannot unmount the root file system, and I do not even have access to the disk device (/dev/xvda), only the partition (xvda1).<br />
<br />
Or you can create a swap file. This is my preferred solution right now.<br />
<br />
# dd if=/dev/zero of=/var/swapfile bs=1M count=2048 &&<br />
# chmod 600 /var/swapfile &&<br />
# mkswap /var/swapfile &&<br />
# echo /var/swapfile none swap defaults 0 0 | tee -a /etc/fstab &&<br />
# swapon -a<br />
<br />
Done. :) I know a lot of people feel icky about using files instead of partitions, but it certainly works well enough as emergency swap space.<br />
<br />
<b>Reference:</b> <br />
<br />
<a href="http://tecadmin.net/add-swap-partition-on-ec2-linux-instance/" target="_blank">http://tecadmin.net/add-swap-partition-on-ec2-linux-instance/</a><br />
<a href="http://stackoverflow.com/questions/17173972/how-do-you-add-swap-to-an-ec2-instance" target="_blank">http://stackoverflow.com/questions/17173972/how-do-you-add-swap-to-an-ec2-instance</a><br />
<a href="http://askubuntu.com/questions/103915/how-do-i-configure-swappiness" target="_blank">http://askubuntu.com/questions/103915/how-do-i-configure-swappiness</a><br />
<a href="http://serverfault.com/questions/218750/why-dont-ec2-ubuntu-images-have-swap" target="_blank">http://serverfault.com/questions/218750/why-dont-ec2-ubuntu-images-have-swap</a><br />
<a href="http://danielgriff.in/2014/add-swap-space-to-ec2-to-increase-performance-and-mitigate-failure/" target="_blank">http://danielgriff.in/2014/add-swap-space-to-ec2-to-increase-performance-and-mitigate-failure/</a>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-22547806951276097642018-09-22T10:13:00.000-07:002018-09-22T10:13:08.157-07:00How to extract deb packageHow to extract deb package<br />
<br />
$ dpkg -x mysql-cluster-community-server_7.6.7-1ubuntu16.04_amd64.deb ~/tmp/out<br />
<br />
or<br />
<br />
$ ar -xv mysql-cluster-community-server_7.6.7-1ubuntu16.04_amd64.deb<br />
<br />
Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-66959818228068639522018-09-22T09:28:00.001-07:002018-09-22T09:28:02.497-07:00Increase MySQL maximum connection limitIncrease MySQL maximum connection limit<br />
<br />
<pre><code class="html">2018-09-22T15:00:42.130042Z 0 [Warning] Changed limits: max_open_files: 1024 (requested 5000)
2018-09-22T15:00:42.130087Z 0 [Warning] Changed limits: max_connections: 214 (requested 500)
2018-09-22T15:00:42.130091Z 0 [Warning] Changed limits: table_open_cache: 400 (requested 2000)</code></pre><br />
<b>Note:</b> Ubuntu has moved from Upstart to Systemd in version 15.04 and no longer respects the limits in /etc/security/limits.conf for system services. These limits now apply only to user sessions.<br />
<br />
# mkdir /etc/systemd/system/mysql.service.d<br />
# vim /etc/systemd/system/mysql.service.d/override.conf<br />
<br />
<pre><code class="html">[Service]
LimitNOFILE=infinity
LimitMEMLOCK=infinity</code></pre><br />
# systemctl daemon-reload<br />
# systemctl restart mysql<br />
<br />
<b>Reference:</b> <br />
<br />
<a href="https://stackoverflow.com/questions/30901041/can-not-increase-max-open-files-for-mysql-max-connections-in-ubuntu-15" target="_blank">https://stackoverflow.com/questions/30901041/can-not-increase-max-open-files-for-mysql-max-connections-in-ubuntu-15</a>Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-15934709563717710772018-09-16T13:10:00.000-07:002018-09-16T13:10:14.007-07:00To let other different users login to Amazon's EC2 instance<b>Solution 1:</b><br />
<br />
<b>On the local machine, get public key for later use:</b><br />
<br />
$ test -f ~/.ssh/id_rsa.pub && cat ~/.ssh/id_rsa.pub || ssh-keygen -t rsa -C "me@example.com" && cat ~/.ssh/id_rsa.pub<br />
<br />
<b>On the remote EC2 instance, create a new user and add the new user to sudo group:</b><br />
<br />
# useradd USER_NAME -m -s /bin/bash -c 'admin user' && usermod -aG sudo USER_NAME<br />
<br />
# visudo<br />
<br />
<pre><code class="html">%sudo ALL=(ALL:ALL) NOPASSWD: ALL</code></pre>or<br />
<pre><code class="html">USER_NAME ALL=(ALL) NOPASSWD: ALL</code></pre><br />
# sudo su - USER_NAME<br />
<br />
$ mkdir ~/.ssh \<br />
&& chmod 700 ~/.ssh \<br />
&& touch ~/.ssh/authorized_keys \<br />
&& chmod 600 ~/.ssh/authorized_keys \<br />
&& vim ~/.ssh/authorized_keys<br />
<br />
<b>On the local machine:</b><br />
<br />
$ ssh -i ~/.ssh/id_rsa -p 22 USER_NAME@1.2.3.4<br />
or<br />
$ mosh --ssh="ssh -i ~/.ssh/id_rsa -p 22" USER_NAME@1.2.3.4<br />
<br />
<b>Solution 2:</b><br />
<br />
# vim /etc/ssh/sshd_config<br />
<br />
<pre><code class="html">PasswordAuthentication = yes</code></pre><br />
# systemctl restart sshd.service<br />
<br />
<b>Solution 3:</b><br />
<br />
<b>Add a new user:</b><br />
<br />
# useradd testuser -m -c 'test user'<br />
<br />
<b>Switch to the new account so that newly created files have the proper ownership:</b><br />
<br />
# sudo su - testuser<br />
<br />
$ mkdir ~/.ssh<br />
<br />
$ chmod 700 ~/.ssh<br />
<br />
<b>Note:</b> this step is very important; without these exact file permissions, you will not be able to log into this account using SSH.<br />
<br />
$ touch ~/.ssh/authorized_keys<br />
<br />
$ chmod 600 ~/.ssh/authorized_keys<br />
<br />
Login to Amazon Web Services console. Then, go to EC2 and create a new key pair: machineName_userName.<br />
<br />
It will generate a machineName_userName.pem file for you to download.<br />
<br />
Upload machineName_userName.pem to your Linux instance.<br />
<br />
<b>Change the permission of the machineName_userName.pem:</b><br />
<br />
# chmod 400 machineName_userName.pem<br />
<br />
<b>Retrieving the Public Key for Your Key Pair on Linux:</b><br />
<br />
# ssh-keygen -y<br />
<br />
When prompted to enter the file in which the key is, specify the path to your .pem file; for example:<br />
<br />
/path_to_key_pair/machineName_userName.pem<br />
<br />
The command returns the public key:<br />
<br />
<pre><code class="html">ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQClKsfkNkuSevGj3eYhCe53pcjqP3maAhDFcvBS7O6V
hz2ItxCih+PnDSUaw+WNQn/mZphTk/a/gU8jEzoOWbkM4yxyb/wB96xbiFveSFJuOp/d6RJhJOI0iBXr
lsLnBItntckiJ7FbtxJMXLvvwJryDUilBMTjYtwB+QhYXUMOzce5Pjz5/i8SeJtjnV3iAoG/cQk+0FzZ
qaeJAAHco+CY/5WrUBkrHmFJr6HcXkvJdWPkYQS3xqC0+FmUZofz221CBt5IMucxXPkX4rWi+z7wB3Rb
BQoQzd8v7yeb7OzlPnWOyN0qFU0XA246RA8QFYiCNYwI3f05p6KLxEXAMPLE</code></pre><br />
<b>Edit the authorized_keys file with your favorite text editor and paste the public key for your key pair into the file:</b><br />
<br />
# sudo su - testuser<br />
<br />
$ vim ~/.ssh/authorized_keys<br />
<br />
<pre><code class="html">ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQClKsfkNkuSevGj3eYhCe53pcjqP3maAhDFcvBS7O6V
hz2ItxCih+PnDSUaw+WNQn/mZphTk/a/gU8jEzoOWbkM4yxyb/wB96xbiFveSFJuOp/d6RJhJOI0iBXr
lsLnBItntckiJ7FbtxJMXLvvwJryDUilBMTjYtwB+QhYXUMOzce5Pjz5/i8SeJtjnV3iAoG/cQk+0FzZ
qaeJAAHco+CY/5WrUBkrHmFJr6HcXkvJdWPkYQS3xqC0+FmUZofz221CBt5IMucxXPkX4rWi+z7wB3Rb
BQoQzd8v7yeb7OzlPnWOyN0qFU0XA246RA8QFYiCNYwI3f05p6KLxEXAMPLE</code></pre><br />
<b>Remove the private key from the server if you do not need it anymore:</b><br />
<br />
# rm /path_to_key_pair/machineName_userName.pem<br />
<br />
<b>Reference:</b> <br />
<br />
<a href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/managing-users.html" target="_blank">http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/managing-users.html</a><br />
<a href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#how-to-generate-your-own-key-and-import-it-to-aws"
target="_blank">http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#how-to-generate-your-own-key-and-import-it-to-aws</a>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-33910781693530317672018-09-15T22:21:00.003-07:002018-09-15T22:21:53.225-07:00Building MySQL from Source CodeBuilding MySQL from Source Code<br />
<br />
# apt-get update && apt-get install build-essential cmake bison -y<br />
<br />
# cd /usr/local/src \<br />
&& git clone https://github.com/mysql/mysql-server.git --depth 1 \<br />
&& mkdir bld \<br />
&& cd bld \<br />
&& cmake ../mysql-server \<br />
-DDOWNLOAD_BOOST=1 \<br />
-DWITH_BOOST=/usr/local/src/bld \<br />
&& makeJun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-74796655805360789212018-09-15T16:54:00.002-07:002018-09-15T16:54:10.586-07:00Persistent key-value database written in GoPersistent key-value database written in Go<br />
<br />
badger - <a href="https://github.com/dgraph-io/badger/" target="_blank">https://github.com/dgraph-io/badger/</a><br />
<br />
<b>Note:</b> <a href="https://blog.dgraph.io/post/badger/" target="_blank">https://blog.dgraph.io/post/badger/</a><br />
<br />
bbolt - <a href="https://github.com/etcd-io/bbolt" target="_blank">https://github.com/etcd-io/bbolt</a>Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-49876233292790897952018-09-15T09:55:00.000-07:002018-09-15T09:55:02.457-07:00MySQL backup and replication toolsMySQL backup and replication tools<br />
<br />
<b>mydumper</b><br />
<br />
<a href="https://github.com/maxbube/mydumper" target="_blank">https://github.com/maxbube/mydumper</a><br />
<br />
<b>Percona XtraBackup</b><br />
<br />
<a href="https://www.percona.com/software/mysql-database/percona-xtrabackup" target="_blank">https://www.percona.com/software/mysql-database/percona-xtrabackup</a><br />
<br />
<b>Percona MySQL-AutoXtraBackup</b><br />
<br />
<a href="https://github.com/Percona-Lab/MySQL-AutoXtraBackup" target="_blank">https://github.com/Percona-Lab/MySQL-AutoXtraBackup</a><br />
<br />
<a href="https://www.percona.com/blog/2017/11/27/perconalab-autoxtrabackup-v1-5-0-release/" target="_blank">https://www.percona.com/blog/2017/11/27/perconalab-autoxtrabackup-v1-5-0-release/</a><br />
<br />
<b>mysqlbinlog</b><br />
<br />
# mysqlbinlog --result-file=test.log /var/log/mysql/mysql-bin.000001<br />
<br />
<b>Note:</b> You may edit the result file and delete any statements you don't want to execut. Then, do mysql -u root -p < test.log<br />
<br />
# mysqlbinlog --result-file=test.log --base64-output=DECODE-ROWS /var/log/mysql/mysql-bin.000001<br />
# mysqlbinlog --result-file=test.log -v /var/log/mysql/mysql-bin.000001<br />
# mysqlbinlog --result-file=test.log -vv /var/log/mysql/mysql-bin.000001<br />
<br />
<b>Note:</b> Do more research on <span style="background-color: yellow;">global transaction identifier (GTID)</span> vs <span style="background-color: yellow;">binary log position</span><br />
<br />
Get binary log from the remote server:<br />
<br />
# mysqlbinlog --read-from-remote-server --host=192.168.101.2 -p mysqld-bin.000001<br />
<br />
<b>Note:</b> <a href="https://www.percona.com/blog/2012/01/18/backing-up-binary-log-files-with-mysqlbinlog/" target="_blank">https://www.percona.com/blog/2012/01/18/backing-up-binary-log-files-with-mysqlbinlog/</a><br />
<br />
The output of mysqlbinlog can be used as the input of the mysql client to redo the statements contained in the binary log:<br />
<br />
# cd /var/log/mysql<br />
# mysqlbinlog mysql-bin.000001 | mysql -u root -p<br />
<br />
<b>canal</b><br />
<br />
<a href="https://github.com/alibaba/canal" target="_blank">https://github.com/alibaba/canal</a><br />
<br />
<b>Pingcap syncer:</b><br />
<br />
<a href="https://github.com/pingcap/docs/blob/master/tools/syncer.md" target="_blank">https://github.com/pingcap/docs/blob/master/tools/syncer.md</a>Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-59835931328491760232018-09-15T09:37:00.000-07:002018-09-15T09:37:00.565-07:00To Embed Static Assets in GoTo Embed Static Assets in Go<br />
<br />
<b>Requirements:</b><br />
<br />
Compression<br />
Optional decompression<br />
Loading from the local file system<br />
Reproducible builds<br />
Config file<br />
http.FileSystem Interface<br />
<br />
vfsgen - <a href="https://github.com/shurcooL/vfsgen" target="_blank">https://github.com/shurcooL/vfsgen</a><br />
fileb0x - <a href="https://github.com/UnnoTed/fileb0x" target="_blank">https://github.com/UnnoTed/fileb0x</a><br />
go-assets - <a href="https://github.com/jessevdk/go-assets" target="_blank">https://github.com/jessevdk/go-assets</a><br />
<br />
<b>Reference:</b> <br />
<br />
<a href="https://tech.townsourced.com/post/embedding-static-files-in-go/" target="_blank">https://tech.townsourced.com/post/embedding-static-files-in-go/</a>Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0tag:blogger.com,1999:blog-2361800974999510068.post-9284274147352630762018-09-08T21:40:00.003-07:002018-09-08T21:40:43.183-07:00Convert string to hexadecimal on command lineConvert string to hexadecimal on command line<br />
<br />
$ echo -n "Hello" | od -A n -t x1<br />
<br />
<pre><code class="html">48 65 6c 6c 6f</code></pre>Jun Hsiehhttp://www.blogger.com/profile/02948505991334024239noreply@blogger.com0