This is just a place for me to store some of the less encountered notes on administering a server. Unless specified otherwise, most notes are for an Ubuntu Server. It will be updated continuously.
Consider using xip.io or nip.io during development (Wildcard DNS)
This way you can avoid editing your hosts file. Quite useful if you didn't own a domain name as well as you can use this temporarily in your Nginx/Apache server configs. Examples from xip.io site:
10.0.0.1.xip.io resolves to 10.0.0.1
www.10.0.0.1.xip.io resolves to 10.0.0.1
mysite.10.0.0.1.xip.io resolves to 10.0.0.1
foo.bar.10.0.0.1.xip.io resolves to 10.0.0.1
Consider vtop as an alternative to htop
Consider setting the timezone to UTC for standardization
- https://serverfault.com/questions/191331/should-servers-have-their-timezone-set-to-gmt-utc
- http://yellerapp.com/posts/2015-01-12-the-worst-server-setup-you-can-make.html
sudo ln -sf /usr/share/zoneinfo/UTC /etc/localtime
Locale Issue on EC2 Ubuntu
When you first SSH into your EC2 instances, it is quite likely to see some locale error message. Here's how to resolve it.
- http://m-schmidt.eu/2013/03/29/fix-locale-setting-on-ubuntu-ec2-instances/
- https://stackoverflow.com/questions/12016318/aws-ec2-en-us-utf-8-issue
sudo apt install language-pack-en
# nano /etc/environment and add in these 2 lines
LANG=en_US.utf-8
LC_ALL=en_US.utf-8
Consider using Nginx mainline branch instead of the stable branch
The mainline branch is stable branch + bug fixes. Nginx maintainers recommend it over the stable branch actually.
sudo add-apt-repository -y ppa:nginx/development
sudo apt update
sudo apt install -y nginx
Consider upgrading your RSA/RSA 1024 SSH keys to Ed25519
Your old keys may not be as secure anymore, time to upgrade
# New Keys
ssh-keygen -o -a 100 -t ed25519 -f ~/.ssh/id_key_name -C you@company.com
# Upgrade Old keys
for keyfile in ~/.ssh/id_*; do ssh-keygen -l -f "${keyfile}"; done | uniq
Database
Consider using utf8mb4 character set in MySQL
UTF8mb4 fully supports Unicode and may even be better for security compared to utf8
MySQL Indexing
Indexes are usually added to fields that you use in the WHERE section of a query. Though sometimes the SELECT fields too.
When creating indexes, the columns order is important. A different order could meant a different in the query performance.
The more specific your indexes are, the better they tend to perform. But the more indexes you create, the more data it takes up in memory & disk space. It's a trade-off.
When a query is slow, first use EXPLAIN on that query to find out more information about it.
EXPLAIN SELECT * FROM users;
To show a table indexes
SHOW INDEXES FROM users;
Generate a more performant MySQL configuration file using Percona Tools
Just go to https://tools.percona.com/wizard, login or create an account and follow the step by step wizard instructions and download your new configuration file. Then compare against your current MYSQL server configuration file (/etc/mysql/mysql.conf.d/mysqld.cnf in Ubuntu) and combine the new parts you wanted.
Percona is the company that wrote the famous "High-Performance MySQL" book. Also check out their Xtrabackup for backing up your database.
Redis
Getting the latest stable version of Redis
# For Ubuntu
sudo apt-add-repository -y ppa:chris-lea/redis-server
sudo apt update
sudo apt install -y redis-server
Apache
Installing the latest stable version of apache
# Ubuntu
sudo add-apt-repository -y ppa:ondrej/apache2
sudo apt update
sudo apt install -y apache2
# Note if you use ppa:ondrej/apache2, the default MPM is 'event' not 'prefork'
Consider HTML 5 Boilerplate Project Server Configs Recommendations
- https://github.com/h5bp/server-configs (Nginx, Apache, IIS, GAE.etc)
Don't forget your strace
strace is a powerful debugging tool when you don't know what's going on on a production server. It has served me well since learning it in my school days. However, I often hesitate to recommend it to co-workers given my position as a junior. It's so useful I bet pretty much every sysadmin knows already.
# Get a list of running php-fpm processes
ps ax | grep php-fpm
# strace on 1 of the pids in this case 12345 to see what system calls pid 12345 makes
strace -r -p 12345
Consider pigz instead of gzip
pigz is the parallel implementation of gzip. Next time you want to gzip files, consider installing pigz if you have multicores
apt install pigz
PHP Specific (PHP-FPM, Opcache.etc). May move to another post
Disable xDebug on production server
Although xDebug is definitely an awesome tool for debugging your PHP code, it has an impact on performance and should be disabled on the production server most of the time.
sudo phpdismod xdebug
# Reload PHP-FPM afterwards
sudo service php7.2-fpm reload
CentOS Specific
Consider installing EPEL, IUS and Webtatic repositories
"EPEL provides only software that is not in the official CentOS and Red Hat repositories, IUS provides newer versions of software (like MySQL and PHP) that exist in the official repositories." - Rackspace Support
For me, I personally prefer Webtatic repositories as they appears to update more frequently for me for Nginx, PHP and MySQL.
Instructions:
- https://support.rackspace.com/how-to/install-epel-and-additional-repositories-on-centos-and-red-hat/ (EPEL, IUS)
- https://webtatic.com/projects/yum-repository/