Notes on Webpage development
nginx configuration
--prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-file-aio --with-ipv6 --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-stream_ssl_preread_module --with-http_addition_module --with-http_xslt_module=dynamic --with-http_image_filter_module=dynamic --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_slice_module --with-http_stub_status_module --with-http_perl_module=dynamic --with-http_auth_request_module --with-mail=dynamic --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module --with-google_perftools_module --with-debug --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' --with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E'
Hexo setup
Online action includes Nginx, SSL certificate and git Hexo installation. SSL certification need to be renewed by Let’s Encrypt. Check Hexo documentation for more info.
Hostname, Nginx
CentOS 7 from Vultr.com
Install git and Nginx:
yum -y update
yum install -y git nginx
vi /index.html
<!DOCTYPE html>
<html>
<head>
<title></title>
<meta charset="UTF-8">
</head>
<body>
<p>Nginx running</p>
</body>
</html>
vi /etc/nginx/nginx.conf
server {
listen 80;
server_name www.xxx.com;
root /;
}
Port 80 with http done.
SSL certification - Let’ s Encrypt
Using certbot by Let’s Encrypt:
yum install certbot
then obtain the ssl certificate:
mkdir -p /.well-known/acme-challenge
certbot certonly --webroot --email xxx@xxx.com -w / -d www.xxx.com -d xxx.com
To renew the certificate using
certbot renew
or automatically
vi /etc/crontab
0 23 28 * * root certbot renew --quiet --deploy-hook "systemctl restart nginx"
crontab /etc/crontab
Then we need to add the ssl certificate in Nginx:
vi /etc/nginx/nginx.conf
server{
listen 80;
server_name www.xxx.com xxx.com *.www.xxx.com;
root /;
add_header Strict-Transport-Security max-age=15768000;
return 301 https://$server_name$request_uri;
# redirect to https and https only
}
server {
listen 443 ssl;
server_name www.xxx.com xxx.com *.www.xxx.com;
root /;
ssl_certificate /etc/letsencrypt/live/xxx.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/xxx.com/privkey.pem;
}
done.
systemctl restart nginx
Now we can use ssl https to visit our index. on your host name or ip address.
Hexo installation on CenOS
initial git lib:
mkdir /GitLibrary
chmod -R 755 /GitLibrary
cd /GitLibrary
git init --bare hexo.git
then configure the hook:
vi /GitLibrary/hexo.git/hooks/post-update.sample
add
git --work-tree=/ --git-dir=/GitLibrary/hexo.git checkout -f
and then
mv post-update.sample post-update
chmod +x /GitLibrary/hexo.git/hooks/post-update
in some cases post-receive instead of post-update.
Now we finish jobs at server to deploy Hexo. But I still want to add shadowsocks to set up VPN as well as Blog on my server.
Shadowsocks
install shadococks, below git install may be already done:
yum install python-setuptools
easy_install pip
pip install --upgrade pip
yum install git
pip install git+https://github.com/shadowsocks/shadowsocks.git@master
create shadowsocks service:
vi /usr/lib/systemd/system/shadowsocks.service
[Unit]
Description=Shadowsocks Server
Documentation=https://github.com/shadowsocks/shadowsocks
After=network.target remote-fs.target nss-lookup.target
[Service]
Type=forking
ExecStart=/usr/bin/ssserver -c /usr/share/nginx/etc/shadowsocks.json -d start
ExecReload=/bin/kill -HUP $MAINPID
ExecStop=/usr/bin/ssserver -d stop
[Install]
WantedBy=multi-user.target
create the directory /usr/share/nginx/etc/ if not exist
vi /usr/share/nginx/etc/shadowsocks.json
and write:
{
"port_password":
{
"8388":"pswd",
"port2":"pswd2"
},
"_comment":
{
"8388":"for Bob",
"port2":"for me"
},
"timeout":300,
"method":"aes-256-gcm",
"fast_open": false
}
notice we choose aes-256-gcm encryption method here.
then run it
systemctl enable shadowsocks
systemctl start shadowsocks
Hexo in my mac
Important: git and node.js are prerequisite.
Then Hexo installation (we install blog system in ~/Blog):
npm install hexo-cli hexo-server hexo-deployer-git -g
hexo init ~/Blog
cd Blog
npm install
in the configuration file _config.yml:
# URL
### If your site is put in a subdirectory, set url as 'http://yoursite.com/child' and root as '/child/'
url: http://XXX.com
......
# Deployment
### Docs: https://hexo.io/docs/deployment.html
deploy:
type: git
repo: root@XXX.com:/GitLibrary/hexo
branch: master
and you can specify your host name, to create CNAME in the /blog/source and write XXX.com.
Then using
hexo clean
hexo generate
hexo server
we can check our blog offline.
To deploy to the remote server:
hexo deploy
visit your XXX.com : )
Trouble shot
Firewall block
CentOS 7 using firewalld and may block some port, to disable firewall, use
systemctl stop firewalld
SSH key in your PC
ssh-keygen
ssh-copy-id root@host
Or you can find the key upload in your Vultr.com console…
Reference and many thanks
Update on the 2019/10/13
重装系统后懒得在服务器上安装git了（其实就是不想再折腾了，印象中挺麻烦的）
每次直接scp /public 文件夹就完了
Hexo start
Create a new post
$ hexo new "My New Post"
More info: Writing
Run server
$ hexo server
More info: Server
Generate static files
$ hexo generate
More info: Generating
Deploy to remote sites
$ hexo deploy
More info: Deployment
MathJax Test
在LHC上，我们可以简单计算一下它的对撞事例率：它的亮度恒定为 $$ L=10^{34}cm^{-2}s^{-1} $$ $L=10^{34}cm^{-2}s^{-1}$
而非弹性散射的强子截面
$$sigma(pp)approx70mb$$
$sigma(pp)approx70mb$
因此它的对撞事例率为$7times 10^8 Hz$。而束流管中的束团（bunches）每25ns对撞一次，因此每一次对撞的有效对撞事件就是$7times2.5 = 17.5$。最后再考虑到束团的非饱和性（80% of the available bunch spaces will be filled）。则每次对撞竟会同时产生17.5/0.8=22个事例，这就是事例堆叠（Pile up）。如何在这22次相同时间的对撞中正确挑选出来自同一个对撞的径迹和衰变粒子，这就需要我们对事例堆叠进行处理。
$$ mu+ $$ J-PARC $mu^+$
Wiki (Sphinx)
pip install sphinx
pip install sphinx-rtd-theme
pip install m2r
### 22-10-11: change to python3 -m pip install m2r2
in the doc directory, type
sphinx-quickstart
to create the first program. Follow the instructions.
In the conf.py:
import os
import sys
sys.path.insert(0, os.path.abspath('/usr/local/lib/python3.7/site-packages'))
import m2r
import sphinx_rtd_theme
extensions = ['sphinx_rtd_theme',
'm2r',
]
html_theme = 'sphinx_rtd_theme'
source_suffix = ['.rst', '.md']
At the beginning, recommonmark module was used (pip install –upgrade recommonmark). But it got some problem in the following building process when .md was linked indirectly.
Update on Otc. 2022
m2r should be changed to m2r2:
https://github.com/crossnox/m2r2
m2r Compatibility: https://bugs.archlinux.org/task/75433
Sphinx: https://github.com/sphinx-doc/sphinx
readthedocs/sphinx_rtd_theme: https://github.com/readthedocs/sphinx_rtd_theme
Write the file with Markdown
In the index.rst:
Welcome to Cedric Zhang's wiki
===============================================
.. toctree::
:maxdepth: 3
:caption: Muon g-2 at J-PARC:
include
.. toctree::
:maxdepth: 3
:caption: Others:
sub2
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search
here, the include.rst is the link to other .md files. Relative directory. So that one need not to move the exsiting .md file in other folder (course notes for me).
Acceleration Course
===============================================
.. mdinclude:: ../../AccCtrlCourse/Timing_200123.md
.. mdinclude:: ../../AccCtrlCourse/200116-Protection_system.md
.. mdinclude:: ../../AccCtrlCourse/191114.md
.. mdinclude:: ../../AccCtrlCourse/191128.md
note here using mdinclude, not typically include.
Everything done.
make html
then scp the entire /html to the remote sever.
One problem is the figures can not be delt with successfully in .md files. Therefore need to scp the figure folds manually. No good solution to me yet….
Math: the same as markdown \(E = mc^2\)
How to change the page max-width
In _templates folder, create layout.html and paste
{% extends "!layout.html" %}
{% block footer %} {{ super() }}
<style>
.wy-nav-content { max-width: none; }
</style>
{% endblock %}
from
https://github.com/readthedocs/sphinx_rtd_theme/issues/295
others:
https://blog.deimos.fr/2014/10/02/sphinxdoc-and-readthedocs-theme-tricks-2/
https://pythonhosted.org/sphinxjp.themes.basicstrap/design.html
Reference
Spinx
https://docs.readthedocs.io/en/stable/intro/getting-started-with-sphinx.html
http://www.sphinx-doc.org/en/master/usage/quickstart.html
http://www.pythondoc.com/sphinx/tutorial.html
https://buildmedia.readthedocs.org/media/pdf/sphinx/1.7/sphinx.pdf
https://www.writethedocs.org/guide/
https://muffinresearch.co.uk/selectively-including-parts-readme-rst-in-your-docs/
https://reinout.vanrees.org/weblog/2010/12/08/include-external-in-sphinx.html
https://lyk6756.github.io/2018/01/30/read_the_docs.html
https://www.ibm.com/developerworks/cn/opensource/os-sphinx-documentation/index.html
Spinx_rtd_theme
https://github.com/readthedocs/sphinx_rtd_theme
https://sphinx-rtd-theme.readthedocs.io/en/latest/
http://siosio.hatenablog.com/entry/2012/06/05/234818
m2r
https://miyakogi.github.io/m2r/index.html
https://github.com/miyakogi/m2r/blob/master/docs/example.md
Recommonmark
https://stackoverflow.com/questions/46278683/include-my-markdown-readme-into-sphinx
Doku wiki introduction
https://www.dokuwiki.org/dokuwiki
The author: https://github.com/splitbrain & https://www.splitbrain.org/
examples:
https://www-he.scphys.kyoto-u.ac.jp/member/n.kamo/wiki/doku.php?id=study:software:root
nginx with SSL has been built in advance, see the previous blog post.
Preparation: GUI on CentOS7
https://www.itzgeek.com/how-tos/linux/centos-how-tos/install-gnome-gui-on-centos-7-rhel-7.html
Install PHP 7.4
https://computingforgeeks.com/how-to-install-php-7-4-on-centos-7/
Step 1: Add EPEL and REMI Repository
sudo yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
sudo yum -y install https://rpms.remirepo.net/enterprise/remi-release-7.rpm
Step 2: Install PHP 7.4 on CentOS 7
sudo yum -y install yum-utils
sudo yum-config-manager --enable remi-php74
Install PHP 7.4 on CentOS 7.
sudo yum update
sudo yum install php php-cli
Use the next command to install additional packages:
sudo yum install php php-cli php-fpm php-mysqlnd php-zip php-devel php-gd php-mcrypt php-mbstring php-curl php-xml php-pear php-bcmath php-json
done.
php --version
php -v
Setup Nginx, php-fpm on Centos 7
Configure php-fpm
php-fpm is bundled wit php7.
vi /etc/php-fpm.d/www.conf
; Start a new pool named 'www'.
[www]
; Unix user/group of processes
; Note: The user is mandatory. If the group is not set, the default user$
; will be used.
; RPM: apache Choosed to be able to access some dir as httpd
user = nginx
; RPM: Keep a group allowed to write in log dir.
group = nginx
listen = 127.0.0.1:9000
listen.owner = nginx
listen.group = nginx
listen.mode = 0660
[...]
Configure Nginx
vi /etc/nginx/nginx.conf
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name www.psi3770.com;
root /home/html;
add_header Strict-Transport-Security max-age=15768000;
return 301 https://$server_name$request_uri;
include /etc/nginx/default.d/*.conf;
location / {
index index.php index.html index.htm;
}
location ~ \.php$ {
try_files $uri =404;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_buffer_size 128k;
fastcgi_buffers 256 16k;
fastcgi_busy_buffers_size 256k;
fastcgi_temp_file_write_size 256k;
include fastcgi_params;
}
# Settings for a TLS enabled server.
#
server {
listen 443 ssl http2 default_server;
listen [::]:443 ssl http2 default_server;
server_name www.psi3770.com psi3770.com *.psi3770.com;
root /home/html;
ssl_certificate "/etc/letsencrypt/live/www.psi3770.com/fullchain.pem";
ssl_certificate_key "/etc/letsencrypt/live/www.psi3770.com/privkey.pem";
location / {
index index.php index.html index.htm;
try_files $uri $uri/ /index.php?$args;
}
error_page 404 /404.html;
location = /40x.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
location ~ \.php$ {
try_files $uri =404;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_buffer_size 128k;
fastcgi_buffers 256 16k;
fastcgi_busy_buffers_size 256k;
fastcgi_temp_file_write_size 256k;
include fastcgi_params;
}
}
Important: in https, 443 ports should also be set. THIS WAS NOT DISCUSSED IN ANY TROUBLE SHOOTING and I spent lots of time on it.
systemctl restart nginx php-fpm
systemctl stop firewalld
check by
systemctl status nginx php-fpm
Other tips:
chmod -cR 777 *
将递归地更改包括子目录在内的所有文件。
User operation:
useradd mynewuser
If you want to remove all the files for the user, then use -r:
userdel -r mynewuser
https://www.liquidweb.com/kb/how-to-remove-delete-a-user-on-centos-7/
WebDAV
Motivation: to build a WebDAV for the use of Zotero library.
The nginx in my server was built long time ago. In the default nginx installation, however, the WebDAV is not included. Therefore, this post is to upgrade nginx with WebDAV, without disdurbing the current nginx configuration.
Prerequisite
yum -y update
# Install the EPEL repository.
yum install epel-release -y
# Development repository to compile the WebDAV dynamic module for Nginx.
yum groupinstall "Development Tools" -y
yum install yum-utils pcre-devel zlib-devel libxslt-devel libxml2-devel -y
Nginx-dav module compile
nginx -V
to see my nginx version is 1.16.1, therefore download the package:
wget http://nginx.org/download/nginx-1.16.1.tar.gz
and also the module
git clone https://github.com/arut/nginx-dav-ext-module
copy the configuration from ($nginx -V), and configure the new nginx:
./configure --CONFIGURATIONS_FROM_NGINX-V --with-http_dav_module --add-dynamic-module=../nginx-dav-ext-module/
During this step, some libs may need to be installed. See this link. E.g.
./configure: error: the HTTP XSLT module requires the libxml2/libxslt
libraries. You can either do not enable the module or install the libraries.
...
./configure: error: the HTTP image filter module requires the GD library. You can either do not enable the module or install the libraries.
In my cases are:
yum -y install libxml2 libxml2-dev
yum -y install libxslt-devel
yum -y install gd-devel
yum -y install perl-devel perl-ExtUtils-Embed
yum -y install gperftools
finally, we can compile only the module:
make modules
and copy the module to current nginx path:
cp objs/ngx_http_dav_ext_module.so /etc/nginx/modules/
Configuration
prepare the root directory for WebDAV:
mkdir /home/html/webdav/
vi /etc/nginx/nginx.conf
Add:
load_module /etc/nginx/modules/ngx_http_dav_ext_module.so;
#.....
# in the https:
location /webdav {
auth_basic "Closed site";
auth_basic_user_file /etc/nginx/.passwords.list;
client_body_temp_path /home/html/webdav/tmp;
dav_methods PUT DELETE MKCOL COPY MOVE;
dav_ext_methods PROPFIND OPTIONS;
create_full_put_path on;
dav_access user:rw group:rw all:rw;
autoindex on;
client_max_body_size 0;
}
Here, my main website is psi3770.com (/home/html/). The WebDAV is under psi3770.com/webdav. Therefore the location is /webdav. Otherwise it can be simply
location / {
#.....
}
The other way is the indivdual .conf file, which shoule be included in nginx.conf:
### in the nginx.conf
### ...
include /etc/nginx/conf.d/*.conf;
vim /etc/nginx/conf.d/webdav.conf
location /webdav {
dav_methods PUT DELETE MKCOL COPY MOVE;
dav_ext_methods PROPFIND OPTIONS; # https://docs.oracle.com/cd/E19857-01/819-0824/agdav.html
create_full_put_path on; # auto create the directory if not exsiting
client_body_temp_path /tmp/nginx/client-bodies; ## temp path
dav_access user:rw group:rw all:rw; # default user:rw
autoindex on;
client_max_body_size 1G; # File size limit for new files, 0 means no limit
auth_basic "closed site";
auth_basic_user_file /etc/nginx/.passwords.list;
}
User and passwd
echo -n 'userName:' | sudo tee -a /etc/nginx/.passwords.list;
openssl passwd -apr1 | sudo tee -a /etc/nginx/.passwords.list;
Finally, check and reload the nginx.
# Use built-in validator to check the configuration file beforun reload.
nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
systemctl reload nginx
Reference
https://blog.ixk.me/recompilation-of-nginx-installed-for-apt-mode-adds-webdav.html
https://www.dazhuanlan.com/2019/11/30/5de19102679f1/
https://docs.jelastic.com/nginx-webdav-module/
http://nginx.org/en/docs/http/ngx_http_dav_module.html
https://deviant.engineer/2015/05/nginx-reverseproxy-centos7/
https://blog.acesheep.com/index.php/archives/834/
Knowledge of linux and groups:
https://www.linode.com/docs/tools-reference/linux-users-and-groups/
Syncthing
Use personal server as file synchronization cloud disk.
local PC (macOS):
Remote server:
wget https://github.com/syncthing/syncthing/releases/download/v0.14.47/syncthing-linux-amd64-v0.14.51.tar.gz tar xzvf syncthing-linux-amd64-v0.14.47.tar.gz cd syncthing-linux-amd64-v0.14.47 cp syncthing /usr/local/bin syncthing # run directly
Ctrl+cto turn off the syncthing and configure it.vim /root/.config/syncthing/config.xml
#<address>127.0.0.1:8384</address> <address>0.0.0.0:8384</address>
then restart the syncthing.
psi3770.com:8384
Systemd
vim /usr/lib/systemd/system/syncthing@.service
[Unit]
Description=Syncthing - Open Source Continuous File Synchronization for %I
Documentation=man:syncthing(1)
After=network.target
[Service]
User=%i
ExecStart=/usr/local/bin/syncthing -no-browser -no-restart -logflags=0
Restart=on-failure
SuccessExitStatus=3 4
RestartForceExitStatus=3 4
[Install]
WantedBy=multi-user.target
systemctl start syncthing@root.service
systemctl status
systemctl stop
systemctl enable # start at login
systemctl disable # start at login