Nginx with Let’s Encrypt on WordPress

使用Lets Encrypt提供免费的ssl证书


获取cerbot项目

git clone https://github.com/certbot/certbot.git

重点关注certbot-auto脚本,因为使用是lnmp进行blog搭建在生成证书时需要指定nginx插件来配合,这样在生成证书后会在nginx配置文件中自动写入证书配置

./certbot-auto --nginx –d “website”

而后会需要选择一些必要的信息注意console输出,需要注意的是由于使用的是aws的实例在执行脚本时会出现错误大概意思就是aws提供的linux系统不被识别,可以通过修改脚本来忽略这个检查,大概在810行处

elif [ -f /etc/redhat-release ]; then

替换为

elif [ -f /etc/redhat-release ] || grep 'cpe:.*:amazon_linux:2' /etc/os-release > /dev/null 2>&1; then

由于在命令中使用了nginx插件脚本在执行时会去找nginx和nginx配置文件,lnmp在安装nginx时没有将nginx.conf放置在/etc这个目录下,解决的方式是创建软连接,通过执行./certbot-auto –nginx –d “website”命令可见

The nginx plugin is not working; there may be problems with your existing configuration.
The error was: MisconfigurationError('Error while running nginx -c /etc/nginx/nginx.conf -t.\n\nnginx: [emerg] open() "/etc/nginx/nginx.conf" failed (2: No such file or directory)\nnginx: configuration file /etc/nginx/nginx.conf test failed\n',)

执行

ln -s /usr/local/nginx/conf/ /etc/nginx

重启lnmp访问站点可发现原有的证书颁发者变成了Let’sEncrypt Authority X3

如果证书过期尝试执行,会根据之前证书生成的记录尝试更新证书

./certbot-auto renew

Install Cmake3

yum remove cmake
yum install gcc-c++
wget https://cmake.org/files/v3.10/cmake-3.10.0.tar.gz
tar -xvzf cmake-3.10.0.tar.gz
cd cmake-3.10.0
./bootstrap
make
make install

AWS Elasticsearch Service初探

创建Elasticsearch域

通过VPC内网访问ES的VPC地址,方法有很多种可以使用负载均衡进行转发访问

访问成功看到熟悉的You Know, for Search

导入日志数据

使用filebeat和elasticsearch的pipeline进行数据导入,注意AWS提供的VPC地址实际上是个负载均衡其elasticsearch所在端口为443,filebeat必须使用filebeat-oss不然会出现x_pack认证问题

使用kibana console创建pipeline

access log pipeline

PUT _ingest/pipeline/weblog_combined
{
    "description": "Ingest pipeline for Combined Log Format",
    "processors": [
      {
        "grok": {
          "field": "message",
          "patterns": [
            """%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "%{WORD:verb} %{DATA:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response:int} (?:-|%{NUMBER:bytes:int}) %{QS:referrer} %{QS:agent}"""
          ]
        }
      },
      {
        "date": {
          "field": "timestamp",
          "formats": [
            "dd/MMM/YYYY:HH:mm:ss Z"
          ]
        }
      },
      {
        "user_agent": {
          "field": "agent"
        }
      }
    ]
}

error log pipeline

PUT _ingest/pipeline/weblog_nginx_error
{
  "description": "Ingest pipeline Nginx error logs",
  "processors": [
    {
      "grok": {
        "field": "message",
        "patterns": [
          """^(?<timestamp>%{YEAR}[./]%{MONTHNUM}[./]%{MONTHDAY} %{TIME}) \[%{LOGLEVEL:severity}\] %{POSINT:pid}#%{NUMBER:threadid}\:( \*%{NUMBER:connectionid})? %{DATA:message}(,|$)( client: %{IPORHOST:client})?(, server: %{IPORHOST:server})?(, request: "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion}))")?(, upstream: "%{DATA:upstream}")?(, host: "%{IPORHOST:vhost}")?"""
        ]
      }
    },
    {
      "date": {
        "field": "timestamp",
        "formats": [
          "YYYY/MM/dd HH:mm:ss"
        ]
      }
    }
  ]
}

修改filebeat配置文件

---
filebeat.inputs:
  -
    enabled: true
    exclude_lines:
      - GET.*ELB-HealthChecker\/2.0
    fields:
      index_name: weblog_access
    paths:
      - /home/wwwlogs/access.log
    pipeline: weblog_combined
    tags:
      - weblogs
      - nginx
    type: log
  -
    enabled: true
    exclude_lines:
      - "newsyslog\\[.*\\]: logfile turned over"
    fields:
      index_name: weblog_nginx_error
    paths:
      - /home/wwwlogs/nginx_error.log
    pipeline: weblog_nginx_error
    tags:
      - weblogs
      - nginx
    type: log
logging.to_files: false
logging.to_syslog: true
logging.level: debug
output.elasticsearch:
  hosts:
    - "https://:443"
  index: "%{[fields.index_name]:logs}-%{+YYYY.MM.dd}-fbt_%{[agent.version]}"
  protocol: https
  ssl.verification_mode: none
setup.template.enabled: false
setup.ilm.enabled: false

启动filebeat观察输出如有错误需要清洗log或者更改pipeline patterns

filebeat -e

from Nginx Logs to Elasticsearch (in AWS) Using Pipelines and Filebeat (no Logstash)

删除文件中多匹配字符串行

grep -F -v test example.txt > example.txt.tmp && mv example.txt.tmp example.txt

其思路为通过字符串查找匹配行然后输出不匹配的行到临时文件,然后再用临时文件覆盖原文件达到删除匹配字符串行的效果.

替换war包中的文件

  • jar tvf 查找war包中的文件
    jar tvf sw-V2.0.1.war | grep AppValues
    
    13908 Tue Jun 30 10:27:42 CST 2020 WEB-INF/classes/cn/com/xx/xx/common/constant/AppValues.class
  • jar xvf 解压完整路径
jar xvf sw-V2.0.1.war WEB-INF/classes/cn/com/xx/xx/common/constant/AppValues.class

inflated: WEB-INF/classes/cn/com/xx/xx/common/constant/AppValues.class
  • 替换文件
  • jar uvf 压缩至war包
    jar uvf sw-V2.0.1.war WEB-INF/classes/cn/com/xx/xx/common/constant/AppValues.class
    
    adding: WEB-INF/classes/cn/com/xx/xx/common/constant/AppValues.class(in = 13894) (out= 5232)(deflated 62%)