AWS Elasticsearch Service初探

创建Elasticsearch域

通过VPC内网访问ES的VPC地址,方法有很多种可以使用负载均衡进行转发访问

访问成功看到熟悉的You Know, for Search

导入日志数据

使用filebeat和elasticsearch的pipeline进行数据导入,注意AWS提供的VPC地址实际上是个负载均衡其elasticsearch所在端口为443,filebeat必须使用filebeat-oss不然会出现x_pack认证问题

使用kibana console创建pipeline

access log pipeline

PUT _ingest/pipeline/weblog_combined
{
    "description": "Ingest pipeline for Combined Log Format",
    "processors": [
      {
        "grok": {
          "field": "message",
          "patterns": [
            """%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "%{WORD:verb} %{DATA:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response:int} (?:-|%{NUMBER:bytes:int}) %{QS:referrer} %{QS:agent}"""
          ]
        }
      },
      {
        "date": {
          "field": "timestamp",
          "formats": [
            "dd/MMM/YYYY:HH:mm:ss Z"
          ]
        }
      },
      {
        "user_agent": {
          "field": "agent"
        }
      }
    ]
}

error log pipeline

PUT _ingest/pipeline/weblog_nginx_error
{
  "description": "Ingest pipeline Nginx error logs",
  "processors": [
    {
      "grok": {
        "field": "message",
        "patterns": [
          """^(?<timestamp>%{YEAR}[./]%{MONTHNUM}[./]%{MONTHDAY} %{TIME}) \[%{LOGLEVEL:severity}\] %{POSINT:pid}#%{NUMBER:threadid}\:( \*%{NUMBER:connectionid})? %{DATA:message}(,|$)( client: %{IPORHOST:client})?(, server: %{IPORHOST:server})?(, request: "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion}))")?(, upstream: "%{DATA:upstream}")?(, host: "%{IPORHOST:vhost}")?"""
        ]
      }
    },
    {
      "date": {
        "field": "timestamp",
        "formats": [
          "YYYY/MM/dd HH:mm:ss"
        ]
      }
    }
  ]
}

修改filebeat配置文件

---
filebeat.inputs:
  -
    enabled: true
    exclude_lines:
      - GET.*ELB-HealthChecker\/2.0
    fields:
      index_name: weblog_access
    paths:
      - /home/wwwlogs/access.log
    pipeline: weblog_combined
    tags:
      - weblogs
      - nginx
    type: log
  -
    enabled: true
    exclude_lines:
      - "newsyslog\\[.*\\]: logfile turned over"
    fields:
      index_name: weblog_nginx_error
    paths:
      - /home/wwwlogs/nginx_error.log
    pipeline: weblog_nginx_error
    tags:
      - weblogs
      - nginx
    type: log
logging.to_files: false
logging.to_syslog: true
logging.level: debug
output.elasticsearch:
  hosts:
    - "https://:443"
  index: "%{[fields.index_name]:logs}-%{+YYYY.MM.dd}-fbt_%{[agent.version]}"
  protocol: https
  ssl.verification_mode: none
setup.template.enabled: false
setup.ilm.enabled: false

启动filebeat观察输出如有错误需要清洗log或者更改pipeline patterns

filebeat -e

from Nginx Logs to Elasticsearch (in AWS) Using Pipelines and Filebeat (no Logstash)

删除文件中多匹配字符串行

grep -F -v test example.txt > example.txt.tmp && mv example.txt.tmp example.txt

其思路为通过字符串查找匹配行然后输出不匹配的行到临时文件,然后再用临时文件覆盖原文件达到删除匹配字符串行的效果.

替换war包中的文件

  • jar tvf 查找war包中的文件
    jar tvf sw-V2.0.1.war | grep AppValues
    
    13908 Tue Jun 30 10:27:42 CST 2020 WEB-INF/classes/cn/com/xx/xx/common/constant/AppValues.class
  • jar xvf 解压完整路径
jar xvf sw-V2.0.1.war WEB-INF/classes/cn/com/xx/xx/common/constant/AppValues.class

inflated: WEB-INF/classes/cn/com/xx/xx/common/constant/AppValues.class
  • 替换文件
  • jar uvf 压缩至war包
    jar uvf sw-V2.0.1.war WEB-INF/classes/cn/com/xx/xx/common/constant/AppValues.class
    
    adding: WEB-INF/classes/cn/com/xx/xx/common/constant/AppValues.class(in = 13894) (out= 5232)(deflated 62%)

在内网如何知道其他ip的MAC地址

 arp -a
? (192.168.1.169) at 48:a1:95:c3:d8:14 [ether] on eno1
? (172.17.0.2) at 02:42:ac:11:00:02 [ether] on docker0
? (192.168.1.243) at 50:7b:9d:0b:4c:52 [ether] on eno1
? (192.168.1.193) at 64:b0:a6:ca:85:d0 [ether] on eno1
? (192.168.1.190) at 50:e5:49:c6:fe:3e [ether] on eno1
? (192.168.1.234) at e0:89:7e:b3:7c:3f [ether] on eno1
gateway (192.168.1.1) at 08:57:00:a9:06:d7 [ether] on eno1
? (192.168.1.241) at 54:bf:64:40:11:c7 [ether] on eno1
? (192.168.1.185) at 80:ad:16:47:a9:01 [ether] on eno1
? (192.168.1.212) at 4a:1d:9b:f9:41:df [ether] on eno1
? (192.168.1.78) at 80:18:44:e8:91:fc [ether] on eno1
? (192.168.1.240) at 7c:76:68:ce:eb:ec [ether] on eno1

注意缓存

Oracle10g

由于之前安装过10g卸载重新安装后出现如下错误

ORA-12560: TNS:protocol adapter error

哎西

检查注册列表是不是没卸载干净

regedit
HKEY_LOCAL_MACHINE\SOFTWARE\ORACLE

看看新安装的ORACLE_SID是不是对的发现也是对的

再看看oracle相关服务正不正常

services.msc

发现也正常

接着看环境变量在系统变量中oracle_sid发现跟重新安装的sid不一样更改成一样的解决问题

~, ^, *在package.json中的含义

node用npm解决依赖的方式是靠package.json中用x.y.z的方式表明版本信息,其中

  • ~锁定y位如~1.2.3表示依赖大于或等于1.2.3但小于1.3.0的任意版本
  • ^锁定x位如^1.2.3表示依赖大于或等于1.2.3但小于2.0.0的任意版本
  • *取决于所在位置如1.*表示依赖大于或等于1.0.0但小于2.0.0的任意版本
    语义化版本

Auto start service on Boot

CentOS or RHEL 6.x

chkconfig --add httpd
chkconfig httpd on
chkconfig --list 可以看到其他启动服务和启动级别

/etc/rc.d/init.d下可观察启动脚本

/etc/centos-release 是否存在观察centOS版本

RHEL or CentOS 7.x

systemctl enable php-fpm

WP with Live2D

引言

blog的浮动窗体小组件挺有意思加之

步骤

GitHub地址https://github.com/xiazeyu/live2d-widget.js

由于是node项目依赖于npm先clone下来项目

git clone https://github.com/xiazeyu/live2d-widget.js

然后

npm install

发现在项目目录下node依赖的文件夹已经出现了node_modules

观察项目目录下的package.json看看怎样启动项目

"scripts": {
    "update:submodule": "git submodule foreach git pull origin master",
    "_esdoc": "./node_modules/.bin/esdoc",
    "_changelog": "conventional-changelog --outfile CHANGELOG.md --release-count 0",
    "_titlechangelog": "sed -i '1i\\# Changelog\\n\\n' CHANGELOG.md",
    "inst:dev": "npm install -g commitizen && npm install -g conventional-changelog-cli && npm install",
    "build:dev": "./node_modules/.bin/webpack --progress --colors",
    "build:prod": "./node_modules/.bin/webpack --env prod --progress --colors && npm run build:module",
    "build:module": "./node_modules/.bin/webpack --env prod --progress --colors --config webpack.config.common.js",
    "build:docs": "git pull && npm run _changelog && git add CHANGELOG.md && npm run update:submodule && git add ghpages",
    "build:esdoc": "cp lib/stats.html ghpages/stats.html -f && npm run _titlechangelog && npm run _esdoc",
    "deploy:doc": "git pull && git checkout master && cd ghpages/ && git status && git add --all && git commit -m \"Update docs\" && git push origin HEAD:master --force && cd ..",
    "deploy": "build:prod && build:docs",
    "lint": "eslint --ext .js ./src",
    "commit": "git pull && git-cz",
    "v:major": "npm version major",
    "v:pmajor": "npm version premajor",
    "v:minor": "npm version minor",
    "v:pminor": "npm version preminor",
    "v:patch": "npm version patch",
    "v:ppatch": "npm version prepatch",
    "test": "echo \"Error: no test specified\" && exit 1",
    "postinstall": "opencollective-postinstall"
  }

执行之

npm run build:dev

启动后由webpack提供动态调试打开dev.html

默认加载的modelizumi可以在项目目录\src\config\defaultConfig.js中的jsonPath修改,目前支持的model可从live2d-widget.jsCurrent supported models:找到,替换下面地址中红色部分即可

https://unpkg.com/live2d-widget-model-izumi@latest/assets/izumi.model.json

替换完成保存后webpack会重新加载配置js

观察dev.html其初始化方式和项目目录下\src\index.js中可配置参数,根据情况进行修改,生成的所需引用的js文件都在lib文件夹下

由于站点使用的是WordPress搭建的使用的是Php先装个叫Snippets的插件然后用其在php代码中插入<script>标签来完成加载生成的js

继续阅读“WP with Live2D”