2022-06-21

linux - crontab

命令格式

1
2
3

MIN HOUR DOM MON DOW CMD
# DOM: day of month
# DOW: day of week

查看

1	$ crontab -l

编辑

1	$ crontab -e

启动时运行

1	@reboot /home/ubuntu/app/consul/start.sh

时间格式

MIN HOUR DOM MON DOW CMD
 *   *    *   *   *   ...
 
# 特殊符号
*	任意值
,	多值分隔符
-	范围分隔符
/	间隔符 (step values)

# 示例
* * * * *       每分钟
30 08 10 06 *   每年的 6 月 10 号 8 点 30 分
30 * * * *      每小时的 30 分
00 11,16 * * *  每天的 11:00 和 16:00
00 09-18 * * *  每天的 9、10 ... 18 点 00 分
00 09-18 * * 1-5 周一至周五每天的 9、10 ... 18 点 00 分
*/10 * * * *    每十分钟

设置 PATH

1 2	$ crontab -e # 开头添加如下内容 PATH=/bin:/sbin:/usr/bin:/usr/sbin:/home/ubuntu/.local/bin

2022-06-19

python►notes

python - generate wheel

安装

1	$ pip3 install wheel setuptools

创建 `setup.py`

from setuptools import setup

setup(
    name='abtest',
    version='1.0',
    packages=['.abtest'],
)

setup.py 示例

import os.path

from setuptools import setup

REQUIREMENTS = 'requirements.txt'
requires = []
if os.path.isfile(REQUIREMENTS):
    with open(REQUIREMENTS, 'r') as f:
        requires = f.read().splitlines()

# VERSION i.e. 1.3
with open('VERSION', 'r') as f:
    v = f.read()
    setup(
        name='abtest',
        version=v.strip(),
        packages=['abtest', 'data', 'model', 'utils'], # 指定 package
        install_requires=requires # 指定依赖基本版本
    )

打包工程

1	python setup.py bdist_wheel --universal

2022-06-19

python►notes

python - request

引入

1	import urllib.request

发送请求

q = urllib.request.Request(url=url, method='GET')
with urllib.request.urlopen(q) as f:
	status = f.status
	data = f.read().decode('utf-8')

2022-06-12

coding►c++

c++ compiler

指定编译器

export PATH=/opt/scylladb/bin:$PATH
export LD_LIBRARY_PATH=/opt/scylladb/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/scylladb/lib:$LD_LIBRARY_PATH
export CC=/opt/scylladb/bin/gcc
export CXX=/opt/scylladb/bin/g++

编译选项

# warning
-Wall			开启警告（all: 应该开启的最小警告集合）
-Wextra		开启扩展警告
-Werror 	所有警告视为错误
-Wno-error=...	关闭某项警告视为错误
-Wno-<...>			关闭某项警告视为错误

附录

`-Wall`

-Waddress -Warray-bounds (only with -O2) -Wc++0x-compat -Wchar-subscripts
-Wenum-compare (in C/Objc; this is on by default in C++) -Wimplicit-int (C and
 Objective-C only) -Wimplicit-function-declaration (C and Objective-C only) 
-Wcomment -Wformat -Wmain (only for C/ObjC and unless -ffreestanding) 
-Wmissing-braces -Wnonnull -Wparentheses -Wpointer-sign -Wreorder -Wreturn-type 
-Wsequence-point -Wsign-compare (only in C++) -Wstrict-aliasing 
-Wstrict-overflow=1 -Wswitch -Wtrigraphs -Wuninitialized -Wunknown-pragmas 
-Wunused-function -Wunused-label -Wunused-value -Wunused-variable 
-Wvolatile-register-var

`-Wextra`

-Wclobbered -Wempty-body -Wignored-qualifiers -Wmissing-field-initializers
-Wmissing-parameter-type (C only) -Wold-style-declaration (C only) -Woverride-init
-Wsign-compare -Wtype-limits -Wuninitialized -Wunused-parameter (only with -Wunused
 or -Wall) -Wunused-but-set-parameter (only with -Wunused or -Wall)

warn 示例配置

-Wall -Wextra -Waggregate-return -Wcast-align -Wcast-qual -Wdisabled-optimization -Wdiv-by-zero -Wendif-labels -Wformat-extra-args -Wformat-nonliteral -Wformat-security -Wformat-y2k -Wimplicit -Wimport -Winit-self -Winline -Winvalid-pch -Wjump-misses-init -Wlogical-op -Werror=missing-braces -Wmissing-declarations -Wno-missing-format-attribute -Wmissing-include-dirs -Wmultichar -Wpacked -Wpointer-arith -Wreturn-type -Wsequence-point -Wsign-compare -Wstrict-aliasing -Wstrict-aliasing=2 -Wswitch -Wswitch-default -Werror=undef -Wno-unused -Wvariadic-macros -Wwrite-strings -Wc++-compat -Werror=declaration-after-statement -Werror=implicit-function-declaration -Wmissing-prototypes -Werror=nested-externs -Werror=old-style-definition -Werror=strict-prototypes

2022-06-12

coding►c++

内存泄漏排查

简述

$ pid=$(pgrep ranker_service)
$ pmap -x $pid
9085:   ./bin/ranker_service --cloud_region=aws-se --gray_cloud_list=none
Address           Kbytes     RSS   Dirty Mode  Mapping
0000000000400000   19180    7236       0 r-x-- ranker_service
00000000018bb000     280     168      24 r---- ranker_service
0000000001901000      92      40      40 rw--- ranker_service
0000000001918000     212     120     120 rw---   [ anon ]
0000023412800000  262144  216280  216280 rw---   [ anon ]
000002342f000000  528384  524292  524292 rw---   [ anon ]
0000023453800000  262144  262144  262144 rw---   [ anon ]
0000023469c00000  262144  262144  262144 rw---   [ anon ]
0000023486000000  262144  262144  262144 rw---   [ anon ]
00000234aec00000  524288  518296  518296 rw---   [ anon ]
00000234db400000  262144  150220  150220 rw---   [ anon ]
0000023546400000 23330816 23188160 23188160 rw---   [ anon ]
0000023b50400000  303104  294920  294920 rw---   [ anon ]
0000023b69000000  475136  458768  458768 rw---   [ anon ]
...
$ gcore $pid

dump

$ gdb bin/ranker_service core.9085
(gdb) dump binary memory result.bin 0x23546400000 0x23586400000  # dump 1G 内存数据

## dump 内存块 0x23546400000 全部数据
# 计算地址
> hex(0x0000023546400000 + 23330816 * 1024) # python3
'0x23ad6400000'
(gdb) dump binary memory result.bin 0x23546400000 0x23ad6400000  # dump 区块全部数据 23330816KB

统计 vtable

1
2

$ hexdump result.bin | awk '{printf "%s%s%s%s\n%s%s%s%s\n", $5,$4,$3,$2,$9,$8,$7,$6}' | sort | uniq -c | sort -nr  > hex.t
$ hexdump core.9085 | awk '{printf "%s%s%s%s\n%s%s%s%s\n", $5,$4,$3,$2,$9,$8,$7,$6}' | sort | uniq -c | sort -nr  | head

参考

2022-06-10

linux►software

nginx creat self-signed SSL certificate

创建证书

# 自认证证书
# 生成 nginx-selfsigned.pass.key
$ openssl genrsa -des3 -passout pass:over4chars -out nginx-selfsigned.pass.key 2048
# 生成 nginx-selfsigned.key
$ openssl rsa -passin pass:over4chars -in nginx-selfsigned.pass.key -out nginx-selfsigned.key
$ rm nginx-selfsigned.pass.key # 可以删除了
# 生成 nginx-selfsigned.csr
$ openssl req -new -key nginx-selfsigned.key -out nginx-selfsigned.csr # 一直回车
# 生成 nginx-selfsigned.crt
$ openssl x509 -req -sha256 -days 365 -in nginx-selfsigned.csr -signkey nginx-selfsigned.key -out nginx-selfsigned.crt

# 需要的文件
- nginx-selfsigned.key
- nginx-selfsigned.crt

或者

1	$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/nginx-selfsigned.key -out /etc/ssl/certs/nginx-selfsigned.crt

设置 nginx 使用 ssl

把生成的整数文件放到 /certs 目录下。

1
2
3

$ sudo mkdir /certs
$ sudo mv nginx-selfsigned.key /certs
$ sudo mv nginx-selfsigned.crt /certs

生成 dhparam.pem 文件。

1	$ openssl dhparam -out /certs/dhparam.pem 4096

创建 /etc/nginx/snippets/self-signed.conf ，输入如下内容。

ssl_certificate /certs/nginx-selfsigned.crt;
ssl_certificate_key /certs/nginx-selfsigned.key;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;
add_header Strict-Transport-Security "max-age=63072000; includeSubdomains";
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;

ssl_dhparam /certs/dhparam.pem;

设置 nginx site 配置。

server {
    listen 80 default_server;
    listen [::]:80 default_server;
    listen 443 ssl http2 default_server;
    listen [::]:443 ssl http2 default_server;

    server_name server_domain_or_IP;
    include snippets/self-signed.conf;
    ...
}

配置 http 跳转 https

按需配置。

server {
    listen 80 default_server;
    listen [::]:80 default_server;
    server_name server_domain_or_IP;
    return 302 https://$server_name$request_uri;
}

参考

https://www.digitalocean.com/community/tutorials/how-to-create-a-self-signed-ssl-certificate-for-nginx-in-ubuntu-16-04

2022-06-09

coding►c++►tools►gperftools

gperftools

安装

1	$ yum install gperftools gperftools-devel -y

Heap Profile

# dump
$ env LD_PRELOAD="/usr/lib64/libtcmalloc_and_profiler.so" HEAPPROFILE="heap-profile" <binary>
$ env LD_PRELOAD="/usr/lib64/libtcmalloc.so" HEAPPROFILE="heap-profile" <binary>
$ env LD_PRELOAD="/usr/lib64/libtcmalloc.so" HEAPPROFILE="/tmp/profile" HEAP_PROFILE_TIME_INTERVAL=60 <binary>

# analysis
$ pprof --text <binary> /tmp/profile....heap
$ pprof --svg <binary> /tmp/profile....heap > heap.svg

2022-06-08

架构►container►k8s

k8s usage

[toc]

节点管理

删除节点

1	$ sudo kubeadm reset cleanup-node # master 节点也可以清除

spec:
	template:
    # 按标签选择机器
    nodeSelector:
      <label-name>: <label-value>
    # 设置机器亲和性
    affinity:
      # 设置 node 亲和性
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
                - key: <label-name>
                  operator: In
                  values:
                    - <label-value>
      # 设置 pod 亲和性
      podAntiAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
                - key: app.kubernetes.io/name
                  operator: In
                  values:
                    - <pod-name>
                    - <pod-name>
            topologyKey: kubernetes.io/hostname

Deployment 管理

1	kubectl delete --all deployments d-n=<ns>

Pod 管理

# 查看 pod  信息
$ kubectl get pods -A  # A = all-namespaces

# 删除 pod
$ kubectl delete pod <name> --namespace=<ns>
# 批量删除
kubectl delete pod --all -n=<ns>

# 查看 pod 详情; ip 等
$ kubectl describe pod <name> --namespace=<ns>

Service

# 列出所有 service
k get services -o wide -A --sort-by=.metadata.name 

# 获取 service 详情; nodeport 等
k describe service kubernetes-dashboard -n kube-system

Token

1	microk8s kubectl create token -n kube-system default --duration=8544h

私有仓库

# docker login
docker login hub.company.com

# 不指定 namespace, 设置 default
kubectl create secret generic regcred \
    --from-file=.dockerconfigjson=~/.docker/config.json \
    --type=kubernetes.io/dockerconfigjson

# 每个 namespace 需要单独设置
kubectl create secret generic regcred \
    --from-file=.dockerconfigjson=~/.docker/config.json \
    --type=kubernetes.io/dockerconfigjson \
    --namespace=ace

# 修改 deployment
apiVersion: v1
kind: Pod
metadata:
  name: private-reg
spec:
  containers:
  - name: private-reg-container
    image: <your-private-image>
  # 添加 imagePullSecrets
  imagePullSecrets:
  - name: regcred

Secret

# 创建
kubectl create secret generic regcred \
    --from-file=.dockerconfigjson=$HOME/.docker/config.json \
    --type=kubernetes.io/dockerconfigjson \
    --namespace=default

# 查看
kubectl get secrets
kubectl get secretss <secret> --output=yaml

# 删除
kubectl delete secret <secret>

Namespace

# 列出所有的 namespace
$ kubectl get namespaces

# 创建命名空间
$ kubectl create namespace <space-name>

登录容器

1	kubectl exec --stdin --tty <pod-instance> -- /bin/bash

工具

k9s，基于 SHELL 的 K8S 客户端工具
k8s Lens，桌面端 K8S 客户端工具

2022-06-04

python►library►matplotlib

matplotlib

参数

alpha

透明度
float [0, 1]

bins

分段区间
可以是一个整数或一个序列
如果是整数n，将会均匀分为n个区间，否则按指定的序列进行分段

配置

subplot

绘制子图。

原型

1	subplot(nrows, ncols, index, **kwargs)

示例

import matplotlib.pyplot as plt

y = [i for i in range(100)]
z = [i * 1.1 for i in range(100)]

plt.subplot(2, 2, 1)
plt.xlabel('x label')
plt.ylabel('y label')
plt.plot(y, label='y')
plt.plot(z, label='z')
plt.legend()

#### no legend
plt.subplot(2, 2, 2)
plt.xlabel('x label')
plt.ylabel('y label')
plt.plot(y, label='y')
plt.plot(z, label='z')

plt.subplot(2, 2, 4)
plt.xlabel('x label')
plt.ylabel('y label')
plt.plot(y, label='y')
plt.plot(z, label='z')
plt.legend()

# no NO.3 subplot 
plt.show()

# 数据来自鸢尾花数据集，绘制散点图
columns = ['Id', 'SepalLengthCm', 'SepalWidthCm', 'PetalLengthCm', 'PetalWidthCm', 'Label']
for i in range(2, 5):
    plt.subplot(1, 3, i - 1)
    plt.scatter(features[:, 1], features[:, i], c=labels, s=50, cmap='viridis')
    plt.title(columns[i])
plt.show()

# 自定义 subplot size
start = 2
end = 5
fg, axs = plt.subplots(end - start, end - start, figsize=(25, 25))
for i in range(start, end):
    for j in range(start, end):
        axs[i - start, j - start].scatter(features[:, 1], features[:, i], c=labels, s=25, cmap='viridis')
        axs[i - start, j - start].set_title(columns[i] + ' - ' + columns[j])

2022-06-04

python►library►matplotlib

matplotlib usage

简介

Matplotlib

数据可视化

安装

1
2
3

sudo pip3 install matplotlib
# or
sudo apt install python3-matplotlib

示例程序

sample 1

import numpy as np
import matplotlib.pyplot as plt


x = np.linspace(0, 10, 1000)    # 作图的变量自变量
y = np.sin(x) + 1               # 因变量y
z = np.cos(x**2) + 1            # 因变量z

plt.figure(figsize=(8, 4))      # 设置图像大小
plt.plot(x, y, label='$\sin x+1$', color='red', linewidth=2)    # 作图, 设置标签, 线条颜色, 线条大小
plt.plot(x, z, 'b--', label='$\cos x^2+1$')
plt.xlabel('Time(s) ')  # x轴名称
plt.ylabel('Volt')      # y轴名称
plt.ylim(0, 2.2)        # 显示的y轴范围
plt.legend()            # 显示图例
plt.show()              # 显示作图结果

运行结果

Sample 2 - subplot

#!/bin/python
import matplotlib.pyplot as plt

# add a subplot
ax1=plt.subplot(2, 2, 1)
# add a subplot with no frame
ax2=plt.subplot(2, 2, 2, frameon=False)
# add a polar subplot
plt.subplot(2, 2, 3, projection='polar')
# add a red subplot that shares the x-axis with ax1
plt.subplot(2, 2, 4, sharex=ax1, facecolor='red')

#delete ax2 from the figure
plt.delaxes(ax2)
#add ax2 to the figure again
plt.subplot(ax2)

plt.show()

Sample 3 - drow

from cycler import cycler
import numpy as np
import matplotlib.pyplot as plt


x = np.linspace(0, 2 * np.pi)
offsets = np.linspace(0, 2*np.pi, 4, endpoint=False)
# Create array with shifted-sine curve along each column
yy = np.transpose([np.sin(x + phi) for phi in offsets])

# 1. Setting prop cycle on default rc parameter
plt.rc('lines', linewidth=4)
plt.rc('axes', prop_cycle=(cycler('color', ['r', 'g', 'b', 'y']) +
                           cycler('linestyle', ['-', '--', ':', '-.'])))
fig, (ax0, ax1) = plt.subplots(nrows=2)
ax0.plot(yy)
ax0.set_title('Set default color cycle to rgby')

# 2. Define prop cycle for single set of axes	color & width
ax1.set_prop_cycle(cycler('color', ['c', 'm', 'y', 'k']) +
                   cycler('lw', [1, 2, 3, 4]))	
ax1.plot(yy)
ax1.set_title('Set axes color cycle to cmyk')

# Tweak spacing between subplots to prevent labels from overlapping
fig.subplots_adjust(hspace=0.3)
plt.show()

Sample 4 - set size && save picture

#### way 1
from matplotlib.pyplot import figure
figure(figsize=(20, 10), dpi=80,)

#### way 2
plt.figure(figsize=(15, 10), dpi=80)

命令格式

查看

编辑

启动时运行

时间格式

设置 PATH

安装

创建 setup.py

打包工程

引入

发送请求

指定编译器

编译选项

附录

-Wall

-Wextra

warn 示例配置

简述

参考

创建证书

设置 nginx 使用 ssl

配置 http 跳转 https

参考

安装

Heap Profile

节点管理

删除节点

标签

Deployment 管理

Pod 管理

Service

Token

私有仓库

Secret

Namespace

登录容器

工具

参数

alpha

bins

配置

subplot

简介

安装

示例程序

sample 1

Sample 2 - subplot

Sample 3 - drow

Sample 4 - set size && save picture

创建 `setup.py`

`-Wall`

`-Wextra`