gitlab 使用

配置

修改外部访问地址

  • 修改配置文件 /etc/gitlab/gitlab.rb
    • external_url 'http://git.sample.com:port'
    • 注意:修改后,gitlab 监听的 port 也会随之更改

修改 clone 地址

  • http
    • 设置路径:Admin -> General -> Visibility and access controls -> Custom Git clone URL for HTTP(S)
  • ssh
    • 修改配置文件:/etc/gitlab/gitlab.rb,注意:host 和 port 要分开修改
      • gitlab_rails['gitlab_ssh_host'] = 'git.sample.com'
      • gitlab_rails['gitlab_shell_ssh_port'] = 2222

Spring注解

简述

Spring注解

注解 对象 说明
@Controller DispatcherServlet会自动扫描注解了此注解的类
@Service -
@Reponsitory -
@Component -
@Autowired 方法、字段 依赖注入
@Configuration 声明类为配置类
@ComponentScan 自动扫描指定包内特定注解并注册Bean,默认扫描@Service、@Component、@Controller、@Repository注解,可自定义规则
@Bean 方法、注解 声明返回值为Bean,可定义init、destroy方法,在Bean创建、销毁时调用
@Aspect 声明切面
@After 方法 后置建言,原方法前执行
@Before 方法 前置建言,原方法前执行
@Around 方法 环绕建言,在原方法执行前执行,在原方法执行后再执行
@PointCut - 声明切点,即定义拦截规则,确定有哪些方法会被切入
@Transactional 声明事务
@Cacheable 声明数据缓存
@Value 值注入,常与Sping EL表达式语言一起使用,注入普通字符,系统属性,表达式运算结果,其他Bean的属性,文件内容,网址请求内容,配置文件属性值等等
@PropertySource 指定文件地址,提供了一种方便的、声明性的机制,向Spring的环境添加PropertySource。与@Configuration一起使用
@PostConstruct 方法 构造函数执行完成之后执行该方法
@PreDestroy 方法 对象销毁之前执行该方法
@Profile 根据配置条件注册/激活
@EnableAsync 配置类 开启异步任务支持
@Async 类、方法 注解在方法上标示这是一个异步方法,在类上标示这个类所有的方法都是异步方法
@EnableScheduling 开启对计划任务的支持
@Scheduled 方法 声明该方法是计划任务,支持多种类型的计划任务:cron、fixDelay、fixRate
@Conditional 条件声明
@RunWith junit注解
@ContextConfiguration 用来加载配置ApplicationContext,其中classes属性用来加载配置类:@ContextConfiguration(classes = {TestConfig.class(自定义的一个配置类)})
@ActiveProfiles 用来声明活动的profile
@EnableWebMvc 配置类 开启SpringMvc的Mvc的一些默认配置:如ViewResolver,MessageConverter等。同时在自己定制SpringMvc的相关配置时需要做到两点:1.配置类继承WebMvcConfigurerAdapter类,2.必须使用这个@EnableWebMvc注解
@RequestMapping 类、方法 映射web请求(访问路径和参数
@ResponseBody 方法、类 将返回值放在response体内
@RequestBody 参数 request的参数在request body内,此注解放置在参数前
@PathVariable 参数 接受路径参数
@RestController 组合了@Controller和@ResponseBody,当我们只开发一个和页面交互数据的控制层的时候可以使用此注解
@ControllerAdvice 声明一个控制器建言,它也组合了@Component注解,会自动注册为Spring的Bean
@ExceptionHandler 方法 异常处理
@ModelAttribute 参数、方法 将键值对添加到全局,所有注解了@RequestMapping的方法可获得此键值对
@WebAppConfiguration 一般用在测试上,注解在类上,用来声明加载的ApplicationContext是一个WebApplicationContext。他的属性指定的是Web资源的位置,默认为src/main/webapp,我们可以修改为:@WebAppConfiguration(“src/main/resources”)
@EnableAutoConfiguration 此注释自动载入应用程序所需的所有Bean,该注解组合了@Import注解,@Import注解导入了EnableAutoCofigurationImportSelector类,它使用SpringFactoriesLoader.loaderFactoryNames方法来扫描具有META-INF/spring.factories文件的jar包。而spring.factories里声明了有哪些Bean自动配置
@ImportResource 用来加载额外配置(xml等定义)
@ConfigurationProperties 将properties属性与一个Bean及其属性相关联,从而实现类型安全的配置
@ConditionalOnBean 条件注解,当容器里有指定Bean的条件下
@ConditionalOnClass 条件注解,当类路径下有指定的类的条件下
@ConditionalOnExpression 条件注解,基于SpEL表达式作为判断条件
@ConditionalOnJava 条件注解,基于JVM版本作为判断条件
@ConditionalOnJndi 条件注解,在JNDI存在的条件下查找指定的位置
@ConditionalOnMissingBean 条件注解,当容器里没有指定Bean的情况下
@ConditionalOnMissingClass 条件注解,当类路径下没有指定的类的情况下
@ConditionalOnNotWebApplication 条件注解,当前项目不是web项目的条件下
@ConditionalOnResource 条件注解,类路径是否有指定的值
@ConditionalOnSingleCandidate 条件注解,当指定Bean在容器中只有一个,后者虽然有多个但是指定首选的Bean
@ConditionalOnWebApplication 条件注解,当前项目是web项目的情况下
@EnableConfigurationProperties 声明开启属性注入,使用@Autowired注入,例:@EnableConfigurationProperties(HttpEncodingProperties.class)。
@AutoConfigureAfter 在指定的自动配置类之后再配置,例:@AutoConfigureAfter(WebMvcAutoConfiguration.class)

JSR 注解

注解 说明 JSR版本
@Resource 依赖注入 JSR250
@Inject 依赖注入 JSR330

对比

Resource & Inject & Autowired

注解 Processor 注入方式
Resource AutowiredAnnotationBeanPostProcessor 默认字段名(field name),若失败退化为类型(type)
Inject AutowiredAnnotationBeanPostProcessor 默认依据类型注入
Autowired CommonAnnotationBeanPostProcessor 默认依据类型注入

参考

机器学习 - 工具

数据处理

工具 用途
pandas 提供大量快速便捷处理数据的函数及高效操作大型数据集所需的工具
numpy 用于存储和处理大型矩阵与维度数组
matplotlib 图标绘图系统
seaborn 基于 Matplotlib 的数据可视化库

预测工具

工具 用途
scikit-learn (sklearn) 常用的数据预测分析工具,机器学习
pytorch 深度学习工具
tensorflow 深度学习工具
paddlepaddle 深度学习工具

其他

工具 用途
jupyter server / jupyter notebook 基于 web 的交互式计算平台
zeppelin 基于 web 的数据驱动、交互式数据分析平台

Jupyter使用

安装

1
$ pip3 install jupyter

运行

1
$ jupyter notebook

设置密码

1
2
3
4
5
# 生成配置文件
$ jupyter notebook --generate-config

# 设置密码, 把密码加密后写入文件, 不会启动服务
$ jupyter notebook password

设置监听和端口

1
$ jupyter notebook --ip 0.0.0.0 --port 18800 

Trouble Shotting

‘charset_normalizer’ has no attribute ‘md__mypyc’

1
AttributeError: partially initialized module 'charset_normalizer' has no attribute 'md__mypyc' (most likely due to a circular import)

解决

1
$ pip3 install --force-reinstall charset-normalizer

推荐

1
$ jupyter lab --notebook-dir=$PWD --ip 0.0.0.0 --port 18800 --NotebookApp.token=token

数据存储格式

行、列存储

行存储和列存储用于描述数据组织方式,通常,行存储存取单位是整条记录,列存储存储单位是数据记录的一列。

姓名 年龄 省份
Li 25 山东
Zhao 23 北京
Zhang 22 辽宁

以上面数据为例,基于行存储,数据可能组织如下。

1
2
3
Li,25,山东
Zhao,23,北京
Zhang,22,辽宁

基于列存储,数据可能组织如下。

1
2
3
Li,Zhao,Zhang
25,23,22
山东,北京,辽宁

两者各有优劣,对比如下。

对比项 行存储 列存储
格式 按行存储 按列,每一列单独存储
粒度 记录 记录单列
索引 需要额外索引提高效率 数据即索引
读取 读取整条记录,可能存在冗余读取 按列多次读取,性能可能会更差
写入 一次写入,数据完整性更容易保证 按列多次写入,性能可能会更差
性能 建立索引、物化视图、冗余读写消耗大量资源 无需索引、视图
扩展 随着数据量增加为维持性能需要大量膨胀数据库 可横向扩展
应用 OLTP OLAP
全量操作 全量更新、聚合字段效率差 粒度更细,效率较高
压缩 压缩率较低 每一列数据同质,无二义性,且顺序存储,可达到较高压缩率

关系型数据库一般采用行存储,存储结构化数据,随着数据量增加,要提升性能需要付出很大的代价。列存储数据库一般具备可伸缩、可分区的特点,可以高效实现数据序列化、存储、检索。

数据库示例

行存储 列存储
MySQL HBase
MongoDB

参考

Docker应用

影音娱乐

  • Plex
  • jellyfin

办公下载

  • FileBrowser
  • Krusader (binhex-krusader)
  • Nextcloud
  • Transmission
  • 可道云
  • 迅雷远程
  • 蚂蚁笔记

Koel

docker command

1
2
3
4
5
6
7
8
9
10
11
12
docker run -d --name koel \
-p 2070:80 \
-e DB_CONNECTION=mysql \
-e DB_HOST=192.168.1.3 \
-e DB_DATABASE=koel \
-e DB_USERNAME=koel \
-e DB_PASSWORD=password \
-v /mnt/user/music:/music \
-v /mnt/user/appdata/koel/koel.env:/var/www/html/.env \
-v /mnt/user/appdata/koel/covers:/var/www/html/public/img/covers \
-v /mnt/user/appdata/koel/search_index:/var/www/html/storage/search-indexes \
phanan/koel

koel.env

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
APP_NAME=Koel
APP_ENV=production
APP_DEBUG=false
APP_URL=http://koel.domain.com

# A comma-separated list of (Koel server) hostnames accepted to access Koel.
# Leave this empty to allow access to Koel with any hostname.
# Example: localhost,192.168.0.1,yourdomain.com
TRUSTED_HOSTS=

# A random 32-char string. You can leave this empty if use php artisan koel:init.
APP_KEY=

# Database connection name, which corresponds to the database driver.
# Possible values are:
# mysql (MySQL/MariaDB - default)
# pgsql (PostgreSQL)
# sqlsrv (Microsoft SQL Server)
# sqlite-persistent (Local sqlite file)
# IMPORTANT: This value must present for `artisan koel:init` command to work.
DB_CONNECTION=mysql

DB_HOST=192.168.1.3
DB_PORT=
DB_DATABASE=koel
DB_USERNAME=koel
DB_PASSWORD=password

# Some providers (e.g. Heroku) provide a "database URL" instead separated config values, which
# you can use here instead.
DATABASE_URL=

# The absolute path to the root CA bundle if you're connecting to the MySQL database via SSL.
MYSQL_ATTR_SSL_CA=


# The storage driver. Valid values are:
# local: Store files on the server's local filesystem.
# sftp: Store files on an SFTP server.
# s3: Store files on Amazon S3 or a S3-compatible service (e.g. Cloudflare R2 or DigitalOcean Spaces). Koel Plus only.
# dropbox: Store files on Dropbox. Koel Plus only.
STORAGE_DRIVER=local


# The ABSOLUTE path to your media. This value can always be changed later via the web interface.
# Required if you're using the local file system to store your media (STORAGE_DRIVER=local).
MEDIA_PATH=


# S3 or S3-compatible service settings. Required if you're using S3 to store your media (STORAGE_DRIVER=s3).
# Remember to set CORS policy to allow access from your Koel's domain (or "*").
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
# For Cloudflare R2, set this to "auto". For S3 and other services, set this to the region of your bucket.
AWS_REGION=
AWS_ENDPOINT=
AWS_BUCKET=


# Dropbox settings. Required if you're using Dropbox to store your media (STORAGE_DRIVER=dropbox)
# Follow these steps to have these values filled:
# 1. Create a Dropbox app at https://www.dropbox.com/developers/apps
# 2. Run `php artisan koel:setup-dropbox` from the CLI and follow the instructions.
DROPBOX_APP_KEY=
DROPBOX_APP_SECRET=
DROPBOX_REFRESH_TOKEN=


# SFTP settings. Required if you're using SFTP to store your media (STORAGE_DRIVER=sftp).
SFTP_HOST=
SFTP_PORT=

# The absolute path of the directory to store the media files on the SFTP server.
# Make sure the directory exists and is writable by the SFTP user.
SFTP_ROOT=

# You can use either a username/password pair…
SFTP_USERNAME=
SFTP_PASSWORD=

# …or private key authentication:
SFTP_PRIVATE_KEY=
SFTP_PASSPHRASE=


# By default, Koel ignores dot files and folders. This greatly improves performance if your media
# root have folders like .git or .cache. If by any chance your media files are under a dot folder,
# set the following setting to false.
IGNORE_DOT_FILES=true


# The maximum scan time, in seconds. Increase this if you have a huge library.
# Note: This setting doesn't have effect when scanning via koel:sync.
APP_MAX_SCAN_TIME=600


# The memory limit, in MB, used by the scanning process.
# For example, if you want to set a memory limit of 2048MB, enter "2048" (without
# quotes) here.
MEMORY_LIMIT=


# The streaming method.
# Can be either 'php' (default), 'x-sendfile', or 'x-accel-redirect'
# See https://docs.koel.dev/usage/streaming for more information.
# Note: This setting doesn't have effect if the media needs transcoding (e.g. FLAC).

# It's HIGHLY recommended to use 'x-sendfile' or 'x-accel-redirect' if
# you plan to use the Koel mobile apps.

STREAMING_METHOD=php


# Full text search driver.
# Koel supports all drivers supported by Laravel (see https://laravel.com/docs/9.x/scout).
# Available drivers: 'tntsearch' (default), 'database', 'algolia' or 'meilisearch'.
# For Algolia or MeiliSearch, you need to provide the corresponding credentials.
SCOUT_DRIVER=tntsearch
ALGOLIA_APP_ID=
ALGOLIA_SECRET=
MEILISEARCH_HOST=
MEILISEARCH_KEY=


# Last.fm API can be used to fetch artist and album information, as well as to
# allow users to connect to their Last.fm account and scrobble.
# To integrate Koel with Last.fm, create an API account at
# https://www.last.fm/api/account/create and set the credentials here.
# Consult Koel's doc for more information.
LASTFM_API_KEY=
LASTFM_API_SECRET=


# Spotify API can be used to fetch artist and album images.
# To integrate Koel with Spotify, create a Spotify application at
# https://developer.spotify.com/dashboard/applications and set the credentials here.
# Consult Koel's docs for more information.
SPOTIFY_CLIENT_ID=
SPOTIFY_CLIENT_SECRET=


# To integrate Koel with YouTube, set the API key here.
# See https://docs.koel.dev/service-integrations#youtube for more information.
YOUTUBE_API_KEY=


# You can also configure Koel to use a CDN to serve the media files.
# This url must be mapped to the home URL of your Koel's installation.
# No trailing slash.
CDN_URL=


# To transcode FLAC to MP3 and stream it on the fly, make sure the following settings are sane.
# If you don't want to transcode FLAC (i.e. to stream it as-is), set this to false.
TRANSCODE_FLAC=false

# The full path of ffmpeg binary.
FFMPEG_PATH=/usr/local/bin/ffmpeg


# The bit rate of the output mp3 stream. Higher value results in better quality,
# but slower streaming and more bandwidth.
OUTPUT_BIT_RATE=128


# Whether to allow song downloading.
# Note that if you're downloading more than one song, Koel will zip them up
# using PHP's ZipArchive. So if the module isn't available in the current
# environment, such a download will (silently) fail.
ALLOW_DOWNLOAD=true


# Whether to create a backup of a song when deleting it from the filesystem.
BACKUP_ON_DELETE=true


# If using SSO, set the providers details here. Koel will automatically enable SSO if these values are set.
# Create an OAuth client and get these values from https://console.developers.google.com/apis/credentials
SSO_GOOGLE_CLIENT_ID=
SSO_GOOGLE_CLIENT_SECRET=
# The domain that users must belong to in order to be able to log in.
SSO_GOOGLE_HOSTED_DOMAIN=your.koel.domain


# Koel can be configured to authenticate users via a reverse proxy.
# To enable this feature, set PROXY_AUTH_ENABLED to true and provide the necessary configuration below.
PROXY_AUTH_ENABLED=false
# The header name that contains the unique identifier for the user
PROXY_AUTH_USER_HEADER=remote-user
# The header name that contains the user's preferred, humanly-readable name
PROXY_AUTH_PREFERRED_NAME_HEADER=remote-preferred-name
# A comma-separated list of allowed proxy IPs or CIDRs, for example, 10.10.1.0/24 or 2001:0db8:/32
# If empty, NO requests will be allowed (which means proxy authentication is disabled).
PROXY_AUTH_ALLOW_LIST=


# Sync logs can be found under storage/logs/. Valid options are:
# all: Log everything (errored-, skipped-, and successfully processed file).
# error: Log errors only. This is the default.
SYNC_LOG_LEVEL=error


# Koel attempts to detect if your website uses HTTPS and generates secure URLs accordingly.
# If this attempt fails for any reason, you can force it by setting this value to true.
FORCE_HTTPS=


# Pusher configuration, for interesting features such as remote controlling.
PUSHER_APP_ID=
PUSHER_APP_KEY=
PUSHER_APP_SECRET=
PUSHER_APP_CLUSTER=


# The following settings are for Koel to send emails, for example to send user invitations and reset passwords.
MAIL_FROM_ADDRESS=email@domain.com
MAIL_FROM_NAME="${APP_NAME}"
MAIL_MAILER=smtp
MAIL_HOST=mailhog
MAIL_PORT=1025
MAIL_USERNAME=null
MAIL_PASSWORD=null
MAIL_ENCRYPTION=null


SQS_PUBLIC_KEY=
SQS_SECRET_KEY=
SQS_QUEUE_PREFIX=
SQS_QUEUE_NAME=
SQS_QUEUE_REGION=


# The variables below are Laravel-specific.
# You can change them if you know what you're doing. Otherwise, just leave them as-is.
BROADCAST_DRIVER=log
CACHE_DRIVER=file
FILESYSTEM_DISK=local
QUEUE_CONNECTION=sync
SESSION_DRIVER=file
SESSION_LIFETIME=120

启动后进终端运行。

1
2
3
4
$ php artisan key:generate # 生成 APP_KEY
# 如果需要重新设置数据库地址,运行如下命令
# NOTE: 可选
$ php artisan koel:init # 忽略前端编译错误

忽略前端编译问题。

最后,账号密码。

1
2
3
4
# Username
admin@koel.dev
# Password
KoelIsCool

systemd使用

添加服务

  • 创建可执行服务程序
  • 创建 Unit File
  • 复制 Unit File 至 /etc/systemd/system
  • 服务启 / 停

示例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# [required] 1. 创建可执行服务程序
## File: test_service.sh
DATE=`date '+%Y-%m-%d %H:%M:%S'`
echo "Example service started at ${DATE}" | systemd-cat -p info

while :
do
echo "Looping...";
sleep 30;
done

# [optional] 2. 复制程序至 /usr/bin
$ sudo cp test_service.sh /usr/bin/test_service.sh
$ sudo chmod +x /usr/bin/test_service.sh

# [required] 3. 创建 Unit File
## File: myservice.service
[Unit]
Description=Example systemd service.

[Service]
Type=simple
ExecStart=/bin/bash /usr/bin/test_service.sh

[Install]
WantedBy=multi-user.target

# [required] 4. 复制 Unit File 至 /etc/systemd/system
$ sudo cp myservice.service /etc/systemd/system/myservice.service
$ sudo chmod 644 /etc/systemd/system/myservice.service

服务启停

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# 启动 
$ sudo systemctl start myservice

# 查看状态
$ sudo systemctl status myservice

# 停止
$ sudo systemctl stop myservice

# 重启
$ sudo systemctl restart myservice

# 开机启动
$ sudo systemctl enable myservice

Neovim

配置

使用 LazyVim 进行配置。

1
git clone https://github.com/LazyVim/starter ~/.config/nvim

安装字体(可选)

如果是通过 ssh 远程登录,需要在本地机器安装,并设置 terminal font。

1
2
wget https://github.com/ryanoasis/nerd-fonts/releases/download/v3.2.1/JetBrainsMono.zip
unzip JetBrainsMono.zip -d ~/.local/share/fonts

ubuntu

1
2
sudo apt install fontconfig
fc-cache -fv

概念

Buffer

NeoVim 以 Buffer List 的方式管理打开的文件,每个打开的文件是一个 Buffer,并且有唯一的 id(:ls 可查看打开的文件列表及 id)。

快捷键

LazyVim

1
<space>e      # 打开 NeoTree

Buffer

1
2
3
4
5
6
7
bd            # 删除当前 buffer
bn # 下一个 buffer
bn{number} # 下面的第 {number} 个
bp # 前一个
bp{number} # 前面的第 {number} 个
b# # 切换到最近激活的 buffer
ls # 列出所有 buffer

NeoTree

1
2
3
4
?							# 打开帮助框
a # 添加文件/文件夹,添加文件夹则以 '/' 结尾,否则添加文件
H # 显式/折叠隐藏文件
. # 设置选定的目录为 root dir

使用

多行编辑

1
<ctrl>-v    # 开启矩阵选择

插件

常见问题

图标显示异常

修改 terminal font,以 iterm2 为例。字体可从 nerd fonts 下载。

image-20240922102745830