Loki日志收集

1、Loki部署

  • 二进制文件下载解压
1
2
3
4
wget https://github.com/grafana/loki/releases/download/v2.8.4/loki-linux-amd64.zip
unzip loki-linux-amd64.zip
mkdir -p /usr/local/loki/{etc,logs,bin}
mv loki-linux-amd64 /usr/local/loki/bin/loki
  • loki配置文件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
vim /usr/local/loki/etc/local-config.yaml

auth_enabled: false

server:
  http_listen_port: 3100

common:
  path_prefix: /loki
  storage:
    filesystem:
      chunks_directory: /loki/chunks
      rules_directory: /loki/rules
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

schema_config:
  configs:
    - from: 2020-10-24
      store: boltdb-shipper
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 24h

ruler:
  alertmanager_url: http://localhost:9093
  • systemd文件
1
2
3
4
5
6
7
8
9
10
11
12
13
vim /usr/lib/systemd/system/loki.service

[Unit]
Description=Loki service
After=network.target

[Service]
Type=simple
User=root
ExecStart=/usr/local/loki/bin/loki -config.file /usr/local/loki/etc/local-config.yaml

[Install]
WantedBy=multi-user.target
  • 启动loki服务
1
systemctl start loki.service

2、Minio作为后端存储

3、Promtail作为收集agent

  • 二进制文件下载
1
2
3
4
wget https://github.com/grafana/loki/releases/download/v2.8.4/promtail-linux-amd64.zip
unzip promtail-linux-amd64.zip
mkdir -p /usr/local/promtail/{etc,logs,bin}
mv promtail-linux-amd64 /usr/local/promtail/bin/promtail
  • promtail配置文件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
vim /usr/local/promtail/etc/config.yaml

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://127.0.0.1:3100/loki/api/v1/push

scrape_configs:
- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: varlogs
      __path__: /var/log/*log

- job_name: system-message
  static_configs:
  - targets:
      - localhost
    labels:
      job: varlogs-message
      __path__: /var/log/messages

- job_name: nginx
  pipeline_stages:
  - replace:
      expression: '(?:[0-9]{1,3}\.){3}([0-9]{1,3})'
      replace: '***'
  static_configs:
  - targets:
      - localhost
    labels:
      job: nginx_access_log
      host: localhost
      agent: promtail
      __path__: /var/log/nginx/*access.log
  • systemd文件
1
2
3
4
5
6
7
8
9
10
11
12
13
vim /usr/lib/systemd/system/promtail.service

[Unit]
Description=Promtail service
After=network.target

[Service]
Type=simple
User=root
ExecStart=/usr/local/promtail/bin/promtail -config.file /usr/local/promtail/etc/config.yaml

[Install]
WantedBy=multi-user.target
  • 启动promtail
1
systemctl start promtail
  • 验证数据
1
curl "127.0.0.1:9080/metrics"

4、Grafana可视化展示

  • docker部署grafana
1
2
3
4
5
6
7
8
9
10
11
12
13
cat docker-compose.yaml

version: "3"
services:
  grafana:
    image: grafana/grafana:10.0.4
    container_name: grafana
    restart: unless-stopped
    ports:
     - '3000:3000'
    environment:
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
1
docker-compose up -d
  • 配置数据源

由于granfana是由docker部署，数据源的URL地址，需要填写服务器IP的地址

loki-2.png

loki-1.png