mrckndt/CS-Repro-Mattermost
1
0
Fork 1
mirror of https://github.com/coltoneshaw/CS-Repro-Mattermost.git synced 2025-12-23 18:11:34 +01:00
Code Releases Activity

Compare commits

base: mrckndt:readme
Branches Tags
mrckndt:main mrckndt:readme mk:linux-test mk:main
..
compare: mk:linux-test
Branches Tags
mk:linux-test mk:main mrckndt:main mrckndt:readme

1 Commits
readme ... linux-test

Author SHA1 Message Date
mk
d0a1dce9a6 test 2024-12-19 12:17:54 +01:00
11 changed files with 63 additions and 1128 deletions
Expand all files Collapse all files

8
.gitignore vendored
View File

@@ -1,4 +1,5 @@
license* license.mattermost
license.txt
## Ignoring keycloak so it can be pulled already setup. ## Ignoring keycloak so it can be pulled already setup.
volumes volumes
@@ -12,8 +13,3 @@ files/postgres/replica/replica_1_postgres.conf
files/postgres/replica/replica_2.conf files/postgres/replica/replica_2.conf
files/postgres/replica/replica_2_postgres.conf files/postgres/replica/replica_2_postgres.conf
**/.claude/settings.local.json
*_backup.json
*.log

92
CLAUDE.md
View File

@@ -1,92 +0,0 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
This repository contains a CS Repro Mattermost environment - a reproduction of a standard Mattermost customer production environment with preconfigured SAML, LDAP, Advanced Logging, Prometheus, Grafana, Elasticsearch, and read replicas. It is designed for testing and reproducing customer environments.
## Environment Setup
### Prerequisites
You need to have Docker and Docker Compose installed on your machine to run this environment.
### Initial Setup
1. Add an enterprise license file named `license.mattermost` to the root directory
2. Run `make run` to start the core services
## Common Commands
### Environment Management
- `make run` - Initialize environment and create containers
- `make run-all` - Spin up all environment containers (with DB replicas and Mattermost HA)
- `make run-db-replicas` - Add database replicas to the environment
- `make run-mm-replicas` - Add Mattermost HA nodes to the environment
- `make run-rtcd` - Launch RTCD service for Mattermost Calls
- `make start` - Start existing containers
- `make stop` - Stop all running containers
- `make restart` - Restart all containers
- `make restart-mattermost` - Restart only Mattermost containers (preferred for testing changes)
- `make backup-keycloak` - Generate a backup of the current Keycloak setup
- `make restore-keycloak` - Restore Keycloak data from an existing backup
- `make reset` - Clean volumes directory and reinitialize the environment
- `make delete-data` - Clear all data within volumes
- `make nuke` - Erase all configurations and data
- `make nuke-rmi` - Complete cleanup including removing Docker images
### Upgrading and Downgrading
To upgrade Mattermost:
1. Modify the image tag in `docker-compose.yml`
2. Run `make restart-mattermost`
To downgrade Mattermost:
1. Modify the image tag in `docker-compose.yml`
2. Run `make downgrade`
## Component Access
### Mattermost
- URL: http://localhost:8065
- Default credentials available in the accounts section below
### LDAP
- Admin access: `cn=admin,dc=planetexpress,dc=com` / `GoodNewsEveryone`
- Commands for LDAP operations are in the README.md
### Grafana
- URL: http://localhost:3000
- Credentials: admin / admin
### Elasticsearch
- URL: http://localhost:9200
### mitmproxy
- URL: http://localhost:8181
## Account Information
| Username | Password | Role in Mattermost | Can use LDAP? | Can use SAML? |
|-----------|-----------|-------------------|---------------|---------------|
| admin | admin | n/a | n/a | n/a |
| professor | professor | Sys Admin | Yes | Yes |
| bender | bender | Member | Yes | Yes |
| hermes | hermes | Sys Admin | Yes | Yes |
| fry | fry | Member | Yes | Yes |
| leela | leela | Member | Yes | Yes |
| zoidberg | zoidberg | Member | Yes | Yes |
| amy | amy | Member | Yes | Yes |
## Database Information
- Primary PostgreSQL: `postgresql://mmuser:mmuser_password@localhost:5432/mattermost`
- Replica 1: `postgresql://mmuser:mmuser_password@localhost:5433/mattermost`
- Replica 2: `postgresql://mmuser:mmuser_password@localhost:5434/mattermost`

23
Makefile
View File

@@ -6,12 +6,6 @@ logs:
@echo "Done" @echo "Done"
setup-mattermost: setup-mattermost:
@cp ./files/mattermost/defaultConfig.json ./volumes/mattermost/config
@cp ./files/mattermost/replicaConfig.json ./volumes/mattermost/config
@cp ./files/mattermost/rtcdConfig.json ./volumes/mattermost/config
@cp ./files/mattermost/samlCert.crt ./volumes/mattermost/config
@cp ./license.mattermost ./volumes/mattermost/config/license.mattermost-enterprise
@./scripts/mattermost.sh setup @./scripts/mattermost.sh setup
check-mattermost: check-mattermost:
@@ -28,7 +22,7 @@ echo-logins:
run: run:
@echo "Starting..." @echo "Starting..."
@make restore-keycloak #@make restore-keycloak
@make run-core @make run-core
@make setup-mattermost @make setup-mattermost
@make echo-logins @make echo-logins
@@ -36,7 +30,7 @@ run:
run-core: run-core:
@echo "Starting the core services... hang in there." @echo "Starting the core services... hang in there."
@docker-compose up -d postgres openldap prometheus grafana elasticsearch mattermost keycloak loki promtail @docker-compose up -d postgres openldap prometheus grafana elasticsearch mattermost keycloak #mitmproxy
run-db-replicas: run-db-replicas:
@echo "Starting with replicas. Hang in there..." @echo "Starting with replicas. Hang in there..."
@@ -50,11 +44,6 @@ run-mm-replicas:
@echo "Starting Mattermost replicas. Hang in there..." @echo "Starting Mattermost replicas. Hang in there..."
@docker exec -it cs-repro-mattermost mmctl config set ClusterSettings.Enable true --local @docker exec -it cs-repro-mattermost mmctl config set ClusterSettings.Enable true --local
@docker-compose down mattermost @docker-compose down mattermost
@cp ./files/mattermost/defaultConfig.json ./volumes/mattermost_2/config
@cp ./files/mattermost/replicaConfig.json ./volumes/mattermost_2/config
@cp ./files/mattermost/rtcdConfig.json ./volumes/mattermost_2/config
@cp ./files/mattermost/samlCert.crt ./volumes/mattermost_2/config
@cp ./license.mattermost ./volumes/mattermost/mattermost_2/license.mattermost-enterprise
@docker-compose up -d mattermost mattermost-2 @docker-compose up -d mattermost mattermost-2
@docker exec -it -u root cs-repro-mattermost-2 /bin/bash update-ca-certificates @docker exec -it -u root cs-repro-mattermost-2 /bin/bash update-ca-certificates
@echo "Should be up and running. Go crazy." @echo "Should be up and running. Go crazy."
@@ -93,14 +82,6 @@ restart-mattermost:
@docker start cs-repro-mattermost @docker start cs-repro-mattermost
@make check-mattermost @make check-mattermost
restart-grafana:
@echo "Restarting Grafana container"
@docker stop cs-repro-grafana
@wait
@echo "Starting Grafana container"
@docker start cs-repro-grafana
@echo "Grafana restarted"
reset: reset:
@echo "Resetting..." @echo "Resetting..."
@make delete-data @make delete-data

11
README.md
View File

@@ -71,7 +71,7 @@ Additionally, the keycloak container can take up to 5 minutes to spin up. If it'
- **`make run`**: Initializes the environment and creates the containers - **`make run`**: Initializes the environment and creates the containers
- **`make run-all`**: Spins up all environment containers with the database replicas and Mattermost HA. - **`make run-all`**: Spins up all environment containers with the database replicas and Mattermost HA.
- You must run `make run` before running the below: - You must run `make run` before running the below:
- **`make run-db-replicas`**: Launches the environment with replicas. Ideal for adding replicas to an existing setup or initializing with replicas from the get-go. - **`make run-db-replica`**: Launches the environment with replicas. Ideal for adding replicas to an existing setup or initializing with replicas from the get-go.
- **`make run-mm-replica`**: Launches an additional Mattermost node and enables HA. - **`make run-mm-replica`**: Launches an additional Mattermost node and enables HA.
- **`make run-rtcd`**: Launches the RTCD service for Mattermost Calls and updates the config to use it correctly. - **`make run-rtcd`**: Launches the RTCD service for Mattermost Calls and updates the config to use it correctly.
- **`make backup-keycloak`**: Generates a backup of the current Keycloak setup in the files directory. Useful for infrequent backups. - **`make backup-keycloak`**: Generates a backup of the current Keycloak setup in the files directory. Useful for infrequent backups.
@@ -79,7 +79,6 @@ Additionally, the keycloak container can take up to 5 minutes to spin up. If it'
- **`make stop`**: Halts all running containers. - **`make stop`**: Halts all running containers.
- **`make restart`**: Restarts all Docker containers in the environment. - **`make restart`**: Restarts all Docker containers in the environment.
- **`make restart-mattermost`**: Specifically restarts only the Mattermost containers for quick testing. - **`make restart-mattermost`**: Specifically restarts only the Mattermost containers for quick testing.
- **`make restart-grafana`**: Specifically restarts only the Grafana container.
- **`make reset`**: Cleans the volumes directory and reinitializes the environment to default settings. - **`make reset`**: Cleans the volumes directory and reinitializes the environment to default settings.
- **`make delete-data`**: Clears all data within volumes, effectively stopping Mattermost. - **`make delete-data`**: Clears all data within volumes, effectively stopping Mattermost.
- **`make nuke`**: Erases all configurations and data, sparing your personal data. - **`make nuke`**: Erases all configurations and data, sparing your personal data.
@@ -91,10 +90,10 @@ Additionally, the keycloak container can take up to 5 minutes to spin up. If it'
1. Modify the line in the `docker-compose.yml` file to be the version you want 1. Modify the line in the `docker-compose.yml` file to be the version you want
You're just replacing the tag at the end, this one is `10.7` for example. It must be a version of Mattermost that exists on Docker. You're just replacing the tag at the end, this one is `7.7` for example. It must be a version of Mattermost that exists on Docker.
```bash ```bash
mattermost/mattermost-enterprise-edition:10.7 mattermost/mattermost-enterprise-edition:release-7.7
``` ```
2. Run `make restart-mattermost` 2. Run `make restart-mattermost`
@@ -107,10 +106,10 @@ Doing this will wipe anything you have in the database and any existing Mattermo
1. Modify the line in the `docker-compose.yml` file to be the version you want 1. Modify the line in the `docker-compose.yml` file to be the version you want
You're just replacing the tag at the end, this one is `10.7` for example. It must be a version of Mattermost that exists on Docker. You're just replacing the tag at the end, this one is `7.7` for example. It must be a version of Mattermost that exists on Docker.
```bash ```bash
mattermost/mattermost-enterprise-edition:10.7 mattermost/mattermost-enterprise-edition:release-7.7
``` ```
2. Run `make downgrade` 2. Run `make downgrade`

77
docker-compose.yml
View File

@@ -46,30 +46,6 @@ services:
volumes: volumes:
- ./files/prometheus.yml:/etc/prometheus/prometheus.yml:ro - ./files/prometheus.yml:/etc/prometheus/prometheus.yml:ro
- ./volumes/prometheus:/prometheus - ./volumes/prometheus:/prometheus
loki:
container_name: cs-repro-loki
image: grafana/loki:3.2.2
restart: unless-stopped
ports:
- 3100:3100
volumes:
- ./files/loki:/etc/loki
- ./volumes/logs-node-1:/logs/node-1
- ./volumes/logs-node-2:/logs/node-2
command: -config.file=/etc/loki/loki-config.yaml
promtail:
container_name: cs-repro-promtail
image: grafana/promtail:3.2.2
restart: unless-stopped
ports:
- 9080:9080
volumes:
- ./files/promtail:/etc/promtail
- ./volumes/logs-node-1:/logs-node-1
- ./volumes/logs-node-2:/logs-node-2
command: -config.file=/etc/promtail/promtail-config.yaml
depends_on:
- loki
grafana: grafana:
container_name: cs-repro-grafana container_name: cs-repro-grafana
image: grafana/grafana:10.0.4 image: grafana/grafana:10.0.4
@@ -90,9 +66,9 @@ services:
depends_on: depends_on:
postgres: postgres:
condition: service_healthy condition: service_healthy
mitmproxy: #mitmproxy:
condition: service_started # condition: service_started
image: mattermost/mattermost-enterprise-edition:10.7 image: mattermost/mattermost-enterprise-edition:release-9.11
restart: unless-stopped restart: unless-stopped
security_opt: security_opt:
- no-new-privileges:true - no-new-privileges:true
@@ -106,24 +82,28 @@ services:
volumes: volumes:
- ./volumes/mattermost/config:/mattermost/config:rw - ./volumes/mattermost/config:/mattermost/config:rw
- ./volumes/mattermost/data:/mattermost/data:rw - ./volumes/mattermost/data:/mattermost/data:rw
- ./volumes/logs-node-1:/mattermost/logs:rw - ./volumes/mattermost/logs:/mattermost/logs:rw
- ./volumes/mattermost/plugins:/mattermost/plugins:rw - ./volumes/mattermost/plugins:/mattermost/plugins:rw
- ./volumes/mattermost/client/plugins:/mattermost/client/plugins:rw - ./volumes/mattermost/client/plugins:/mattermost/client/plugins:rw
- ./volumes/mattermost/bleve-indexes:/mattermost/bleve-indexes:rw - ./volumes/mattermost/bleve-indexes:/mattermost/bleve-indexes:rw
#- ./license.mattermost:/mattermost/config/license.mattermost-enterprise:ro
#- ./files/mattermost/samlCert.crt:/mattermost/config/saml-cert.crt
#- ./files/mattermost/defaultConfig.json:/mattermost/config/defaultConfig.json:ro
#- ./files/mattermost/replicaConfig.json:/mattermost/config/replicaConfig.json:ro
#- ./files/mattermost/rtcdConfig.json:/mattermost/config/rtcdConfig.json:ro
## Files are required for the mitmproxy on the box ## Files are required for the mitmproxy on the box
- ./files/mitmproxy/mitmproxy-ca.pem:/etc/ssl/certs/mitmproxy-ca.pem - ./files/mitmproxy/mitmproxy-ca.pem:/etc/ssl/certs/mitmproxy-ca.pem
- ./files/mitmproxy/mitmproxy-dhparam.pem:/etc/ssl/certs/mitmproxy-dhparam.pem - ./files/mitmproxy/mitmproxy-dhparam.pem:/etc/ssl/certs/mitmproxy-dhparam.pem
environment: environment:
# - HTTP_PROXY=http://cs-repro-mitmproxy:8080 #- HTTP_PROXY=http://cs-repro-mitmproxy:8080
# - HTTPS_PROXY=http://cs-repro-mitmproxy:8080 #- HTTPS_PROXY=http://cs-repro-mitmproxy:8080
- MM_SqlSettings_DriverName=postgres - MM_SqlSettings_DriverName=postgres
- MM_SqlSettings_DataSource=postgres://mmuser:mmuser_password@cs-repro-postgres:5432/mattermost?sslmode=disable&connect_timeout=10&binary_parameters=yes - MM_SqlSettings_DataSource=postgres://mmuser:mmuser_password@cs-repro-postgres:5432/mattermost?sslmode=disable&connect_timeout=10&binary_parameters=yes
- MM_SAMLSETTINGS_IDPCERTIFICATEFILE=/mattermost/config/saml-cert.crt - MM_SAMLSETTINGS_IDPCERTIFICATEFILE=/mattermost/config/saml-cert.crt
- MM_ServiceSettings_EnableLocalMode=true - MM_ServiceSettings_EnableLocalMode=true
- MM_ServiceSettings_LocalModeSocketLocation=/var/tmp/mattermost_local.socket - MM_ServiceSettings_LocalModeSocketLocation=/var/tmp/mattermost_local.socket
- MM_ServiceSettings_LicenseFileLocation=/mattermost/config/license.mattermost-enterprise - MM_ServiceSettings_LicenseFileLocation=/mattermost/config/license.mattermost-enterprise
- MM_LogSettings_Directory=
## mysql Settings ## mysql Settings
# - MM_SqlSettings_DriverName=mysql # - MM_SqlSettings_DriverName=mysql
@@ -134,7 +114,8 @@ services:
keycloak: keycloak:
container_name: cs-repro-keycloak container_name: cs-repro-keycloak
restart: unless-stopped restart: unless-stopped
image: quay.io/keycloak/keycloak:18.0 platform: linux/amd64
image: quay.io/keycloak/keycloak:latest
volumes: volumes:
- ./volumes/keycloak:/opt/keycloak/data:rw - ./volumes/keycloak:/opt/keycloak/data:rw
environment: environment:
@@ -223,9 +204,9 @@ services:
depends_on: depends_on:
postgres: postgres:
condition: service_healthy condition: service_healthy
mitmproxy: #mitmproxy:
condition: service_started # condition: service_started
image: mattermost/mattermost-enterprise-edition:10.7 image: mattermost/mattermost-enterprise-edition:release-9.11
restart: unless-stopped restart: unless-stopped
security_opt: security_opt:
- no-new-privileges:true - no-new-privileges:true
@@ -239,14 +220,14 @@ services:
volumes: volumes:
- ./volumes/mattermost/config:/mattermost/config:rw - ./volumes/mattermost/config:/mattermost/config:rw
- ./volumes/mattermost/data:/mattermost/data:rw - ./volumes/mattermost/data:/mattermost/data:rw
- ./volumes/logs-node-2:/mattermost-1/logs:rw - ./volumes/mattermost_2/logs:/mattermost/logs:rw
- ./volumes/mattermost_2/plugins:/mattermost/plugins:rw - ./volumes/mattermost_2/plugins:/mattermost/plugins:rw
- ./volumes/mattermost_2/client/plugins:/mattermost/client/plugins:rw - ./volumes/mattermost_2/client/plugins:/mattermost/client/plugins:rw
- ./volumes/mattermost_2/bleve-indexes:/mattermost/bleve-indexes:rw - ./volumes/mattermost_2/bleve-indexes:/mattermost/bleve-indexes:rw
# - ./license.mattermost:/mattermost/config/license.mattermost-enterprise:ro - ./license.mattermost:/mattermost/config/license.mattermost-enterprise:ro
# - ./files/mattermost/samlCert.crt:/mattermost/config/saml-cert.crt - ./files/mattermost/samlCert.crt:/mattermost/config/saml-cert.crt
# - ./files/mattermost/defaultConfig.json:/mattermost/config/defaultConfig.json:ro - ./files/mattermost/defaultConfig.json:/mattermost/config/defaultConfig.json:ro
# - ./files/mattermost/replicaConfig.json:/mattermost/config/replicaConfig.json:ro - ./files/mattermost/replicaConfig.json:/mattermost/config/replicaConfig.json:ro
## Files are required for the mitmproxy on the box ## Files are required for the mitmproxy on the box
- ./files/mitmproxy/mitmproxy-ca.pem:/etc/ssl/certs/mitmproxy-ca.pem - ./files/mitmproxy/mitmproxy-ca.pem:/etc/ssl/certs/mitmproxy-ca.pem
@@ -260,7 +241,6 @@ services:
- MM_ServiceSettings_EnableLocalMode=true - MM_ServiceSettings_EnableLocalMode=true
- MM_ServiceSettings_LocalModeSocketLocation=/var/tmp/mattermost_local.socket - MM_ServiceSettings_LocalModeSocketLocation=/var/tmp/mattermost_local.socket
- MM_ServiceSettings_LicenseFileLocation=/mattermost/config/license.mattermost-enterprise - MM_ServiceSettings_LicenseFileLocation=/mattermost/config/license.mattermost-enterprise
- MM_LogSettings_Directory=/mattermost/logs
mattermost-rtcd: mattermost-rtcd:
container_name: cs-repro-mattermost-rtcd container_name: cs-repro-mattermost-rtcd
platform: linux/amd64 platform: linux/amd64
@@ -278,11 +258,16 @@ services:
- "8443:8443/udp" - "8443:8443/udp"
- "8443:8443/tcp" - "8443:8443/tcp"
- "8045:8045" - "8045:8045"
mitmproxy: #mitmproxy:
container_name: cs-repro-mitmproxy # container_name: cs-repro-mitmproxy
image: mitmproxy/mitmproxy # image: mitmproxy/mitmproxy
command: mitmweb --web-host 0.0.0.0 --set confdir=/certs --mode transparent # command: mitmweb --web-host 0.0.0.0 --set confdir=/certs --mode transparent
# Named volumes section is removed since we're using host path mounts # volumes:
# - ./files/mitmproxy:/certs
# ports:
# - "8180:8080"
# - "8181:8081"
# restart: unless-stopped
# mysql: # mysql:
# container_name: cs-repro-mysql # container_name: cs-repro-mysql
# image: mysql:8 # image: mysql:8

BIN
files/.DS_Store vendored Normal file
View File

Binary file not shown.

827
files/grafana/dashboards/enhanced_logs.json
View File

@@ -1,827 +0,0 @@
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": {
"type": "grafana",
"uid": "-- Grafana --"
},
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"links": [],
"liveNow": false,
"panels": [
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"gridPos": {
"h": 12,
"w": 24,
"x": 0,
"y": 0
},
"id": 7,
"options": {
"dedupStrategy": "none",
"enableLogDetails": true,
"prettifyLogMessage": false,
"showCommonLabels": false,
"showLabels": true,
"showTime": true,
"sortOrder": "Descending",
"wrapLogMessage": false
},
"targets": [
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"editorMode": "builder",
"expr": "{filename=~\"/logs-node-.*/(mattermost|advancedLogs).log\"} | json | level=~\"${log_level}\" | line_format \"{{.level}} [{{.caller}}] {{.msg}}\"",
"queryType": "range",
"refId": "A"
}
],
"title": "Filtered Logs",
"type": "logs"
},
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"axisCenteredZero": false,
"axisColorMode": "text",
"axisLabel": "",
"axisPlacement": "auto",
"barAlignment": 0,
"drawStyle": "line",
"fillOpacity": 20,
"gradientMode": "none",
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
},
"lineInterpolation": "smooth",
"lineWidth": 2,
"pointSize": 5,
"scaleDistribution": {
"type": "linear"
},
"showPoints": "never",
"spanNulls": false,
"stacking": {
"group": "A",
"mode": "none"
},
"thresholdsStyle": {
"mode": "off"
}
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 24,
"x": 0,
"y": 12
},
"id": 1,
"options": {
"legend": {
"calcs": [
"sum"
],
"displayMode": "table",
"placement": "right",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"targets": [
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"editorMode": "code",
"expr": "sum(count_over_time({filename=~\"/logs-node-.*/(mattermost|advancedLogs).log\"} | json | level=~\"(debug|info|warn|error)\" [$__interval])) by (level)",
"legendFormat": "{{level}}",
"queryType": "range",
"refId": "A"
}
],
"title": "Log Levels Over Time",
"type": "timeseries"
},
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "yellow",
"value": 5
},
{
"color": "red",
"value": 10
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 4,
"x": 0,
"y": 20
},
"id": 2,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"textMode": "auto"
},
"pluginVersion": "10.0.4",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"editorMode": "code",
"expr": "sum(count_over_time({filename=~\"/logs-node-.*/(mattermost|advancedLogs).log\"} | json | level=\"error\" [$__range]))",
"queryType": "instant",
"refId": "A"
}
],
"title": "Error Count",
"type": "stat"
},
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "yellow",
"value": 10
},
{
"color": "red",
"value": 20
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 4,
"x": 4,
"y": 20
},
"id": 3,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"textMode": "auto"
},
"pluginVersion": "10.0.4",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"editorMode": "code",
"expr": "sum(count_over_time({filename=~\"/logs-node-.*/(mattermost|advancedLogs).log\"} | json | level=\"warn\" [$__range]))",
"queryType": "instant",
"refId": "A"
}
],
"title": "Warning Count",
"type": "stat"
},
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 4,
"x": 8,
"y": 20
},
"id": 4,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"textMode": "auto"
},
"pluginVersion": "10.0.4",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"editorMode": "code",
"expr": "sum(count_over_time({filename=~\"/logs-node-.*/(mattermost|advancedLogs).log\"} | json | level=\"info\" [$__range]))",
"queryType": "instant",
"refId": "A"
}
],
"title": "Info Count",
"type": "stat"
},
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "blue",
"value": null
}
]
}
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 5,
"x": 12,
"y": 20
},
"id": 9,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"reduceOptions": {
"calcs": [
"lastNotNull"
],
"fields": "",
"values": false
},
"textMode": "auto"
},
"pluginVersion": "10.0.4",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"editorMode": "code",
"expr": "sum(count_over_time({filename=~\"/logs-node-.*/(mattermost|advancedLogs).log\"} | json | level=\"debug\" [$__range]))",
"queryType": "instant",
"refId": "A"
}
],
"title": "Debug Count",
"type": "stat"
},
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "palette-classic"
},
"custom": {
"hideFrom": {
"legend": false,
"tooltip": false,
"viz": false
}
},
"mappings": []
},
"overrides": []
},
"gridPos": {
"h": 8,
"w": 7,
"x": 17,
"y": 20
},
"id": 6,
"options": {
"displayLabels": [
"name",
"percent"
],
"legend": {
"displayMode": "table",
"placement": "right",
"showLegend": true,
"values": [
"value"
]
},
"pieType": "pie",
"reduceOptions": {
"calcs": [
"sum"
],
"fields": "",
"values": false
},
"tooltip": {
"mode": "single",
"sort": "none"
}
},
"targets": [
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"editorMode": "code",
"expr": "sum by(level) (count_over_time({filename=~\"/logs-node-.*/(mattermost|advancedLogs).log\"} | json [30m]))",
"legendFormat": "{{level}}",
"queryType": "range",
"refId": "A"
}
],
"title": "Log Level Distribution",
"type": "piechart"
},
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"custom": {
"align": "auto",
"cellOptions": {
"type": "auto"
},
"inspect": false
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": [
{
"matcher": {
"id": "byName",
"options": "count"
},
"properties": [
{
"id": "custom.width",
"value": 80
}
]
},
{
"matcher": {
"id": "byName",
"options": "component"
},
"properties": [
{
"id": "custom.width",
"value": 200
}
]
}
]
},
"gridPos": {
"h": 9,
"w": 12,
"x": 0,
"y": 28
},
"id": 5,
"options": {
"cellHeight": "sm",
"footer": {
"countRows": false,
"fields": "",
"reducer": [
"sum"
],
"show": false
},
"showHeader": true
},
"pluginVersion": "10.0.4",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"editorMode": "code",
"expr": "topk(10, sum by(caller) (count_over_time({filename=~\"/logs-node-.*/(mattermost|advancedLogs).log\"} | json | level=~\"warn|error\" [30m])))",
"legendFormat": "{{caller}}",
"queryType": "range",
"refId": "A"
}
],
"title": "Top 10 Error Sources (30m)",
"transformations": [
{
"id": "reduce",
"options": {
"includeTimeField": false,
"mode": "reduceFields",
"reducers": [
"sum"
]
}
},
{
"id": "organize",
"options": {
"excludeByName": {},
"indexByName": {},
"renameByName": {
"Field": "component",
"Value": "count"
}
}
},
{
"id": "sortBy",
"options": {
"fields": {},
"sort": [
{
"desc": true,
"field": "count"
}
]
}
}
],
"type": "table"
},
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"description": "Displays errors grouped by error message",
"fieldConfig": {
"defaults": {
"color": {
"mode": "thresholds"
},
"custom": {
"align": "auto",
"cellOptions": {
"type": "auto"
},
"filterable": true,
"inspect": false
},
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
}
},
"overrides": [
{
"matcher": {
"id": "byName",
"options": "Count"
},
"properties": [
{
"id": "custom.width",
"value": 80
}
]
},
{
"matcher": {
"id": "byName",
"options": "Error Message"
},
"properties": [
{
"id": "custom.width",
"value": 500
}
]
}
]
},
"gridPos": {
"h": 9,
"w": 12,
"x": 12,
"y": 28
},
"id": 8,
"options": {
"cellHeight": "sm",
"footer": {
"countRows": false,
"fields": "",
"reducer": [
"sum"
],
"show": false
},
"showHeader": true,
"sortBy": [
{
"desc": true,
"displayName": "Count"
}
]
},
"pluginVersion": "10.0.4",
"targets": [
{
"datasource": {
"type": "loki",
"uid": "${DS_LOKI}"
},
"editorMode": "code",
"expr": "topk(10, sum by(error) (count_over_time({filename=~\"/logs-node-.*/(mattermost|advancedLogs).log\"} | json | level=\"error\" [30m])))",
"queryType": "range",
"refId": "A"
}
],
"title": "Top 10 Error Messages",
"transformations": [
{
"id": "reduce",
"options": {
"includeTimeField": false,
"mode": "reduceFields",
"reducers": [
"sum"
]
}
},
{
"id": "organize",
"options": {
"excludeByName": {},
"indexByName": {},
"renameByName": {
"Field": "Error Message",
"Value": "Count"
}
}
}
],
"type": "table"
}
],
"refresh": "10s",
"schemaVersion": 38,
"style": "dark",
"tags": [
"mattermost",
"logs"
],
"templating": {
"list": [
{
"current": {
"selected": false,
"text": "loki",
"value": "loki"
},
"hide": 0,
"includeAll": false,
"label": "Datasource",
"multi": false,
"name": "DS_LOKI",
"options": [],
"query": "loki",
"queryValue": "",
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"type": "datasource"
},
{
"current": {
"selected": true,
"text": [
"info",
"warn",
"error",
"debug"
],
"value": [
"info",
"warn",
"error",
"debug"
]
},
"hide": 0,
"includeAll": true,
"label": "Log Level",
"multi": true,
"name": "log_level",
"options": [
{
"selected": false,
"text": "All",
"value": "$__all"
},
{
"selected": true,
"text": "debug",
"value": "debug"
},
{
"selected": true,
"text": "info",
"value": "info"
},
{
"selected": true,
"text": "warn",
"value": "warn"
},
{
"selected": true,
"text": "error",
"value": "error"
}
],
"query": "debug,info,warn,error",
"queryValue": "",
"skipUrlSync": false,
"type": "custom"
}
]
},
"time": {
"from": "now-1h",
"to": "now"
},
"timepicker": {},
"timezone": "",
"title": "Mattermost Enhanced Logs",
"uid": "mattermost-logs",
"version": 2,
"weekStart": ""
}

42
files/grafana/provisioning/datasources/automatic.yml
View File

@@ -1,31 +1,17 @@
apiVersion: 1 apiVersion: 1
datasources: datasources:
- name: mattermost - name: mattermost
type: prometheus type: prometheus
access: proxy access: proxy
url: http://prometheus:9090 url: http://prometheus:9090
password: password:
user: user:
database: database:
isDefault: true isDefault: true
jsonData: jsonData:
tlsAuth: false tlsAuth: false
tlsAuthWithCACert: false tlsAuthWithCACert: false
httpMethod: "POST" httpMethod: "POST"
version: 1 version: 1
editable: true editable: true
- name: loki
type: loki
access: proxy
url: http://loki:3100
password:
user:
database:
isDefault: false
jsonData:
tlsAuth: false
tlsAuthWithCACert: false
httpMethod: "POST"
version: 1
editable: true

50
files/loki/loki-config.yaml
View File

@@ -1,50 +0,0 @@
auth_enabled: false
server:
http_listen_port: 3100
grpc_listen_port: 9096
common:
instance_addr: 127.0.0.1
path_prefix: /tmp/loki
storage:
filesystem:
chunks_directory: /tmp/loki/chunks
rules_directory: /tmp/loki/rules
replication_factor: 1
ring:
kvstore:
store: inmemory
query_range:
results_cache:
cache:
embedded_cache:
enabled: true
max_size_mb: 100
schema_config:
configs:
- from: 2020-10-24
store: tsdb
object_store: filesystem
schema: v13
index:
prefix: index_
period: 24h
ruler:
alertmanager_url: http://localhost:9093
# By default, Loki will send anonymous, but uniquely-identifiable usage and configuration
# analytics to Grafana Labs. These statistics are sent to https://stats.grafana.org/
#
# Statistics help us better understand how Loki is used, and they show us performance
# levels for most users. This helps us prioritize features and documentation.
# For more information on what's sent, look at
# https://github.com/grafana/loki/blob/main/pkg/analytics/stats.go
# Refer to the buildReport method to see what goes into a report.
#
# If you would like to disable reporting, uncomment the following lines:
#analytics:
# reporting_enabled: false

41
files/promtail/promtail-config.yaml
View File

@@ -1,41 +0,0 @@
server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /tmp/positions.yaml
clients:
- url: http://loki:3100/loki/api/v1/push
scrape_configs:
- job_name: mmlogs-1
static_configs:
- targets:
- localhost
labels:
job: mmlogs-1
__path__: /logs-node-1/*.log
- job_name: mmlogs-2
static_configs:
- targets:
- localhost
labels:
job: mmlogs-2
__path__: /logs-node-2/*.log
- job_name: mmlogs-local-1
static_configs:
- targets:
- localhost
labels:
job: mmlogs-local-1
__path__: /logs-local-1/*.log
- job_name: mmlogs-local-2
static_configs:
- targets:
- localhost
labels:
job: mmlogs-local-2
__path__: /logs-local-2/*.log

2
scripts/general.sh
View File

@@ -9,8 +9,6 @@ logins () {
echo " - All Mattermost Grafana charts are setup." echo " - All Mattermost Grafana charts are setup."
echo " - For more info https://github.com/coltoneshaw/CS-Repro-Mattermost#use-grafana" echo " - For more info https://github.com/coltoneshaw/CS-Repro-Mattermost#use-grafana"
echo "- Prometheus: http://localhost:9090" echo "- Prometheus: http://localhost:9090"
echo "- Loki: http://localhost:3100/ready"
echo "- Promtail: http://localhost:9080"
echo "- PostgreSQL" "localhost:5432" with 'mmuser' / 'mmuser_password' echo "- PostgreSQL" "localhost:5432" with 'mmuser' / 'mmuser_password'
echo echo
echo =========================================================== echo ===========================================================