feat(v2.0.0): tsdb (#1162)

* feat: tsdb

* fix(ci): remove --parseGoList=false from swag init to fix dependency resolution

* fix(ci): fix swag init directory and temporary remove s390x support due to cgo issues

* fix(ci): fix swag init output directory to cmd/dashboard/docs

* fix(ci): set GOTOOLCHAIN=auto for gosec

* feat: add system storage maintenance for SQLite and TSDB

* shit

* feat: add s390x support and improve service monitoring

* ci: upgrade goreleaser-cross image to v1.25

* ci: add libzstd-dev:s390x for cross-compilation

* ci: build libzstd for s390x from source

* ci: add libzstd_linux_s390x.go for gozstd linking

* ci: use vendor mode for s390x gozstd build

* ci: clone zstd source for s390x build

* refactor(tsdb): rename MaxDiskUsageGB to MinFreeDiskSpaceGB and optimize queries

- Rename config to accurately reflect VictoriaMetrics behavior: minimum free disk space threshold
- Add QueryServiceHistoryByServerID for batch query optimization
- Fix hasStatus to avoid false status counting when only delay data exists
- Fix service aggregation boundary: use successCount*2 >= count
- Fix serviceID parsing with strconv.ParseUint error handling
- Add TagFiltersCacheSize for better query performance

* feat(api): add server metrics endpoint and simplify service history response

- Add /server/:id/metrics API for querying TSDB server metrics
- Simplify getServiceHistory by removing redundant data conversion
- Change AvgDelay type from float32 to float64
- Remove generated swagger docs (to be regenerated)
- Update TSDB query, writer and tests

* chore: 临时禁用不支持前端

* ci: cache zstd build for s390x to speed up CI

* fix(tsdb): fix race conditions, data correctness and optimize performance

- Fix TOCTOU race between IsClosed() and write/query by holding RLock
- Fix delay=0 excluded from stats by using hasDelay flag instead of value > 0
- Fix fmt.Sscanf -> strconv.ParseUint for server_id parsing with error logging
- Fix buffer unbounded growth by flushing inside lock when over maxSize
- Split makeMetricRow into makeServerMetricRow/makeServiceMetricRow
- Extract InitGlobalSettings() from Open() for VictoriaMetrics globals
- Remove redundant instance/GetInstance/SetInstance singleton
- Add error logging for silently skipped block decode errors
- Optimize WriteBatch* to build all rows in single write call
- Optimize downsample to use linear scan instead of map for sorted data
- Optimize query slice reuse across block iterations

* 服务添加DisplayIndex (#1166)

* 服务添加DisplayIndex

* 根据ai建议修改

---------

Co-authored-by: huYang <306061454@qq.com>

* fix(tsdb): restore SQLite fallback and monthly status reload on restart

- Restore ServiceHistory model and SQLite write fallback when TSDB is disabled
- Reload monthlyStatus (30-day) and serviceStatusToday from TSDB/SQLite on startup
- Add SQLite fallback query for /service/:id/history and /server/:id/service
- Remove breaking GET /service/:id endpoint, keep /service/:id/history only
- Add QueryServiceDailyStats to TSDB for per-day aggregation
- Add tests for monthly status and today stats loading from both TSDB and SQLite
- Migrate ServiceHistory table only when TSDB is disabled

* ci: exclude false-positive gosec rules G117, G703, G704

* feat(api): expose tsdb_enabled in setting response

* ci: restore G115 exclusion accidentally dropped in previous commit

* fix: update version numbers for OfficialAdmin and Official templates

* chore: upgrade frontend

* chore: upgrade frontend

---------

Co-authored-by: 胡说丷刂 <34758853+laosan-xx@users.noreply.github.com>
Co-authored-by: huYang <306061454@qq.com>

.github/workflows/release.yml

@@ -15,17 +15,18 @@ jobs:
         goos: [linux, windows]
         goarch: [amd64]
         include:
-          - goos: linux
-            goarch: s390x
           - goos: linux
             goarch: arm64
+          - goos: linux
+            goarch: s390x
 
     name: Build artifacts
     runs-on: ubuntu-latest
     container:
-      image: goreleaser/goreleaser-cross:v1.24
+      image: goreleaser/goreleaser-cross:v1.25
     steps:
-      - run: |
+      - name: Install dependencies
+        run: |
           apt update && apt install unzip curl -y
           wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/bin/yq
           chmod +x /usr/bin/yq
@@ -53,7 +54,45 @@ jobs:
       - name: generate swagger docs
         run: |
           go install github.com/swaggo/swag/cmd/swag@latest
-          swag init --pd -d . -g ./cmd/dashboard/main.go -o ./cmd/dashboard/docs --parseGoList=false
+          swag init --pd -d cmd/dashboard -g main.go -o cmd/dashboard/docs
+          go mod tidy
+
+      - name: Cache zstd for s390x
+        if: matrix.goarch == 's390x'
+        id: cache-zstd
+        uses: actions/cache@v4
+        with:
+          path: /tmp/zstd-s390x
+          key: zstd-s390x-v1.5.7
+
+      - name: Build zstd for s390x
+        if: matrix.goarch == 's390x'
+        run: |
+          # gozstd doesn't ship prebuilt libzstd for s390x, build it manually
+          go mod vendor
+
+          if [ -f /tmp/zstd-s390x/libzstd.a ]; then
+            echo "Using cached libzstd.a"
+          else
+            # Clone zstd source and build for s390x
+            ZSTD_VERSION=v1.5.7
+            git clone --depth 1 --branch ${ZSTD_VERSION} https://github.com/facebook/zstd /tmp/zstd
+            cd /tmp/zstd/lib
+            CC=s390x-linux-gnu-gcc \
+              CXX=s390x-linux-gnu-g++ \
+              AR=s390x-linux-gnu-ar \
+              ZSTD_LEGACY_SUPPORT=0 \
+              make clean libzstd.a
+            mkdir -p /tmp/zstd-s390x
+            cp libzstd.a /tmp/zstd-s390x/libzstd.a
+          fi
+
+          # Copy to vendor directory
+          GOZSTD_VENDOR=${GITHUB_WORKSPACE}/vendor/github.com/valyala/gozstd
+          cp /tmp/zstd-s390x/libzstd.a ${GOZSTD_VENDOR}/libzstd_linux_s390x.a
+          
+          # Create the Go file to link the library
+          printf '//go:build linux && s390x\n\npackage gozstd\n\n/*\n#cgo LDFLAGS: ${SRCDIR}/libzstd_linux_s390x.a\n*/\nimport "C"\n' > ${GOZSTD_VENDOR}/libzstd_linux_s390x.go
 
       - name: Build with tag
         if: contains(github.ref, 'refs/tags/')

.github/workflows/test.yml

@@ -34,7 +34,7 @@ jobs:
           go install github.com/swaggo/swag/cmd/swag@latest
           touch ./cmd/dashboard/user-dist/a
           touch ./cmd/dashboard/admin-dist/a
-          swag init --pd -d . -g ./cmd/dashboard/main.go -o ./cmd/dashboard/docs --parseGoList=false
+          swag init --pd -d cmd/dashboard -g main.go -o cmd/dashboard/docs
 
       - name: Unit test
         run: |
@@ -46,5 +46,7 @@ jobs:
       - name: Run Gosec Security Scanner
         if: runner.os == 'Linux'
         uses: securego/gosec@master
+        env:
+          GOTOOLCHAIN: auto
         with:
-          args: --exclude=G104,G402,G115,G203 ./...
+          args: --exclude=G104,G115,G117,G203,G402,G703,G704 ./...