Skip to content

Push monitor retries not reset after a heartbeat is receivedΒ #6586

@jdfranel

Description

@jdfranel

πŸ“‘ I have found these related issues/pull requests

None

πŸ›‘οΈ Security Policy

πŸ“ Description

Hi there,

I recently migrated to v2.0.2, migration when nicely (took a long time but eventually finished without issues) and then I used kumadb-migrator to move data to a mariadb database. Everything seemed to work correctly and I let i run. Not sure it is related to my issue but I put it there just to have the whole picture.

The issue is, since v2.0.2, I have a couple of PUSH monitors that are monitoring a daily job. The heartbeat duration is set to 1 hour 25 retries. But event when an UP heartbeat is pushed the retries are still incrementing until the monitor goes DOWN.

Here is what it looks on the dashboard:
Image

πŸ‘Ÿ Reproduction steps

  • Migrate to v2.0.2
  • (maybe move to mariadb)

πŸ‘€ Expected behavior

Same behaviour that I had before v2. As soon as the UP heartbeat is pushed, the retry counter should be reset.

πŸ˜“ Actual Behavior

The retry counter sometimes still keep counting event though an UP heartbeat is registered.

Here is the database extract where the heartbeat 27383803 does not reset the retries counter.

MariaDB [kumadb]> SELECT * FROM heartbeat WHERE monitor_id = 92 ORDER BY end_time DESC LIMIT 100;
+----------+-----------+------------+--------+----------------------------------------------------------------------------------------------+---------------------+------+----------+------------+---------------------+---------+
| id       | important | monitor_id | status | msg                                                                                          | time                | ping | duration | down_count | end_time            | retries |
+----------+-----------+------------+--------+----------------------------------------------------------------------------------------------+---------------------+------+----------+------------+---------------------+---------+
| 27418319 |         0 |         92 |      0 | No heartbeat in the time window                                                              | 2026-01-05 09:00:05 | NULL |     3600 |          0 | 2026-01-05 09:00:05 |      31 |
| 27413391 |         0 |         92 |      0 | No heartbeat in the time window                                                              | 2026-01-05 08:00:05 | NULL |     3600 |          0 | 2026-01-05 08:00:05 |      30 |
| 27408463 |         0 |         92 |      0 | No heartbeat in the time window                                                              | 2026-01-05 07:00:05 | NULL |     3600 |          0 | 2026-01-05 07:00:05 |      29 |
| 27403536 |         0 |         92 |      0 | No heartbeat in the time window                                                              | 2026-01-05 06:00:05 | NULL |     3600 |          0 | 2026-01-05 06:00:05 |      28 |
| 27398606 |         0 |         92 |      0 | No heartbeat in the time window                                                              | 2026-01-05 05:00:05 | NULL |     3600 |          0 | 2026-01-05 05:00:05 |      27 |
| 27393671 |         1 |         92 |      0 | No heartbeat in the time window                                                              | 2026-01-05 04:00:05 | NULL |     3600 |          0 | 2026-01-05 04:00:05 |      26 |
| 27388740 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-05 03:00:05 | NULL |     3600 |          0 | 2026-01-05 03:00:05 |      25 |
| 27383807 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-05 02:00:05 | NULL |     3602 |          0 | 2026-01-05 02:00:05 |      24 |
| 27383803 |         0 |         92 |      1 | Backup in file 2026-01-05_03.00.00_+0100-mealie.dump.gz (248K) succeeded in 0.716991 seconds | 2026-01-05 02:00:02 |  716 |     3598 |          0 | 2026-01-05 02:00:03 |       0 |
| 27378866 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-05 01:00:04 | NULL |     3601 |          0 | 2026-01-05 01:00:04 |      23 |
| 27373938 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-05 00:00:04 | NULL |     3601 |          0 | 2026-01-05 00:00:04 |      22 |
| 27369010 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 23:00:04 | NULL |     3601 |          0 | 2026-01-04 23:00:04 |      21 |
| 27364082 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 22:00:04 | NULL |     3601 |          0 | 2026-01-04 22:00:04 |      20 |
| 27359154 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 21:00:04 | NULL |     3601 |          0 | 2026-01-04 21:00:04 |      19 |
| 27354226 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 20:00:04 | NULL |     3601 |          0 | 2026-01-04 20:00:04 |      18 |
| 27349298 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 19:00:04 | NULL |     3601 |          0 | 2026-01-04 19:00:04 |      17 |
| 27344370 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 18:00:04 | NULL |     3600 |          0 | 2026-01-04 18:00:04 |      16 |
| 27339442 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 17:00:04 | NULL |     3600 |          0 | 2026-01-04 17:00:04 |      15 |
| 27334514 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 16:00:04 | NULL |     3600 |          0 | 2026-01-04 16:00:04 |      14 |
| 27329586 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 15:00:04 | NULL |     3600 |          0 | 2026-01-04 15:00:04 |      13 |
| 27324658 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 14:00:04 | NULL |     3600 |          0 | 2026-01-04 14:00:04 |      12 |
| 27319730 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 13:00:04 | NULL |     3600 |          0 | 2026-01-04 13:00:04 |      11 |
| 27314802 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 12:00:04 | NULL |     3600 |          0 | 2026-01-04 12:00:04 |      10 |
| 27309869 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 11:00:04 | NULL |     3600 |          0 | 2026-01-04 11:00:04 |       9 |
| 27304941 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 10:00:04 | NULL |     3600 |          0 | 2026-01-04 10:00:04 |       8 |
| 27300013 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 09:00:04 | NULL |     3600 |          0 | 2026-01-04 09:00:04 |       7 |
| 27295085 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 08:00:04 | NULL |     3600 |          0 | 2026-01-04 08:00:04 |       6 |
| 27290157 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 07:00:04 | NULL |     3600 |          0 | 2026-01-04 07:00:04 |       5 |
| 27285229 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 06:00:04 | NULL |     3600 |          0 | 2026-01-04 06:00:04 |       4 |
| 27280298 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 05:00:04 | NULL |     3600 |          0 | 2026-01-04 05:00:04 |       3 |
| 27275368 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 04:00:04 | NULL |     3601 |          0 | 2026-01-04 04:00:04 |       2 |
| 27270437 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 03:00:03 | NULL |     3602 |          0 | 2026-01-04 03:00:03 |       1 |
| 27265497 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 02:00:02 | NULL |     3601 |          0 | 2026-01-04 02:00:03 |      24 |
| 27265496 |         0 |         92 |      1 | Backup in file 2026-01-04_03.00.00_+0100-mealie.dump.gz (248K) succeeded in 0.842704 seconds | 2026-01-04 02:00:02 |  842 |     3600 |          0 | 2026-01-04 02:00:02 |       0 |
| 27260563 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 01:00:02 | NULL |     3601 |          0 | 2026-01-04 01:00:02 |      23 |
| 27255635 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-04 00:00:02 | NULL |     3601 |          0 | 2026-01-04 00:00:02 |      22 |
| 27250707 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-03 23:00:02 | NULL |     3601 |          0 | 2026-01-03 23:00:02 |      21 |
| 27245779 |         0 |         92 |      2 | No heartbeat in the time window                                                              | 2026-01-03 22:00:02 | NULL |     3601 |          0 | 2026-01-03 22:00:02 |      20 |

🐻 Uptime-Kuma Version

2.0.2

πŸ’» Operating System and Arch

Debian 12

🌐 Browser

Brave 1.85.118 (Official Build) (64-bit)

πŸ–₯️ Deployment Environment

  • Runtime Environment:
    • Docker: Version 28.0.4 (Build b8034c0)
    • Docker Compose: Version 2.34.0
    • MariaDB: Version 11.8 (LTS: Yes)
  • Database:
    • MariaDB: External
  • Database Storage:
    • Filesystem:
      • Linux: ext4
    • Storage Medium: NVMe
  • Uptime Kuma Setup:
    • Number of monitors: 118

Docker compose file

services:
  uptime-kuma:
    image: louislam/uptime-kuma:${UPTIMEKUMA_IMAGE_VERSION:-latest}
    restart: unless-stopped
    networks:
      - default
      - traefik
    environment:
      - TZ=Europe/Zurich
      - UPTIME_KUMA_DB_TYPE=mariadb
      - UPTIME_KUMA_DB_HOSTNAME=db
      - UPTIME_KUMA_DB_NAME=${MYSQL_DATABASE}
      - UPTIME_KUMA_DB_USERNAME=${MYSQL_USER}
      - UPTIME_KUMA_DB_PASSWORD=${MYSQL_PASSWORD}
    volumes:
      - ./data/uptime-kuma:/app/data
    labels:
      - traefik.enable=true
      - traefik.http.routers.${SERVICE_NAME:-uptime-kuma}.entrypoints=websecure
      - traefik.http.routers.${SERVICE_NAME:-uptime-kuma}.tls.certresolver=${TRAEFIK_CERT_RESOLVER-le-tls}
      - traefik.http.routers.${SERVICE_NAME:-uptime-kuma}.rule=Host(`${SERVICE_NAME:-uptime-kuma}.${BASE_URL}`)
      - traefik.http.services.${SERVICE_NAME:-uptime-kuma}.loadbalancer.server.port=3001
      - com.centurylinklabs.watchtower.monitor-only=false

  db:
    image: mariadb:${MARIADB_IMAGE_TAG:-latest}
    restart: unless-stopped
    environment:
      - TZ=Europe/Zurich
      - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
      - MYSQL_DATABASE=${MYSQL_DATABASE}
      - MYSQL_USER=${MYSQL_USER}
      - MYSQL_SPASSWORD=${MYSQL_PASSWORD}
    volumes:
      - dbdata:/var/lib/mysql
    labels:
      - com.centurylinklabs.watchtower.monitor-only=false

πŸ“ Relevant log output

uptime-kuma-1  | 2026-01-03T21:00:02+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 18 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-03T22:00:02+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 19 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-03T23:00:02+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 20 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T00:00:02+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 21 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T01:00:02+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 22 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T02:00:02+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 23 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T03:00:03+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 24 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T04:00:03+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 1 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T05:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 2 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T06:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 3 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T07:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 4 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T08:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 5 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T09:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 6 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T10:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 7 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T11:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 8 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T12:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 9 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T13:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 10 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T14:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 11 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T15:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 12 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T16:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 13 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T17:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 14 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T18:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 15 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T19:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 16 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T20:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 17 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T21:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 18 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T22:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 19 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-04T23:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 20 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-05T00:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 21 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-05T01:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 22 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-05T02:00:04+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 23 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-05T03:00:05+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 24 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-05T04:00:05+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Pending: No heartbeat in the time window | Max retries: 25 | Retry: 25 | Retry Interval: 3600 seconds | Type: push
uptime-kuma-1  | 2026-01-05T05:00:05+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Failing: No heartbeat in the time window | Interval: 3600 seconds | Type: push | Down Count: 0 | Resend Interval: 0
uptime-kuma-1  | 2026-01-05T06:00:05+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Failing: No heartbeat in the time window | Interval: 3600 seconds | Type: push | Down Count: 0 | Resend Interval: 0
uptime-kuma-1  | 2026-01-05T07:00:05+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Failing: No heartbeat in the time window | Interval: 3600 seconds | Type: push | Down Count: 0 | Resend Interval: 0
uptime-kuma-1  | 2026-01-05T08:00:05+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Failing: No heartbeat in the time window | Interval: 3600 seconds | Type: push | Down Count: 0 | Resend Interval: 0
uptime-kuma-1  | 2026-01-05T09:00:05+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Failing: No heartbeat in the time window | Interval: 3600 seconds | Type: push | Down Count: 0 | Resend Interval: 0
uptime-kuma-1  | 2026-01-05T10:00:05+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Failing: No heartbeat in the time window | Interval: 3600 seconds | Type: push | Down Count: 0 | Resend Interval: 0
uptime-kuma-1  | 2026-01-05T11:00:05+01:00 [MONITOR] WARN: Monitor #92 'DBBackup - mealie': Failing: No heartbeat in the time window | Interval: 3600 seconds | Type: push | Down Count: 0 | Resend Interval: 0

Metadata

Metadata

Assignees

No one assigned

    Labels

    A:monitorIssues or PRs related to monitorsbugSomething isn't workingtype:enhance-existingfeature wants to enhance existing monitor

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions