PostgresqlRestarted #

Postgresql restarted

alert: PostgresqlRestarted
annotations:
  description: |-
    Postgresql restarted
      VALUE = {{ $value }}
      LABELS = {{ $labels }}    
  runbook: https://srerun.github.io/prometheus-alerts/runbooks/postgres-exporter/postgresqlrestarted/
  summary: Postgresql restarted (instance {{ $labels.instance }})
expr: time() - pg_postmaster_start_time_seconds < 60
for: 0m
labels:
  severity: critical

Meaning #

The PostgresqlRestarted alert is triggered when the Postgresql instance restarts. This alert indicates that the database server has been restarted, which may cause temporary unavailability of the database and potential data loss.

Impact #

The impact of this alert can be significant, as it may:

• Cause temporary unavailability of the database, leading to errors or timeouts for applications that rely on it.
• Result in potential data loss or corruption if the database was not properly shut down.
• Require manual intervention to assess and address the cause of the restart.

Diagnosis #

To diagnose the issue, follow these steps:

1. Check the Postgresql logs to determine the reason for the restart.
2. Verify that the Postgresql instance is running and available.
3. Check for any error messages or alerts related to the restart.
4. Review system logs for any signs of system crashes or other issues that may have contributed to the restart.

Mitigation #

To mitigate the issue, follow these steps:

1. Verify that the Postgresql instance is running and available.
2. Run a database consistency check to ensure that the database is intact.
3. Review system logs and Postgresql logs to determine the root cause of the restart.
4. Take corrective action to prevent future restarts, such as addressing system crashes or configuration issues.
5. Consider implementing automatic failover or high availability solutions to minimize the impact of future restarts.
6. Notify stakeholders and application owners of the outage and provide an estimated time to recovery.

Note: For detailed instructions and troubleshooting steps, refer to the PostgresqlRestarted runbook.