SoftCat Healthcheck Documentation

SoftCat Healthcheck is a service for monitoring cron jobs (see guide) and similar periodic processes:

  • SoftCat Healthcheck listens for HTTP requests ("pings") from your cron jobs and scheduled tasks.
  • It keeps silent as long as pings arrive on time.
  • It raises an alert as soon as a ping does not arrive on time.

SoftCat Healthcheck works as a dead man's switch for processes that need to run continuously or on a regular, known schedule. Some examples of jobs that would benefit from SoftCat Healthcheck monitoring:

  • filesystem backups, database backups
  • task queues
  • database replication monitoring scripts
  • report generation scripts
  • periodic data import and sync jobs
  • periodic antivirus scans
  • DDNS updater scripts
  • SSL renewal scripts

SoftCat Healthcheck is not the right tool for:

  • monitoring website uptime by probing it with HTTP requests
  • collecting application performance metrics
  • log aggregation

Concepts

A Check represents a single service you want to monitor. For example, when monitoring cron jobs, you would create a separate check for each cron job to be monitored. Each check has a unique ping URL, schedule, and associated integrations. For the available configuration options, see Configuring checks.

Each check is always in one of the following states, depicted by a status icon:

New. A newly created check that has not received any pings yet. Each new check you create will start in this state.
Up. All is well. The last "success" signal has arrived on time.
Late. The "success" signal is due but has not arrived yet. It is not yet late by more than the check's configured Grace Time.
Down. The "success" signal has not arrived yet, and the Grace Time has elapsed. When a check transitions into the "Down" state, SoftCat Healthcheck sends alert messages via the configured integrations.
Paused. You can manually pause the monitoring of specific checks. For example, if a frequently running cron job has a known problem, and a fix is in the works but not yet ready, you can pause monitoring the corresponding check temporarily to avoid unwanted alerts about a known issue.
Additionally, if the most recent received signal is a "start" signal, this will be indicated by three animated dots under the check's status icon.

Ping URL. Each check has a unique Ping URL. Clients (cron jobs, background workers, batch scripts, scheduled tasks, web services) make HTTP requests to the ping URL to signal a start of the execution, a success, or a failure.

SoftCat Healthcheck supports two ping URL formats:

  • https://healthcheck.softcat.org/ping/<uuid>
    The check is identified by its UUID. Check UUIDs are assigned automatically by SoftCat Healthcheck, and are guaranteed to be unique.
  • https://healthcheck.softcat.org/ping/<project-ping-key>/<name-slug>
    The check is identified by project's Ping key and check's slug (user-chosen, URL-friendly identifier). Optionally supports auto-provisioning: if you ping a slug value that does not have a corresponding check, SoftCat Healthcheck can create the check automatically.

You can append /start, /fail or /<exitcode> to the base ping URL to send "start" and "failure" signals. The "start" and "failure" signals are optional. You don't have to use them, but you can gain additional monitoring insights if you do use them. See Measuring script run time and Signaling failures for details.

You should treat check UUIDs and project Ping keys as secrets. If you make them public, anybody can send telemetry signals to your checks and mess with your monitoring.

Read more about Ping URLs in Pinging API.

Grace Time is one of the configuration parameters you can set for each check. It is the additional time to wait before sending an alert when a check is late. Use this parameter to account for minor, expected deviations in job execution times. If you use "start" signals to measure job execution time, Grace Time also sets the maximum allowed time gap between "start" and "success" signals. If a job sends a "start" signal but does not send a "success" signal within grace time, SoftCat Healthcheck will assume failure and send out alerts.

An Integration is a specific method for delivering monitoring alerts when a check's change states. SoftCat Healthcheck supports many types of integrations: email, webhooks, SMS, Slack, PagerDuty, etc. You can set up multiple integrations. For each check, you can specify which integrations it should use.

For more information on integrations, see Configuring notifications.

Project. To keep things organized, you can group checks and integrations in Projects. Your account starts with a single default project, but you can create additional projects as needed. You can transfer existing checks between projects while preserving their configuration and ping URLs.

Each project has a configurable name, a separate set of API keys, and a separate project team. The project's team is the set of people you have granted read-only or read-write access to the project.

For more information on projects, see Projects and teams.