App health and Rails

App health answers four plain questions about an app you deploy with Kamal:

• Is it running the version you expect, with database migrations applied?
• Are requests mostly succeeding, at a normal speed?
• Are background jobs moving?
• Did the last deploy make any of that worse?

It is not a tracing or error-reporting product. Upstate keeps counts, timings in fixed buckets, and the names of controller actions and job classes. It does not keep request records, parameters, SQL, paths with IDs, client IP addresses, user agents, exception messages, or job arguments.

Request health with nothing to install

Every request to your app passes through kamal-proxy. The Upstate observer on your server reads the proxy's request log and sends one summary per minute and container: how many requests, how many returned 4xx and 5xx, and how long they took.

Open a project from Projects and find App health for the environment. It shows requests per minute, the share of server errors (5xx), and the 95th-percentile response time (p95) as small charts. Use the range buttons to switch between the last hour, 24 hours, and 7 days.

• No change to your app, image, or deploy configuration is needed.
• Requests to /up and /upstate/health are counted separately as excluded health checks and never count towards errors or speed.
• Nothing is exposed publicly. The observer reads the log on your server and reports it with its normal host report.
• If the page says there is no request data yet, update the observer on that server and send some traffic through kamal-proxy. See the host observer for how reports work.

Response times are reported as the upper edge of a bucket (for example 250 ms or 1 s), so treat p95 as a good indicator rather than an exact measurement.

Before and after each deploy

About 15 minutes after a deployment receipt reaches a final state, Upstate compares the 15 minutes before the deploy started with the 15 minutes after it finished. The receipt gets an After the deploy section with the error rate and p95 for both periods.

Upstate flags a regression when, after the deploy, there are at least 100 requests and either:

• the 5xx rate is more than twice the earlier rate and more than one percentage point higher, or
• p95 is more than double the earlier p95 and above 300 ms.

A regression adds a warning to the receipt and sends one message to your deploy notification destinations. If there is too little traffic to judge, the receipt says Not enough traffic to compare. That is not a pass or a fail.

Add Rails health with the gem

For Rails apps, the optional upstate-rails gem adds what request logs cannot see (Phoenix apps use the upstate_phoenix plug instead; see App health and Phoenix): whether the app booted on the expected version, pending migrations, Puma and database pool pressure, background jobs, and the slowest actions.

1. On the project page, open App health and choose Generate token under Add Rails health. Only owners and admins can do this. The token is shown once, so copy it before you leave the page.

2. Add the gem to your app:

   bundle add upstate-rails

3. Add the token to .kamal/secrets, and list it under env.secret in config/deploy.yml:

   UPSTATE_HEALTH_TOKEN=paste-the-token-here

   env:
     secret:
       - UPSTATE_HEALTH_TOKEN

4. Deploy with Kamal as usual.

The observer then asks each container for /upstate/health from inside your server's private network and sends the answer with its next report. The first result usually appears within a couple of minutes. Do not commit the token.

To replace a token, choose Rotate token, update .kamal/secrets, and deploy. To stop collecting Rails health, choose Revoke token.

What is sent

• App version, Rails and Ruby versions, whether YJIT is on, and when the app booted.
• Whether the database responds and whether migrations are pending.
• Puma worker and thread counts and backlog, and database pool size, busy, and waiting connections.
• Job counts: waiting, oldest waiting age, processed and failed in the last five minutes.
• Per minute, the top controller actions by name with counts, errors, timing buckets, and database time.
• Cache store name and whether it responds.

What is never sent

Request or response bodies, parameters, headers, cookies, paths with IDs, SQL, client IP addresses, user agents, exception classes or messages, job arguments, and environment variables. The token itself is never written to Upstate's logs or audit trail.

Background jobs

The gem reads job numbers from the shared store, so containers that only run jobs are covered too. Supported adapters are Solid Queue, Sidekiq, and GoodJob. Any other adapter is shown as not read yet.

Retention by plan

The last-hour chart uses per-minute data. The 24-hour and 7-day charts use hourly data, and ranges are shortened to what your plan keeps.

Troubleshooting

The Rails block on the project page shows what each container last reported and how long ago. When a container cannot be read, it explains why.

If you also set up the older Rails health endpoint check, it keeps working and is shown as Legacy health endpoint.