On the apps:
A camera that's offline will look something like this on the app's Cameras screen:
If you try to watch the camera's feed while it's offline, you'll get something like this:
On the website:
A camera that's offline will look like this on your Cameras page:
If you happen to be watching a camera's feed and it loses connection, you'll get something like this: