Viewing device logs

ClearBlade IoT Core can send device activity logs to Cloud Logging. They include information such as device connections and errors. These are called events.

Authentication failures are not logged. A device must be authenticated to ClearBlade IoT Core to generate event logs.

Device events lifecycle

All ClearBlade IoT Core devices follow a similar lifecycle with the following events. Some of these events and their underlying details are captured in device logs.

  1. Connecting to ClearBlade IoT Core.

  2. Sending and receiving telemetry and state events.

  3. Disconnecting from or dropping the connection to ClearBlade IoT Core.

Whether the devices use the MQTT or HTTP bridge determines how these events occur and are logged.

Logging device activity

ClearBlade IoT Core uses log levels to determine which device events are sent to Cloud Logging. You can set a registry’s device log levels and all its devices or individual devices. A registry’s log level applies to all the registry’s devices. The log level set for an individual device overrides its registry's setting.

This table describes the available device log levels:

Log level

Description

Log level

Description

None

No device event logs are collected.

ERROR

Captures all error events, such as failed publishes.

INFO (MQTT only)

Captures all info events, such as connections and disconnections over MQTT. It also captures all ERROR events.

DEBUG

Captures all debug events, such as publishes, subscribes, and heartbeats. It is helpful in pinpointing problems with specific devices and captures all ERROR and INFO events. See Device debug logging for more information.

Important: Device log entries are created on a best-effort basis. Entries may not be logged during heavy traffic periods, even if the maximum number of device activity logs has yet to be reached.

Logged device event list

The following tables show:

  • Which device events are logged

  • The event’s eventType as logged in Cloud Logging

  • Whether successes, failures, or both are logged for each event

  • The log level that must be set on a registry or device to log the event

MQTT bridge

Device event

eventType

Success

Failure

Device event

eventType

Success

Failure

Authenticate to server

N/A

Not logged*

Not logged

Connect to server

CONNECT

INFO

ERROR

Disconnect from server

DISCONNECT

INFO

Not logged

Attach a device to a gateway

ATTACH_TO_GATEWAY

INFO

ERROR

Detach a device from a gateway

DETACH_FROM_GATEWAY

INFO

ERROR

Publish a telemetry event or state to the server

PUBLISH (on /devices/{device-id}/events or /devices/{device-id}/state MQTT topic)

DEBUG

ERROR

Receive a configuration update

PUBLISH (on /devices/{device-id}/config MQTT topic)

DEBUG

ERROR

Subscribe to Pub/Sub configuration topic

SUBSCRIBE

DEBUG

ERROR

Unsubscribe from Pub/Sub configuration topic

UNSUBSCRIBE

DEBUG

ERROR

PUBACK received from server

PUBACK

DEBUG

ERROR

Keep-alive ping sent to server

PINGREQ

DEBUG

ERROR

Command sent to device from server

PUBLISH

DEBUG

ERROR

Command PUBACK sent to server

PUBACK

DEBUG

ERROR

* If MQTT authentication succeeds, the device connects to ClearBlade IoT Core. A CONNECT event is logged rather than an authenticate to server event.

HTTP bridge

Device event

methodName

Success

Failure

Device event

methodName

Success

Failure

Authenticate to server

N/A

Not logged**

Not logged

Publish a telemetry event

google.cloud.iot.v1.PublishEvent

DEBUG

ERROR

Set device state

google.cloud.iot.v1.SetDeviceState

DEBUG

ERROR

Receive a configuration update

google.cloud.iot.v1.GetDeviceConfig

DEBUG

ERROR

** If HTTP authentication succeeds, the device connects to ClearBlade IoT Core. The relevant request message (GET if a configuration is received or POST if a message is published) is logged.

Enabling, changing, and disabling device logging

You can choose what logs are reported to Cloud Logging when you create or edit a registry or device in the console.

Setting a registry’s log levels

Registry log settings automatically apply to all the registry’s devices.

To create or edit a registry and set its log level:

  1. Go to the Registries page.

  2. At the top of the page, click Create Registry.

    To edit an existing registry, click its ID on the Registries page and click Edit registry at the top.

  3. Under Cloud Logging, select a log level.

  4. Click Create (if creating a new registry) or Update (if editing an existing registry).

Setting a device’s log levels

A device's log level overrides its registry's log level.

To set a new or existing device’s log level:

  1. Go to the console’s Registries page.

  2. Click the device registry ID.

  3. In the left registry menu, click Devices.

  4. To create a new device, click Create a device.

    To edit an existing device, click its ID on the Devices page and click Edit device at the top.

  5. Under Cloud Logging, select a log level.

  6. Click Submit (if creating a new device) or Update (if editing an existing device).

Viewing logs

You can view your project’s device activity logs in the Logs Explorer. Select device_activity in the log name menu.

For more details, see Using the Logs Explorer.

Exporting device logs

You can export device logs the same way as other kinds of logs.

These examples describe why you might want to export device logs:

  • You can export your device log copies to Cloud Storage, BigQuery, or Pub/Sub to keep device logs longer or to use more powerful search capabilities. Using Pub/Sub, you can export to other applications, other repositories, and to third parties.

  • To manage your device logs across an entire organization, you can create aggregated export sinks that can export logs from any or all projects in the organization.

Device logging limits

Device log quotas and limits apply at the project level. They are measured separately from and do not count toward other Cloud Logging quotas and limits. Cloud Logging quotas for other services are unaffected if device log quotas are exhausted. The reverse is also true.

Device log quotas are based on two factors:

  • The number of device events logged per second. Events include publishes, connections, disconnections, and so forth.

  • The total size (in bytes) of device events logged per minute.

If either device log quota is exceeded, device logs stop temporarily.

Here is a device log quota and limit list.

Best practices

Device debug logging

Setting the DEBUG log level for a registry or an individual device can generate large amounts of logging information. As a result, if you turn on debug logging for a registry with a large device fleet, logging records may be dropped due to the high log velocity and quantity.

For example, suppose you have a registry with 100,000 devices, and the registry’s debug log level is set. If each device publishes one telemetry event every second, only 1,000 out of 100,000 will be logged every second. This is because, as shown in Quotas and limits, the maximum number of logged entries is 1,000 per second.

For best results, enable debug logging for only a short time or a few devices.