Syncing LTE Beacon events with Estimote Cloud

Arguably the greatest power of the LTE Beacon is that it can talk directly to Estimote Cloud.

The way this works however is somewhat different than, for example, your computer, constantly connected to the Internet. That’s because being online uses energy, and that’s something the battery-powered LTE beacon has a limited amount of.

Instead, the LTE beacon spends most of its time offline. It’ll store data to be sent to Estimote Cloud in its memory. And once in a while, it’ll connect to the network to actually send that data. Or, in the LTE beacon’s micro-app terminology, it’ll enqueue events to be synced later. (For time-sensitive events, you can also force an immediate sync.)

What’s ahead (aka Table of Contents)

Enqueueing events

Each event has a name, and can also have some JSON data attached to it. Let’s check this example:

timers.repeat('1 hour', () => {
    var currentTemperature = sensors.temp.get();
    cloud.enqueue('temperature-checkpoint', {temperature: currentTemperature});
});

This code says that every 1 hour, the beacon will read the temperature from its sensor, and generate a new “temperature-checkpoint” event with the temperature attached to it. That event is then stored in the beacon’s memory until we sync it to Estimote Cloud.

Note: The events are stored in a persistent flash memory, so they’ll survive things like running out of power, or unexpected restarts.

Syncing events with Estimote Cloud

There are two main ways to do that:

  • You can configure your micro-app to do the sync periodically—for example, every 1 day.

or

  • Your micro-app can “force” a sync at any given time.

Periodic syncing

Out-of-the-box, the LTE Beacon will sync with Estimote Cloud every 1 month. Your micro-app can change that:

sync.setSyncPeriod('1 day');
// supported suffixes are:
// ms, s/sec/secs, m/min/mins, h/hour/hours, d/day/days, w/week/weeks
//
// min allowed sync period is 2 minutes (= 120 seconds = 120,000 ms), max is 90 days

Combined with the hourly temperature checkpoints, we have an app that measures the temperature every hour, and syncs the measurements once a day.

Important: The LTE Beacon doesn’t have persistent configuration—all the configuration is expected to be performed during the start of the beacon and the micro-app. So keep the code that changes the sync period in your app!

Forced syncing

Micro-apps can also force a sync. For example, maybe there are some exceptional situations when we can’t wait until the daily sync we’ve set above. We could code our micro-app to force a sync with a button press:

io.press(() => {
    sync.now();
});

This way, we get the benefit of automated, daily syncs, with a “fail-safe” option to force a sync with a button press.

Important consideration: battery life

In our temperature-measuring example, why not just sync every time we read from the temperature sensor? Why do the whole, “save events for later” dance?

The thing is, the LTE radio in the LTE beacon uses quite a lot of energy. The more often you use it, the shorter the battery life of your LTE Beacon. This means you need to find the right balance between how often you want to sync, and how often you’ll have to recharge the beacon. Here are some rough numbers to give you an idea:

sync period built-in battery life
daily 15 months
hourly 3 weeks
every 5 minutes < 2 days

Keep in mind: These numbers are just for LTE syncs. If you program your beacon to also do other energy-intensive operations, like GPS tracking or Bluetooth scanning, this will further reduce its battery life.

Important consideration: memory

Storing events to sync them in bulk later is a great little trick, but the storage space isn’t unlimited. If the events queue gets full and you try to enqueue more events, the LTE Beacon will:

  1. Start removing the oldest events, in order to make space for the new ones.

    If you don’t want your event to ever be automatically “overwritten” this way, you can mark it as non-volatile:

    cloud.enqueue('my-event', myData, {volatile: false})
    
  2. If there’s still not enough space (for example, because there’s too many non-volatile events in the queue), the beacon will attempt to sync the events to Estimote Cloud, in order to empty the queue. However, until that operation finishes and succeeds, enqueue will be throwing Not enough storage space errors, and the new events will be lost!

  3. Since (1) involves shuffling things around in the filesystem, and (2) involves network operations, you can expect the beacon and the micro-app to slow down considerably during that time.

As you see, letting the queue get full means potential trouble, so it’s best to never let that happen! We strongly recommended to find the right balance between the amount of events you generate and how often you sync.

Estimating the number of events you can store

The beacon has about 96 kB of flash memory allocated for “user data”, which is used for the events queue, but also for the micro-app and the data you manage via the storage.* API. A simple micro-app (like the one we’ve seen in Enqueueing events) is about 15–20 kB.

Each event is about 45 bytes for some metadata + 1 byte per each character of the event name + the payload encoded as MessagePack.

Note: The upper limit for the size of an event is about 4 kB, but this may change in the future.

So if we wanted to roughly estimate how many temperature checkpoints we can fit in the memory:

  • each event is about 45 bytes for metadata + 22 bytes for the “temperature-checkpoint” string + 22 bytes for the {"temperature": 21.2} payload encoded as MessagePack = a total of about 89 bytes
  • assuming our micro-app weighs 15 kB, and since we don’t use any storage.*, there’s 96 kB - 15 kB = 81 kB of “user space” left for the events
  • 81 kB = 82,944 bytes, divided by 89 bytes = 931 events
  • now, in reality, it might be a bit less, since no filesystem is 100% efficient
  • most importantly though, this is a very “ball-parky” math, so treat it as such: something to give you a rough understanding of the memory limitations, and not as something that will always yield a correct answer!

Say we sync daily, and we generate the temperature checkpoints:

  • every hour = 24 events per day = OK
  • every 5 minutes = 288 events per day = still most likely OK
  • every 1 minute = 1440 events per day, so we’d need to at least double the sync frequency (720 events per sync)

Important: Most of the things mentioned in this paragraph are something we consider implementation details, and may change at any time. We’ll try to keep this section up-to-date though!

Important consideration: data usage

LTE Beacon subscription comes with a cap on the amount of syncs and data usage, so maybe try not to do syncs every 5 seconds for days on end. And if you’re building a production app and plan to scale up with time, designing around the data usage and the costs it incurs from the very beginning is likely a good idea.

We’d be happy to geek out with you about how to architect your app for maximum efficiency, and do some back-of-the-envelope math to help you estimate the costs. You can reach us at contact@estimote.com.

Sending messages from Estimote Cloud to beacon

The LTE Beacon can receive so-called user messages from the Estimote Cloud. You can use this mechanism to send configuration or commands to your beacons. First, you need to add a handler for incoming messages to your micro-app code:

cloud.onReceive(message => {
  if (message.type === "my_type") {
    print("I received new text: " + message.payload.text);
    io.led(true);
    io.setLedColor(message.payload.color)
  }
});

This code registers a handler for incoming messages. If the beacon receives a user message with type my_type it will print text from the message, turn on the LED, and change its color.

Note: Each time you call cloud.onReceive the previous handler will be overwritten. By default your beacon will ignore user-messages, you have to program it to react to your messages explicitly. In our example the beacon will only be interested in messages with type my_type.

Now you can enqueue a user message using the Estimote Cloud user messages REST API. The Estimote Cloud will store your message and send it to the beacon during its next synchronization.

curl -X POST 'https://cloud.estimote.com/v3/lte/user_messages' \
-u 'YOUR_SDK_APP_ID:YOUR_SDK_APP_TOKEN' \
-H "Content-Type: application/json" \
-d '
{
  "device_identifier": "YOUR_DEVICE_IDENTIFIER",
  "type": "my_type",
  "payload": {
     "text": "change the LED color to red.",
     "color": [1,0,0] 
  }
}'

The Estimote Cloud stores your user messages in a queue (similarly to how cloud.enqueue stores device events), so you can create many messages that will be sent in a batch during next synchronization.

Keep in mind: remember to set the synchronization period so that you maintain a healthy balance between beacon’s responsiveness and battery life.

Final notes

Syncing via Web IDE & Bluetooth

It’s worth nothing that when you’re connected to the Web IDE, syncs are routed via Bluetooth, to the Estimote IDE web-app, and from there to Estimote Cloud.

The most important implication is, if you’re in an area without LTE-M coverage, you may see syncs work for you when connected to the Web IDE, but fail when not.

Syncing pauses when battery is low

If the battery is low on charge (specifically, low enough that the voltage drops below ~ 3.4 V), syncing (and any cellular operations for that matter) will be disabled.

The beacon might still be operational and you micro-app might continue to do non-cellular operations (for example, scanning via Bluetooth and queueing events), and once you recharge the beacon, syncing will be re-enabled automatically.

Note: The battery voltage also drops in low temperatures, so you may also experience no syncs, for example, if the LTE Beacon is deployed outside in Winter.