Store-and-Forward: Not Losing Process Data When the Network Drops
A blank live screen during a network drop is obvious. The damage that lasts is elsewhere: the historian has a hole, the shift report cannot be rebuilt, and a quality or energy dispute has nothing to replay. Store-and-forward is the mechanism that keeps the archive usable after the link returns — local buffering at the gateway, then automatic backfill of process curves covering the interruption. It is not a convenience feature. It is a clause that can be written into a technical agreement and tested on site. This article sets out what is actually lost during an outage, the conditions the mechanism has to meet, how to accept it, and how it sits beside 24/7 platform operation and energy baselines.
What an outage actually takes out of the archive
Operators notice the blank screen first. The cost shows up later, when someone asks for the curve of a batch that ran during the drop, or when a production count and a quality event no longer line up.
A communication outage does not stop the process. The press still cures, the pump still runs, the meter still turns. What stops is the path that writes those values into the historian. After the link returns, live values resume — but the minutes or hours in between are missing unless the edge held them.
- Process curves: temperature, pressure, current, speed — the traces used for quality replay and dispute handling
- Production counts and yields: cycle counts, good/NG, output per shift — totals that no longer match the physical product
- Alarm sequence: first-out, duration, reset — the order of events that tells you what actually happened
- Shift and energy reports: any aggregation that reads the historian will either skip the gap or invent a number from incomplete samples
The live picture recovers by itself. The archive does not. That is the distinction store-and-forward is built to close.
What store-and-forward has to do to be more than a buffer
Buffering locally is not enough. If timestamps are rewritten on upload, if out-of-order packets overwrite newer values, or if backfill only covers some tags, the historian still cannot be trusted. The mechanism has to survive the outage as a time-stamped record, then merge it without damaging data that arrived later.
On delivered projects of ours the store-and-forward sub-item already has technical-agreement and acceptance evidence: after a communication outage is resolved, process curves from the interruption period are backfilled automatically. Field devices are collected at the gateway, held in local cache, and sent upstream over dual MQTT and REST channels with gateway, client and self-signed certificate options. That combination — cache plus dual channel plus written acceptance — is what makes the clause testable.
- Local cache at the gateway: values continue to be sampled while the uplink is down, within a declared retention window, rather than being discarded at the device
- Original timestamps preserved: each sample keeps the time it was taken at the edge, not the time it arrived at the server
- Automatic backfill after restore: reconnecting the link starts the catch-up without an operator exporting files by hand
- Dual-channel MQTT and REST: if one path is degraded, the other can still carry the buffered data; channel security is certificate-based
- Completeness that can be checked: tag list, sampling interval and the outage window are known, so missing points can be counted rather than asserted
None of this is a claim that every packet survives every failure. Cache depth, disk wear, power loss at the gateway and the length of the outage all bound what can be recovered. Those bounds belong in the technical agreement, not in marketing copy.
How to accept store-and-forward on site
"Data is complete" cannot be tested. An acceptance clause is useful only if a third person with no project history can run a procedure and get a yes or a no. The four steps below are the ones worth writing down before signing.
- Induced outage: disconnect the uplink for a declared interval while the process is running, record wall-clock start and end, then restore the link and wait for backfill to finish
- Time synchronisation: confirm NTP or an equivalent time source at the gateway and the server before the test, so backfilled samples do not land in the wrong hour
- Out-of-order write: after restore, verify that samples from the gap insert at their original timestamps and do not overwrite values collected after the link returned
- Report recalculation: rebuild the shift report, energy total or quality curve that covers the outage window, and compare it with the values recorded independently at the machine or meter
Pass means the historian contains a continuous series for the agreed tags across the outage window, timestamps line up with the independent record, and the rebuilt report matches within the tolerance written in the agreement. Fail means a gap, a time shift, or a report that still cannot be explained.
Fault scenario, with and without backfill
The table is written to be copied into a technical agreement as a discussion aid. Fill the third column with the actual cache window and tag list for the site, rather than treating it as a generic promise.
| Fault scenario | Without backfill | With backfill |
|---|---|---|
| Plant WAN or 4G drop for minutes to hours | Live screens freeze; historian has a hole; shift report and energy total cannot be rebuilt | Gateway keeps sampling; after restore, process curves for the interruption are backfilled automatically |
| Brief Ethernet flap on a single segment | A few samples missing; alarms may fire twice or not at all | Buffered samples fill the flap; alarm sequence can be reconstructed from original timestamps |
| Cloud or MQTT broker unreachable, field network still up | Edge keeps running but the historian stops; overnight production is invisible upstairs | REST or the second MQTT path, plus local cache, carries the backlog when the broker returns |
| Gateway reboot or power cycle | Buffer contents depend on whether cache was on persistent storage | Persistent cache survives a reboot within the declared window; volatile cache does not — state this in the agreement |
| Clock drift between gateway and server | Backfill lands in the wrong interval; reports double-count or skip | Time sync is a precondition of the test; out-of-order merge uses the sample timestamp, not arrival time |
Store-and-forward covers communication interruptions between the gateway and the upper system. It does not replace UPS at the gateway, RAID on the server, or a redundant SCADA host. Those are separate clauses.
Where this sits next to 24/7 operation and energy baselines
24/7 operation and store-and-forward answer different failures. Hot standby, RAID and a redundant host keep the SCADA server available when a machine fails. Store-and-forward keeps the historian usable when the field link fails while the process is still running. Specifying one does not cover the other.
Energy work starts only after collection is stable. A baseline with holes cannot support specific consumption, shift comparison or ISO 50001 records, because any gap is an argument waiting to happen. The same applies to OEE and quality replay: a missing curve is a missing proof. Treat store-and-forward as a data-chain precondition, then design 24/7 availability and energy metering on top of a historian that can actually be rebuilt.
Summary
A network drop takes the live picture first and the archive second. Store-and-forward is the gateway-side cache plus automatic backfill of process curves once communication returns — MQTT and REST dual channel, original timestamps, a completeness check that can be run on site.
Write the cache window, the tag list and the four acceptance steps into the technical agreement. That is what turns a feature name into a clause both sides can test.
Put store-and-forward on the acceptance list
Send us the tag list, sampling interval and the longest outage you need to cover, and we can propose a cache window, dual-channel path and an on-site acceptance procedure.
Contact Us