Eclipse ioFog - Real World Patterns for the Edge

Abstract

Edge Computing will be one of the Mega technology trends for the next two decades. The transformation of our global IT infrastructure to capitalize on the explosion of data and devices at the Edge will require new approaches. Learn about these new Patterns for the Edge and get started with Eclipse ioFog.

Overview

The world is going through a transformation. Data is exploding at an unprecedented scale. By some estimates, the amount of data created in the world every year will balloon from 16ZB to 160ZB by 2025. This is largely being fueled by the explosive growth of the Internet of Things (IoT), currently estimated to approach 50 Billion in number by 2020. Whereas most computing devices were previously largely general purpose, Things are now usually single-purpose vertical products that are all generating and emitting an astounding amount of data.

What is the Edge?

Today's computing architectures were not designed to handle this amount of data. The recent market migration towards centralized Cloud solutions has resulted in a pattern where applications are required to push their data to the Cloud for processing. This is not scalable at the rate of data growth generated by the Things. New approaches to computing are required to dramatically reduce the cost and TCO of storing huge amounts of data.

Additionally, the pattern of use of data is changing. Traditionally, data storage was expensive and deriving insight and making decisions the purview of the database or data warehouse. The advent of Big Data changed all that, ushering in a new era of mass data storage where decisions are now made in real-time on massive fuzzy data sets by machine learning algorithms (rather than by humans).

Even if there was enough storage in the Cloud [there isn't!] to store all the data, there is no way near enough network to transport all the data to the Cloud. The advent of 5G will only increase the amount of Things creating data, while also creating a further imbalance in the network. The hop from Phone or Thing to cellular base station will be faster than the hop from cellular network to Cloud.

Inherently then, existing Cloud architectures exhibit a set of limitations:

1. Inefficient use of bandwidth, as not all IoT data needs to be streamed and stored in the Cloud, given that only a limited subset of it has actual business value
2. Network latency, as the interaction of IoT devices with the Cloud, is delayed, and therefore inappropriate for real-time applications (like autonomous cars)
3. Waste of storage, since excess data without essential business value is stored in the Cloud (e.g. in cases of sensors recording values that do not change frequently)
4. Limited privacy-friendliness, as IoT/Cloud deployments provide no easy way to isolate private sensitive datasets

These factors all lead us to examine the need for a new architectural strategy to support the data and application requirements of tomorrow. Collectively, this architectural shift has been termed Edge Computing. Rather than attempt to pull all of the data back to the Cloud, it makes much more sense to process the data at the point of creation. Instead of a Cloud-centric storage architecture, what's needed is an Edge-centric compute architecture. Instead of moving the data, move the compute, harnessing the processing power and resources of all of the Things.

Patterns for the Edge

One such approach is Eclipse ioFog. ioFog is a universal Edge Compute Platform. ioFog provides a standardized way to develop and remotely deploy secure microservices to Edge computing devices. ioFog makes deployment simple, by abstracting the underlying hardware to provide a common compute platform that enables the same software to run on any device. This is achieved by allowing developers to package and deploy their applications to each device using containerization technologies such as Docker and Unikernels.

ioFog is optimized to run on constrained hardware which is most commonly found at the edge. With a one-line install, ioFog Engine can be downloaded onto any piece of hardware that runs Linux (from Yocto to RedHat). Built with constrained resources in mind, ioFog can run on devices as small as a Raspberry Pi Zero, smart cameras, autonomous vehicles, LTE base stations, Wi-Fi routers, cable boxes, smart lighting, and IoT gateways.

Developers are able to easily deploy advanced AI and machine learning algorithms at the edge, to perform tasks such as real-time video streaming analysis, natural language processing, streaming analytics, etc.

Beyond One Device

If you've got one edge device chances are you've probably got more. Here, ioFog makes it easy to deploy and manage multiple devices (or nodes) at the same time as an Edge Compute Network (ECN). Since your ECN is likely distributed - composed of many different nodes across multiple networks, each with potentially differing microservices - ioFog manages them all automatically, even across complicated network configurations, and can be used to maintain and optimize the entire fleet of devices. Using a protocol-agnostic MIME-type-like approach to describing data, ioFog applications and nodes are able to handle and route any data type, even if they don't understand the actual payload of the data. ioFog also supports native Geofencing of data, nodes, and routing.

Securing the Edge

When dealing with the Edge, Cloud-based PKI security just doesn't work. Cloud-based PKI presupposes that devices are out of reach and that physical security at data centers assures only authorized personnel have access to the computing devices. The edge provides no such guarantee. ioFog's security model is inspired by Blockchain. Each node in your ECN is a member of a distributed trust network. Each node is constantly validating a comprehensive set of security rules with all the other nodes, looking for minor deviations or signals of rogue nodes. When a rogue node is found, they are automatically quarantined from the ECN. If rogue nodes do not pass stringent security checks to re-enter the network, they can be remotely wiped of all software and data. All communication and data transfer between nodes happens via session based MicroVPNs that are created on the fly by ioFog, making it easy for developers to focus on their application and automatically take advantage of the inherent capabilities of the ioFog platform.

Getting Started

If you want to play around with ioFog, getting started for is extremely simple. Assuming you have a Linux development environment with Docker installed, just enter these commands:

   $ curl -sSf https://iofog.org/quick-start/linux.sh | sh
   $ docker-compose up

Once Docker Compose has started, you can now use 'docker ps' to show the status of the running containers:

 $ docker ps
    CONTAINER ID    IMAGES                         etc...     NAMES
    ...             iofog-env_iofog-agent                     iofog-agent
    ...             iofog-env_iofog-controller                iofog-controller
    ...             iofog-env_iofog-connector                 iofog-connector

And that's it - you now have a three node ECN running locally!

Learning More

The Eclipse ioFog project has been in development for close to four years. It was initiated and contributed by Edgeworx, and is being used extensively by many companies across the Cloud, Healthcare, Oil & Gas and Telco industries, to name a few. If you want to learn more about ioFog, build ECNs and start developing microservices for your Edge devices, you can find out more at the ioFog website. Or, get stuck right in with the Getting Started guide and learn how to deploy one of the many prebuilt microservices from the ioFog Catalog. If you're more inclined to learn from the code, you can browse the repository or get involved in our forums.