Why open source
In many cases, complex enterprise solutions need to use solutions with specific requirements and possibilities. Proprietary IoT is easy to use and scale but is limited in specific ways of extending. MQTT broker can be constrained in functionality and implementation of MQTT standard, edge device libraries can lack necessary functionality, cloud services can be insufficient, and vendor lock-in is always a problem. Open-source opens up the possibility to use any software for IoT or open remote IoT, if it supports the interface of our used solution. We can always switch to a better one if any used service does not provide enough functionality. Different open source IoT management platforms provide different ranges and depths of individual IoT layers implementation. Therefore, it is essential to choose which IoT platforms open source or open remote IoT to use to specify solution requirements and compare it to what individual IoT management platforms can offer.
We will use IoT World Forum IoT Reference Model to touch most of the single parts of IoT.
What is an open source platform
IoT open source platforms contains multiple conceptual layers for how to compound a IoT platform or system.
Physical Devices and Controllers – This layer consists of things. Things are sensors, Edge nodes, and things with integrated sensors. Edge nodes serve as sensors’ gateway and provide local data processing and decision-making functionality independent of cloud connection.
Connectivity – Provides secure connection to the cloud to transfer all the necessary data. In deployment, this layer can use a single solution or multiple technologies, depending on the needs. For example, field Area Networks (FANs) alternatives can include wired, cellular, LPWAN, and many other wireless options, as well as multi-tiered solutions, and can be built out of private, public, or a mix of private and public transport solutions.
Edge Computing – This layer can also be called “Cloud Edge” or “Cloud Gateway” computing. This layer receives data from edge devices and routes it to further data processing cloud services.
Data Accumulation – Data from IoT devices are coming to the cloud in great size and variety, so there is a need to accumulate data for subsequent processing, normalization, integration, and preparation for upstream applications. This layer may be implemented in simple SQL or require a more sophisticated Hadoop & Hadoop File System, Mongo, Cassandra, Spark, or other NoSQL solutions.
Data Abstraction – In this layer, we organize incoming data from the data lake into the appropriate schema and flows for upstream processing, as data destined for downstream layers is reformatted appropriately for device interaction and queued for processing. A key architecture element for larger high-performance deployments is a publish/subscribe.
Application Layer – Here, application logic is executed, such as monitoring, process optimization, alarm management, statistical analysis, control logic, and logistics.
Collaboration and Processes – This is the main presentation layer for users to use data and interact with the system.
IoT platforms open source creation support
For supporting the creation of open source IoT platforms, two organizations aim to maintain an architectural model of cooperating services by defining interfaces between them. Services can be custom implemented or used existing services options or platforms for internet of things in the market:
LF Edge (Linux Foundation) – https://www.lfedge.org/
LF Edge is an umbrella organization that aims to establish an open, interoperable framework for edge computing independent of hardware, silicon, cloud, or operating system. By bringing together industry leaders, LF Edge will create a common framework for hardware and IoT software platforms standards and best practices to sustain current and future generations of IoT software platforms and edge devices.
We are fostering collaboration and innovation across multiple industries, including industrial manufacturing, cities and government, energy, transportation, retail, home and building automation, automotive, logistics, and health care — all of which stand to be transformed by edge computing.
Eclipse IoT (Eclipse Foundation) – https://iot.eclipse.org/
The Eclipse IoT Working Group is a collaboration of industry and academic partners who are building open-source technology that will be the foundation for the Internet of Things (IoT).
The collaboration focuses on building 1) open-source implementations of IoT standards and protocols, 2) open-source frameworks and services that will be used by IoT solutions, and 3) tools for IoT developers.
This wiki page tracks the activities of the working group.
The iot.eclipse.org website is intended to be the source of information for people who want to use Eclipse IoT technology.
For the implementation of the IoT service platform, we can use several projects. Most notable are:
OpenStack use case of Edge Computing: https://www.openstack.org/use-cases/edge-computing/
OpenStack provides fundamental infrastructure building blocks that can be deployed anywhere, including the edge of the network. The flexible and modular nature of OpenStack means you can efficiently run the minimal services required at the edge, yet provide robust support for bare metal, container technologies and virtual machines.
OpenStack is already the most highly distributed infrastructure software, running in thousands of data centers worldwide. Many users in the telecom and retail industries are working now to advance the edge computing use case with OpenStack.
StarlingX – https://www.starlingx.io/
Complete cloud infrastructure software stack for the edge used by the most demanding applications in industrial IoT, telecom, video delivery and other ultra-low latency use cases.
With deterministic low latency required by edge applications, and tools that make distributed edge manageable, StarlingX provides a container-based infrastructure for edge implementations in scalable solutions that is ready for production now.
Eclipse IoT – https://iot.eclipse.org/
Eclipse foundation manages IoT aimed projects that cover different areas of IoT platforms.
EdgeX Foundry is a vendor-neutral open-source platform hosted by the Linux Foundation, providing a common framework for industrial IoT edge computing. At the core, is a set of loosely coupled microservices organized in different layers.
The most ambitious platforms are Eclipse IoT and EdgeX Foundry, so we chose to investigate them and get a taste of what it is all about.
EdgeX Foundry is a highly flexible open-source software framework. The IoT World Forum Reference model holds the role of the Physical Devices and Controllers layer and can be used to implement Edge device functionalities. It refers to individual vendor-neutral modular services and interfaces on how they communicate. It radically speeds developer and technology providers’ time to market by device-data ingestion, normalization, analysis and sharing in support of new IoT data services and advanced edge computing applications, including AI and automation.
The main purpose of the edge device is to collect data, transform them, execute some basic logic, and then send commands to sensors or send transformed filtered data to the cloud with the option to push them to a queue in case of network unavailability. EdgeX ensures it by working in multiple layers, each designed to its specific purpose. Individual services are built into Docker containers and published to the Docker Hub. Then they can be booted up in any container host application like Docker and additionally orchestrated by any container orchestration tool such as Rancher or OpenYurt.
The device layer contains the specification of device services for sensory device integration. The purpose of device services is to support sensor connection by specific protocol. EdgeX has already prebuilt services for HTTP or MQTT. Core services implement data storage, command propagation, metadata and registry and configuration services. The next supporting services layer contains services for Rules processing, Scheduling, Alerts and Logging.
Export services layer contains the specification for services to register and send data into the cloud. Custom implementation of these services is needed, and this way, any cloud solution is supported by this platform. Core services for running EdgeX devices can be custom implemented, or we can use existing free, open-source products compatible with EdgeX specifications. For it to be a full-fledged solution, features like edge devices orchestration, updates, monitoring, and already implemented sensors integration are missing even though other open-source solutions can provide it. So, to add some important missing features, IoTech built a paid solution on top of EdgeX.
Its software for IoT that helps to connect and acquire real-time sensor data, run edge intelligence, integrate with any cloud with complete deployment flexibility and manage edge applications and nodes at scale.
Covers the full spectrum of secure hard and soft real-time edge computing requirements and adds Enterprise level support to enable EdgeX to be used in big industry solutions.
A comprehensive set of edge management focused capabilities which include:
- Centralized management for IoT edge applications and hardware nodes
- Host on any cloud or on-premise environment
- Robust and tolerant
- Handles connectivity losses and unexpected reboots
- Communication instigated by Edge Nodes
- Communication is all northbound; nodes only need to know how to contact the controller
- Handles connectivity losses and unexpected reboots
- Simplified 3rd party integration via CLI and open REST APIs
- GUI for user friendly operation
- Manage any containerized or native Linux binary applications
- Application deployment, configuration, updates and lifecycle management (start/stop/restart)
- Manage edge nodes
- Node onboarding / registration
- Install / update firmware, infrastructure and security certificates
- Node onboarding / registration
- Seamless integration with IOTech’s edge platforms, Edge Xpert and Edge XRT
- Monitor nodes and applications
- Visualization of alerts and metrics
- Explore live and historical data
- Use familiar tools (Grafana, Portainer, etc.) when desired
- Visualization of alerts and metrics
- Mange users, roles and permissions
- OT device / sensor provisioning
- Debugging and troubleshooting
- Remote node access via WebSSH reverse tunnel
- Define and execute a script on a node to perform maintenance tasks
- Remote node access via WebSSH reverse tunnel
EdgeXpert is EdgeX Foundry based Framework adding a lot of missing functionality and provides business ready solution with commercial grade support.
When speaking about Java technologies and the Eclipse development environment, IoT is covered by Eclipse Foundation. It is approached as an open-source initiative of multiple teams and companies to deliver different solutions for different problems of IoT that can be used together as a full-fledged solution.
The Eclipse IoT Working Group enables collaboration on developing open-source implementations of IoT standards and protocols, frameworks and services used by IoT solutions, and tools for IoT developers for commercial-grade IoT. It is supported and used by Bosch in their IoT platform.
Eclipse IoT unites different projects for the implementation of IoT platform layers. The most interesting are:
|4diac||Infrastructure for distributed industrial process measurement and control systems|
|AGILE||Modular hardware and software Gateway for IoT. (WiFi, BLE, ZigBee, Zwave..) to cloud|
|Californium||CoAP framework for back-end services communicating with IoT devices|
|Concierge||Small-footprint implementation of the OSGi Core Specifications R5 optimized for mobile and embedded devices.|
|Ditto™||Eclipse Ditto™ is a framework for providing the “Digital Twin” pattern for IoT applications in order to interact with IoT devices. That means that Ditto mirrors physical devices|
|fog05||Decentralized infrastructure for provisioning and managing compute, storage, communication and I/O resources available anywhere across the network.|
|hawkBit||Domain independent back-end framework for rolling out software updates to constrained edge devices as well as more powerful controllers and gateways connected to IP based networking infrastructure.|
|hono||Remote service interfaces for connecting large numbers of IoT devices to a back end and interacting with them in a uniform way regardless of the device communication protocol.|
|ioFog™||All of the pieces needed to build and run applications at the edge at enterprise scale. The diversity and complexity of edge hardware is abstracted by ioFog Agent software. The management and orchestration of edge microservices is performed by ioFog Controller and its supporting set of components.|
|Kanto||Eclipse Kanto™ is a modular IoT edge software that enables devices for IoT with all essentials like cloud connectivity, digital twins, local communication, container management, and software updates - all configurable and remotely manageable by an IoT cloud ecosystem of choice.|
|Kapua||Modular IoT cloud platform to manage and integrate devices and their data. A solid integrated foundation of IoT services for any IoT application|
|Keyple||Transforms complex contactless ticketing, transportation and event access systems into a simple integration to manage terminal readers and portals that use smart cards and mobile applications to enter and exit.|
|KUKSA||Shared building blocks for the Software Defined Vehicles that can be shared across the industry. That millions lines of code should go into generating customer value and not reinvented wheels. KUKSA tries to provide you with a solid set of wheels that can act a solid foundation for a variety of competing products and services. In that sense KUKSA components encourage cooperation on the plumbing, enabling competition and faster innovation cycles on the customer-value creating procelain.|
|Kura||Extensible open source IoT Edge Framework based on Java/OSGi for implementing IoT Gateway. Kura offers API access to the hardware interfaces of IoT Gateways (serial ports, GPS, watchdog, GPIOs, I2C, etc.). It features ready-to-use field protocols (including Modbus, OPC-UA, S7), an application container, and a web-based visual data flow programming to acquire data from the field, process it at the edge, and publish it to leading IoT Cloud Platforms through MQTT connectivity.|
Examples of how to use Eclipse IoT projects to implement IoT Platform requirements is to use:
- Kanto as Edge device with custom implemented sensors connectors
- Cloud connector to connect to Azure, Bosh IoT Suite
- Eclipse IoT powered cloud:
- Hono as cloud message broker
- Ditto as digital twin service to structure sensory data
- HawkBit to push software updates to edge
Each IoT platform is well implemented and maintained and can be used based on project scenario and requirements. Edge implementation has its differences and can be summarized as:
|Foundation||Eclipse Foundation||Linux Foundation|
|Github Repo.||Kura||EdgeX Go|
|OS Support||Ubuntu Core,Raspbian, Linux (Ubuntu/Ubuntu Core, Yocto-based linux distribution), Windows.||Ubuntu Core,Raspbian, Linux (Ubuntu/Ubuntu Core, Yocto-based linux distribution), Windows.|
|Web based Administration Console allows user to do network, sensor related configurations.||EdgeX GUI is a work in progress. Checkout this Github repository.|
|Database support for local persistence||Supports H2, the Java SQL database for storing telemetry as well as configurations locally in IoT Gateway.||Supports MongoDB and Redis at the persistence layer.|
|IoT Cloud Platforms support||Eurotech Everyware Cloud™ platform, Amazon AWS IoT™ platform, Azure IoT Hub™ platform, Eclipse Kapua™ platform, Eclipse Hono™ platform||Supports integration with Google IoT Core out of the box. User can write their own “Export Service” to integrate with other cloud platforms.|
|Docker support||Docker Hub link||Docker Hub link|
|Remote Device Management Support||Supports remote diagnostics over pre-defined MQTT topics.|
|Field Protocol Support||1. OPC-UA
|Quality Assurance||Unit and Integration Tests are part of the main repository. As a developer, you should checkout Kura’s Github Wiki on testing|
|Contributors||Eurotech, Redhat||DellEMC, VMWare, Mainflux among other companies who fund development of EdgeX. See this list of member companies.|
|License||Eclipse Public License. This means that any developer/company using this project needs to disclose source code with the same license.||Apache 2.0. This does not warrant you to disclose source code. Developer/company only needs to preserve copyright and license notices|
Each solution has different services and applications to cover individual layers of IoT. Advanced data processing and machine learning have to be implemented custom way, most often by using the cloud.
Services implement layers:
|Collaboration and Processes|
|Application||hawkBit, ioFog||Edge Builder|
|Data Abstraction||Ditto||Edge Builder|
|Data Accumulation||Ditto||Edge Builder|
|Edge Computing||Ditto||Edge Builder|
|Physical Devices and Controllers||Kanto, Kura||EdgeX Foundry, EdgeXpert|
Now we can propose any of these solutions based on IoT platform requirements. We have a very good starting point in product implementation as we touched both technologies and understood how these services work and what is needed to make them work together.