Join the Insider! Subscribe today to receive our weekly insights
The Operational Data Platform is our vision for the future of OT data management: a cloud-native, contextualized data store that bridges the gap between IT and OT.
Welcome to Part 2 of our series on IT and OT Data.
In Part 1, we introduced the IT world of Data Lakes and Data Warehouses and reviewed the OT perspective on time series data. We also highlighted the importance of various forms of context. If you haven’t read Part 1, make sure to catch up!
In this part, we will discuss the concept of an Operational Data Platform.
Revisiting Data Lakes and Data Warehouses
As you remember, a Data Lake stores raw data from different sources, while a Data Warehouse holds processed and structured data. In the OT world, raw sensor data is typically stored in a Historian, which runs on a local server.
Let’s revisit the cookie factory, Sweet Harmony Treats, from Part 1.
John, the process engineer, is optimizing the baking process, and Emma, the maintenance engineer, is detecting faulty settings and predicting equipment failures. They both use the Historian to access their sensor data, often using spreadsheets to analyze the data. For example, Emma might extract times from her logbooks when maintenance was performed and cross-reference this with the Historian. John might use data from the MES system to find start and end times of production batches.
While they strive to do their best, manually linking data to events is time-consuming, error-prone, and unscalable. They are missing a crucial element: context.
For instance, if John is asked which type of cookie has the highest energy consumption, or wants to compare cookies baked in different seasons, he needs various types of context:
- Batch and/or order number
- Cookie type or product name
- Identifiers of raw materials used
- Quality data from lab technicians
- Maintenance reports from Emma’s team
This data is usually available in a Manufacturing Operations Management (MOM/MES) system, a relational database. The challenge is combining both data sources to answer questions such as:
- What was the average temperature during batch 2023.45A.11?
- How much energy was consumed to produce brownies last month?
- Which shift team has the lowest amount of rework?
Moving to the Cloud
The adoption of cloud-native Data Platforms for OT data is still limited. While primary operations use cases are slow to move, those involving less massive time series data and benefiting from additional data integration or contextualization are more likely to shift first. There are several reasons for this slow adoption:
Added Value: Cloud vendors often emphasize unburdening users, but end-users seek added value beyond maintenance—what can they do now that they couldn’t before?
IT/OT Convergence: Cloud is typically seen as an IT domain, while historians belong to OT. Bridging this gap requires cooperation between IT and OT, which we discussed in our article on IT/OT cooperation models.
Technological Challenges: Most IT cloud systems are based on relational data models or require interpolated/equidistant data sets. Cloud-native Time Series stores exist but often only hold a copy of on-prem data, offering limited added value.
Other Factors: Costs, security, and other considerations also play a role in the slow adoption.
Introducing Context
Like a Data Warehouse, we need to introduce context into our OT environment to make sense of time series data. For example, with the context of our cookie factory, we could compare energy consumption during each heat step over time with just one click. Relevant contexts in an industrial environment include:
- Asset Context: Insight into the physical assets in the plant, often found in Engineering or Master Data systems.
- Example: A temperature sensor in a specific section of the oven.
- Production Context: Links data to the actual manufacturing process, found in an MES.
- Example: Calculating average temperature and gas consumption for a specific batch.
- Maintenance Context: Direct insights into the OEE of equipment.
- Example: Understanding the relationship between maintenance actions and process conditions.
Other possible contexts include Financial Context (e.g., energy prices) and Quality Context (e.g., correlation between product quality and process parameters).
Crystal Ball: The Future of an Operational Data Platform
The diagram envisions a coexistence between existing Enterprise Data Platforms and Operational Data Platforms. In the long run, we expect full convergence, but let’s take it step by step:
Now: Most systems holding data are isolated. OT systems are local, while IT systems are cloud-centric. Powerful OT systems/historians are emerging but real platforms are still lacking.
Mid-Term (1-5 years): The Operational Data Platform will be cloud-native, enabling exponential scaling of future use cases. Integration between IT and OT platforms will be key, and providing both public and private cloud options will be crucial for security-sensitive industries.
Future (+5 years): IT and OT technologies and cultures will merge. While this seems like sci-fi today, the differences between IT and OT will eventually disappear.
Outlook
A fully operational Operational Data Platform is not yet available, but several vendors are moving in this direction. Start by diving into this topic to see how an ODP can benefit you. However, be prepared for challenges related to Data Governance, Quality, and Observability. Balancing data duplication with direct sourcing is also an ongoing issue.
Conclusion: The Operational Data Platform is the future of OT data management, bridging the gap between IT and OT with cloud-native, contextualized solutions. Embrace this vision to enhance data accessibility and drive digital transformation in your organization.
In Part 3, we cover Data Governance, Quality, and Observability!
