Data explosion – dealing with a data overload

Big data challenges

Big data is associated with three well-known challenges, which were explained by KWR researcher Tessa Pronk. Big data refers to huge volumes of data, which are rapidly or continuously generated and in a great variety of forms. First and foremost, the architecture within which the data are captured has to meet certain requirements. The speed of data transfer can present a particular obstacle here. Innovative solutions to get around this problem include ‘Linked Data’ or a ‘Data Train’ for example. With a Data Train the software travels along the data, instead of the other way around. A second challenge has to do with making the data usable. When the data stream is so large, rapid and diverse, it is difficult to effectively safeguard its quality. Lastly, the use of the data is itself an added challenge. There is a large number of possible uses for the data. One needs to specify the purpose and clearly define the objective upfront.

Figure 1 – Improving data transfer.

Waternet data strategy

One of the main purposes of the knowledge exchange meetings is to share operational experiences. Eljakim Koopman from Waternet outlined his water utility’s data strategy. Waternet works with an Object Type Library (OTL) which forms the basis of its data strategy. In the OTL, all assets, such as pumps and dikes, are described as objects in information models, with all the information necessary for each object. The Linked Data in the OTL can flexibly interconnect all kinds of information. This drastically reducesthe number of separate applications, which were previously necessary for the data storage. With the data located in a single place, an adjustment to data only needs to be made once in case of changes. Data that are generated by a sensor for instance, can also be described in this manner. An Application Programming Interface (API) is used for the simple, rapid and automated acquisition of sensor data.

Figure 2 – Overview of the IT product within Waternet.

Differences among water utilities

The meeting concluded with an interactive discussion about various aspects of data management at the water utilities. This dealt primarily with experiences and bottlenecks. It turned out that some water utilities already have had experience with data management strategies, while others are still in the early stages. For those at the beginning, it is important to start with a clear conception of definitions and responsibilities. By investing sufficiently in this fundament, one ultimately saves on time and money. An insight from the Flemish water utility showed that data legislation by the public authorities can help set this process in motion.

Time, money and human resources

The discussion further revealed that the structuring of data demands a big investment in time, money and human resources. The participants agreed that it is difficult to draw up a financial cost-benefit analysis for a structured data management plan. The point is that the opportunities offered by a data strategy are hard to express in money terms. Nevertheless, it was concluded that in this day and age it would be unthinkable not to have a data management plan. If water utilities exchange their best practices in structured data management, this can help clarify potential added value. Starting small can lead to more.  The ‘Waterschapshuis’, which is a overarching initiative of the Dutch Water Authorities, was mentioned as an example of a centralised management of information models. An approach of this type would bring a big boost in efficiency for the water utilities.

Management of security risks

Lastly, there was also a discussion of the contribution of data management plans in addressing catastrophes such as a data-infarct or a security breach. By limiting the number of installed applications in a water utility, the security risks remain manageable. Following the discussion of all these different facets of data management, it was established that a clear strategy plan is needed to shape the digital transformation of water utilities.