PANORAMO

Current information

News, Events, Jobs

Artificial Intelligence for Sewer Inspection

IBAK Helmut Hunger GmbH & Co. KG is working on software to ease the inspector's workload during inspection data capture. The artificial intelligence (AI) of the software whose aim is to enable standard features and defects to be automatically identified offers great potential. IBAK has already made some remarkable progress in the development of this extensive AI project.

(This article was published in a German professional journal.)

1. Types of Artificial Intelligence
AI systems can be classified as weak or strong AI. Weak or narrow AI refers to AI systems that are developed and trained for a specific task. Virtual personal assistants such as Apple's Siri are weak AI applications. IBAK is also working on an AI system of this type. With the software development project under the name of ArtIST (Artificial Intelligence Software Tool), we are not talking about fantasies for the far-off future but about a practical application of AI in the wastewater industry. 'The heart of artificial intelligence consists in automatically processing large volumes of data, identifying underlying patterns without human assistance and making autonomous decisions and/or predictions on the basis of this.'1 With the requirements of the wastewater industry in mind, the identification of standard features and defects in sewers is a practical, recurring task

2. Demand in the Wastewater Industry
In Germany, the publicly owned sewer systems alone have an estimated length of more than 594,000 km.2 Moreover, the publicly-owned sewer systems are as a rule the biggest assets of municipalities. The time and effort required to inspect these subterranean assets regularly is considerable and qualified personnel is scarce. As a result, a demand exists in the wastewater industry to combat this challenge by means of an autonomously learning system.
The aim of the IBAK ArtIST software development project is to ease the inspector's workload during inspection data capture by automatically identifying standard features and defects in sewers. Thanks to the assistance with routine assessments, the inspector can then concentrate on the analysis of the defects. This is expected to have an enormous time-saving effect on his extensive workload and at the same yield results of very high quality.

3. How the Data Affects the Quality
To be able to develop AI-supported applications, data is required for testing and training the algorithms.3 The IBAK PANORAMO camera system has now been capturing sewer inspection data for more than 15 years. This 360° camera technology supplies optimum image data for the AI software because it ensures 100% coverage of the inner pipe walls. The highly efficient scanner technology therefore provides comprehensive basis data because there is no possibility of overlooking and therefore not recording relevant pipe segments. Part of the extensive PANORAMO data that has been assessed since 2002 has been used, and will continue to be used, as learning matter for training the AI software.
More important than the volume of data is its quality and in particular the context of the data must be suitable.4 To make the image data usable, it must be supplemented with further information. Assessments and decisions are required to enable them to be identified as underlying patterns or principles in a volume of data by machine learning. The quality of the AI software depends decisively on the data that is made available to the AI system. If it is supplied with inadequate or inconsistent information, it will learn to make inadequate assessments.

4. Supervised Learning
As the pioneer and originator of the sewer inspection industry, IBAK is in possession of the know-how necessary to ensure the quality of the data at the decisive stage of development of supervised learning. The AI developers at IBAK are trained systematically on the subject of sewer inspection and take part regularly in advanced training courses and relevant international symposia. The incoming data is checked on the basis of several decades of extensive experience in sewer inspection at IBAK and the specific knowledge of the expert IT personnel. In addition to this, the data is crosschecked and verified externally. This means that the quality of the data is always evaluated by experts who were not involved in capturing it. This ensures that the targeted high quality of the results provided by the IBAK ArtIST software can be achieved sustainably and objectively.
At IBAK, a team of 6 developers are working on the AI project. The AI team is supported by another 9 employees who are doing the annotations, that is the assignment of information to an image. This task accounts for approximately 80 percent of the time required for an AI project.5 PANORAMO scans that have already been assessed are used initially as specimen images for ArtIST. These are categorised by the associated defect code according to the European standard DIN EN 13508-2.6 This standard includes a coding system for the description of the features and defects that can be observed inside sewers during optical inspections. Defined symbols are allocated to the various types of defect and are specified as the correct answers with the specimen images. In this way, the program learns what connections, cracks, intruding roots and other defects look like. If the AI software is then shown a new inspection film, it can analyse this according to the learned pattern.

5. Great Additional Benefit with the Focus on the Essential
For the AI development project, several hundred kilometres of sewers have been analysed and processed by IBAK to date. Among other things, connections, joints (joint displacements) and cracks were intensively trained in the course of this work. So IBAK has first of all put the focus on the features and defects which an inspector encounters most frequently when capturing the inspection data. Capturing the mentioned features and defects automatically contributes considerably to easing the inspector's workload, as shown below.
A DWA survey provides representative information on the state of sewers in Germany.7 According to this, 'intruding connection' and 'defective connection' are the most frequent occurrences and account for 21% of the defects found in sewers. Even if you only take the capture of defects into consideration, it is already evident that connections represent the highest percentage of occurrences. The time and effort required by an inspector to capture this data is even greater. Connections without any defects are not included in the DWA survey but must be processed during the inspection. These can be identified and documented by the IBAK ArtIST software tool.
To determine what proportion of the inspector's work requires the most time and effort, not only the defects codes but also the master data and control codes such as pipe connection points and connections without any defects must be taken into consideration. An example of this is the percentage distribution of features and defects in the sewer database of a city with a population of over 600,000 (see fig. 1). According to this, the main code BCA (connection) alone accounts for 52 % of all features. Together with the associated main codes BAG (intruding connection) and BAH (defective connection), connections constitute 58 % (shown in light blue) of an inspector's capture work.

The IBAK AI development project can already support the inspector with over half of the input he has to perform solely by automatically identifying the connections. The connections (BCA) automatically identified by the IBAK AI development software are output fully coded according to DIN EN13508 with the main code and the characterization. The automatically identified joints (BAJ) are indicated in the B and C characterizations with the complete code, the position (on the clock-face) and the quantification.

If you also take into consideration the defects codes, master data and control codes that have already been examined by IBAK, some 80 % of an inspector's data capture work is already being covered during the training of the AI software.8

With the trained features and defects, IBAK reaches high success rates. Both the identification rates and the proportion of correctly assigned condition codes are greatly on the increase. Thus, it is already apparent at the present stage of development of the IBAK ArtIST software tool that it will provide considerable benefit for the work of the inspectors. The system learns new data every day. Further training is required to enable it to distinguish complex defects and to identify defects that do not often occur.

6. The Workflow
IBAK's team of developers is also working on incorporating the IT support into the inspector's working procedures. Envisaged is the following procedure: first of all a PANORAMO scan will be performed usual to capture the optical data. This film will then be loaded into the ArtIST cloud via the IKAS evolution sewer analysis software. This IT infrastructure will be provided via the internet. Sufficient storage space, the necessary computing power and the ArtIST software tool will be made available there. The ArtIST software tool will then create an inspection report document as a service. The report document with DWA M-149-29 coding will then be returned from the ArtIST cloud to IKAS evolution. The inspector will find this in the form of an observations list which is filled in with features and defects. The inspector will now be able to check the list of features and defects and complete this if necessary.

7. Future Prospects
The information given above has shown that the automatic capture of the most frequent features and defects can already make a considerable contribution to easing the inspector's workload. This was initially IBAK's main focus during the AI development work. The training performed up to now covers roughly 80% of the entries that are regularly made by an inspector during the capture of standard features and defects in an urban environment. The aim is not to restrict input to the main code but to indicate the features and defects together with the associated characterizations and quantifications.
Although as a first step the inspector no longer has to make the most frequent entries himself, but only needs to check them professionally, the increase in efficiency achieved is already of great benefit. At the present stage of development, IBAK is now starting on the first practical tests and pilot applications. At the same time, the supervised learning phase will be further continued in order to ensure high success rates and to cover more complex defects. Finally training will be extended to defects that do not often occur.
Right from the beginning of the AI development project, IBAK has had in mind various possible applications that could be beneficial to the sewer inspection industry in future. To already create a basis for this, IBAK is working on a flexible AI model and is giving it thorough and meticulous training. All future applications will be based on the stage of development the project has reached at the time of their implementation. The AI system then only has to learn what it does not already know. So the experience gained from training with the PANORAMO data will also be of use when IBAK extends the AI system in the future with MPEG data. Preparations are already in progress for augmenting the AI system to include the assessment of video data captured with a pan and rotate camera so that so that it will be possible to press ahead efficiently with further improvements.
The pioneer of the sewer inspection has thus laid all the foundations for making this innovative key technology available to the wastewater industry.

----------------------------------------

  1. Kreutzer, R. T./ Sirrenberg, M. (2019): Understanding Artificial Intelligence: Fundamentals, Use Cases and Methods for a Corporate AI Journey, Springer Gabler, p. 9
  2. de.statista.com/statistik/daten/studie/152743/umfrage/laenge-des-kanalnetzes-in-deutschland-im-jahr-2007/ (Survey on length of sewers)
  3. Simon, W. (2019): Künstliche Intelligenz. Was man wissen muss. Was kann sie? Wie funktioniert sie? Was sind die Folgen?, BoD, Norderstedt, p. 67 (Artificial Intelligence - what you need to know)
  4. German Federal Ministry for Economic Affairs and Energy, BMWi (2019): Technologieszenario 'Künstliche Intelligenz in der Industrie 4.0' , p. 22 (Artificial Intelligence in Industry)
  5. Simon, W. (2019): Künstliche Intelligenz. Was man wissen muss. Was kann sie? Wie funktioniert sie? Was sind die Folgen?, BoD, Norderstedt, p. 60 (Artificial Intelligence – what you need to know)
  6. Editor's note: 'Categorised' is used here in the context of data processing. 'Assessment' and 'Classification' are the terms commonly used in the wastewater industry for the evaluation of the results of optical inspections with regard to the any need for action. This is generally performed by sewage contractors.
  7. Cf.. C. Berger, C. Falk, F. Hetzel, J. Pinnekamp, S. Roder, J. Ruppelt: 'State of the Sewer System in Germany – Results of the DWA survey 2015'  Korrespondenz Abwasser, Abfall 2016 (63) Nr. 6
  8. 86.13% is the sum of the percentage of connections (BCA, BAG, BAH): 57.66%; cracks (BAB): 8.88%; joints (BAJ, joint displacements): 1.52%; connection points (BCD, BCE): 18.13%
  9. The coding system can be selected in the project configurations.