Digitalization is also finding its way into research. Our customer, ZEA-3, is a facility at Forschungszentrum Jülich with a focus on analysis services in the area of composition analysis.
‍
Using various methods, procedures and technologies, ZEA-3 analyzes physical samples from around the world. The results of the laboratory processes serve as the basis for scientific studies and thus form an important part of the research. The data is also essential for external customers to work on new innovations and products.

These scientific laboratory processes are laborious and complex. In order for the samples to be analyzed, they must first be prepared for the procedures used. In addition, the manufacturers of individual laboratory devices have specific formatting of data sets. This brings with it a high number of results and formats.

For uniform structuring, the data is transferred to Excel tables. This presentation of the data is all the more important because different research areas need access to the results.
‍
Therefore, error-free assignment of the specific results is crucial. This is associated with a clear structure, as several hundred samples are often examined in parallel. These individual procedures and results build on each other and form a coherent laboratory process.

The complex processing and preparation of the measurement results is time-consuming and error-prone. Therefore, ZEA-3 turned to SIDESTREAM to develop an automated solution. The goal was to develop comprehensive software that would make it possible to carry out the entire examination process fully autonomously in the long term. It is particularly important that no errors occur. Because the measurement results meet scientific standards and the institute's customers trust the accuracy of the data. The efficiency and user-friendliness of the software must therefore not come at the expense of data governance.

Our approach was to analyze the overall process and identify important fundamental process steps. Close communication with the scientists and high test coverage of the end product were of great importance.
‍
The focus of our work was on data quality and the FAIR principles. The acronym FAIR stands for Findable, Accessible, Interoperable and Reusable.
‍
It applied:
‍
• to output the individual formats of the analysis technologies and procedures without errors,

• calculate measured values correctly,

• and adhere to scientific standards.

We have managed to prepare the various formats and results clearly in a web application. The basis for this was the various result files of the individual processes. We used these to reverse engineer and derive input, output and the process function. Based on our findings, we programmed a first version of an analysis procedure. An important aspect of this project was the test runs with the scientists. In addition to automated data processing, the application presents the results and formats in a clear and structured manner so that the institute's employees can use the data efficiently. In the future, the entire analysis process will be fully automated so that scientists can concentrate on interpreting the data.

In addition to the large number of steps and data sets, repetition loops also make the entire analysis process very complex. The process steps are not strictly linear. Certain measurement results from the laboratory instruments may lead to repetitions of the respective analysis. These verification loops must be covered in the software to guarantee the accuracy of the results. To do this, we modeled the entire process as a Finite State Machine (FSM). This allows absolute control over the process. Because FSMs cannot be avoided. The process will only continue once the correct conditions have been fully met. This approach is essential for success, particularly in data-centric processes.

This FSM-based solution is a perfect example of a human-in-the-loop process. The scientists only carry out a few expert steps and enter the final results back into the system. The system consists of several Docker microservices. These were deployed locally in the research center. The data is managed in a PostgreSQL database. The majority of the code forms a Python backend, while the user application is a modern VueJS web application. The entire code is checked by a high test coverage. At least 90 percent must be achieved at every point in the code. Otherwise, the automatic test and merge process does not allow the introduction of new features (“merge in master”). This eliminates potential errors at an early stage and enables an application that meets scientific standards.