Cloud Computing and Geodata

There are good reasons why cloud computing plays an important role in working with geodata. The cloud offers the ability to quickly and flexibly scale resources for processing large geodata sets. This allows companies to work more efficiently and avoid bottlenecks. Scalability is also an important requirement to allow many users simultaneous access to the geodata. This promotes collaboration and facilitates the exchange of information between different teams and departments. By using cloud computing, companies can reduce the costs of building and maintaining their own infrastructures for storing and processing geodata. Instead, they only pay for the resources actually used in the cloud.

For efficient processing, analysis, and provision of geodata in the cloud, we have developed our own control and storage components that have been further developed as open-source projects for several years and are used in various projects.

Steep is a cloud-based workflow management system that orchestrates your microservices to process big data resource- and cost-efficiently. With Steep, we offer a very powerful and robust solution for effective control of complex data processing chains in the cloud. The calculation of these processing and analysis chains can run very long, sometimes over days or weeks. The use of Steep ensures a high level of automation that allows the process to run efficiently, robustly, and with as little user interaction as possible. Steep also helps to intelligently parallelize workflows to make optimal use of available computing capacities. Through the feature of ‘Required Capabilities’, Steep is able to request exactly the cloud resources needed by your services. This ensures that all workflows are processed as cost-efficiently as possible.

To the Steep website

Publications:

• Würz, H. M., Kocon, K., Pedretscher, B., Klien, E., and Eggeling, E. (2023). A Scalable AI Training Platform for Remote Sensing Data, AGILE GIScience Ser., 4, 53, https://doi.org/10.5194/agile-giss-4-53-2023
• Krämer, M. (2021). Efficient Scheduling of Scientific Workflow Actions in the Cloud Based on Required Capabilities. In S. Hammoudi, C. Quix, & J. Bernardino (Eds.), Data Management Technologies and Applications. Communications in Computer and Information Science (Vol. 1446, pp. 32–55). Springer. https://doi.org/10.1007/978-3-030-83014-4_2

With GeoRocket, we set new standards for high-performance storage of objects with semantics and spatial reference in a cloud-based data storage. GeoRocket can store large INSPIRE datasets, GeoJSON files, 3D city models, and all other XML-based geodata. It offers high-performance storage and supports multiple cloud-based back-ends such as Amazon S3, MongoDB, or distributed file systems.

The storage and indexing methodology developed in GeoRocket works schema-agnostic, format-preserving, cloud-based, event-driven – and fast. The software is also easily integrable into existing infrastructures.

To the Georocket website

Publications:

• Bormann, P., Krämer, M., & Würz, H. M. (2022). Working efficiently with large geodata files using ad-hoc queries. Proceedings of the 11th International Conference on Data Science, Technology, and Applications DATA, 438–445. https://doi.org/10.5220/0011291200003269
• Krämer, M. (2020). GeoRocket: A scalable and cloud-based data store for big geospatial files. SoftwareX, 11. https://doi.org/10.1016/j.softx.2020.100409