GEMA – Central process control with Camunda
GEMA orchestrates more than 580 cross-team processes with millions of records centrally using Camunda. cimt developed the process control and supported the migration to Google Cloud.
OTTO
Big Data Analytics
OTTO – Big Data Analytics
Customer
Company: Otto GmbH & Co KG
Industry: E-commerce
Headquarters: Hamburg
Your contact
Rouven HomannBoard Member, cimt Group
Starting point
“Driven by data, inspired by our customers” – that is OTTO’s mission.
Business intelligence is a key driver in becoming a “data-driven company”. OTTO uses the latest BI technologies and makes the customer journey and core processes transparent through data science. Data generates insights that enable ever better strategies and processes for a successful digital future.
cimt actively supports OTTO in designing and implementing BI solutions:
Assignment
Building a platform for developing and operating analytical models based on big data technologies
- Creating a virtual working environment with a direct connection to the Cloudera Hadoop cluster
- Providing tools, languages and libraries for the data science department (including HDFS, Hive, Impala, Python, Scala, Spark, IntelliJ IDEA)
- Developing a self-service application for transferring data from the classic DWH into the Hadoop cluster
- Designing and implementing logically separated development, test and production environments for analytical processes
- Developing an integrated application for defining, building, publishing and deploying analytical processes as artefacts, and for running these artefacts in different logical environments
Challenges
- Building a flexible analysis environment
- Independent development of analyses (self-service) by business units and data scientists without support from software developers
- Defining quality gates for the transition of analyses from lab → prototype → factory
Technologies and methods
Scrum, Kanban, Cloudera (HDFS, Yarn, Hive, Impala, Spark), IntelliJ IDEA, Talend Studio DI, Java, Python, PySpark, Jupyter, Zeppelin, Scala, Teradata, Oracle, HP ALM, Jira, Confluence, Grafana, Git, Nexus, JFrog Artifactory, Docker, Jenkins, Maven and shell programming
Results
- In close collaboration with the business unit, an environment was created in which data scientists can independently develop analyses and bring them to production readiness. New (as yet unknown) libraries, modules and Spark-based programming languages can be tested while ensuring stable operations.
- The individual releases were delivered on time.
- Automation of build and deployment processes
References
More business cases
Panasonic Europe – Scalable approval processes with Camunda and DMN
Panasonic Europe digitalises hundreds of approval processes with one generic BPMN model and DMN decision tables in Camunda; cimt is responsible for the methodological design and supports the implementation.
GEMA – New data strategy with Databricks
GEMA used SQL, Excel and manual reconciliation for its reporting. cimt supported the build-up of a lakehouse platform with Databricks, through which business units now receive their data automatically – in under two hours instead of two to three days.