Data Engineer (m/f/d)

Schwarz Digits creates the technological foundation for digital sovereignty in Europe. As the IT and digital division of the Schwarz Group, we develop and manage the IT infrastructures for the retail divisions Lidl and Kaufland, as well as Schwarz Production and PreZero. At the same time, we operate as an independent provider in the external market to support companies across Europe in their digital transformation. We bundle our core services in the areas of Cloud, Cyber Security, Data & AI, Communication, and Workspace. The Impact You Will Create As a Data Engineer, you will build and scale the data backbone that powers tailored recommendations for millions of customers daily. You will design high-throughput data pipelines, operationalize Machine Learning models on Google Cloud Platform (GCP), and build robust microservices to serve real-time personalized experiences. If you thrive on handling massive retail datasets, leveraging Databricks, and establishing modern software engineering practices in production, this role is for you. Data Pipeline Engineering: Design, build, and optimize large-scale batch and streaming data pipelines using PySpark on Databricks to process complex retail and customer interaction datasets. Pipeline Deployment & Orchestration: Package, deploy, and maintain robust data workflows, heavily leveraging modern patterns like Databricks Asset Bundles (DABs) for standardized lifecycle management. Microservices & API Development: Develop and maintain high-performance, low-latency APIs in Python (FastAPI) and Kotlin to deliver recommendations downstream to consumer-facing applications. Infrastructure & Automation: Provision and manage scalable cloud infrastructure using Infrastructure as Code (IaC) and orchestrate containerized workloads on Kubernetes (GKE). Database Management: Structure, query, and optimize non-relational storage layers utilizing MongoDB for fast data lookup and feature retrieval. CI/CD & DevOps: Build, test, and automate CI/CD release pipelines using GitLab CI/CD, ensuring strict code quality, zero-downtime deployments, and system reliability. Experience and Skills You will Need Big Data Stack: Proven, hands-on experience developing production-grade data pipelines with PySpark on Databricks. Experience with Databricks Asset Bundles is strongly preferred. Cloud Infrastructure: Strong hands-on experience in Google Cloud Platform (GCP), including cloud-native storage, compute, and networking services. DevOps & IaC: Demonstrated proficiency with Kubernetes deployment and management, alongside Infrastructure as Code frameworks (e.g., Terraform). API Development: Strong programming skills in Python (FastAPI preferred) and/or Kotlin for building scalable backend services. Database Experience: Practical experience working with MongoDB or similar NoSQL databases for real-time applications. CI/CD & Source Control: Deep familiarity with version control and pipeline design using GitLab. Mindset: Strong problem-solving skills, focus on code maintainability, and enthusiasm for scaling recommendation engines in a fast-paced retail ecosystem. Our Offer 25 days annual leave + 1 day annual leave after 5 years in the company Meal tickets Additional health insurance A good work life balance with flexible working time A pleasant and diverse environment with regular events, team buildings and stimulating activities A huge array of tools & technologies available on the spot and ready for a steady personal development Variety of opportunities with one of the strongest and largest retail companies in the world You will be part of an international team composed of people from different countries and backgrounds Onboarding and support/mentoring