Data Engineer at DSF
Job Location: Pune, India
Vanderlande’s warehousing solutions are the first choice for major e-commerce players across the globe, helping them to fulfil their promise of same-day delivery for billions of orders. In addition, nine of the 15 largest global food retailers rely on Vanderlande’s efficient and reliable solutions. Our airport’s baggage handling systems are installed in over 600 airports around the globe.
All those systems generate data. Do you see a challenge in building data-driven services for our customers using that data? Do you want to contribute to the fast growing Vanderlande Technology Department on its journey to become more data driven? If so, then join our Digital Service Factory (DSF) team!
As a Data Engineer, you will work in the newly created Warehouse Process optimization team, collaborating with data scientists, an architect, and a product owner. The WS Process Optimization tool is a new product under development, and upon hitting milestones, the team will be further expanded with more people to support the successful scaling and productization.
In your role, you’ll flexibly work on creating and scaling a prototype, by creating a backend to embed the models developed by the data scientists and the data ingestion connectors/pipelines to run the model. You will need to communicate with multiple technical stakeholders to agree on the requirements and interfaces. The team architect will be your sparring partner and you will collaborate on creating an end to end tool.
- You participate in end to end development through prototyping, validation, implementation, launch, and further iterations. You participate in architecture and roadmap discussions.
- You develop new features for our products and services which provide our customers with actionable information through multiple channels.
- You participate in the development of our back-end applications, (relational and graph) databases, data pipelines and APIs.
- You ensure that applications are running in a cloud-based environment.
- Collaborate together with front-end developers and data scientists in the team to reach the shared goal.
- You will develop scalable data pipelines to transform and aggregate data for business use, following software engineering best practices. For these data pipelines you will make use of the best frameworks available for data processing like Spark.
- You develop our data services for customer sites towards a product, using (test & deployment) automation, componentization, templates and standardization in order to reduce delivery time of our projects for customers.
- You design and build a CI/CD pipeline, including (integration) test automation for data pipelines. In this process you strive for an ever-increasing degree of automation.
- You will work with infrastructure engineer to extend storage capabilities and types of data collection (e.g. streaming)
- You enjoy rapid prototyping and getting your hands dirty.
- You are a team player and do not shy away from opportunities to help the team and product to grow.
- You are proactive and a strong communicator.
- You have a Bachelor or Master’s degree in Computer Science or equivalent.
- You can deal with the uncertainty of a new product introduction: you can take ownership of unclear requirements and support refining them, and you are flexible if the requirements change on the way.
- You have experience developing online applications, especially those containing a lot of data. Experience with Even Driven Architectures is a strong plus.
- You have a strong knowledge of web and cloud technologies such as: web services, REST API’s, messaging protocols (MQTT, AMQP, MQ), security (oAuth, OpenID, Keycloak).
- You have experience with Azure, including serverless concepts and CI/CD pipelines. Experience working with Kubernetes and Docker is a plus.
- You have experience creating and deploying SQL and noSQL databases. You have strong experience with Graph databases (e.g. deployment, querying, ETLs, …).
- You have experience building and optimizing data pipelines, architectures and data sets. Knowledge of Scalable data processing frameworks (e.g. Spark) is desired. Python fluency is desired.
- You are fluent in English.