Azure Data Platform Specialist
The data platform and the data platform team:
- The data platform is created to enable data engineers, analysis, scientists, and other types of data users to perform their data work in an easier, faster and more reliable and secure way.
- The platform scope is to provide services and applications to ensure that data people can focus on data work and do not need to deal with underlying technical complexity
- In order to achieve what stated above the platform team will perform ingestion of raw data in what we call source data products, and create apps, templates, or other software solution with the goal of enabling data platform users.
- Although the team build also pilots, the reliability of the platform, ability to work faster, in an automated way and to reduce ops overhead are the main technical objectives when designing solutions
Role of the candidate in the team:
- This is a hands-on development role
- Perform ingestion from various types of source system (batch or streaming)
- Create ETL pipeline, simple from a logic perspective, but technically advanced (fully automated, configurable, scalable, cost efficient, etc)
- Design and develop advanced platform solutions in Azure to solve complex engineering/data needs (software libraries, templates, reusable mini-applications, APIs, catalogs, security frameworks, data encryptions frameworks, monitoring solutions, etc)
Important technical skills required:
Software engineering skills:
Experience working with source code and version repository
Understanding of the software lifecycle
Experience with both developing and operating applications
Experience with CI/CD and more in general DevOps practices
Understanding of Software engineering best practices
Understanding of batch and streaming architectures
Understanding of event-based orchestrations
Creation and management of APIs
Azure knowledge, in particular:
Extensive experience on Azure ADF
Experience with Azure DevOps
Ability to work with Blob Storage
Ability to work with several types of data structures (relational, file based, nosql)
Experience with SQL databases and/or other types of databases, like Cosmos DB, Redis
Important Soft skills required:
Ability to work independently as well as in a team Good communication skills (English)
Nice to have skills:
C# and .NET knowledge Python and Spark knowledge Understand the basics of distributed systems Knowledge of algorithms and data structures Ability to think at scale and in operational way
Experience and other characteristics:
- Min 2 – 3 years of experience on some topics.
- Passion for good software and data engineering practices
- Willingness to learn more