Lead Data Engineer
Role Overview
We are looking for a Lead Data Engineer with strong experience in Azure Data Engineering, Python/PySpark, SQL, and Data Warehousing. The role involves designing scalable data pipelines, leading Azure ETL solutions, and providing technical guidance to the team.
Key Responsibilities
- Design and develop scalable data pipelines using Python/PySpark.
- Architect and implement Azure ETL/ELT solutions using Databricks, Data Factory, Blob Storage, Synapse, Azure SQL, and Lakebase.
- Work with cross-functional teams to translate business requirements into technical solutions.
- Lead code reviews, ensure engineering best practices, and maintain technical documentation.
- Mentor junior engineers and provide technical leadership.
- Troubleshoot data, pipeline, and performance issues.
- Use Git, Azure DevOps, and Jira for source control and project delivery.
Required Skills
- 8+ years of experience in Data Engineering/Data Warehousing.
- 5+ years of experience with Python/PySpark.
- 8+ years of experience with SQL, including complex queries, stored procedures, and functions.
- Strong experience with Azure Databricks, Data Factory, Blob Storage, Synapse, Azure SQL, Lakebase, and Unity Catalog.
- Knowledge of Azure Functions, Logic Apps, Azure VMs, Git, and Azure DevOps.
- Strong problem-solving, communication, and leadership skills.
Preferred
- Azure Data Engineer certification is a plus.
- Experience with Lakehouse architecture and data governance.
Role Details
Role: Lead Data Engineer
Employment: Full-Time, Permanent
Experience: 8+ Years
Education: B.Tech/B.E., BCA, B.Sc., or any relevant postgraduate qualification