Data Engineer

Azure Data Engineer with 6+ years of experience designing, developing, and optimizing enterprise-scale data platforms using Microsoft Azure and Snowflake

Proven expertise in building scalable data pipelines, data warehouses, and analytics solutions leveraging Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, Snowflake, Python, SQL, and Spark

Experienced in data modeling, ETL/ELT development, cloud migration, performance tuning, data governance, and implementing CI/CD practices

Strong background in integrating data from multiple sources and enabling business intelligence and advanced analytics solutions

Key Skills:

Azure Data Factory (ADF)

Azure Databricks

Azure Synapse Analytics

Snowflake Data Cloud

Python, PySpark, SQL

Azure Data Lake Storage (ADLS Gen2)

Azure Functions

Delta Lake

Data Warehousing & Data Modeling

ETL / ELT Development

CI/CD (Azure DevOps, Git)

Performance Tuning & Optimization

Data Governance & Security

Power BI Integration

Agile & Scrum Methodologies

What you will do:

Design, develop, and maintain scalable data engineering solutions on Azure and Snowflake platforms

Build and optimize ETL/ELT pipelines using Azure Data Factory, Databricks, and Snowflake to ingest, transform, and load data from various structured and unstructured sources

Develop and maintain enterprise data warehouses, data marts, and analytical datasets supporting reporting, BI, and advanced analytics initiatives

Implement data transformation and orchestration processes using PySpark, SQL, Python, and Snowflake stored procedures

Design and manage cloud-based data lakes using Azure Data Lake Storage (ADLS Gen2) and Delta Lake architecture

Collaborate with business stakeholders, data analysts, and architects to gather requirements and translate them into scalable data solutions

Monitor and optimize data pipeline performance, ensuring reliability, scalability, and cost efficiency

Implement data quality frameworks, validation rules, and monitoring processes to ensure accurate and trusted data

Manage Snowflake objects including databases, schemas, warehouses, tasks, streams, and data-sharing capabilities

Develop and maintain dimensional and normalized data models following industry best practices

Implement security controls, access management, encryption, and governance policies across Azure and Snowflake environments

Support cloud migration and modernization initiatives from on-premises or legacy platforms to Azure and Snowflake

Automate deployment processes using Azure DevOps, Git repositories, and CI/CD pipelines

Troubleshoot production issues, perform root cause analysis, and provide timely resolution for data-related challenges

Participate in Agile ceremonies, code reviews, and technical design discussions to ensure high-quality solution delivery

Create technical documentation, architecture diagrams, operational runbooks, and knowledge-transfer materials

Preferred Certifications:

Microsoft Certified: Azure Data Engineer Associate (DP-203)

SnowPro Core Certification

Originally posted on Himalayas