Data Engineer II

Posting title: Data Engineer II

Experience: 6+ Years

Location: Bangalore

Work mode: Hybrid

Primary skills: ETL, Data Warehousing, Reporting, SQL, Azure services such as Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Storage, etc.

Qualification: Any Engineering/ Computers degree

Roles & Responsibilities:

-Design, implement, and manage scalable and efficient data architecture on the Azure cloud platform.

-Develop and maintain data pipelines for efficient data extraction, transformation, and loading (ETL) processes.

-Perform complex data transformations and processing using Azure Data Factory, Azure Databricks, Snowflake's data processing capabilities, or other relevant tools.

-Develop and maintain data models within Snowflake and related tools to support
reporting, analytics, and business intelligence needs.

-Collaborate with cross-functional teams to understand data requirements and design appropriate data integration solutions.

-Integrate data from various sources, both internal and external, ensuring data quality and consistency.

-Ensure data models are designed for scalability, reusability, and flexibility.

-Implement data quality checks, validations, and monitoring processes to ensure data accuracy and integrity across Azure and Snowflake environments.

-Adhere to data governance standards and best practices to maintain data security and compliance.

-Handling performance optimization in ADF and Snowflake platforms

-Collaborate with data scientists, analysts, and business stakeholders to understand data needs and deliver actionable insights.

-Provide guidance and mentorship to junior team members to enhance their technical Skills.

-Maintain comprehensive documentation for data pipelines, processes, and architecture within both Azure and Snowflake environments including best practices, standards, and procedures.

Skills & Knowledge

-Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

-5 + Years of experience in Information Technology, designing, developing and executing solutions.

-3+ Years of hands-on experience in designing and executing data solutions on Azure cloud platforms as a Data Engineer.

-Strong proficiency in Azure services such as Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Storage, etc.

-Familiarity with Snowflake data platform would be an added advantage.

-Hands-on experience in data modelling, batch and real-time pipelines, using Python, Java or JavaScript and experience working with Restful APIs are required.

-Expertise in DevOps and CI/CD implementation.

-Hands-on experience with SQL and NoSQL databases.

-Hands-on experience in data modelling, implementation, and management of OLTP and OLAP systems.

-Experience with data modelling concepts and practices.

-Familiarity with data quality, governance, and security best practices.

-Knowledge of big data technologies such as Hadoop, Spark, or Kafka.

-Familiarity with machine learning concepts and integration of ML pipelines into data workflows

-Hands-on experience working in an Agile setting.

-Is self-driven, naturally curious, and able to adapt to a fast-paced work environment.

-Can articulate, create, and maintain technical and non-technical documentation.

-Public cloud certifications are desired