Senior Data Engineer

About Mrikal: Mrikal is a leading product studio specializing in digital solutions across Pharma, Retail, Oil & Gas, and Edtech industries. Our multidisciplinary team excels in design, development, deployments, data engineering, and AI. We are committed to delivering world-class solutions through good architecture, data practices, embedded AI, user empathy, and growth hacks.

About the Role: We are looking for a talented Data Engineer to join our team and play a pivotal role in designing, building, and maintaining the cloud-based data infrastructure for our clients. In this role, you will collaborate closely with data architects, analysts, and cross-functional teams to develop robust and scalable data pipelines, ensuring the seamless flow of data across various systems and platforms.

Responsibilities:
• Data Ingestion: Design and develop data ingestion pipelines to extract data from diverse sources, including databases, data lakes, and streaming platforms, into cloud-based data repositories.
• ETL/ELT Processes: Build and maintain ETL/ELT processes using cloud-native services and tools (e.g., Google Cloud Dataflow, Azure Data Factory, AWS Glue) to transform and load data into data warehouses or data lakes.
• Data Processing Workflows: Implement and optimize data processing workflows using distributed computing frameworks like Apache Spark, Apache Beam, or cloud-native services such as Azure Databricks, Google Dataproc, or Amazon EMR.
• Data Storage Solutions: Develop and maintain data storage solutions, including data lakes (e.g., Azure Data Lake Storage, Google Cloud Storage, Amazon S3) and data warehouses (e.g., Azure Synapse Analytics, Google BigQuery, Amazon Redshift).
• Collaboration: Work closely with data architects, data scientists, and analysts to understand data requirements and implement efficient data models and schemas.
• Data Quality: Ensure data quality, integrity, and security by implementing data validation, monitoring, and governance processes.
• Automation and Orchestration: Automate and orchestrate data pipelines using cloud-native tools (e.g., Azure Data Factory, Google Cloud Composer, AWS Step Functions) to ensure efficient and reliable data processing.
• Optimization: Optimize data pipelines and infrastructure for performance, scalability, and cost-effectiveness, leveraging cloud-native services and best practices.
• Technical Support: Provide technical support and documentation for data processing solutions and infrastructure.

Qualifications:
• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• Minimum of 5 years of experience as a Data Engineer, with a strong focus on cloud technologies.
• Proven expertise in at least one major cloud platform (Azure, GCP, or AWS) and its data services and tools. Preferably Google Cloud Platform (GCP), using services like Cloud Dataflow, Cloud Dataprep, Cloud Dataproc, and BigQuery.
• Proficient in Python, SQL, and scripting languages such as Bash or PowerShell.
• Experience with data processing frameworks and libraries like Apache Spark, Apache Beam, pandas, etc.
• Knowledge of data warehousing concepts, data modeling techniques (e.g., star schema, dimensional modeling), and ETL/ELT processes.
• Familiarity with big data technologies and frameworks like Apache Hadoop, Apache Kafka, etc.
• Understanding of data governance, data security, and compliance best practices.
• Experience with containerization technologies like Docker and Kubernetes.
• Strong problem-solving, analytical, and troubleshooting skills.
• Excellent communication and collaboration skills, with the ability to work effectively in cross-functional teams.
• Familiarity with agile methodologies and DevOps practices.

Preferred Expertise: While expertise in Google Cloud Platform (GCP) is preferred, candidates with experience in other cloud platforms like AWS or Azure will also be considered, as the core data engineering principles and technologies are transferable across cloud providers.

Why Join Us: At Mrikal, we pride ourselves on fostering a fun and inclusive culture that encourages creativity and innovation. As a Data Engineer, you'll have the opportunity to develop your skills, contribute to impactful projects, and be part of a team that values creativity, collaboration, and continuous learning.