Data Engineer

Responsibilities:

Data Pipeline Development:

Assist in designing and building scalable ETL/ELT pipelines using tools.
Participate in the development and maintenance of data processing workflows, ensuring data accuracy and reliability.
Collaborate with the team to implement real-time and batch data solutions that meet business requirements.

Open-Source Tool Integration:

Contribute to the integration and optimization of open-source data engineering tools.
Work on integrating and managing databases to support various applications.
Stay engaged with new developments in open-source technologies and bring innovative ideas to the team.

Collaboration and Learning:

Work closely with senior engineers and data scientists to understand data requirements and translate them into technical solutions.
Engage in code reviews, pair programming, and team discussions to continuously improve your skills and contribute to the team's success.
Actively participate in team meetings and share your insights to help solve technical challenges.

System Monitoring and Improvement:

Monitor the performance of data systems and pipelines, identifying and addressing any issues proactively.
Contribute to the ongoing optimization of data systems for better performance, scalability, and cost-efficiency.
Help implement best practices in data governance, security, and documentation.

Requirements:

1. Education: Bachelor's degree in Computer Science.
2. 3-5 years of experience in software development and data engineering.
3. Hands-on experience with open-source data pipeline tools.
4. Strong programming skills in languages like NodeJs, Golang, and Java.
5. Proficiency in SQL and experience with both relational and NoSQL databases.
6. Solid understanding of data warehousing concepts, ETL processes, and data modeling.
7. Experience with CI/CD pipelines, containerization, and orchestration tools.
8. Strong problem-solving skills and a willingness to learn and adapt in a dynamic environment.
9. Excellent communication skills, with the ability to work collaboratively in a fast-paced, team- oriented environment.