Big Data Developer

Job Duties And Responsibilities

Build processes that support data transformation, data structures, metadata, dependency and workload management

Build and Implement data ingestion and curation process developed using Big data tools such as Spark (Scala/python), Hive, Spark, HDFS, Sqoop, Hbase, Kerberos, Sentry and Impala etc.

Handle ingestion of huge volumes data from various platforms for Analytics needs and writing high-performance, reliable and maintainable ETL code Strong SQL knowledge and data analysis skills for data anomaly detection and data quality assurance.

Write shell scripts. Complex SQL queries, Hadoop commands and Git.

Create Database, Schemas, Hive tables (External and Managed) with various file formats (Orc, Parquet, Avro and Text etc.)

Monitor performance of production jobs and advising any necessary infrastructure changes.

Work on Code version using Bitbucket and CI/CD pipeline.

Requirements: Bachelor’s degree in Computer Science, IT, Engg or related with at least 60 months of experience. Send resume to: [email protected] including the JOB ID. Travel and relocation to various unanticipated client locations throughout the United States may be required.Equal Opportunity Employer.