Data Engineer, Web Scraping

Posted: 4 months ago• Recruiter recently active

Job Location

Visa Sponsorship

Not Available

RelocationNot Allowed

Hiring contact

Olivia Quill

Employee

Charleston

About the job

In this role, you will:

Design, implement, and optimize end-to-end data pipelines for scraping and processing structured and unstructured data using Google Cloud Platform (or similar) and best practices;
Conduct ad hoc web scraping and data collection to support research and intelligence initiatives;
Prepare data for further analysis, including data cleaning, transformation, anonymization, and masking;
Contribute to the development of internal and external APIs, following best practices;
Collaborate with ML engineers, other data engineers, and software developers to deliver actionable insights and functional tools, including internal and external dashboards, APIs, and data dumps; and
Drive other critical initiatives.

Requirements:

Degree (or equivalent work experience) in Computer Science, Engineering, Information Science, Data Science or a related field (graduate degree preferred)
2+ years of professional experience in data engineering or a closely related field
Ability to communicate complex technical ideas clearly to non-technical audiences
Proficiency in Python, SQL
Experience with web scraping/crawling (e.g., Beautiful Soup, Selenium, Scrapy)
Experience with Google Cloud Platform (or similar), including storage and database services (e.g., Cloud Storage, CloudSQL, Cloud Spanner) and workflow orchestration (e.g., Cloud Composer/Airflow, Cloud Run, Pub/Sub)
Experience building and managing data pipelines, especially for text data
Comfort working in fast-moving, high-impact environments, such as startups, AI research labs, or security-focused teams

Compensation & Benefits:

Salary Range: $105K–$125K, depending on experience and location
Bonus: Performance-based annual bonus
Professional Development: Support for conferences, continuing education, or leadership training
Work Environment: Fully remote, U.S.-based
Health Benefits: Comprehensive health, dental, and vision coverage
Time Off: Generous PTO and paid holiday schedule

Retirement: 401(k) plan