Data Science Intern (web scrapper)

As a Web Scraping focused Data Engineer, you will be responsible for extracting and ingesting data from websites/other sources using web crawling tools. In this role you will own the creation process of these tools, services, and workflows to improve crawl/ scrape analysis, reports and data management. We will rely on you to test the data and the scrape to insure accuracy and quality. You will own the process to identify and rectify any issues with breaks as well as scale scrapes as needed.

What's required

• Experience running large scale web scrapes
• Solid Python knowledge
• Familiarity with Mongodb, feature engineering
• Familiarity with techniques and tools for crawling, extracting and processing data (e.g. Scrapy, pandas, mapreduce, SQL, BeautifulSoup, etc).
• Experience with system monitoring/administration tools.
• Experience with applications of LLMs.
• Experience with version control, open source practices, and code review
• Experience with applications designed to display archived web content
• Great communication skills (written and Spoken in English)