Data Engineer

At Lazarus, we’re building foundation models to help people solve the world’s toughest problems. Enterprise and government clients use our APIs to power products and workflows that keep humans focused on the things we do best. We're a venture-backed team headquartered in Boston, MA who share a passion for innovation, growth, and a commitment to the responsible use of AI.

You can check out a demo of what we do here: https://www.loom.com/share/cf119144b2f5482d9eed650da055389e

Job Overview

We are seeking a highly motivated and technically adept Data Engineer interested in contributing to our Machine Learning products in a hybrid role encompassing Testing and Data Engineering. The ideal candidate can take ownership of managing various data sets that we use for training, evaluation, and proving new capabilities while leveraging scripting and python skills to implement and automate testing for our range of document processing capabilities. The data needs to be stored securely (per company policies) while providing uniform easy access to anyone within the Lazarus team who needs access. You will work closely with our QA and testing engineers as they seek to automate regression and other testing for formal product releases, ensuring that the data sets can evolve over time (via versioning) and managing both the data and any ground truth labels associated with the data and extending their utilization through the Model Development lifecycle. You will also bring best practices for data management to the team including building a data catalog so we always understand the data available and creating the foundation for an internal data mart supporting platform monitoring and a full range of business intelligence operations while building infrastructure for automating and scaling testing of capabilities.

Responsibilities

• Contribute to discussions and decisions around infrastructure options for data storage, both offline for training as well as for production system
• Build a data catalog capturing what data we have, its description and usage, and any licensing status/issues/privacy concerns for specific data sets
• Work with ML team and Implementation/customer-facing team(s) to add new data from customers, standard tests, open sources, and synthetic data
• Version control data sets, including metadata and truth labels associated with data sets, track evolution of open source/external data
• Apply standard normalization as well as pre-processing to data sets
• Learn how to run our Synthetic Document Generation tool to construct or augment data sets
• Develop and extend testing scripts and infrastructure to standardize and automate elements of testing and validation across our products
• Build a pipeline to collect and generate near-real-time metrics around production performance

Requirements

• BS in STEM field or significant experience
• 2+ years of experience Python coding in a development environment (proficient to expert in Python)
• Experience working with Google Cloud Platform
• Experience managing large data sets
• Test scripting or test automation experience
• Experience working with Google Cloud Platform
• Secure data storage experience

Nice-to-haves

• REST interfaces and using Postman to exercise endpoints
• Pipeline implementation experience with tools like AirFlow or DBT
• SQL and NoSQL querying
• Experienced in building apps leveraging APIs

Benefits

• Comprehensive benefits package, including health, dental, and vision insurance, as well as retirement savings plans
• Flexible working from home arrangements
• Opportunities for growth and professional development
• A collaborative and supportive company culture
• Access to cutting-edge technology and resources for research and development
• Salary range: $100,000 - $130,000

Lazarus AI is an equal opportunity employer. We are committed to equal employment opportunity and nondiscrimination for all employees and qualified applicants without regard to a person's race, color, gender, age, religion, national origin, ancestry, disability, veteran status, genetic information, sexual orientation or any characteristic protected under applicable law. We do not tolerate discrimination or harassment of any kind. This applies to every aspect of employment at Lazarus, including, but not limited to, employment, training, promotion, demotion, transfer, leaves of absence and termination.