Senior Data & AI Systems Engineer (SDE — ML / Data Engineering)

Lead Data & AI Systems Engineer (SDE — ML / Data Engineering)

On-Site | Hyderabad, India | Full-Time

About StepsAI

We build AI-powered agentic chatbots for websites, WhatsApp, Instagram, and Messenger. Our growth runs on a large-scale cold outreach engine targeting 2,00,000+ companies/month across India, USA, Canada, and globally. We're hiring someone to architect, build, and own the entire lead generation and cold emailing platform end-to-end — not just the data pipelines, but the full web application that the team uses daily to manage, monitor, and operate millions of leads across scraping, enrichment, verification, campaign execution, and analytics. You'll build it, ship it, host it, and keep it running.

The Problem

We have millions of raw store records. Our sending infra handles lakhs of leads/month. Today everything runs on spreadsheets, manual CSV exports, and laptop-by-laptop processing. Your job: replace all of this with a single, production-grade web application backed by autonomous pipelines.

What You'll Build

The Application — Full-Stack Lead Generation Platform (30%)

• Design, develop, and deploy a production web application (React/Next.js frontend + FastAPI/Python backend + PostgreSQL) for the entire lead generation and cold emailing operation
• Dashboard views: real-time funnel visualization, source-wise ingestion stats, enrichment progress, email verification rates, campaign health, domain/sender scores, cost-per-lead analytics
• Lead management UI: search, filter, segment, tag, and export leads across companies, people, emails, and phone numbers
• Campaign management module: create campaigns, assign email IDs, set sequences (primary + follow-ups), configure geo/industry/persona targeting, monitor sends/bounces/replies in real-time
• Admin panels for scraper management (start/stop/schedule jobs, view logs, manage proxy pools), enrichment tool configuration (API keys, credit tracking, waterfall priority), and domain/email-ID health monitoring
• Host and maintain the application on AWS/GCP with CI/CD, proper auth, role-based access, and production-grade reliability

Data Engineering & Scraping (30%)

• Production web scrapers (Scrapy, Playwright, Apify) for Google Maps, IndiaMART, JustDial, Yelp, BBB, government registries (MCA, FSSAI, US SoS portals)
• ETL pipelines to ingest, deduplicate, and normalize millions of records from 8+ parallel sourcing channels
• PostgreSQL database architecture for 24L+ cumulative company/people/email/phone records
• Website contact crawlers extracting emails, phone numbers, and WhatsApp links at scale
• Proxy rotation, anti-bot evasion, rate limiting for sustained large-scale scraping

GenAI & Agentic Systems (20%)

• Waterfall enrichment pipelines: Apollo → Hunter → Snov.io → RocketReach with fallback logic and credit optimization
• Multi-agent workflows (LangChain/LangGraph) for lead qualification, persona classification, and enrichment routing
• LLM-powered hyper-personalized cold email generation using company context, tech stack, and persona data
• Pattern-based email generation + bulk verification via MillionVerifier/ZeroBounce APIs

Infrastructure & Automation (20%)

• Autonomous campaign scheduling engine: queue management → AI email generation → capacity allocation → health monitoring
• Cron jobs, event-driven triggers, and workflow orchestration (n8n/Temporal) for fully autonomous 24/7 operation
• Email infrastructure management: warm-up ramp logic, auto-pause on high bounce, capacity redistribution across 127 domains / 381 email IDs
• CI/CD pipelines, Docker deployments, monitoring, logging, alerting
• Integration with existing StepsAI platform (Next.js frontend, NestJS backend) where needed

Required Skills

• Full-stack development: React/Next.js frontend + FastAPI/Python backend — must be able to build, ship, and host a complete production web application independently
• Strong Python: Scrapy, Playwright, pandas, BeautifulSoup, regex-based extraction, API integrations
• PostgreSQL: schema design for millions of records, indexing, deduplication, query optimization
• LLM/GenAI: hands-on with OpenAI/Anthropic APIs, LangChain/LangGraph, agentic workflows, RAG architectures
• Web scraping at scale: proxy management, anti-bot techniques, handling diverse data formats (HTML, JSON, CSV, government portals)
• DevOps fundamentals: Docker, AWS/GCP deployment, CI/CD, domain management, SSL, basic Kubernetes
• Data quality mindset: deduplication, normalization, validation, audit logging
• Git, cron-based automation, webhook integrations

Strong Plus

• MCP (Model Context Protocol) experience for tool integration
• Workflow orchestration: n8n, Temporal, Airflow, Prefect
• B2B data enrichment platforms: Apollo, Hunter, Clay, BuiltWith, Snov.io
• Email deliverability knowledge: domain warm-up, SPF/DKIM/DMARC, sender reputation
• OSINT tools: theHarvester, Maltego, SpiderFoot
• Government data scraping (India MCA/GST/Udyam, US SEC/SAM.gov)
• Experience building internal tools or SaaS dashboards from scratch

Qualifications

• B.Tech/M.Tech/MS in CS, Data Engineering, AI/ML, or related field
• 1–4 years experience (strong freshers with exceptional project portfolios welcome)
• Must work on-site from Hyderabad

What We Offer

• Own an entire platform from Day 1 — architecture, tech choices, UI design, deployment, everything
• Work directly with co-founders on a system that directly drives revenue
• Existing frontend and backend teams available to support — but you should be capable of building independently

To Apply: Send your resume with links to any projects (especially full-stack apps, scraping tools, or data platforms you've built and deployed) and a brief note on how you'd approach building a web application that handles 2L leads/month end-to-end from scraping to campaign delivery.