Sr. Python Data Engineer

As a Senior Python Data Engineer at UrbanSDK, you will play a pivotal role in designing, developing, and maintaining both processing and backend components of our web applications.
You will leverage your expertise in Python for backend services, ensuring seamless integration and performance.

A key aspect of this role will involve designing and developing workflow tasking systems to manage complex processes and tasks efficiently. Additionally, you will work with big data technologies to handle and process large-scale data efficiently.

You will also work extensively with PostGIS for geospatial data management and utilize a range of technologies and tools to deliver high-quality software solutions.

Key Responsibilities:
Backend Development: Develop and maintain backend services using Python, leveraging Flask for web services, SQLAlchemy for ORM, and other Python packages like GeoPandas, Pandas, and PySpark for data processing and analysis.

Workflow Tasking System: Design and develop workflow tasking systems to automate and manage complex workflows to build analytics based on anomalies or thresholds.

Data Handling: Work with csv,geojson, shapefiles and other geospatial data, ensuring proper handling and integration within our applications.

Big Data Processing: Utilize big data technologies such as Apache Spark (via PySpark) to process and analyze large datasets. Implement data processing pipelines to handle large file processing.

Database Management: Design and optimize flat file data lakes for efficient data management and retrieval in big data contexts.

Integration and Deployment: Utilize Docker and Kubernetes for containerization and orchestration in our CI/CD pipelines.

Cloud Services: Manage and deploy applications on AWS, leveraging cloud services to enhance application scalability and performance.

DevOps Practices: Collaborate with the DevOps team to maintain and enhance infrastructure, ensuring efficient and reliable application operations.

Collaboration: Work closely with cross-functional teams including UX/UI designers, data scientists, and other developers to deliver high-quality software solutions.

Mentorship: Provide technical leadership and mentorship to junior developers, fostering a collaborative and innovative work environment.

Qualifications:

Education: Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
Experience: Minimum of 8 years of experience in full stack development with a strong focus on frontend and backend technologies.

Must Haves:
Backend Skills: Expertise in Python with hands-on experience in Flask, SQLAlchemy, and data processing using Pandas, GeoPandas, and PySpark.
Workflow Tasking Systems: Proven experience in designing and developing workflow tasking systems, including task automation and management.
Big Data Technologies: Experience with big data technologies, particularly Apache Spark (PySpark), and handling large-scale data processing.
Database Skills: Extensive experience with PostgreSQL and PostGIS for geospatial data management.
Data Formats: Familiarity with shapefiles and their integration into applications.
Soft Skills: Strong problem-solving abilities, excellent communication skills, and the ability to work effectively in a team environment.

Nice to haves:
Frontend Skills: Proficiency in React and experience with libraries such as Mapbox and Mapbox-GL.
Containerization and Orchestration: Experience with Docker and Kubernetes.
Cloud Platforms: Experience with AWS and cloud service management.
Messaging Systems: Knowledge of RabbitMQ for messaging and task queuing.