Big Data Engineer

About the role

We’re Hiring: Big Data Engineer

Are you passionate about building scalable big data solutions and high-performance data pipelines? Join our team and work with cutting-edge technologies to develop enterprise-grade data platforms.

Experience Required

• 4+ years of software engineering experience developing Big Data pipelines.
• 3+ years of hands-on experience programming in Scala and PySpark.
• 3+ years of experience building solutions on AWS, including AWS Glue.
• 3+ years of experience working with Big Data file formats, including Parquet.
• 1+ years of experience with Data Validation, Data Profiling, or building Rule-Based Data Engines.
• 1+ years of experience leveraging AI coding assistants or code-generation tools (e.g., GitHub Copilot, Codex) in daily software development tasks.

Key Responsibilities

• Design, build, and optimize a highly scalable, rule-based data validation engine.
• Develop robust, high-performance Big Data pipelines using Scala, PySpark, and AWS Glue.
• Write clean, well-tested code utilizing optimized formats like Parquet for efficient storage and retrieval.
• Perform detailed data validation and profiling to ensure data quality and consistency across data systems.
• Use enterprise-approved AI tools to streamline software development workflows and automate repetitive tasks.
• Collaborate with cross-functional teams to translate business requirements into scalable technical solutions.
• Evaluate emerging AI and Big Data technologies to drive innovation.
• Participate in code reviews, testing, debugging, and performance optimization.

Required Skills

• Scala
• PySpark
• AWS
• AWS Glue
• Parquet
• Big Data Pipelines
• Data Validation
• Data Profiling
• AI Coding Assistants (GitHub Copilot/Codex)

What you'll do

• Own delivery for client-facing web, mobile, AI, cloud or digital transformation work based on the role.
• Collaborate with project managers, designers, QA and client stakeholders through discovery, build, launch and support.
• Write clear documentation, communicate risks early and keep delivery quality high.
• Contribute to reusable engineering practices, reviews and mentoring inside the WebSenor team.

What we're looking for

• Strong hands-on experience relevant to this role and a portfolio of shipped production work.
• Clear communication, ownership mindset and ability to work with global clients.
• Comfort with delivery discipline: estimation, reviews, testing, documentation and release readiness.
• Ability to learn fast and work across modern product, engineering and AI-enabled workflows.

What you'll get

• Work on real enterprise client projects with measurable business outcomes.
• Collaborate with experienced engineering, design, QA and delivery teams.
• Growth through mentoring, project ownership and exposure to global clients.
• HR-approved compensation, benefits and policies as finalized during the offer process.

About the team

You will work with WebSenor project managers, UI/UX designers, QA engineers, cloud specialists and senior developers focused on quality delivery for enterprise clients.

Our hiring process

• Application Review: HR and delivery leadership review your profile for role fit.
• Technical Discussion: Meet the technical team for skill, communication and project-readiness evaluation.
• Final Conversation: Discuss compensation, joining timeline and team fit.
• Offer and Onboarding: Receive the offer, documentation checklist and onboarding plan.