Principal Data Engineer (Big Data - Scala/Pyspark) – C13/VP - Chennai

The Role

We are looking for a hands-on Principal Data Engineer who is passionate about solving business problems through innovation and engineering practices. As a Principal Data Engineer, you will leverage your deep technical knowledge to drive the creation of high-quality software products. You will also be expected to mentor other engineers, share your technical expertise, and promote a culture of technical excellence within the team. The Principal Data Engineer will report to an Engineering Manager and will be a floating member of multiple engineering teams. There is an expectation to contribute to the codebase and deliver solutions against the sprint-level commitments.

Responsibilities

·         Code contributing member of multiple Agile teams, working to deliver sprint goals.

·         Demonstrating deep technical knowledge and expertise in software development, including programming languages, frameworks, and best practices. Providing guidance and mentorship to junior team members

·         Actively contributes to the implementation of critical features and complex technical solutions. Write clean, efficient, and maintainable code that meets the highest standards of quality.

·         Collaborate with other Principal Engineers to define and evolve the overall system architecture and design.

·         Provide guidance on scalable, robust, and efficient solutions that align with business requirements and industry best practices.

·         Offer expert engineering guidance and support to multiple teams, helping them overcome technical challenges, make informed decisions, and deliver high-quality software solutions. Foster a culture of technical excellence and continuous improvement.

·         Stay up to date with emerging technologies, tools, and industry trends. Evaluate their potential impact on the organization and provide recommendations for technology adoption and innovation.

Required Qualifications

·         10+ years’ experience of implementing data-intensive solutions using agile methodologies.

·         Proficient in one or more programming languages commonly used in data engineering such as Scala or Pyspark

·         Experience with Hadoop for data storage and processing is valuable, as is exposure to modern data platforms such as Snowflake and Databricks.

·         Proven experience of providing technical vision and guidance to a data team

·         Experience of modelling data for analytical consumers

·         Strong proficiency in working with relational databases and using SQL for data querying, transformation, and manipulation.

·         Clear understanding of Data Structures and Object-Oriented Principles.

·         Multiple years of experience with software engineering best practices (unit testing, automation, design patterns, peer review, etc.)

·         Experience in cloud native technologies and patterns (AWS, Google Cloud)

·         Multiple years of experience architecting and building horizontally scalable, highly available, highly resilient, and low latency applications

·         Multiple years of experience with Cloud-native development and Container Orchestration tools (Serverless, Docker, Kubernetes, OpenShift, etc.)

·         Ability to automate and streamline the build, test and deployment of data pipelines.

·         Thrives in a dynamic environment, capable of managing multiple tasks simultaneously while maintaining a high standard of work.

·