Principal Data Engineer

All about Zeta Suite : <br/>Zeta is the world’s first and only Omni Stack for banks and fintechs. We are rethinking payments from core to the edge, led by the vision to augment the purpose of money and banking with technology. A single, modern software stack comprising processing, loans, customizable mobile and web apps, a fraud engine, and rewards for retail banking.We are a new-age, high-growth startup (&amp; a unicorn!) founded in 2015 by two visionary leaders, <a href="https://www.zeta.tech/in/about-us">Bhavin Turakhia &amp; Ramki Gaddipati</a>, whose entrepreneurial legacy &amp; excellence has put us on top of the global fintech ecosystem. Zeta counts amongst its customers over 10 banks and 25 fintechs across 8 countries - some of our notable clients include Sodexo - a leading issuer of employee benefits &amp; rewards with over 30 million global users, and HDFC Bank - the 14th largest bank by market cap in the world. <a href="https://www.zeta.tech/in/zeta-manifesto">Learn more about our manifesto &amp; beyond</a>. <h3>What would you do here? </h3> <ul> <li>Lead the design, development, and maintenance of data architecture and data models to support data ingestion, transformation, and analysis using tools such as Debezium, Kafka, Apache Spark, Flink, Airflow, Trino and DBT.</li><li>Develop, maintain and optimize ETL pipelines and data integration workflows using programming languages like Python, Java, and Scala to ensure high quality and timely delivery of data.</li><li>Monitor and troubleshoot data quality and performance issues, identifying opportunities for improvement and implementing best practices using tools like Prometheus and Grafana.</li><li>Design and implement data security and privacy measures to ensure data is protected and compliant with relevant regulations using tools such as Apache Ranger and AWS IAM.</li><li>Stay up-to-date with industry trends and emerging technologies to continuously improve the data engineering capabilities of the organization.</li><li>Mentor and guide junior data engineers, sharing your expertise in data engineering best practices, tools, and technologies.</li><li>Work with stakeholders to define and document data governance policies and ensure adherence to them.</li><li>Collaborate with data scientists, analysts, and business stakeholders to understand their data needs and provide scalable solutions</li></ul> <h3>What are we looking for?</h3> <ul> <li>Bachelor's degree in Computer Science, Information Systems, or related field.</li><li>8+ years of experience in data engineering, with a strong focus on data architecture, ETL, and data modeling using tools such as Apache Spark, Trino (formerly Presto), Airflow, DBT, and Python.</li><li>Experience leading the design and implementation of complex data systems and architectures, with a deep understanding of data warehousing concepts and cloud-based data platforms (e.g., AWS, GCP, Azure) and associated technologies such as S3, Redshift, and BigQuery.</li><li>Expertise in programming languages like Python, Java, and Scala, as well as one or more big data technologies (e.g., Hadoop, Spark, Kafka, Airflow). Excellent SQL skills are mandatory.</li><li>Experience with data security and privacy practices, as well as regulatory compliance (e.g., GDPR, CCPA) and tools such as Apache Ranger and AWS IAM.</li><li>Familiarity with streaming data architectures and tools like Debezium for change data capture and Apache Kafka for messaging.</li><li>Experience with distributed query engines like Trino (formerly Presto) for high performance querying of big data.</li><li>Strong leadership skills and experience mentoring and guiding junior data engineers.</li><li>Excellent problem-solving skills and ability to work independently or as part of a team.</li><li>Strong communication and interpersonal skills to collaborate effectively with cross-functional teams.</li></ul>Life At ZetaAt Zeta, we want you to grow to be the best version of yourself by unlocking the great potential that lies within you. This is why our core philosophy is ‘People Must Grow.’ We recognize your aspirations; act as enablers by bringing you the right opportunities, and let you grow as you chase disruptive goals. <br/><br/>#LifeAtZeta is adventurous and exhilarating at the same time. You get to work with some of the best minds in the industry and experience a culture that values the diversity of thoughts. If you want to push boundaries, learn continuously and grow to be the best version of yourself,  Zeta is the place to be!  <a href="https://careers.zeta.tech/">Explore the life at zeta </a><br/>Zeta is an equal opportunity employer.  At Zeta, we are committed to equal employment opportunities regardless of job history, disability, gender identity, religion, race, marital/parental status, or another special status. We are proud to be an equitable workplace that welcomes individuals from all walks of life if they fit the roles and responsibilities.