Senior Data Engineer

Key Responsibilities:

• Validate and rationalize estimation practices (story points, t-shirt sizing, function points) by aligning team size, velocity, and backlog management to actual delivery outcomes.
• Develop a framework/checkpoints for estimation validation to ensure forecasts for new data products are realistic and do not lead to unchecked growth in headcount.
• Analyze existing data ingestion processes and recommend standardized, reusable patterns for onboarding new data sources (e.g., metadata-driven pipelines, automation, ingestion templates) that reduce one-off solutions.
• Assess DevOps and CI/CD pipelines for bottlenecks and propose improvements that speed delivery without sacrificing quality.
• Collaborate with the Solution Architect on updating the current-state architecture and drafting future-state recommendations (integration, scalability, accelerators).
• Provide technical inputs to the Project Estimation Validation Report (current practices, gap analysis, skills assessment, accuracy improvements).
• Recommend best in class estimation discipline, backlog predictability, and scalable data engineering practices.

Required Skills & Experience:

• 8–10 years in data engineering with strong exposure to enterprise-scale data warehouses and analytics platforms.
• Hands-on expertise with Hadoop/Spark, Databricks (PySpark, Delta Lake), Snowflake, and AWS data services.
• Demonstrated experience in Agile estimation techniques (velocity management, story points, effort estimation models).
• Ability to correlate delivery estimates with actuals and recommend right-sizing approaches for teams and projects.
• Proven track record of designing scalable ingestion frameworks (e.g., metadata-driven ETL/ELT, reusable connectors, automation accelerators).
• Strong SQL and Python development skills; experience with orchestration (Airflow, dbt) and CI/CD for data pipelines.
• Consulting and communication skills to explain estimation methodology, team capacity modeling, and ingestion strategies to both technical and executive stakeholders.

Bachelor Degree in relevant field with 6- 8 years of relevant experience