Lead IT Data Engineer (SQL/Python)

Job Details:

The Lead IT Data Engineer owns and drives the organization's data strategy, setting the technical direction for data engineering, enterprise data modeling, and agentic AI adoption. This role is responsible for platform architecture, tool selection, cloud strategy, budget and capacity planning, and the design of enterprise AI agent systems — all while establishing governance frameworks for responsible data and AI practices and mentoring the engineering team. 

Essential Duties and Responsibilities 

• Own and drive the overall data strategy, including the multi-quarter technical roadmap, platform architecture, and data engineering standards. 

• Architect end-to-end data solutions across cloud platforms (AWS, GCP, or Azure), setting standards for orchestration (Airflow, Dagster), transformation (dbt), streaming (Kafka, Flink), and storage (Delta Lake, Iceberg, Snowflake, BigQuery). 

• Own the enterprise data modeling strategy — crafting scalable models using dimensional, multi-dimensional, and advanced normalization techniques, with enterprise-wide documentation and metadata governance. 

• Define API design standards and data contracts to ensure reliable, well-governed interfaces between data producers and consumers. 

• Establish enterprise-level data governance, security, and compliance frameworks across all data and AI systems, including access controls, cataloging, and lineage. 

• Define and enforce CI/CD standards for data pipelines, containerized architectures (Docker, K8s), and infrastructure as code (Terraform). 

• Drive data observability practices and platform reliability at enterprise scale. 

• Drive build-vs-buy evaluations for data and AI tools, considering TCO, vendor lock-in, scalability, and organizational fit; manage vendor relationships. 

• Own or co-own infrastructure budget and capacity planning for data platform resources; optimize cloud costs at the organizational level. 

• Define and drive the organization's agentic AI strategy, architecting enterprise- scale multi-agent systems, autonomous data pipelines, and RAG/knowledge graph platforms. 

• Establish AI governance frameworks, including ethics policies, bias detection, safety guardrails, security standards (prompt injection, data exfiltration, PII), and compliance with emerging regulations (e.g., EU AI Act). 

• Establish LLMOps practices at scale — model deployment, prompt versioning, A/B testing, performance monitoring, drift detection, and cost optimization. 

• Design human-in-the-loop escalation paths for critical AI-driven decisions, ensuring appropriate oversight. 

• Lead AI platform evaluation and integration, including TCO analysis, data residency, and SLA requirements for agentic frameworks. 

• Set software engineering best practices — code review standards, design patterns, technical debt management, and documentation. 

• Advocate for and lead adoption of data mesh and data-as-a-product principles. 

• Mentor the engineering team on data engineering, data modeling, and AI best practices. 

• Perform other assigned job-related duties that align with our organization's vision, mission, and values and fall within your scope of practice. 

Qualifications 

Education: Bachelor's Degree o