Senior Data Engineer

About Sand

Sand Technologies is a fast-growing enterprise AI company that solves real-world problems for large blue-chip companies and governments worldwide.

We’re pioneers of meaningful AI: our solutions go far beyond chatbots. We are using data and AI to solve the world’s biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on the world. For example, our AI systems help to manage the water supply for the entire city of London. We created the AI algorithms that enabled the 7th largest telecommunications company in the world to plan its network in 300 cities in record time. And we built a digital healthcare system that enables 30m people in a country to get world-class healthcare despite a shortage of doctors.

We’ve grown our revenues by over 500% in the last 12 months while winning prestigious scientific and industry awards for our cutting-edge technology. We’re underpinned by over 300 engineers and scientists working across Africa, Europe, the UK and the US. 

About the role

Sand Technologies build data-intensive systems that enable insight, intelligence, and informed decision-making. We typically work with hybrid data architectures with centralised lakehouses or data warehouses and distributed data products on top. Our stack includes tools such as Databricks, dbt, Docker, Python, SQL, and PySpark. We primarily work in cloud-native environments across AWS, Azure, and GCP, while occasionally supporting self-hosted open-source deployments.

A Senior Data Engineer is responsible for designing, building, and maintaining scalable data architecture that underpins our decision-support applications. Our decision-support applications range from traditional Analytics (data warehouse), to Machine Learning, to Digital Twins and on occasion serving LLMs and Agentic workflows, and as such your data architecture should support various use cases. You will work closely with cross-functional teams and contribute to the strategic direction of our data initiatives.

We operate with a strong code-first, “data as a product” mindset, where testing, reliability, observability, and performance are non-negotiable.

Specific Responsibilities

• Architect and build a secure, scalable urban data platform integrating multi-agency and infrastructure datasets at scale.
• Design resilient cloud-native architectures supporting batch, streaming, and near-real-time operational workloads.
• Lead development of high-performance ingestion and transformation pipelines across legacy systems, APIs, IoT/telemetry, and structured data sources.
• Implement distributed and event-driven processing systems (e.g., Spark, Kafka or equivalent) for large-scale analytical and operational use cases.
• Establish platform reliability standards, including observability, automated data quality validation, lineage, monitoring, and defined SLAs/SLOs.
• Design and enforce strong data governance and access control frameworks, including RBAC, encryption, auditability, and secure data handling practices.
• Build modern lakehouse or equivalent architectures that enable advanced analytics, GIS, and production-grade machine learning.
• Partner closely with data scientists, ML engineers, and senior stakeholders to operationalize AI and analytics at scale.
• Optimize platform performance, scalability, and cost efficiency as adoption grows.
• Contribute to long-term architectural direction and mentor engineering team members.

 

Requirements - Essential

• 6+ years designing and operating large-scale semi-distributed data platforms (hybrid centralised and distributed) in cloud or hybrid environments.
• Proven experience architecting modern data systems (lakehouse, data mesh, or equivalent) supporting both analytical (descriptive and predictive) and operational workloads.
• Deep hands-on expertise with distributed processing frameworks (e.g., Spark) and streaming/event systems (e.g., Kafka or similar).
• Strong experience building secure, governed data environments with robust access controls, encryption, lineage, and audit capabilities.
• Experience designing secure data platforms in regulated or government environments, with strong understanding of compliance, auditability, and data protection standards.
• Experience integrating heterogeneous data sources, including legacy systems, APIs, telemetry/IoT systems, and relational databases.
• Demonstrated ability to design highly available, observable, production-grade data systems.
• Experience enabling machine learning and advanced analytics through robust data i