Data Architecture Team Lead

Our Company

We’re Hitachi Digital Services, a global digital solutions and transformation business with a bold vision of our world’s potential. We’re people-centric and here to power good. Every day, we future-proof urban spaces, conserve natural resources, protect rainforests, and save lives. This is a world where innovation, technology, and deep expertise come together to take our company and customers from what’s now to what’s next. We make it happen through the power of acceleration.

Imagine the sheer breadth of talent it takes to bring a better tomorrow closer to today. We don’t expect you to ‘fit’ every requirement – your life experience, character, perspective, and passion for achieving great things in the world are equally as important to us.

The team

We are looking for a seasoned Lead Data Architect with deep hands-on expertise in designing and delivering event-driven architectures and real-time streaming systems. The ideal candidate will have extensive experience with Apache Kafka, Apache Spark Structured Streaming, Apache Flink, and messaging queues, and a strong background in building highly resilient IoT data platforms on AWS

The role

Architecture & Design

• Architect and implement scalable, fault-tolerant, and low-latency data pipelines for real-time and batch processing.
• Design event-driven systems using Kafka, Flink, and Spark Structured Streaming.
• Define data models, schemas, and integration patterns for IoT and telemetry data.

Technical Leadership

• Lead the technical direction of the data engineering team, ensuring best practices in streaming architecture and cloud-native design.
• Provide hands-on guidance in coding, debugging, and performance tuning of streaming applications.
• Collaborate with product, engineering, and DevOps teams to align data architecture with business needs.

Implementation & Delivery

• Build and deploy real-time data processing solutions using Apache Flink and Spark Structured Streaming.
• Integrate messaging systems (Kafka, Kinesis, RabbitMQ, etc.) with cloud-native services on AWS.
• Ensure high availability, scalability, and resilience of data platforms supporting IoT and telemetry use cases.

Innovation & Optimization

• Continuously evaluate and improve system performance, latency, and throughput.
• Explore emerging technologies in stream processing, edge computing, and cloud-native data platforms.
• DevOps,CI/CD, and infrastructure-as-code practices.

 

What you’ll bring

• Mandatory Expertise:
• Apache Flink (real-time stream processing)
• Apache Spark Structured Streaming
• Apache Kafka or equivalent messaging queues (e.g., RabbitMQ, AWS Kinesis)
• Event-driven architecture design
• AWS services: S3, Lambda, Kinesis, EMR, Glue, Redshift
• Additional Skills:
• Strong programming skills in Pyspark, Java, or Python
• Experience with containerization (OpenShift)
• Familiarity with IoT protocols and resilient data ingestion patterns
• Knowledge of data lake and lakehouse architectures (Iceberg) S3 storage

Preferred Qualifications:

• Experience in building large-scale IoT platforms or telemetry systems.
• AWS Certified Data Analytics or Solutions Architect.

About us

We’re a global, team of innovators. Together, we harness engineering excellence and passion to co-create meaningful solutions to complex challenges. We turn organizations into data-driven leaders that can make a positive impact on their industries and society. If you believe that innovation can bring a better tomorrow closer to today, this is the place for you.

 

 

Championing diversity, equity, and inclusion 

Diversity, equity, and inclusion (DEI) are integral to our culture and identity. Diverse thinking, a commitment to allyship, and a culture of empowerment help us achieve powerful results. We want you to be you, with all the ideas, lived experience, and fresh perspective that brings. We support your uniqueness and encourage people from all backgrounds to apply and realize their full potential as part of our team.

How we look after you

We help take care of your today and tomorrow with industry-leading benefits, support, and services that look after your holistic health and wellbeing. We’re also champions of life balance and offer flexible arrangements that work for you (role and location dependent). We’re always looking for new ways of working that bring out our best, which leads to unexpected ideas. So here, you’ll experience a sense o