Data Engineer

We are seeking a highly skilled Data Engineer to support enterprise data platforms that enable analytics, AI-driven insights, member communications, and business intelligence capabilities. This role requires deep hands-on expertise in Google Cloud Platform (GCP), modern data engineering frameworks, cloud-native architectures, and enterprise data governance.

The ideal candidate will have experience delivering scalable data solutions in GCP, building large-scale batch and streaming pipelines, and supporting healthcare or financial services organizations. As a Data Engineer, you will collaborate with architects, product owners, data scientists, and business stakeholders to design, develop, and optimize secure, reliable, and high-performing data platforms in a regulated environment.

Key Responsibilities

Data Engineering & Pipeline Development

• Design, develop, and maintain scalable batch and real-time data pipelines using GCP services including BigQuery, Dataflow, Pub/Sub, Cloud Composer, Cloud Functions, and Cloud Run.

• Build reusable and maintainable ingestion, transformation, and orchestration frameworks using Python, SQL, Spark/PySpark, Apache Beam, .

• Develop data integration solutions leveraging APIs, event-driven architectures, and cloud-native services.

Cloud Data Platform Development

• Build and support enterprise-grade data lake, data warehouse, and streaming solutions across GCP and AWS cloud platforms.

• Independently design, develop, and deploy cloud-native data products supporting analytics, reporting, AI/ML, and operational workloads.

• Participate in enterprise cloud migration initiatives and modernization efforts, helping define migration strategies and implementation roadmaps.

Data Architecture & Modeling

• Design and implement logical and physical data models using Star Schema and Snowflake Schema methodologies.

• Develop scalable solutions supporting BigQuery, Snowflake, Redshift, Teradata, and other enterprise analytical platforms.

• Optimize partitioning, clustering, indexing, and storage strategies to improve performance and cost efficiency.

Streaming & Real-Time Processing

• Build and maintain high-throughput streaming data solutions using Pub/Sub, Kafka, Dataflow, Spark Structured Streaming, Apache Beam, and Apache NiFi.

• Support near real-time data processing requirements across business and operational domains.

Data Governance & Compliance

• Implement enterprise-grade governance solutions using tools such as Collibra, Privacera, Unity Catalog, and related governance frameworks.

• Ensure compliance with HIPAA, GDPR, PCI-DSS, and corporate security standards through robust access controls, lineage tracking, auditing, encryption, and data protection practices.

• Support metadata management, data quality monitoring, and data lineage initiatives across the enterprise.

Performance, Reliability & Operations

• Monitor, troubleshoot, and optimize enterprise data pipelines and cloud infrastructure.

• Implement observability using Cloud Monitoring, logging, alerting, and automated recovery solutions.

• Create data flow documentation, technical design artifacts, production support procedures, and root cause analyses (RCA).

Collaboration & Delivery

• Collaborate with business stakeholders, architects, product managers, and engineering teams to deliver scalable data solutions.

• Participate in Agile and SAFe delivery frameworks across onshore and offshore teams.

• Communicate complex technical concepts effectively to both technical and non-technical stakeholders.

Core Technical Skills

Cloud Platforms

• Google Cloud Platform (GCP)

• BigQuery

• Dataflow

• Pub/Sub

• Cloud Composer

• Cloud Functions

• Cloud Run

• Cloud Storage

• Cloud Monitoring

Data Engineering & Processing

• Python

• SQL

• PySpark