Data Engineer (SR-5253733)

Remote – United States

The scope of the proposed services will include the following:

• Assess feasibility and technical requirements for LINKS → DataLake integration.

• Collaborate with OPH Immunization Program, OPH Bureau of Health Informatics and STChealth on data specifications and recurring ingestion pipelines.

• Build and optimize ETL workflows for LINKS and complementary datasets (Vital Records, labs, registries).

• Design scalable data workflows to improve data quality, integrity, and identity resolution.

• Implement data governance, observability, and lineage tracking across all pipelines.

• Mentor engineers, support testing, and enforce best practices in orchestration and architecture.

• Document and communicate technical solutions to technical and non-technical stakeholders.

Expertise and/or relevant experience in the following areas are mandatory:

• 3 years of experience in data engineering and/or data architecture

• 2 years of experience with Python for ETL and automation (pandas, requests, API integration).

• 2 years hands-on experience with SQL queries, stored procedures, performance tuning (preferable Oracle, SQL Server, MySQL)

• 1 year experience with ETL orchestration tools (Prefect, Airflow or equivalent).

• 1 year experience with cloud platforms (Azure, AWS, or GCP), including data onboarding/migration.

• 1 year exposure to data lake / medallion architecture (bronze, silver, gold)

• 2 years of experience providing written documentation and verbal communication for cross-functional collaboration.

Expertise and/or relevant experience in the following areas are desirable but not mandatory:

• 5+ years of experience in data engineering roles

• Experience integrating or developing REST/JSON or XML APIs

• Familiarity with CI/CD pipelines (GitHub Actions, Azure DevOps, etc.).

• Exposure to Infrastructure as Code experience (Terraform, CloudFormation).

• Experience with data governance and metadata tools (Atlan, OpenMetadata, Collibra).

• Public health/healthcare dataset or similar experience, including PHI/PII handling.

• Familiarity with SAS and R workflows to support epidemiologists and analysts.

• Experience with additional SQL platforms (Postgres, Snowflake, Redshift, BigQuery).

• Familiarity with data quality frameworks (Great Expectations, Deequ).

• Experience with real-time/streaming tools (Kafka, Spark Streaming).

• Familiarity with big data frameworks for large-scale transformations (Spark, Hadoop).

• Knowledge of data security and compliance frameworks (HIPAA, SOC 2, etc.).

• Agile/SCRUM team experience.

Note: Expected Start/End Dates: 12/15/2025 - 11/30/2026. This position will not be extended beyond June 30, 2029.

Hours worked should be within the State’s core hours of 7:00 AM to 6:00 PM CT. Contractor personnel may be required to work on State holidays, weekends and hours outside of the State’s core business hours in order to meet critical deadlines.