Sr. Data Engineer – EHR Data Migration

We are eClinicalWorks. We are a privately held leader in healthcare IT, providing comprehensive, cloud based EHR/PRM solutions to medical professionals worldwide to improve workflows and reduce the risk of physician burnout. We care. We are committed to positive change. And that’s where you come in. Do you value creativity and innovation? Great, so do we. At eClinicalWorks, we share a passion for improving healthcare through dedication, education, and teamwork. Everyone has that one thing they’re really good at. We value your talent and want you to join our fast-paced, fun, and culturally diverse environment. Ready to make a difference? Apply today.

 

 

Summary
We’re hiring a hands-on Sr. Data Engineer to lead large-scale EHR → eClinicalWorks (eCW) data migrations. You’ll combine advanced SQL, Python + AI engineering, and Azure-native services to deliver ingestion, mapping, transformation, validation, and document conversion at scale. You’ll architect resilient ADF pipelines, Azure Functions, and AKS workloads, and apply LLM-powered data processing to accelerate mapping and improve quality.
Key Responsibilities
Client Engagement & Requirements  

• Partner with customers to capture migration scope, workflows, and data quality expectations; lead technical discoveries, schema deep-dives, and gap analysis.
• Present migration strategies, risks, and timelines; set and manage SLAs/SLOs for each phase.

Data Analysis, Mapping & Loading (EHR → eCW)  

• Analyze complex schemas across SQL Server/MySQL/Oracle/APIs, design staging and canonical models.
• Build deterministic and AI-assisted mappings.

AI Engineering & Automation  

• Use Python with Copilot, Cursor, Azure OpenAI/Claude or Codex to accelerate coding, tests, and playbooks.
• Build LLM/RAG services for mapping suggestions, anomaly detection, context- aware transformations, design prompts, guardrails, and evaluation harnesses.
• Implement retrieval pipelines (embeddings, vector search) and maintain prompt/test corpora.

Azure Cloud Data Platform  

• Architect and operate ADF orchestrations; implement Azure Functions-based
  microservices; manage Blob/ADLS Gen2, Azure SQL/MI, Key Vault, and
  VNETs/private endpoints.
• Enable observability using Azure Monitor/Log Analytics/App Insights; publish run
  health, lineage, and cost dashboards.
• Establish CI/CD with Azure DevOps/GitHub Actions and IaC (Bicep/Terraform) enforce environments, approvals, and policy-as-code.

Document Migration & Unstructured Data Processing  

• Convert and migrate millions of documents (DOC/DOCX/RTF/TXT/IMG) into PDF.
• Run conversion at scale on AKS with containers/Helm, HPA/KEDA autoscaling, and
  Service Bus/Event Hubs for queue-driven orchestration.
• Ensure output quality (completeness, orientation, readability), retries/backoff,
  idempotency, and poison-message handling; track throughput/error budgets.

Quality, Testing & Troubleshooting  

• Implement data quality checks (Great Expectations/dbt tests) and automated
  reconciliation; own root-cause analysis for schema, mapping, and pipeline issues.
• Maintain playbooks/runbooks; drive continuous improvement and pattern reuse.

Security & Compliance  

• Ensure HIPAA compliance, PHI protection, encryption, secure data handling, and
  proper access controls.
• Follow internal security and auditing requirements when working with clinical
  datasets.

Collaboration & Leadership  

• Partner with Clinical Informatics, Product, Customer Success, and Engineering to
  deliver smooth migrations.
• Contribute to internal frameworks, tools, process improvements, and
  documentation.
• Mentor engineers; champion code reviews, testing, and an automation/AI-first
  engineering culture.

Required Skills & Qualifications  

• 3+ years of data engineering experience.
• Expert SQL (T‑SQL, window functions, indexing, query plan analysis, stored procedures/functions) on large datasets.
• Adv