Data Engineer – External Data Integrations

<p></p> <p>OTR Solutions is an innovator in the transportation industry providing a suite of factoring, fuel, and business management solutions. We help new and established companies get fast access to the funds they need for daily operations. As a Private Equity-backed FinTech company, we are scaling our technology and AI capabilities to deliver next-generation automation and document intelligence solutions for the freight and financial services industries.</p> <p><strong><em>OTR has been recognized as a “Top Workplace” by the Atlanta Journal-Constitution since 2016!</em></strong></p> <p>As a <strong>Data Engineer – External Data Integrations</strong>, you will design, build, and maintain scalable batch-oriented data pipelines that enable secure and reliable data exchange between OTR and external systems. You will play a key role in developing integrations with third-party platforms, supporting internal data operations, and ensuring data quality and observability across mission-critical workflows.</p> <p>This role combines data engineering and integration development, requiring close collaboration with Engineering, Business Intelligence, and business stakeholders to deliver scalable solutions that support automation, reporting, and business growth initiatives.</p> <p><strong>Please note: We do not sponsor work-related visas.**</strong></p> <p><strong>Responsibilities:</strong></p> <ul> <li>Design and build scalable batch ingestion pipelines from external systems, including APIs, SFTP sources, and databases</li> <li>Develop outbound data pipelines to securely publish structured data to external partners and third-party platforms</li> <li>Build, maintain, and optimize Azure Synapse pipelines and notebook-based transformations using PySpark and Spark SQL</li> <li>Orchestrate workflows using Azure Logic Apps, including scheduling, retries, monitoring, and alerting</li> <li>Design and implement data quality validation, observability, and monitoring frameworks to ensure pipeline reliability and SLA adherence</li> <li>Define and maintain schema standards, versioning strategies, and partner-facing data contracts</li> <li>Build secure integrations with third-party systems using modern authentication methods including OAuth and Azure Key Vault</li> <li>Partner with Engineering, Product, and Business Intelligence teams to define integration requirements and scalable data exchange patterns</li> <li>Troubleshoot complex integration issues and continuously improve performance, reliability, and maintainability of data pipelines</li> <li>Identify opportunities to automate and streamline data workflows across systems</li> </ul> <p><strong>What we look for:</strong></p>