Senior Data Engineer

Required Experience and Skills:

We are seeking a highly skilled and motivated Data Engineer with expertise in SQL development, CI/CD, and custom development for database and ETL solutions. This role is ideal for candidates with hands-on experience in DevOps practices, Flyway-based database versioning, and data engineering within Oil & Gas operations (upstream, midstream, and downstream).

Minimum 9+ years experience

• TSQL
• SQL Server
• SSIS
• Oil and Gas Industry experience

 Nice to have:

• Database Administration
• Python

Key Responsibilities:

Database Architecture & Design

• Design scalable and efficient SQL database architectures tailored to upstream, midstream, and downstream oil and gas operations.
• Develop and maintain logical and physical data models to support data warehousing and business reporting needs.

SQL Development & Optimization

• Write complex T-SQL code including stored procedures, views, functions, and triggers.
• Optimize large-volume queries for production logs, telemetry, and field operations to improve performance and reliability.

Data Integration & ETL Pipelines

• Build and manage robust ETL/ELT pipelines for structured and unstructured data from sources such as SCADA, historians, ERP systems (e.g., SAP), CMMS, drilling reports, and IoT devices.
• Ensure data validation, integrity, and transformation align with business and regulatory standards.

CI/CD & DevOps Integration

• Develop and manage Flyway-based SQL migration scripts to version control database changes.
• Integrate database deployment automation into CI/CD pipelines using tools like Azure DevOps or GitHub Actions.
• Ensure consistent deployment processes across development, test, staging, and production environments.

Custom Development & Platform Support

• Leverage Visual Studio and .NET tools to build supporting utilities or integrate with existing applications.
• Provide performant SQL backends for reporting platforms such as Power BI, Spotfire, or Tableau.

Collaboration & Support

• Collaborate with data analysts, architects, and business stakeholders to translate operational requirements into scalable data solutions.
• Support application teams and business users during feature rollouts, deployments, and upgrades.

Documentation & Best Practices

• Document database structures, processes, and technical design decisions.
• Enforce and advocate for SQL development best practices, code reviews, and maintainability standards.

Required Qualifications:

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related field.
• 8+ years of experience in SQL development and database architecture.
• Strong command of Microsoft SQL Server, SSIS, and T-SQL.
• Experience in Oil & Gas domains (e.g., well lifecycle, production volumes, pipeline operations).
• Hands-on experience with Flyway for database versioning and change management.
• Proficiency in Visual Studio and custom software development tools.
• Familiarity with data modeling tools (ER/Studio, Erwin) and version control systems (Git).

Preferred Qualifications:

• Experience with cloud data platforms (Azure SQL, AWS RDS, Snowflake).
• Understanding of WITSML standards and Oil & Gas-specific data structures.
• Exposure to Python, PowerShell, or other scripting languages for data automation.
• Experience with data governance, MDM, and data quality frameworks.

Soft Skills:

• Analytical mindset with strong problem-solving capabilities for data-intensive operations.
• Excellent communication skills, with the ability to explain technical solutions to business users.
• Team-oriented and collaborative, working efficiently with cross-functional groups.
• Detail-oriented with a focus on data accuracy, validation, and compliance.
• Self-driven and accountable, capable of managing workload independently in fast-paced environments.
• Flexible and adaptable, able to pick up new tools and domains quickly as project needs evolve.
• Proactive thinker who anticipates potential issues and proposes effective solutions.
• Strong organizational and time management skills with the ability to juggle multiple priorities effectively.

Datavail is a leading provider of data management, applicatio