About the Role:

Grade Level (for internal use):

Job Description 

S&P Dow Jones Indices is seeking a Sr. Lead Data Engineer with expertise in Databricks/AWS EMR and distributed big data processing technologies to be a key player in the implementation and support of data platforms for S&P Dow Jones Indices. This role requires a seasoned technologist who contributes to application development and maintenance. The candidate should actively evaluate new products and technologies to build solutions that streamline business operations. The candidate must be delivery-focused with solid financial applications experience. The candidate will assist in day-to-day support and operations functions, design, development, and unit testing. 

Responsibilities and Impact: 

• Lead the design and implementation of distributed data processing workloads using Python, including data access from relational databases and cloud storage technologies. 

• Ensure performance, security, and reliability; coordinate with application teams to define database design. 

• Collaborate with cross-functional teams to support data-driven initiatives. 

• Mentor junior team members and promote best practices. 

• Oversee maintenance and troubleshooting of distributed data processing systems. 

• Drive innovation by evaluating and integrating new technologies. 

• Produce system design documents and participate in technical walkthroughs. 

• Perform application and system performance tuning and troubleshoot performance issues. 

• Effectively interact with global customers, business users, and IT staff. 

Compensation/Benefits Information (US Applicants Only):

S&P Global states that the anticipated base salary range for this position is $127,000 - $185,000. The final base salary for this role will be based on the individual’s geographic location, experience level, skill set, training, licenses, and certifications.

In addition to base compensation, this role is eligible for an annual incentive plan.  This role is eligible to receive additional S&P Global benefits. For more information on the benefits that we provide to our employees, please click here.

What We’re Looking For: 

Basic Required Qualifications: 

• Bachelor’s degree in Computer Science, Information Systems, Engineering, or equivalent work experience. 

• 10+ years of IT experience in application support or development. 

• Design & build ingestion pipelines using ETL/ELT and schema-on-read in data lake technologies. 

• Experience in designing transactional systems, data warehouses, data lakes, and data integrations within a big data ecosystem, particularly leveraging cloud technologies.

• Provide technical expertise in the design and implementation of data ingestion pipelines with modern AWS cloud and other technologies such as S3, Hive, Databricks, Scala, Python, and large-scale data analytics tools. 

• Strong experience in managing distributed data processing workloads, including configurations. 

• Experience in working with multi-threaded, high-performance, low-latency messaging systems. 

• Experience in AWS cloud-based technologies. 

• Experience using system tools, source control systems, utilities, and third-party products. 

• Excellent com