Lead Software Engineer – Databricks / PySpark / AWS

Job Summary:

We have an exciting and rewarding opportunity for you to take your software engineering career to the next level. 

As a Lead Software Engineer at JPMorgan Chase within the Corporate Technology, you play a crucial role in an agile team dedicated to enhancing, building, and delivering trusted, market-leading technology products in a secure, stable, and scalable manner. As a key technical contributor, you are tasked with implementing critical technology solutions across multiple technical domains, supporting various business functions to achieve the firm's business objectives.

Job Responsibilities:

• Design solutions at the appropriate level of detail and drive consensus among peers as needed.
• Champion software engineering best practices within the team.
• Collaborate with software engineers and cross-functional teams to design and implement deployment strategies using AWS Cloud and Databricks pipelines.
• Lead the design, development, testing, and implementation of application solutions.
• Partner with technical experts, stakeholders, and team members to resolve complex technical challenges.
• Proactively address issues to support leadership objectives and prevent customer impact.
• Design, develop, and maintain robust data pipelines for ingesting, processing, and storing large volumes of data from diverse sources.
• Implement ETL (Extract, Transform, Load) processes to ensure data quality and integrity using tools such as Apache Spark and PySpark.
• Monitor and optimize the performance of data systems and pipelines.
• Apply best practices for data storage, retrieval, and processing.
• Maintain comprehensive documentation for data systems, processes, and workflows.

 

Required Qualifications, Capabilities, and Skills:

• Formal training or certification on software engineering concepts and 5+ years applied experience.
• Proficiency in programming languages such as Python and PySpark.
• Experience designing and implementing data pipelines in cloud environments.
• Strong background in design, architecture, and development using AWS Services, Databricks, Spark, Snowflake, and related technologies.
• Experience with CI/CD tools such as Jenkins, GitLab, or Terraform.
• Familiarity with containerization and orchestration technologies including ECS, Kubernetes, and Docker.
• Ability to troubleshoot issues related to Big Data and Cloud technologies.

 

Preferred Qualifications, Capabilities, and Skills:

• 5+ years of experience leading and developing data solutions in AWS Cloud.
• 10+ years of experience building, implementing, and managing data pipelines using Databricks on Spark or similar cloud technologies

 

JPMorganChase, one of the oldest financial institutions, offers innovative financial solutions to millions of consumers, small businesses and many of the world’s most prominent corporate, institutional and government clients under the J.P. Morgan and Chase brands. Our history spans over 200 years and today we are a leader in investment banking, consumer and small business banking, commercial banking, financial transaction processing and asset management.

We offer a competitive total rewards package including base salary determined based on the role, experience, skill set and location. Those in eligible roles may receive commission-based pay and/or discretionary incentive compensation, paid in the form of cash and/or forfeitable equity, awarded in recognition of individual achievements and contributions. We also offer a range of benefits and programs to meet employee needs, based on eligibility. These benefits include comprehensive health care coverage, on-site health and wellness centers, a retirement savings plan, backup childcare, tuition reimbursement, mental health support, financial coaching and more. Additional details about total compensation and benefits will be provided during the hiring process. 

We recognize that our people are our strength and the diverse talents they bring