Senior HPC Linux Clusters Engineer (Hybrid Eligible)

Requisition Id 10657 

Overview: 

The National Center for Computational Sciences (NCCS) at Oak Ridge National Lab (ORNL), which hosts several of the world’s most powerful computer systems, is seeking highly qualified individuals to play a key role in improving the security, performance, and reliability of the NCCS computing environments.  

 

In this role, you will work within the HPC Clusters Group inside of the NCCS Systems Section to support numerous activities of the center.

 

The HPC Clusters Group administers and supports the division’s HPC computing infrastructure, which includes system installation, deployment, acceptance, performance testing, upgrades, problem diagnosis, and troubleshooting.

 

The Systems Section within National Center for Computational Sciences Division (NCCS).  The HPC Systems Section administers and supports the division’s computing, networking, and storage systems.

 

The NCCS provides state-of-the-art computational and data science infrastructure, coupled with dedicated technical and scientific professionals, to accelerate scientific discovery and engineering advances across a broad range of disciplines. NCCS hosts the Oak Ridge Leadership Computing Facility, one of DOE’s National User Facilities.

 

Major Duties/Responsibilities: 

• Work with the team to define and implement best practices and standards within the organization
• Automate systems administration tasks utilizing open-source configuration management tools
• Assist with designing, building, and maintaining HPC Linux systems
• Tune operating systems and applications to increase performance and reliability of services
• Diagnose system operational problems quickly and effectively
• Document system administration procedures for routine and complex tasks
• Participate in an on-call rotation providing 24-hour, 7-day support and off-hours maintenance windows
• Coordinate with vendors to resolve hardware and software problems
• Deliver ORNL’s mission by aligning behaviors, priorities, and interactions with our core values of Impact, Integrity, Teamwork, Safety, and Service. Promote diversity, equity, inclusion, and accessibility by fostering a respectful workplace – in how we treat one another, work together, and measure success.

 

Basic Qualifications:

• Bachelor’s Degree in computer science or closely related field and a minimum of 8 years of experience in Linux/UNIX Systems Administration.
• An equivalent combination of education and experience will be considered.

 

Preferred Qualifications:

• Excellent interpersonal/communication skills, and the ability to work as part of a team
• Strong working knowledge of Unix system fundamentals and common network protocols.  Knowledge or experience working with Infiniband and parallel filesystems (Lustre, GPFS) is a plus.
• Experience managing Linux/UNIX operating systems in a heterogeneous environment.
• Solid understanding of networked computing environment concepts
• Ability to develop and maintain programs and scripts that aid in the operation and automation of administrative tasks using various shell and scripting languages
• Ability to identify requirements and to define, plan, and implement requisite solutions
• Ability to plan organize, prioritize tasks, and complete assigned projects with minimal supervision
• Experience with continuous integration and continuous deployment software methodologies and how they apply to systems engineering
• An understanding of site reliability engineering fundamentals
• Familiarity with code review processes / best practices and familiarity with tools like Git, GitHub and GitLab
• Experience using tools such as Grafana and Nagios to monitor systems, metrics, and create dashboards
• Experience designing and implement highly available systems/services
• Experience participating in an open-source community with patches accepted upstream
• Experience with developing systems tools, preferably using Python, Golang, Ruby, or BASH
• Experience deploying and maintaining automated configuration management software such as: Puppet, Ansible
• Experience building custom packages for RHEL/Rocky using RPM
• Familiarity or direct experience with various container strategies and orchestration tools such as Docker, PodMan, Singularity, OpenShift/Kubernetes
• Familiarity or direct experience installing and utilizing job management and benchmarking tools such as SLURM, STREAM/OSU, MPI benchmarks, IOR, etc.
• Experience implementing systems level security technologies like SELinux and following security best-practices

 

Special Requirements:

• Visa Sponsorship: Visa s