Senior HPC Systems Administrator

About the Opportunity

Job Summary:

The Research Computing (RC) team at Northeastern University seeks a motivated, self-starting individual to be a member of our dynamic team as a Senior HPC Systems Administrator. The successful candidate will help lead the administration, development, and expansion of  the on-premise HPC resources, research computing  environment (VMs, storage, software, etc.),  and potentially leveraging cloud-computing resources.

As the Senior HPC Systems Administrator, your primary objective will be to setup, monitor, operate, and support high-performance clusters and solutions for faculty research and teaching. You will assist research groups in taking full advantage of Northeastern’s HPC cluster installed at the Massachusetts Green High Performance Computing Center. 

In addition, you will help ensure researchers needs are met, and develop new and novel solutions to support research and teaching, as they pertain to RC systems and solutions.

Minimum Qualifications:

• A bachelor’s degree in engineering or computational sciences with strong Linux systems experience, or 5 years of professional experience in Linux system administration.
• Minimum 5 years Unix/Linux system administration.
• Minimum 5 years managing an HPC environment.
• Strong work ethic and ability to communicate complex ideas to stakeholders with diverse background (both oral and written communication).
• Demonstrated experience in scripting and coding: bash, python, ruby, C/C++.
• Experience with multiple programming languages (including Fortran, C, C++, Perl, Rust).
• Experience with large scale parallel filesystems (GPFS, Lustre, etc).
• Experience configuring and managing batch schedulers, specifically Slurm.
• Experience with common HPC software stacks and management tools (e.g., CMake, Spack) 
•  Experience with Containers (e.g., docker, singularity, podman)
• Knowledge of standard networking, security practices, and user management in a large computing environment.
• Experience with identifying and debugging system problems and performance issues at scale, esp. for GPU-accelerated systems.

Preferred Qualifications

• 7+ years of experience as HPC systems administrator at a technology company or research institution.
• Experience with managining and operating HPC clusters at other leading institutions, DOE/DOD/NSF-funded HPC centers, or HPC-focused cloud computing entities. 
• Experience implementing scientific software on GPU-based HPC systems, esp. with AI/ML workload heavy systems.
• Regular use of identity management software like LDAP/Active Directory.
• Knowledge of virtualization environments (Xen, Eucalyptus, VMWare), cloud computing environments (AWS, Azure, GCP, OpenNebula), and container environments (Docker, Singularity, Shifter).
• Experience using configuration management tools such as Puppet, Chef, Ansible, or Salt.
• Knowledge of database concepts and basic administration for MySQL, Postgres, MongoDB, Microsoft SQL, Oracle.
• Knowledge of a variety of filesystems and storage deployments: NFS, SMB, Lustre, GPFS, Ceph, S3, etc.
• Strong track-record in working with academic researchers funded by federal sponsors and adapting to their evolving needs. 

Key Responsibilities & Accountabilities:

• Manage, monitor, and maintain HPC systems to meet the needs of diverse set of Northeastern researchers and students.
• Quickly identify operational issues, debug performance problems, and optimize HPC systems.
• Build system software infrastructure, in collaboration with other RC staff, to move jobs among on-premise Northeastern clusters,  offload to non-Northeastern resources (e.g., HPC clusters at other academic institutions, national labs, or cloud computing).
• Interact with and support faculty with  diverse computational needs and backgrounds
• Develop and build novel systems/frameworks and workflows to meet researcher’s changing needs.
• Communicate progress and participate in reviews with technical staff and senior management.
• Anticipate and communicate computational needs and challenges proactively to the Northeastern research computing leadership. 
• Document and track progress of multiple ongoing needs/open issues.
• Effectively participate in external collaborations (locally/regionally) and funding opportunities.
• Work with RC staff to develop and maintain technical documentation, both internal (admin facing) and external (user facing) documentation.
• Work with the research computing leadership on short and long term strategies for expanding RC support and solutions.
• Att