GenAI DevOps Engineer

Job Description

Position: GenAI DevOps Engineer 

About the Role

As a GenAI DevOps Engineer on our AI Engineering team, you’ll be responsible for building, deploying, and maintaining the infrastructure and automation that powers our generative AI products. You’ll work closely with software engineers and data scientists to ensure scalable, secure, and reliable operations across cloud environments, with a strong focus on CI/CD, observability, and orchestration.

Core Responsibilities

CI/CD & Workflow Automation

• Design and maintain automated CI/CD pipelines for GenAI services using GitHub Actions, Azure DevOps, or Jenkins.
• Implement DevOps best practices including code linting, testing, and validation gates to ensure high-quality and secure deployments.
• Automate build, test, and release processes for GenAI components such as prompt orchestration scripts, vector retrieval services, and APIs.
• Integrate infrastructure updates into CI/CD workflows using Terraform or CloudFormation for version-controlled provisioning.

Deployment & Runtime Management

• Containerize GenAI services using Docker and deploy to Kubernetes clusters (AKS/EKS/GKE) for scalable and resilient operations.
• Deploy serverless applications using platforms like Cloud Run, AWS Lambda, or Azure Functions to optimize cost and performance.
• Implement safe rollout strategies (canary, blue/green deployments) to minimize risk during updates.
• Manage runtime environments and ensure high availability and scalability of GenAI endpoints.

Monitoring & Alerting

• Instrument services with key metrics (latency, throughput, error rates) and structured logs.
• Build dashboards and alerts using Prometheus, Grafana, Datadog, or cloud-native tools to monitor performance and detect anomalies.
• Support drift detection and regression monitoring for GenAI workflows.

Cloud Infrastructure & Automation

• Operate and optimize compute resources on Azure, AWS, or GCP, including managed services like Databricks, SageMaker, or Vertex AI.
• Maintain Infrastructure-as-Code (IaC) for provisioning clusters, storage, and networking components.
• Ensure secure and compliant infrastructure with secrets management, encryption, and access controls.

GenAI Workflow Orchestration & Retrieval

• Use orchestration frameworks (e.g., LangGraph, LangChain, Langfuse) to automate GenAI workflows and prompt chaining.
• Support embedding-based retrieval pipelines, including vector index maintenance and refresh processes.

Collaboration & Documentation

• Collaborate with cross-functional teams to integrate GenAI capabilities into production systems.
• Produce clear documentation including runbooks, architecture diagrams, and operational guides for on-call support and incident response.

Growth Areas & Nice-to-Have Skills

• Security & Compliance: API authentication (OAuth/JWT), secrets management (Key Vault, AWS KMS), and data encryption.
• Testing & Validation: Data-quality checks, adversarial testing, and automated validation gates.
• Scalability & Cost Optimization: Load testing (Locust, JMeter), spot-instance usage, and caching strategies (Redis).
• Reliability Engineering: On-call rotations, post-mortems, SLOs, and chaos testing fundamentals.
• Experimentation Lifecycle: Tracking experiments, configuration sweeps, and supporting A/B testing.
• Tooling Flexibility: Familiarity with alternative DevOps frameworks (Kubeflow, TFX) and observability stacks (OpenTelemetry).

Required Qualifications

• 5 years in DevOps roles, with at least 1 year supporting GenAI or AI-driven solutions.
• Hands-on experience with one major cloud provider (Azure, AWS, or GCP).
• Strong proficiency in Docker, Kubernetes, and serverless deployment strategies.
• Proven ability to build and maintain CI/CD pipelines and Infrastructure-as-Code.
• Experience with monitoring and alerting stacks (Prometheus/Grafana, Datadog,