Senior Engineer I - Observability - Pipelines & Visualization

CoreWeave is the AI Hyperscaler™, delivering a cloud platform of cutting edge services powering the next wave of AI. Our technology provides enterprises and leading AI labs with the most performant, efficient and resilient solutions for accelerated computing. Since 2017, CoreWeave has operated a growing footprint of data centers covering every region of the US and across Europe. CoreWeave was ranked as one of the TIME100 most influential companies of 2024.

As the leader in the industry, we thrive in an environment where adaptability and resilience are key. Our culture offers career-defining opportunities for those who excel amid change and challenge. If you’re someone who thrives in a dynamic environment, enjoys solving complex problems, and is eager to make a significant impact, CoreWeave is the place for you. Join us, and be part of a team solving some of the most exciting challenges in the industry.  

CoreWeave powers the creation and delivery of the intelligence that drives innovation. 

About the role:

The Observability Pipelines and Visualization team manages the doors – both entry and exit – to the telemetry databases that are managed by the Observability Department. The Pipelines and Visualization team ensures that CoreWeave engineers can effectively produce and consume telemetry for their services. This involves building pipes to ingest and route data in predictable, composable ways as well as visualizing that data after the fact.

We are seeking a Senior Engineer who has experience across observability stacks, understands the unique problems that come with moving cloud-level volumes of telemetry data at scale, and is excited at the prospect of ensuring both CoreWeavers as well as CoreWeave customers have eyes into the same underlying telemetry data to run their services as efficiently as possible. Knowledge of and experience running Grafana at scale, serving a wide variety of customers with varying degrees of access, is a strong plus. As you work closely with a team of 3-6 engineers focused on pipelines and visualization, you will have the opportunity to:

• Collaborate with Logging, Tracing, and Metrics SMEs at a cloud scale.
• Ensure CoreWeavers from teams across the organization have easy access to the telemetry they need for compliance, security, billing, and myriad other specialized use cases.
• Build low-touch, automated tooling that enables CoreWeave engineers to clearly articulate which pieces of telemetry, visualizations, and dashboards should be presented to customers.
• Develop and enforce best practices on alerting and tracking the health of telemetry ETL pipelines.
• Improve the performance, security, reliability, and scalability of observability services while participating in the team’s on-call rotation.
• Enable and evangelize the best practices around alerting. Collaborate with teams to establish observability standards.
• Work with telemetry data at enormous scale, ingesting data from industry-leading GPU clusters.
• Grow, change, invest in your teammates, be invested-in, share your ideas, listen to others, be curious, have fun, and, above all, be yourself.

Wondering if you’re a good fit? We believe in investing in our people, and value candidates who can bring their own diversified experiences to our teams – even if you aren't a 100% skill or experience match. Here are some qualities we’ve found compatible with our team. If a portion of this resonates with you, we’d love to talk. 

• You have six or more years of experience in a software or infrastructure engineering industry.
• You enjoy helping your colleagues achieve more with less effort.
• You are customer obsessed, ecstatic to provide infrastructure as a service, and default to adopting a product lens when evaluating platform scale problems.
• You have experience with observability platforms in production and at scale, and are versed in reliability engineering concepts such as the different types of testing, progressive deployments, error budgets, the role observability plays, and fault-tolerant design.
• You have experience using Kubernetes, a conceptual understanding of its major components, and/or have operated Kubernetes clusters at scale for both event-driven and stateful orchestration.
• You have experience using Temporal as an event-based workflow management tool.
• You’re familiar with various logging and metrics systems like ClickHouse, Elastic, Loki, Victoria Metrics, Prometheus, Thanos and Grafana. You have experience with designing and operating these systems at scale.
• You are familiar with PromQL, any other querying language and enjoy understanding the data model for observability systems. 
• You are excited at the prospect of zero-downtime, transparent migratio