Loading job details…
Pay
Pay not listed
The employer did not disclose pay.
NVIDIA is looking for a Senior Network and Fabric Operations Solutions Architect to join its NVIDIA Infrastructure Specialist Team. Academic and commercial organizations around the world are using NVIDIA products to redefine deep learning and data analytics, and to power next-generation data centres. Join the team building and advising on the networks behind many of the largest and fastest AI/HPC systems in the world!
We are looking for someone who combines deep networking expertise with strong consulting and communication skills. This role will engage directly with customers, partners and cross-functional teams to assess, architect and operate the compute, storage and scale-up fabrics that underpin large GPU estates—serving as both a trusted advisor and a hands-on technical leader. You will own the fabric side of NVIDIA’s Cloud Partner (NCP) operating model across the full Day 1 to Day 2 lifecycle: from InfiniBand/UFM and Ethernet validation at handover, through NVLink partition operations, to a production-stable fabric running at maximum goodput. NCP estates are open-source-first and heterogeneous—Cumulus Linux, SONiC, Spectrum-X, DOCA/DPU services, upstream Kubernetes and Prometheus/Grafana—so this role is deliberately tool-agnostic: you will meet each partner on the fabric they actually run rather than on a single proprietary product.
What You’ll Be Doing:
Own Day 2 fabric operations across NCP fleets: NVLink/NVSwitch partition management (NMX-C / NMX-M) on GB200/GB300 NVL72-class systems, maintenance-partition isolation, and safe partition-change workflows for multi-tenant estates.
Own the switch software and firmware lifecycle—Cumulus Linux, SONiC and switch-OS upgrades, CPLD and field-notice rollout campaigns—with staged, tested upgrade plans that keep partner fabrics off stale releases without disrupting production workloads.
Support Day 1 fabric validation and acceptance: InfiniBand and UFM bring-up, Spectrum-X/RoCE Ethernet configuration, cabling and link-health verification, routing and congestion-control validation, and multi-day, multi-rack burn-in against agreed MTBI and goodput targets.
Minimise the time from cluster handover to first production workload, removing duplicated fabric validation across hardware bring-up, managed-service intake and the partner’s own network operations teams.
Drive fabric reliability at fleet scale: telemetry for link flaps, retransmits and congestion, fault detection through to remediation, root-cause analysis of network-induced job failures, and measurable improvement of MTBI and job goodput.
Provide consultative guidance and hands-on troubleshooting across the fabric stack—NICs and DPUs, switch OS, routing and congestion control, host networking and Kubernetes network integration—and act as the technical leader for assigned accounts, running structured knowledge transfer and producing runbooks so partner teams can operate advanced fabric configurations independently.
What We Need to See:
BS/MS/PhD in Computer Science, Electrical/Computer Engineering, Physics, Mathematics, or related fields, or equivalent experience.
5+ years of professional experience in data centre networking, fabric engineering or large-scale network operations roles.
Fabric & Data Centre Networking: Deep understanding of data centre network architectures and RDMA fabrics—InfiniBand and RoCE/Ethernet—including topology and routing design (fat-tree, rail-optimised), congestion control, lossless/QoS configuration, and troubleshooting across NICs, switches and high-speed interconnects.
NVIDIA Fabric Technologies: Hands-on experience with InfiniBand and UFM, Spectrum-X Ethernet, Cumulus Linux and/or SONiC, ConnectX and BlueField NICs/DPUs, and NVLink/NVSwitch systems—ideally including NMX-C / NMX-M partition operations on NVL72-class platforms.
Linux & Switch Platforms: Deep knowledge of Linux (RedHat, Ubuntu) and switch operating systems, network OS internals, host networking and OS-level security, together with an understanding of the storage and compute traffic patterns of HPC/AI clusters.
Automation, GitOps & Observability: Proficiency in Python and Bash scripting, configuration management and Infrastructure-as-Code tools (e.g. Ansible, Terraform), GitOps-based network configuration and firmware/upgrade management for large fleets, and observability stacks (Grafana, Loki, Prometheus) applied to fabric telemetry.
Fleet Reliability & Customer Engagement: Demonstrated ability to measure and improve MTBI and job goodput on large GPU clusters—fault detection, drain and remediation workflows, SLO/error-budget definition and post-incident review—combined with a strong consultative background leading architectural reviews and presenting to executive stakeholders.
Ways to Stand Out from the Crowd:
Experience operating Kubernetes networking in GPU clusters—CNI plugins, multi-network attachment, SR-IOV and RDMA device plugins—and with the NVIDIA Network and GPU Operators for automated resource lifecycle management.
Expertise with DOCA and DPU infrastructure services (e.g. DOCA DPF) for offloaded networking, security and storage services.
Familiarity with fabric and GPU health telemetry—DCGM and XID diagnostics, link-level and switch counters, node-level health agents, and fleet-wide reliability intelligence.
Knowledge of large-scale training traffic behaviour: collective communication (NCCL) tuning, rail alignment, and diagnosing network-bound performance regressions in distributed training jobs.
Experience delivering multi-tenant network isolation for GPU-as-a-service estates—VRF/VLAN/EVPN segmentation, tenant partitioning and secure fabric handover between tenants.
Know someone for this role?
Share and earn credits toward your CV and other Handmework features.
Sign in to get your linkLocation
Pay not listed
Senior Cloud Infrastructure and Network Operations Solutions Architect