Design and maintain software solutions for real-time monitoring of server infrastructure and GPU health.
Develop and refine diagnostic suites to identify pre-failure indicators and performance bottlenecks in GPU clusters.
Build and optimize monitoring pipelines, ensuring alerts are actionable and noise is minimized.
Create automated recovery workflows to manage hardware faults without manual intervention.
Partner with hardware vendors and site reliability teams to integrate low-level hardware telemetry into high-level monitoring platforms.
Minimum qualifications:
Bachelor’s degree or equivalent practical experience.
2 years of experience with software development in one or more
programming languages (such as Go, C++, Python) with systems-level integration and distributed systems.
Experience with monitoring and observability stacks like Prometheus, Grafana, Elasticsearch, Logstash, Kibana (ELK), or VictoriaMetrics.
Experience with Linux internals, server hardware components, and out-of-band management (e.g., Intelligent Platform Management Interface (IPMI), Redfish).
Preferred qualifications:
3 years of experience with data structures and algorithms in an academic or industry settings.
Experience with GPU architecture and management libraries (e.g., DCGM, or ROCm).
Experience in working with large-scale data centers or cloud provider infrastructure, and contributions to open-source monitoring or infrastructure projects.
Experience with containerization and orchestration (e.g., Docker, Kubernetes).
Knowledge of hardware telemetry protocols and data ingestion pipelines, and distributed systems and how to monitor them.
