НидерландыSite Reliability Engineer - Data Platform
4 нед. назад
НидерландыSenior
hdfskafkadremiokuberneteslinux
Experienced Site Reliability Engineer to manage and scale distributed data platforms for critical data services.
О компании
- IMC is a research-driven trading firm where quantitative modeling, machine learning, and engineering shape how modern markets are traded. A stabilizing force in markets since 1989, we provide liquidity across trading venues, delivering the best outcome in value and risk management to investors. Using our own technology and capital, we build proprietary systems and algorithms that operate across global markets. Our researchers, traders, and engineers work as a collective, combining rapid experimentation, advanced infrastructure, and real-time feedback to turn insight into execution and execution into advantage.
Обязанности
- As an SRE within IMC you will join a sub-team that takes a central role in all the data needs and you’ll be working to:
- Design, implement and operate our data platforms.
- Improve observability so we catch issues before our users do.
- Build automation to reduce toil and allow our systems to scale
- Support and own reliability of critical services (e.g. HDFS, Kafka and Dremio)
- Drive long-term architectural improvements, not just fixing issues, but preventing them.
Другое
- IMC operates on the cutting-edge use of technology to create a competitive edge over the competition. We also grow quick and have plenty of complex technical challenges. We're looking for an experienced SRE with strong background in managing distributed data systems on both bare metal Linux and Kubernetes. We want someone who can help us standardize deployments, elevate observability, and improve automation as we scale our data platform and other critical data services.
- You will join our Data Platform team, part of our local data team that builds and runs the systems that are used by traders, quant researchers and engineering teams, for all their data needs. The Data Platform team is responsible for the foundational platform that our data frameworks and tooling is built on top of. This includes observability, scalability and supporting standardised deployments.
- Strong experience managing distributed data platforms (e.g. Kafka, Hadoop, Spark, Dremio); including full installation, debugging and performance tuning
- Hands on experience deploying, configuring and orchestrating software on Linux and Kubernetes, with proven ability to troubleshoot issues in both environments
- Strong experience with infrastructure as code (Ansible preferred) and best practices
- Proficient programming experience in Python
- Ability to read, write and tune SQL queries
- Comfortable reading Java source code, tuning and debugging running JVMs.
- Familarity with data lakehouse technologies (e.g. Iceberg or Delta Lake) as well as query engine technologies (e.g. Dremio, Presto or Trino)
- Exposure to workflow orchestration tools like Airflow or Dagster
- A proactive mindset: you're not just fixing issues but preventing them.
- Comfortable working across teams, with minimal oversight.
- Data Tools: Hadoop (HDFS), Kafka, Dremio, Iceberg, Clickhouse, Spark, Airflow, Flink
- Infrastructure Automation: Ansible, Puppet, Kubernetes (ArgoCD, Helm, Kustomize)
- Observability: Prometheus, Grafana, AlertManager
- Scripting: Python, Bash, SQL
- Others : PCAP infrastructure