Data Platform Engineer – Inventory & Observability

Другое

• Qube Research & Technologies (QRT) is a global quantitative and systematic investment manager, operating in all liquid asset classes across the world. We are a technology and data driven group implementing a scientific approach to investing. Combining data, research, technology and trading expertise has shaped QRT’s collaborative mindset which enables us to solve the most complex challenges. QRT’s culture of innovation continuously drives our ambition to deliver high-quality returns for our investors.
• As QRT’s infrastructure estate grows, maintaining accurate data on its resources, ownership and performance is increasingly important. In this hands-on role, you will build and operate the data foundations underpinning our infrastructure inventory and observability capabilities. The focus extends beyond dashboard design to creating an accurate, consistent and reliable view of the platform, its ownership and its behaviour.
• Your future role within QRT:
• Design and build data models for infrastructure and operational data, defining canonical representations for resources, ownership, capacity and utilisation.
• Integrate inventory, configuration, security and observability data from multiple systems into a coherent, queryable view.
• Keep key counts and metrics consistent across different consumers, and make data lineage, ownership and limitations explicit.
• Design data structures that are fast to query and easy to reason about at the scale of QRT’s estate.
• Automate data processing, validation and delivery, and build the tooling to detect data-quality, freshness and performance issues before consumers do.
• Investigate and resolve data-quality, freshness and performance issues, solving root causes rather than symptoms.
• Collaborate closely with infrastructure, security, engineering and observability teams to onboard new sources and improve the accuracy of the platform view.
• Take ownership of systems end-to-end, and adapt the platform as scale, requirements and underlying technologies change.
• Your present skillset:
• Experience building and operating production-grade data platforms, analytical data models, or large-scale data-processing applications.
• Strong SQL and dimensional or analytical data modelling experience, including the design of schemas that are maintainable, performant and easy for consumers to reason about.
• Strong Python software engineering experience, including the design of maintainable, tested production applications and scalable ETL/ELT pipelines.
• Demonstrated rigour around data correctness: establishing provenance, reconciling conflicting or incomplete sources, and reasoning carefully about identity, ownership and source of truth.
• Solid working understanding of infrastructure and observability concepts, and comfort working across data, infrastructure and observability domains.
• Experience operating business-critical pipelines and diagnosing data, performance and reliability issues in production.
• Familiarity with knowledge graphs, ontologies and taxonomies, and their application to modelling heterogeneous operational data.
• A strong bias towards automating repetitive work, and pragmatism in balancing correctness, performance and delivery.
• Excellent problem-solving and communication skills, and the ability to own outcomes across research, engineering and infrastructure teams.
• Preferred: A background in data science, statistics or applied machine learning, particularly applied to incomplete, noisy or inconsistent data, or to inferring missing information and identifying unusual behaviour.
• Experience with SQL-based analytical platforms, dbt, object storage and data catalogues.
• Experience with infrastructure inventory or CMDB data, ownership and capacity/utilisation modelling.
• Experience with metrics, logs and vulnerability data, and with observability tooling.
• Experience with Superset or comparable BI and data-exploration tools.
• Experience with Linux, containers and GitLab CI/CD.
• Experience with AWS services and cloud SDKs.