ВеликобританияSenior Data Engineer
1 мес. назад
ВеликобританияSenior
awssnowflakedata engineeringevent-driven architecture
Senior Data Engineer role at a global fintech platform focusing on zero processing fees for merchants.
О компании
- Customer obsessed: We only succeed when our customers do.
- Move fast : Build, test and improve quickly. Progress matters more than perfection.
- Own it : Be accountable, solve problems, and make it happen.
- Be open : Act with honesty and respect. Transparency builds trust.
- Win together : Collaboration beats ego every time.
Обязанности
- We run an event-driven data platform on AWS and Snowflake. Events flow from our product services through EventBridge, Kinesis Firehose, S3, and are auto-ingested into Snowflake via Snowpipe; batch and third-party data lands through Snowpark-deployed Python procedures. All infrastructure is defined in Terraform/Terragrunt. Python (Poetry, Pydantic, pytest) is our pipeline language; GitHub Actions runs CI/CD. Datadog gives us end-to-end pipeline observability.
Требования
- AWS & Event-Driven Architecture: Hands-on experience with streaming/event ingestion (Kinesis, EventBridge or equivalents), S3 partitioning, and queue/DLQ patterns.
- Infrastructure as Code: Terraform experience required; Terragrunt a plus. Comfortable owning modules across environments, not just consuming them.
- Snowflake & Warehouse Engineering: Strong SQL for Snowflake, including Snowpipe, Tasks and Snowpark.
- Dbt : Proven experience in building well tested SQL and python models in dbt.
- CI/CD & Data Security: Experience with GitHub Actions (or equivalent), and secrets/key management, PII and sensitive data handling in Snowflake.
- Python Coding: Python skills for pipeline and infrastructure code (typed models, testing, CI-deployed services), not just scripting or analysis.
Условия
- Tax advantage Share Options
- Flexible working model
- Work from home set up
- Learning & Development opportunities
- Contributory Pension Scheme
- Free Team lunch (Tues & Thurs) and social evenings
- Comprehensive PMI & x4 Life Insurance
- Your birthday off, plus one Revival day
- If you are excited about sharing the adventure, joining a growing team with big ambitions and you are really great at what you do, then apply now!
- Please let us know if you require any reasonable adjustments at any point during the application and/or recruitment process.
Другое
- Super Payments is the only global fintech platform providing 0% processing fees for merchants. Our mission is to use data and AI to make payments free for businesses, so that everyone wins .
- At Super, we are disrupting payments, like Spotify did to music and Robinhood did to trading. We don't make money from transaction fees; instead, we monetise through value-added services such as lender commissions and FX. Our platform supports multiple payment methods — from traditional cards to direct bank transfers via open banking, as well as Apple Pay, Google Pay, and our own Buy Now Pay Later solution.
- Super has raised $66M from leading investors including Accel, Union Square Ventures and Local Globe (the same investors behind Spotify, Stripe, Monzo) and founded by Samir Desai CBE, co-founder of Funding Circle.
- Already trusted by thousands of businesses and more than 4 million customers , Super is processing at a run rate of £1.5B and growing 4x YOY
- Event Pipeline Ownership: Build and evolve the EventBridge → Firehose → S3 → Snowpipe ingestion path, including partitioning and throughput tuning as event volumes grow across streams like fraud, payments and merchant.
- Batch & Third-Party Integrations: Design and deploy Snowpark-based Python procedures for file-based and third-party data loads, scheduled and monitored as Snowflake Tasks.
- Schema Evolution: Work with product engineers to manage upstream schema changes and late-arriving data without breaking downstream consumers.
- Infrastructure as Code: Own and extend Terraform/Terragrunt modules for ingestion infrastructure across multiple environments.
- Pipeline Observability: Design Datadog monitors and alerting for Firehose, EventBridge, DLQs and Snowpipe; triage and resolve ingestion failures.
- Security & Compliance: Manage PII handling (hashing, secure deletion), masking, data governance and access in Snowflake. Support audit-ready, reconciled data for Finance and Compliance.
- Dbt: Build privacy-first data pipelines using dbt and Snowflake Dynamic Tables to safely expose sensitive data for downstream analytics and reporting.
- Data Quality: Build testing and monitoring for ingestion and transformation logic to catch issues before they reach the business or the Analytics Engineering team.
- Stakeholder Collaboration: Partner with the wider Engineering team and Analytics Engineering team as a downstream consumer, to ensure pipelines deliver accurate, audit-ready data.
- Technical Communication: A pragmatic, systems-level mindset with the ability to explain pipeline failures, infrastructure trade-offs, or data-flow issues clearly to both engineers and non-technical stakeholders.
- Ownership & Initiative : A strong willingness to work independently, take initiative, and take full ownership of the pipelines and infrastructure you build, from design and implementation through to production reliability.
- Production Mindset : Proactively monitoring, debugging, and improving pipelines and infrastructure in production, not just building them and moving on.
- *The stated experience and background is a guide and does not preclude applications from candidates with more or less experience, provided the requisite skills can be demonstrated.