Senior Data Engineer (Databricks, Spark)

3 нед. назад

BrazilLATAMSenior

databrickssparketldata pipelinejsondata warehouse

Design, develop and maintain scalable data transformation pipelines for a leading Latin American digital banking platform.

Требования

Extensive professional experience in Data Engineering or Analytics Engineering.
Strong hands-on experience with Databricks, including Workflows, Jobs, and Unity Catalog.
Advanced experience developing with Apache Spark (PySpark or Scala).
Experience processing large-scale datasets containing hundreds of millions of records.
Strong knowledge of Delta Lake and/or Apache Iceberg.
Advanced SQL skills with experience writing highly optimized queries.
Strong understanding of dimensional data modeling, including: Fact and Dimension tables, Slowly Changing Dimensions (SCD), Grain definition.
Experience building incremental and idempotent data pipelines.
Experience with Git, CI/CD pipelines, automated testing, and modern software engineering best practices.
Strong analytical and problem-solving skills.

Будет плюсом

Experience supporting multi-cloud migration initiatives (GCP to AWS).
Experience with AWS services including S3, Glue, and Lake Formation.
Familiarity with Google Cloud Platform (GCP).
Experience working with sensitive HR or People data while ensuring compliance with LGPD, GDPR, or similar regulations.
Understanding of Semantic Layer architecture and modern Lakehouse concepts.
Experience integrating REST APIs and processing JSON data.

Условия

We’re an AI-first global tech company with 25+ years of engineering leadership, 2,000+ team members, and 500+ active projects powering Fortune 500 clients, including HBO, Microsoft, Google, and Starbucks.
From AI platforms to digital transformation, we partner with enterprise leaders to build what’s next. What powers it all? Our people are ambitious, collaborative, and constantly evolving.
benefits vary by location and contract type. Your recruiter will fill you in on the details.
International projects
In-office, hybrid, or remote flexibility
Medical healthcare
Recognition program
Ongoing learning & reimbursement 
Well-being program
Team events & local benefits 
Sports compensation 
Referral bonuses 
Top-tier equipment provision

Другое

A leading global digital banking platform and one of the largest neobanks in Latin America, headquartered in Brazil. The company provides a mobile-first suite of financial services—including credit cards, savings accounts, and investments—designed to eliminate bureaucracy and high fees. By leveraging advanced data analytics and a customer-centric digital infrastructure, the organization delivers transparent and accessible financial solutions to millions of users across Brazil, Mexico, and Colombia.
Design, develop, and maintain scalable data transformation pipelines integrating SaaS platforms, APIs, batch processing, and enterprise data sources.
Process, standardize, and optimize structured and semi-structured (JSON) datasets from heterogeneous systems.
Design and implement dimensional data models, including Fact, Dimension, and Event tables following Data Warehouse best practices.
Build trusted analytical datasets and business metrics that serve as the organization's Single Source of Truth.
Develop incremental, idempotent, and scalable data pipelines using Databricks and Apache Spark.
Optimize Spark workloads for large-scale distributed data processing.
Implement automated data quality validation, monitoring, alerting, and operational support processes.
Validate data reconciliation between legacy and modern cloud platforms throughout migration initiatives.
Produce technical documentation and facilitate knowledge transfer to internal engineering teams.
Collaborate with cross-functional stakeholders to deliver scalable, high-quality data solutions.
Advanced English proficiency
Work Type: Hybrid
Location: São Paulo, Brazil
is proud to be an Equal Opportunity Employer committed to inclusion across minority, gender identity, sexual orientation, disability, age, and more
Reasonable accommodations are available to enable individuals with disabilities to perform essential functions
Please note: this job description is not exhaustive. Duties and responsibilities may evolve based on business needs
Compensation details are shared with candidates at the early stage of the recruitment process.
The offer is not binding until a signed contract is in place.
We lead with trust, respect, and purpose. We believe in open dialogue, creative freedom, and mentorship that helps you grow, lead, and make a real difference. Ours is a culture where ideas are challenged, voices are heard, and your impact matters.