BrazilSenior Data Engineer
1 мес. назад
BrazilLATAMSenior
sparkdbtetldata transformationdata ingestion
Responsible for implementing and maintaining components of the data platform including ingestion jobs, dbt models, Spark transformations, and ensuring data quality and security compliance.
Обязанности
- Implement and maintain individual components of the data platform—for example, ingestion jobs, dbt models, Spark transformations, CDC tasks, matching rules, or deduplication logic.
- Make implementation decisions within a component : schema mapping, transformation logic, join strategy, and similar choices bounded to that unit of work.
- Fix defects in transformations, ingestion jobs, or entity resolution logic when issues are identified.
- Ensure component outputs match the expected schema, data contracts, and downstream expectations.
- Improve a component’s performance, data quality checks, or reliability when gaps or incidents require it.
- Follow existing ETL and MDM standards and team patterns rather than inventing parallel approaches.
- Apply security and compliance expectations to your components: handle sensitive and personal data according to classification, retention, and minimization rules; avoid logging, samples, or exports that over-collect or expose regulated fields beyond what the use case requires.
- Use approved identity, access, and secrets patterns for jobs and services (for example, role-based access, managed identities, or vault-backed credentials)— not hard-coded secrets or ad hoc shared accounts.
- Support auditability of changes and data movement as the team defines it (for example, clear job ownership, metadata, lineage hooks, or evidence packs for controls) so security and compliance reviews can trace what the pipeline does.
- What you'll need:
- Bachelor's degree in Computer Science, Computer Engineering, Information Systems, Systems Analysis and Development, or similar.
- Intermediate English.
- Code quality: write clear, readable, modular code; follow team naming and formatting conventions; avoid unnecessary duplication in your own changes; prefer changes that can be understood without a verbal walkthrough.
- Verification: add required unit or transformation-level tests; validate schema assumptions and basic data quality conditions; ensure changes do not break existing behavior.
- Delivery: submit well-structured pull requests that include a clear description of the change, context, and expected impact, and evidence of testing.
- Stack (typical): Python, SQL, and data processing with PySpark and/or Scala as used in the team’s pipelines.
- Pipelines: practical experience building or maintaining batch/stream components with orchestration (for example, Apache Airflow, Databricks Workflows, or similar) and version control (Git).
- Data work: comfortable with transformation, cleansing, aggregation, and basic performance tuning for SQL and Spark workloads, given volume and complexity.
- Cloud: familiarity with services on a major provider (AWS, Azure, or Google Cloud) in the way the team deploys and runs jobs.
- Security baseline for data engineering: follow least-privilege IAM and service principals for pipelines; prefer encryption in transit and at rest where the platform provides it; keep dependencies and images within approved channels and address high-severity findings from scanners or security tooling when they affect your components.
- Compliance-aware delivery: When a change touches regulated data, new integrations, or new exports, document data purpose, flows, and safeguards in the PR or linked ticket so risk and compliance partners can assess impact without guesswork.
- More about you:
- Hands-on with transformation tooling and data contracts in a shared warehouse.
- APIs or event interfaces used for data exchange between systems.
- Infrastructure-as-code or CI/CD (for example, Azure DevOps, Terraform, GitHub Actions) for job deployment.
- Familiarity with data governance tooling (catalog, quality, policy tags) or vulnerability / secret scanning in CI for data repos and pipelines.
- What we offer
- Performance-based bonus*
- Attendance bonus*
- Private pension plan
- Meal allowance
- Casual office and dress code
- Days off*
- Health, dental, and life insurance plans
- Discounts on medications
- Partnership with WellHub
- Childcare assistance
- Discounts on Ambev products*
- Clube Ben partnership
- Scholarship program*
- School supplies support
- Language learning platforms and training
- Transportation allowance
- *Rules applied
Другое
- About us
- AB InBev is the leading global brewer and one of the world’s top 5 consumer product companies. With over 500 beer brands, we’re number one or two in many of the world’s top beer markets, including North America, Latin America, Europe, Asia, and Africa.
- About
- At , our ambition is – and always will be – to put customers at the heart of everything we do, making their lives easier and their businesses more profitable. Through our B2B e-commerce and SaaS platform, we bring the power of digital to small and medium-sized retailers, unlocking new growth opportunities for all.
- AB InBev Growth Group is proud to be an Equal Opportunity and Affirmative Action employer. We do not discriminate based upon of race, color, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other applicable legally protected characteristics.
- The following fields are optional, but anticipate the information for your registration*.
- Remember: your data will never be used as elimination criteria in selection processes. With them, AB InBev Growth Group is able to analyze diversity and reduce biases in selection processes. We want to contribute to changing this reality by being an inclusive company.
- For more information: