175. Senior Data Engineer

1 нед. назад

EcuadorLATAMSenior

databricksawspythonsparkdelta lakedata modellingdata ingestion

Senior Data Engineer to design, build, and maintain data ingestion pipelines on Databricks/AWS for healthcare-focused client.

We're growing a data engineering team for a US-based client whose entire data platform runs on Databricks on AWS. The immediate priority is building out and scaling their data ingestion layer — bringing in data from a wide range of source systems reliably, efficiently, and in a way that holds up as the platform grows. You'll work closely with two client-side teams: The DBA team, who own data governance and are the authority on source systems, access, and data stewardship.
 
Design, build, and maintain ingestion pipelines into Databricks from a variety of source systems.
Build and optimize Spark workloads in Python — batch and streaming.
Model and structure data in Delta Lake following medallion-style patterns (bronze / silver / gold).
Partner with the DBA team on governance, access, and data quality expectations, and with the Data & Analytics team so ingested data actually serves their reporting and strategy needs.
Monitor, troubleshoot, and tune pipelines for reliability, cost, and performance.
Document pipelines, data contracts, and design decisions so the work is maintainable by the team.

5+ years of experience as a Data Engineer
Strong, demonstrable Databricks experience. Someone who knows the platform well in production, not just conceptually. Delta Lake, workflows/jobs, clusters and compute configuration, Unity Catalog.
Advanced Python for data engineering.
Strong Apache Spark — including an understanding of how to diagnose and fix performance problems, not just write working code.
AWS fundamentals as they relate to a data platform (S3, IAM, networking basics, secrets management).
SQL at a strong level.
Experience designing ingestion pipelines from multiple heterogeneous source systems.
Senior-level autonomy: able to take an ambiguous requirement, ask the right questions, and deliver.
B2+ or superior english level

Experience connecting Databricks to on-premises systems (networking, private connectivity, gateways, CDC from on-prem databases).
Experience with other cloud data platforms — Snowflake, BigQuery, Synapse, Redshift.
Streaming and CDC tooling (Structured Streaming, Auto Loader, Debezium, Kafka).
Infrastructure as code (Terraform, Databricks Asset Bundles) and CI/CD for data pipelines.
Orchestration beyond Databricks Workflows (Airflow, dbt).
Databricks certifications (Data EngineerAssociate / Professional).
Prior experience working with governance-heavy or regulated data environments.

✔ Permanent contract ✔ High-impact projects in the healthcare industry ✔ Learning and continuous growth environment 🚀 ✔ Competitive salary based on experience 💰 ✔ Collaborative and international team
 
📍  Apply only if you reside in Ecuador 📍
 
At  Source Meridian , you’ll be part of a high-impact  tech-health  company, building products that truly make a difference.
If you meet the profile — or know someone who might be interested —  apply now!
We’d love to meet you 💬

Source Meridian is a development software company that works to solve the industry’s most challenging problems in healthcare practices. We are laser focused on specific technologies in the healthcare and life science industries: Healthcare technology, artificial intelligence, and healthcare interoperability.