Lead Data & Integration Engineer

1 мес. назад

SingaporeWorldwideLead

apissftpbatch pipelinesdata ingestiondata transformationdatamappingintegration

Lead design and implementation of data flows and system integrations supporting data movement and preparation for AI use cases.

Обязанности

Analyse business/technical requirements and translate them into data flows and integration designs
Work with upstream and downstream teams to define data contracts and interfaces
Identify gaps, inefficiencies and risks in current data movement processes
Propose pragmatic solutions balancing speed, quality and maintainability
Integration & Data Movement
Design and implement data movement across systems using:
APIs
SFTP and file based transfers
Batch pipelines
Coordinate integrations across systems in the DataLake ecosystem (Informatica, Cloudera, etc.)
Ensure data is correctly transformed, mapped and delivered to target systems
Troubleshoot integration issues across environments
Data Preparation for GenAI
Support data ingestion and preparation for GenAI use cases:
document ingestion
data aggregation
enrichment and transformation
Work with structured and unstructured data
Ensure data is usable for downstream AI workflows (RAG, search, investigation flows)
You are not asking them to build models, just make data usable for them.
Delivery & Coordination
Work across multiple teams:
data platforms
application teams
infrastructure
security
Support SIT, UAT and production rollouts
Ensure integration reliability, error handling and monitoring
Document flows, mappings and interfaces clearly
Key Requirements
Below are the key skillsets that will be required for all relevant tasks mentioned:
10 years of experience in system analysis, integration engineering, data engineering or technical delivery roles.
Strong ability to translate requirements into system flows, data flows, interface specifications and implementation plans.
Experience working with upstream and downstream teams to define and deliver enterprise integrations.
Practical experience with REST APIs, SFTP, batch processing, file based integration and data pipeline orchestration.
Good understanding of data mapping, transformation, aggregation, reconciliation and data quality controls.
Good SQL skills and basic to moderate Python skills for data handling, scripting, automation and troubleshooting.
Exposure to Java
Exposure to Informatica, Cloudera or similar enterprise data platforms.
Working knowledge of Git , branching, pull requests, code reviews and controlled release practices.
Familiarity with CI/CD, Jira, Confluence and enterprise deployment processes.
Experience with Control M or equivalent scheduling tools.
Familiarity with logging (OTEL) and monitoring tools such as Splunk Elastic Stack.
Exposure to GenAI concepts such as document ingestion, RAG, embeddings and data preparation for AI workflows.
Strong communication skills, with the ability to challenge weak designs and coordinate across business, application, data, infrastructure and security teams.
Key Domain:
Data Engineering,
System Integrations,
Python, SQL

Другое

 
 
Encora is a global company that offers Software and Digital Engineering solutions. Our practices include Cloud Services, Product Engineering & Application Modernization, Data & Analytics, Digital Experience & Design Services, DevSecOps, Cybersecurity, Quality Engineering, AI & LLM Engineering, among others.
At Encora, we hire professionals based solely on their skills and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality