Senior DataOps Engineer

2 нед. назад

АзербайджанГрузияТурция+1Senior

awscloud computingdata analyticsai

Responsible for accelerating Data & AI initiatives and modernizing IT estate through cloud migration and secure hybrid cloud foundation on AWS for a telecommunications client.

Обязанности

Design, implement, and maintain the AWS Data Platform foundation, including Amazon S3 lake layout, Apache Iceberg table format standardization, and AWS Glue Data Catalog integration.
Build and optimize scalable batch and streaming data pipelines using Amazon EMR (Serverless and EMR-on-EKS), Apache Spark/PySpark, Apache Flink, and Amazon Athena with workgroup cost caps.
Implement data tokenization and de-identification pipelines on the data side (PA-T) using Protegrity/Spark UDFs for batch, Spark Streaming for micro-batch, and Kafka Connect SMT for streaming PII masking prior to cloud transit.
Build and manage real-time streaming architectures with Amazon MSK (Managed Streaming for Apache Kafka), MSK Replicator/MirrorMaker2, and Schema Registry integration.
Implement Medallion Architecture (Bronze, Silver, Gold layers) for lakehouse data modeling, automating Iceberg table registration and backfill frameworks.
Execute data migration waves across non-PII and PII datasets using AWS DataSync, automated register steps, and streaming migration configs.
Configure data access control and governance models using AWS Lake Formation, cross-engine authorization, AWS Macie for PII detection, and Informatica Data Catalog (Axon, EDC, BDQ) integration.
Build automated schema-drift identification components, GitLab CI/CD pipeline integration for dataset synchronization, and automated data contracts/circuit breakers.
Implement DataOps observability, centralizing logging via Amazon CloudWatch, configuring FinOps cost & anomaly monitoring, and setting up automated alerts.
Author technical documentation, operational runbooks, disaster recovery (DR) procedures, and cutover/rollback playbooks for data platform hardening.

Требования

4+ years of hands-on experience as a Data Engineer or DataOps Engineer building enterprise-grade data platforms and pipelines.
Strong expertise with AWS Data Analytics stack: Amazon S3, AWS Glue Data Catalog, AWS Lake Formation, Amazon EMR (EMR-on-EKS / Serverless), Amazon Athena, AWS DataSync, and Amazon MSK.
Proficient in Apache Spark / PySpark and Spark Streaming for batch, micro-batch, and real-time data processing.
Strong experience in Medallion Lakehouse Architecture design and implementation (Bronze, Silver, Gold layers).
Practical experience in implementing data tokenization and encryption at scale (e.g., Protegrity, Thales, FPE, or Spark UDF-based de-identification pipelines).
Solid knowledge of event-driven architectures & streaming: Apache Kafka / Amazon MSK, Kafka Connect (SMT), Schema Registry, and MirrorMaker2 / MSK Replicator.
Expertise with Relational Databases (Amazon RDS, PostgreSQL, Oracle) and data synchronization techniques.
Hands-on experience with DataOps CI/CD & Automation: GitLab CI/CD, Infrastructure-as-Code (Terraform / AWS CDK), schema-drift detection, and data contract validation.
Familiarity with data governance tools and enterprise data catalogs (e.g., Informatica Axon/EDC, AWS Lake Formation).

Будет плюсом

AWS Certified Data Analytics – Specialty or AWS Certified Data Engineer – Associate.
Experience with Apache Iceberg table format, cataloging, compaction, and schema evolution.
Experience in telecom domain data models, CDR processing, and high-throughput real-time telemetry.
Experience with cloud-side tokenization/detokenization via Athena UDFs / AWS Lambda.
Familiarity with containerization (Docker, EKS, Kubernetes) for big data runtimes.
Experience with AWS Macie and FinOps cost-allocation/anomaly-detection frameworks.

Условия

Flexible working format - remote, office-based or flexible
A competitive salary and good compensation package
Personalized career growth
Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
Active tech communities with regular knowledge sharing
Education reimbursement
Memorable anniversary presents
Corporate events and team buildings
Other location-specific benefits
*not applicable for freelancers

Другое

N-iX is a global software development company founded in 2002, connecting over 2,400+ tech professionals across 40+ countries. We deliver innovative technology solutions in cloud computing, data analytics, AI, embedded software, IoT , and more to global industry leaders and Fortune 500 companies. Join us to create technology that drives real change for businesses and people across the world. We are looking for Senior DataOps Engineer to join the team.
Our client is an Azerbaijani telecommunications company, the largest mobile network operator in Azerbaijan. The main products are: Fixed telephony, Mobile telephony, Internet services, Wireless broadband, and Value-added services.
The primary goal is to accelerate the client’s Data & AI initiatives via a secure, hybrid cloud foundation on AWS while systematically modernizing the IT estate as part of the cloud migration.
Cloud Foundation & Landing Zone: Deploy target hybrid network architectures, establishing a secure Landing Zone and hybrid Data/AI platforms on AWS.
Security, Compliance & Governance: Operationalize on-prem tokenization (achieving zero raw PII in the cloud), resolve policy blockers to include AWS in the ISMS, and establish a Cloud Center of Excellence (CCoE) to govern Cloud adoption.
AI Chatbot & Voicebot Design & Implementation: Develop and operationalize a flagship Customer Care Chatbot and Voicebot as the first hybrid-setup consumer.
Strong critical thinking, problem-solving, and analytical skills.
Excellent communication and collaboration skills to work closely with cross-functional teams (Data Science, Cloud/Platform, Security, Governance).
Results-oriented, proactive mindset with strong ownership of deliverables within an Agile / Scrum framework.
Upper-Intermediate+ English level (written and spoken).