استخدام senior data platform engineer

توضیحات آگهی

We are building a modern on premise data platform from the ground up to power analytics risk management compliance and AI driven insights across our brokerage operations This is a rare opportunity to architect a greenfield lakehouse platform end to end from ingestion through governance to an AI powered analytics agent in a regulated financial environment with real budget and headcount to build it right
We re looking for a hands on Senior Data Platform Engineer who has done this before someone who can make sound architectural trade offs mentor a growing team and isn t afraid to operate complex distributed systems in production
What You ll Work On
Design and operate a Lakehouse architecture Medallion Bronze Silver Gold on Apache Iceberg with Nessie as a versioned catalog backed by on premise object storage MinIO
Build and maintain CDC pipelines from multiple OLTP databases using Debezium and Kafka Connect
Own the Kafka based streaming backbone serving both batch ingestion and real time sub second use cases such as market data and trading event processing
Develop stateful stream processing with Apache Flink for real time risk exposure calculations anomaly detection and alerting
Build and maintain transformation pipelines with dbt orchestrated via Airflow
Support analytical query workloads on Trino and BI delivery through Superset
Design and enforce data governance cataloging and lineage OpenMetadata DataHub fine grained access control Apache Ranger and audit logging suitable for a regulated environment
Deploy and operate platform components across a hybrid Kubernetes VM footprint stateless services on K8s stateful services such as Kafka and MinIO on VMs
Contribute to capacity planning on call practices and platform reliability as the system scales toward production critical status
Must Have Qualifications
5 years of experience building and operating production data infrastructure
Strong hands on experience with Apache Kafka including Kafka Connect and CDC patterns ideally Debezium
Practical experience with a modern table format Apache Iceberg or Delta Lake and lakehouse patterns
Proficiency with a distributed SQL query engine Trino Presto or similar
Experience with dbt for data transformation and modeling
Experience with workflow orchestration tools Airflow or equivalent
Solid understanding of Kubernetes for deploying and operating stateful and stateless workloads
Strong SQL and at least one of Python Scala Java for pipeline development
Demonstrated experience designing data architectures with data governance security and auditability as first class concerns
Strongly Preferred
Experience with Apache Flink or another stream processing engine for stateful low latency workloads
Experience with data catalog lineage tools OpenMetadata DataHub or Amundsen
Experience with fine grained access control systems Apache Ranger or equivalent
Prior experience in financial services trading or another regulated industry with strict data residency requirements
Experience operating fully on premise non cloud infrastructure at scale
Experience with Nessie or git like data versioning for lakehouse catalogs
What Makes This Role Different
Greenfield build you ll shape the architecture not inherit legacy debt
Strict on premise requirement means solving real distributed systems problems without leaning on managed cloud services
Direct exposure to both the batch analytical side and the real time streaming side of the platform
شرکت توسعه فناوری گروه فارابی Farabi Group Tech Development Co
دسته بندی IT DevOps Server
نحوه همکاری تمام وقت
نوع همکاری حضوری
مدرک تحصیلی کارشناسی
سابقه کار بیش از شش سال
حقوق توافقی
جنسیت مهم نیست
مهارت ها kafka Apache Kafka SQL
شهر تهران تهران