نشان کن
کد آگهی: KP6581834785

As a Data Engineer, you are responsible for building and operating reliable, scalable, and observable data pipelines that transform raw operational and event data into trusted analytical datasets. Your work directly supports data scientists and chapter-based data analysts by ensuring data is timely, accurate, and performant.You work across relational sources, event streams, and analytical stores, with strong ownership over data quality and system reliability.What You Drive ForwardDesign, build, and maintain large-scale ETL/ELT data pipelines orchestrated with Apache Airflow.Model, load, and optimize analytical datasets in ClickHouse for high-performance querying and aggregations.Ingest and transform data from relational databases (SQL Server and Mysql).Build and maintain event-driven ingestion pipelines using Kafka, including topic consumption and schema handling.Index and manage analytical and search-oriented datasets in Elasticsearch.Implement data validation, idempotency, and retry mechanisms to ensure pipeline correctness.Monitor data pipelines and infrastructure using Prometheus and Grafana, owning freshness and reliability metrics.Collaborate closely with data scientists on feature-ready datasets and with data analysts on consumable data models.Continuously improve performance, scalability, and operational robustness of the data platform.Required Qualifications3–5 years of experience in data engineering or a closely related role.Strong SQL skills and hands-on experience with relational databases (e.g., SQL Server).Solid experience designing and operating analytical workloads in ClickHouse or similar columnar databases.Practical, production experience with Apache Airflow.Build and optimize real-time streaming systems using Kafka.Experience working with object storage systems such as MinIO for analytical or data platform workloads.Strong understanding of data modeling, data warehousing concepts, and ETL/ELTbest practices.Experience with monitoring, alerting, and observability using Grafana and Prometheus.Experience with version control (Git) and CI/CD practices.Nice to HaveExperience with Delta Lake, Apache Hudi, or Apache Iceberg.Familiarity with lakehouse-style architectures.Exposure to schema evolution, data contracts, or CDC-based ingestion.

دیجی کالا
در تهران
در وبسایت ایران استخدام  (3 روز پیش)
اطلاعات شغل:
نوع همکاری:  تمام‌وقت
نیاز به سابقه:  حداقل 3 سال
ساعت کاری:  تمام وقت
متن کامل آگهی:
As a Data Engineer, you are responsible for building and operating reliable, scalable, and observable data pipelines that transform raw operational and event data into trusted analytical datasets. Your work directly supports data scientists and chapter-based data analysts by ensuring data is timely, accurate, and performant.
You work across relational sources, event streams, and analytical stores, with strong ownership over data quality and system reliability.
What You Drive Forward
Design, build, and maintain large-scale ETL/ELT data pipelines orchestrated with Apache Airflow.
Model, load, and optimize analytical datasets in ClickHouse for high-performance querying and aggregations.
Ingest and transform data from relational databases (SQL Server and Mysql).
Build and maintain event-driven ingestion pipelines using Kafka, including topic consumption and schema handling.
Index and manage analytical and search-oriented datasets in Elasticsearch.
Implement data validation, idempotency, and retry mechanisms to ensure pipeline correctness.
Monitor data pipelines and infrastructure using Prometheus and Grafana, owning freshness and reliability metrics.
Collaborate closely with data scientists on feature-ready datasets and with data analysts on consumable data models.
Continuously improve performance, scalability, and operational robustness of the data platform.
Required Qualifications
3–5 years of experience in data engineering or a closely related role.
Strong SQL skills and hands-on experience with relational databases (e.g., SQL Server).
Solid experience designing and operating analytical workloads in ClickHouse or similar columnar databases.
Practical, production experience with Apache Airflow.
Build and optimize real-time streaming systems using Kafka.
Experience working with object storage systems such as MinIO for analytical or data platform workloads.
Strong understanding of data modeling, data warehousing concepts, and ETL/ELT
best practices.
Experience with monitoring, alerting, and observability using Grafana and Prometheus.
Experience with version control (Git) and CI/CD practices.
Nice to Have
Experience with Delta Lake, Apache Hudi, or Apache Iceberg.
Familiarity with lakehouse-style architectures.
Exposure to schema evolution, data contracts, or CDC-based ingestion.

این آگهی از وبسایت ایران استخدام پیدا شده، با زدن دکمه‌ی تماس با کارفرما، به وبسایت ایران استخدام برین و از اون‌جا برای این شغل اقدام کنین.

هشدار
توجه داشته باشید که دریافت هزینه از کارجو برای استخدام با هر عنوانی غیرقانونی است. در صورت مواجهه با موارد مشکوک،‌ با کلیک بر روی «گزارش مشکل آگهی» به ما در پیگیری تخلفات کمک کنید.
گزارش مشکل آگهی
تماس با کارفرما
این آگهی رو برای دیگران بفرست
نشان کن
گزارش مشکل آگهی
جستجوهای مرتبط
جمعه 18 بهمن 1404، ساعت 08:41