متن کامل آگهی:
As a Data Engineer, you are responsible for building and operating reliable, scalable, and observable data pipelines that transform raw operational and event data into trusted analytical datasets. Your work directly supports data scientists and chapter-based data analysts by ensuring data is timely, accurate, and performant.
You work across relational sources, event streams, and analytical stores, with strong ownership over data quality and system reliability.
What You Drive Forward
Design, build, and maintain large-scale ETL/ELT data pipelines orchestrated with Apache Airflow.
Model, load, and optimize analytical datasets in ClickHouse for high-performance querying and aggregations.
Ingest and transform data from relational databases (SQL Server and Mysql).
Build and maintain event-driven ingestion pipelines using Kafka, including topic consumption and schema handling.
Index and manage analytical and search-oriented datasets in Elasticsearch.
Implement data validation, idempotency, and retry mechanisms to ensure pipeline correctness.
Monitor data pipelines and infrastructure using Prometheus and Grafana, owning freshness and reliability metrics.
Collaborate closely with data scientists on feature-ready datasets and with data analysts on consumable data models.
Continuously improve performance, scalability, and operational robustness of the data platform.
Required Qualifications
3–5 years of experience in data engineering or a closely related role.
Strong SQL skills and hands-on experience with relational databases (e.g., SQL Server).
Solid experience designing and operating analytical workloads in ClickHouse or similar columnar databases.
Practical, production experience with Apache Airflow.
Build and optimize real-time streaming systems using Kafka.
Experience working with object storage systems such as MinIO for analytical or data platform workloads.
Strong understanding of data modeling, data warehousing concepts, and ETL/ELT
best practices.
Experience with monitoring, alerting, and observability using Grafana and Prometheus.
Experience with version control (Git) and CI/CD practices.
Nice to Have
Experience with Delta Lake, Apache Hudi, or Apache Iceberg.
Familiarity with lakehouse-style architectures.
Exposure to schema evolution, data contracts, or CDC-based ingestion.