Senior Data Engineer in our company is responsible for designing, developing, and maintaining data infrastructure that plays a key role in the organization’s strategic decision-making. This position requires a deep technical knowledge, a structured problem-solving approach, the ability to collaborate across teams, and a focus on data quality and system architecture.
The main goal of this role is to build a reliable, scalable, and up-to-date data ecosystem that provides a solid foundation for analysis, reporting, and data-driven growth within the organization.
Key Responsibilities:
- Design and develop data pipelines.
- Implement data transfer and processing systems with Apache Airflow and Kafka.
- Build pipelines that are scalable, easy to monitor, and maintain.
- Construct advanced data architecture.
- Develop a structured and data-driven Data Warehouse based on data architecture principles.
- Define optimized data models to improve data storage and retrieval performance.
- Manage data quality.
- Continuously monitor the accuracy, consistency, and freshness of data.
- Design automated processes to identify and correct defective or inconsistent data.
- Collaborate across teams to create reliable data infrastructure.
- Work with product, analysis, and technical teams to understand data needs.
- Provide structured data for use in analysis, dashboarding, and decision-making projects.
Required Skills and Qualifications:
Technical Knowledge:
- Proficiency in Python and data processing tools.
- Practical experience with Apache Airflow for pipeline orchestration.
- Familiarity with Kafka for real-time data transfer.
- Expertise in working with PostgreSQL and ClickHouse.
- Strong understanding of Data Architecture and Data Modeling concepts.
- Knowledge of Data Quality principles and data monitoring.
Infrastructure and DevOps:
- Familiarity with Docker and CI/CD principles for implementing modern, stable structures.
Personal and Interpersonal Skills:
- High attention to detail.
- Ability to solve problems in complex and dynamic environments.
- Strong communication skills and ability to collaborate effectively with various teams.
Required Experience and Background:
- At least 1 year of relevant work experience in Data Engineering or software development.
- Experience designing and implementing data pipelines in real-world projects.
- Experience in cross-team collaboration on data-driven projects.
- Experience working with technology or product teams is a plus.