متن کامل آگهی:
We’re seeking a Senior Data Engineer to enhance our Data Science Team, focusing on implementing and managing data workflows that support machine learning models and large-scale analytics. This role involves designing ETL processes, ensuring data quality, and deploying ML models to production.
The ideal candidate will have a strong computer science background, advanced Python knowledge, and experience with SQL/NoSQL databases, and Docker/Kubernetes.
You’ll work closely with our data science, business and product teams to drive insights and innovations.
Responsibilities:
Design and implement ETL processes for data transformation and preparation,
Deploy machine learning models to production environments,
Manage data pipelines for analytics and operational use,
Ensure data accuracy and integrity across multiple sources and systems,
Collaborate with data scientists to support ML algorithms and analytics.
Requirements:
4+ years of experience in data engineering within a production environment,
Design and work with big data frameworks (PySpark, Airflow, kafka),
Design, build and maintain ETL pipelines,
Writing effective and scalable Python codes,
Produce clean, consistent, logical code based on designs; submit to GitLab repositoryGenerating infrastructure that allows big data to be accessed and analyzed,
Refactoring existing frameworks, functions and paradigms to optimize their functioning,
Ensuring data quality,
Create and maintain various database systems (MySQL, Oracle, MS SQL Server, Clickhouse),
Experience with Oracle ODI,
Provide knowledge transfer to team members and support staff through application demos, walkthroughs, and documentation,
Experience with ETL process and pipelines,
Strong proficiency in SQL and experience with relational databases and NoSQL databases like MongoDB,
Confident with version control systems (GitHub, GitLab, etc.),
Experience with pandas, numpy, sklearn or other data science frameworks,
Advanced knowledge of application, data, and infrastructure architecture disciplines,
Knowledge of data warehousing concepts and experience with data modeling and schema design,
Problem Solving and communication skills,
Detail Oriented and decision making.