Middle Data Engineer

Colombia

Medellin

**Middle Data Engineer**:
Fintech project, the purpose of which is to create and maintain banking platforms. Users are bank employees, which is about 10,000. Various platforms are used to serve clients of the bank who would like to invest their money in one of Canada’s largest banks and also internal employees of the bank.

As a key member of our data engineering team, you will play a crucial role in the development and implementation of the Data Platform.

**Requirements**:

- 3-5 years of experience building batch and real-time data pipelines leveraging big data technologies.
- **Data Processing**: Experience with big data technologies and distributed data processing tools like Spark, Hadoop, Airflow, NiFi, and Kafka.
- **ETL & Data Pipelines**: Hands-on experience designing, developing, and maintaining ETL processes and data pipelines.
- **Programming**: Proficiency in Python and/or Scala, with the ability to optimize data processing scripts.
- **SQL & Data Modeling**: Strong knowledge of writing SQL queries and designing scalable data models.
- **Cloud Platforms**: Familiarity with cloud-based data platforms such as Snowflake, Databricks, AWS, Azure, or GCP.
- **CI/CD**: Experience using CI/CD tools and working with Docker and Kubernetes for data pipeline automation. DataOps, Monitoring & Performance
- **Optimization**: Ability to troubleshoot and optimize data workflows for reliability and performance.
- **Data Governance & Security**: Basic understanding of data governance principles, data quality frameworks, and compliance requirements.
- English
- Upper-Intermediate level. _ Nice-to-Have:_
- Experience with OpenShift, Trino, Ranger, and Hive.
- Knowledge of machine learning and data science concepts and tools.
- Familiarity with BI & Analytics tools such as Tableau and Superset.
- Good communication skills, attention to detail, Agile mindset, and flexibility.

**Responsibilities**:

- Work with business stakeholders and cross-functional teams to understand data requirements and deliver scalable data solutions.
- Design, develop, and maintain ETL processes to extract, transform, and load data from various sources into our data platform.
- Build batch and event-driven data pipelines using cloud and on-premises hybrid data platforms.
- Collaborate with data architects to review solutions and data models, ensuring adherence to best practices.
- Take ownership of assigned deliverables, ensuring high-quality implementation and timely completion.
- Implement data quality standards while collaborating with governance teams to ensure compliance with data policies.
- Optimize data integration workflows for performance and reliability.
- Troubleshoot and resolve data integration and processing issues.
- Follow best practices in continuous integration and delivery using DataOps pipelines.
- Stay informed about emerging technologies and trends in data engineering.

We offer:
**Well-being**:

- 10 working days of Paid Day Off within an individual year.
- Up to 15 working days of Unpaid days off within an individual year.

**Professional Growth**:

- Sombra University courses - enjoy a range of learning opportunities through Sombra University. It offers many educational courses, as well as educational lectures on a variety of topics.
- Sombra Around Tech - community attendance - Sombra unites engineers and experts in several areas: Front-end, Back-end, QA, DevOps, and Вusiness Analysis.
- Mentorship program - available on request.
- UDEMY online course platform - stay up-to-date with the latest technologies and programming languages.
- English courses and Speaking Club - attend English classes twice a week in small groups.

**Added advantages**:

- Work equipment (Laptop, monitor, and small devices compensation).
- Sombra’s referral program.
- If you know someone you believe is a good fit for our cooperation, you can recommend them and get a reward.
- Public Holidays - celebrate 18 statutory holidays in Colombia.
- Sombra events - Join Sombra’s traditional events (both online and offline).

**Ruslana**Murha**:
Recruiter