Data Orchestration Engineer

hace 2 días

Barranquilla, Atlántico, Colombia JobFinder Spain A tiempo completo

Full time | AgileEngine | Colombia

Posted On 01/16/2025

Job Information

City Barranquilla

State/Province Atlantico

080016

IT Services

Job Description

AgileEngine is one of the Inc. 5000 fastest-growing companies in the U and a top-3 ranked dev shop according to Clutch. We create award-winning custom software solutions that help companies across 15+ industries change the lives of millions.

If you like a challenging environment where you're working with the best and are encouraged to learn and experiment every day, there's no better place - guaranteed :)

What you will do
  1. Design a clear and lean data model that outlines data sources and transformations over this data on top of DAGs and data orchestration tools like Dagster or Airflow.
  2. Data validation and data model testing on each DAG step.
  3. Insights Layer Ownership: Build data models and algorithms to generate first-party data using statistical and machine learning techniques, including LLMs and natural language processing. Generate derived insights and determine accurate values from error-prone sources (e.g., headcount information).
  4. Data Product Development: Develop and enhance data products to improve the discoverability of meaningful knowledge and information in our database. Continuously improve similarity, relevance, normalization, and tagging algorithms that power our search engine.
  5. Pipeline Maintenance: Oversee data pipelines' maintenance and health to ensure accurate, efficient, and optimal data transformations by avoiding repetitive tasks or operations within the data.
  6. Team Collaboration: Collaborate with the team to devise product goals, outline milestones, and execute plans with minimal guidance.
  7. Data Warehouse Design: Contribute to the design of a robust data warehouse architecture by following best practices and industry standards; transferring data from S3, loading data with different schedules, and managing different data pipelines on top of a unique warehouse architecture.
  8. Collaborate with our platform team to make design decisions on the optimal middle-layer database flow improving DAG execution times and costs.
Must haves
  1. +4 years of experience as a Data Engineer.
  2. Programming Languages: Python, SQL.
  3. Data Warehouses: Snowflake, Databricks.
  4. Containerization: Docker.
  5. DevOps: AWS.
The benefits of joining us
  1. Professional growth: Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps.
  2. Competitive compensation: We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities.
  3. A selection of exciting projects: Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands.
  4. Flextime: Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office – whatever makes you the happiest and most productive.
#J-18808-Ljbffr

  • Senior Data Orchestration Professional

    hace 4 días

    Barranquilla, Atlántico, Colombia beBee Careers A tiempo completo

    We're looking for a talented Data Orchestration Engineer to join our team. As a key member of our data engineering group, you will be responsible for designing and implementing data pipelines, ensuring data quality, and collaborating with cross-functional teams to drive business outcomes.

  • Senior Data Orchestration Specialist

    hace 6 días

    Barranquilla, Atlántico, Colombia beBee Careers A tiempo completo

    This is an exciting opportunity for a Senior/Lead Data Orchestration Engineer to join our team and contribute to the design and implementation of data models that drive business insights.The ideal candidate will have experience designing and implementing data models using DAGs and data orchestration tools like Dagster or Airflow.You will work closely with...

  • Data Orchestration Engineer

    hace 2 días

    Barranquilla, Atlántico, Colombia JobFinder Spain A tiempo completo

    Job InformationBarranquilla, Atlantico, 080016IT ServicesJob DescriptionWe are one of the Inc. 5000 fastest-growing companies in the U.S. and a top-3 ranked dev shop according to Clutch.We create award-winning custom software solutions that help companies across 15+ industries change the lives of millions.If you like a challenging environment where you're...

  • Senior Data Engineer: Data Pipeline Specialist

    hace 3 días

    Barranquilla, Atlántico, Colombia beBee Careers A tiempo completo

    Senior Data Engineer PositionWe're seeking a seasoned data engineer to contribute to our team's success. The ideal candidate will have expertise in ingesting, structuring, and analyzing unstructured data sources.Ingesting, structuring, and analyzing diverse data sources is a key responsibility of this role.The successful candidate will design, maintain, and...

  • Data Engineering Leader

    hace 6 días

    Barranquilla, Atlántico, Colombia beBee Careers A tiempo completo

    As a Senior/Lead Data Orchestration Engineer, you will play a pivotal role in designing and implementing data models that drive business insights.This position involves creating clear and lean data models outlining data sources and transformations using DAGs and data orchestration tools like Dagster or Airflow.You will validate data and test data models on...

  • Data Model Architect

    hace 4 días

    Barranquilla, Atlántico, Colombia beBee Careers A tiempo completo

    We are seeking a highly skilled Senior/Lead Data Orchestration Engineer to join our team and contribute to the design and implementation of data models that drive business insights.The ideal candidate will have experience designing and implementing data models using DAGs and data orchestration tools like Dagster or Airflow.You will work closely with the team...

  • Big Data Engineer

    hace 4 días

    Barranquilla, Atlántico, Colombia beBee Careers A tiempo completo

    Data Engineering ChallengesWe're seeking an experienced engineer to build and maintain declarative data models, develop complex data transformations, and implement data ingestion integrations with various sources such as Postgres, Kafka, and DynamoDB. Your work will be instrumental in enabling data-driven decision-making across the organization.

  • Data Engineer Leader

    hace 5 días

    Barranquilla, Atlántico, Colombia beBee Careers A tiempo completo

    Scala or Java Data Engineer (Senior/Lead)Data engineering plays a crucial role in unlocking business value from data. As a Senior/Lead Data Engineer, you will be responsible for designing, developing, and maintaining highly scalable data engineering solutions leveraging AWS services.You will design and build scalable pipelines with a clear focus on data...

  • Scala or Java Data Engineer

    hace 4 horas

    Barranquilla, Atlántico, Colombia JobFinder Spain A tiempo completo

    Job Title: Scala or Java Data Engineer (Senior/Lead)A challenging opportunity for a skilled data engineer to design, develop, and maintain highly scalable data engineering solutions leveraging AWS services.About the RoleWe are seeking an experienced data engineer to work collaboratively with application engineers, data scientists, product managers, and...

  • Lead Data Engineer Job

    hace 3 días

    Barranquilla, Atlántico, Colombia beBee Careers A tiempo completo

    Data Engineer Job Description">We are seeking an experienced Data Engineer to join our team and contribute to the design, development, and implementation of scalable data architectures and pipelines.The successful candidate will have a strong background in data engineering, with experience in ingesting, processing, and analyzing large datasets using...