Data Engineer Python

hace 3 semanas

Cali, Colombia JobFinder Spain A tiempo completo

AgileEngine is one of the Inc. 5000 fastest-growing companies in the U and a top-3 ranked dev shop according to Clutch. We create award-winning custom software solutions that help companies across 15+ industries change the lives of millions.

If you like a challenging environment where you’re working with the best and are encouraged to learn and experiment every day, there’s no better place - guaranteed

What you will do
  1. Web Scraping & Data Extraction: Design, develop, and optimize web scraping strategies for large-scale data extraction from dynamic websites; identify and assess relevant data sources, ensuring alignment with business objectives; implement automated web scraping solutions using Python and libraries like Scrapy, BeautifulSoup, and Selenium; build resilient and adaptable scrapers that can handle website structure changes, rate limits, and anti-scraping measures.
  2. Data Processing & Integration: Cleanse, validate, and transform extracted data to ensure accuracy, consistency, and usability; store and manage large volumes of scraped data using best-in-class storage solutions; develop ETL pipelines to integrate scraped data into data warehouses and analytics platforms; collaborate with cross-functional teams, including data scientists and engineers, to make scraped data actionable.
  3. Web Scraping & Optimization: Optimize scraping procedures to improve efficiency, reliability, and scalability across multiple data sources; implement solutions for bypassing CAPTCHAs, rotating user agents, and managing proxy services; continuously monitor, troubleshoot, and maintain scraping scripts to minimize disruptions due to site changes.
  4. Compliance & Documentation: Stay up to date with legal, ethical, and compliance considerations related to web scraping and data collection; ensure data collection processes align with best practices and regulatory requirements; maintain clear and detailed documentation of scraping methodologies, data pipelines, and best practices.
Must haves
  1. 5+ years of hands-on experience in web scraping, data extraction, and integration.
  2. Strong proficiency in Python and web scraping frameworks (Scrapy, BeautifulSoup, Selenium).
  3. Experience working with large-scale data storage solutions and optimizing retrieval performance.
  4. Strong grasp of ETL processes, data pipelines, and data warehousing.
  5. Familiarity with APIs for data extraction and integration from public and restricted sources.
  6. Strong problem-solving skills with an ability to debug and adapt to changing web structures.
  7. Solid understanding of web scraping ethics, legal implications, and compliance guidelines.
Nice to haves
  1. Bachelor’s degree in Computer Science, Data Science, Information Technology, or a related field.
  2. Experience with cloud-based distributed scraping systems (AWS, GCP, Azure).
  3. Knowledge of big data frameworks and experience handling high-volume datasets within Snowflake.
  4. Familiarity with machine learning techniques for data extraction and natural language processing (NLP).
  5. Experience working with JSON, XML, CSV, and other structured data formats.
  6. Proficiency with version control systems (Git).
The benefits of joining us
  1. Professional growth: Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps.
  2. Competitive compensation: We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities.
  3. A selection of exciting projects: Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands.
  4. Flextime: Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office – whatever makes you the happiest and most productive.
Next Steps After You Apply

The next steps of your journey will be shared via email within a few hours. Please check your inbox regularly and watch for updates from our Internal Applicant site, LaunchPod, which will guide you through the process.

#J-18808-Ljbffr

  • Data Engineer

    hace 1 semana

    Cali, Valle del Cauca, Colombia Interfell A tiempo completo

    Nuestro cliente aliado es una empresa chilena enfocada en brindar servicios de soluciones integrales a sus clientes, ellos apoyan en las áreas de Transformación Digital, Infraestructura, gestión y desarrollo de proyectos.Actualmente buscan un Data Engineer que se integraría en una célula de trabajo para colaborar en un proyecto basado en Snowflake que...

  • Data Engineer

    hace 2 semanas

    Cali, Valle del Cauca, Colombia Smart Talent It A tiempo completo

    **Un cordial saludo, en Smart Talent tenemos una vacante denominada "Data Engineer" que te puede interesar a ti o a alguno de tus compañeros, así que no dejes de compartirla.100% Trabajo remoto.****Salario hasta COP $ 12.000.000 según conocimientos y experiência, la cual debe ser mínimo de 3 años.****Perfil**:**Buscamos un Data Engineer para que haga...

  • Data Engineer

    hace 5 días

    Cali, Colombia BairesDev A tiempo completo

    At BairesDev®, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works remotely on roles that drive significant impact worldwide. Data Engineer at...

  • Data Engineer Python

    hace 4 semanas

    Cali, Colombia Jobfinder Spain A tiempo completo

    AgileEngine is one of the Inc. 5000 fastest-growing companies in the U and a top-3 ranked dev shop according to Clutch.We create award-winning custom software solutions that help companies across 15+ industries change the lives of millions.If you like a challenging environment where you're working with the best and are encouraged to learn and experiment...

  • Data Engineer Python

    hace 3 semanas

    Cali, Colombia JobFinder Spain A tiempo completo

    AgileEngine is one of the Inc. 5000 fastest-growing companies in the U and a top-3 ranked dev shop according to Clutch. We create award-winning custom software solutions that help companies across 15+ industries change the lives of millions. If you like a challenging environment where you’re working with the best and are encouraged to learn and experiment...

  • Data Engineer Python

    hace 2 semanas

    Cali, Valle del Cauca, Colombia JobFinder Spain A tiempo completo

    AgileEngine is one of the Inc. 5000 fastest-growing companies in the U and a top-3 ranked dev shop according to Clutch. We create award-winning custom software solutions that help companies across 15+ industries change the lives of millions. If you like a challenging environment where you're working with the best and are encouraged to learn and experiment...

  • Data Engineer Python

    hace 3 semanas

    Cali, Colombia JobFinder Spain A tiempo completo

    AgileEngine is one of the Inc. 5000 fastest-growing companies in the U and a top-3 ranked dev shop according to Clutch. We create award-winning custom software solutions that help companies across 15+ industries change the lives of millions. If you like a challenging environment where you’re working with the best and are encouraged to learn and experiment...

  • Data Engineer Python

    hace 2 semanas

    Cali, Valle del Cauca, Colombia JobFinder Spain A tiempo completo

    AgileEngine is one of the Inc. 5000 fastest-growing companies in the U and a top-3 ranked dev shop according to Clutch. We create award-winning custom software solutions that help companies across 15+ industries change the lives of millions. If you like a challenging environment where you're working with the best and are encouraged to learn and experiment...

  • Data Engineer

    hace 3 semanas

    Cali, Colombia Endava A tiempo completo

    **Responsibilities**: - Design and implement Data architectures on cloud or on-premises. - Develop logical data models for OLTP and OLAP processing. - Develop data pipelines for extracting, cleansing, transforming, and enhancing data. - Discovers and elicits business and data requirements. - Team-oriented attitude and the ability to work well with others to...

  • Full-Stack Python Engineer/ Senior Consultant

    hace 2 semanas

    Cali, Colombia Provectus IT, Inc. A tiempo completo

    **Full-stack Python Engineer/ Senior Consultant**: San José, San Jose, Medellín, Antioquia, Bogotá, Capital District, Cali, Valle del Cauca **About project** We're a premier AI consultancy that transforms how organizations leverage artificial intelligence. Our team combines deep technical expertise with a consultative approach to deliver innovative AI...

  • Full-stack Python Engineer/ Senior Consultant

    hace 2 semanas

    Cali, Colombia Provectus IT, Inc. A tiempo completo

    Full-stack Python Engineer/ Senior Consultant About project We're a premier AI consultancy that transforms how organizations leverage artificial intelligence. Our team combines deep technical expertise with a consultative approach to deliver innovative AI solutions that create lasting business impact. Join Provectus as a Senior Full-stack Python Engineer/...

  • Data Engineer

    hace 2 semanas

    Cali, Colombia Orbidi A tiempo completo

    **Sobre nosotros**:En ORBIDI buscamos talentosos profesionales apasionados por la tecnología y el marketing para unirse a nuestro equipo y ayudar a las pequeñas y medianas empresas a crecer y mantenerse competitivas en el mercado.Utilizamos inteligencia artificial, modelos predictivos y tecnologías avanzadas para ayudar a las Pymes a maximizar su...

  • Python Data Engineer

    hace 3 semanas

    Cali, Colombia JobFinder Spain A tiempo completo

    This is a remote position and we are hiring candidates from the whole country. AgileEngine is one of the Inc. 5000 fastest-growing companies in the US and a top-3 ranked dev shop according to Clutch. We create award-winning custom software solutions that help companies across 15+ industries change the lives of millions. If you like a challenging environment...

  • Python Data Engineer

    hace 3 semanas

    Cali, Colombia JobFinder Spain A tiempo completo

    This is a remote position and we are hiring candidates from the whole country. AgileEngine is one of the Inc. 5000 fastest-growing companies in the US and a top-3 ranked dev shop according to Clutch. We create award-winning custom software solutions that help companies across 15+ industries change the lives of millions. If you like a challenging environment...

  • Python Data Engineer

    hace 2 semanas

    Cali, Valle del Cauca, Colombia JobFinder Spain A tiempo completo

    This is a remote position and we are hiring candidates from the whole country. AgileEngine is one of the Inc. 5000 fastest-growing companies in the US and a top-3 ranked dev shop according to Clutch. We create award-winning custom software solutions that help companies across 15+ industries change the lives of millions. If you like a challenging environment...

  • Cloud Data Architect

    hace 12 horas

    Perímetro Urbano Santiago de Cali, Colombia Endava Limited A tiempo completo

    Job SummaryWe are seeking an experienced data engineer to join our team. As a senior data engineer, you will be responsible for designing and building data infrastructure, data transformation, and metadata to meet business requirements and platform needs.ResponsibilitiesYou will work closely with our data analyst/data scientist to understand evolving needs...

  • Data Engineer Analyst

    hace 3 semanas

    Cali, Colombia World Business Lenders, Llc A tiempo completo

    3 weeks ago Be among the first 25 applicants Get AI-powered advice on this job and more exclusive features.About World Business Lenders ( World Business Lenders (WBL) provides general purpose short-term real estate collateralized commercial loans to a broad customer base comprised of small and medium sized businesses throughout the United States that lack...

  • Data Engineer Specialist

    hace 2 días

    Perímetro Urbano Santiago de Cali, Colombia World Business Lenders, LLC A tiempo completo

    World Business Lenders, LLC seeks a skilled Data Engineer Analyst to join our team. This role plays a critical part in ensuring the accuracy and reliability of data used in credit risk models and financial planning.Job DescriptionThe successful candidate will be responsible for maintaining and updating databases that store key financial and operational data....

  • Data Engineer Analyst

    hace 3 semanas

    Cali, Colombia World Business Lenders, LLC A tiempo completo

    3 weeks ago Be among the first 25 applicants Get AI-powered advice on this job and more exclusive features. About World Business Lenders ( World Business Lenders (WBL) provides general purpose short-term real estate collateralized commercial loans to a broad customer base comprised of small and medium sized businesses throughout the United States that lack...

  • Data Engineer Analyst

    hace 3 semanas

    Cali, Colombia World Business Lenders, LLC A tiempo completo

    3 weeks ago Be among the first 25 applicants Get AI-powered advice on this job and more exclusive features. About World Business Lenders ( World Business Lenders (WBL) provides general purpose short-term real estate collateralized commercial loans to a broad customer base comprised of small and medium sized businesses throughout the United States that lack...