Design, develop, and maintain data extraction and processing solutions, and API-based backend services with security, scalability, and observability standards; while stabilizing and improving existing scraping products to ensure operational continuity and accelerate deliveries in an expanding team.Key ResponsibilitiesTranslate business requirements into maintainable technical solutionsDesign resilient and cost-efficient architectures on cloud platformsDevelop robust scrapers (proxy rotation, CAPTCHA handling, authentication, DOM change tolerance)Design and implement REST APIs with authentication, pagination, and security best practicesOrchestrate and monitor data pipelines (scheduling, retries, alerts, logging, metrics)Normalize and transform data (ETL/ELT) and design storage schemasReduce downtime when target sources change (self-healing capabilities)Elevate quality and performance standards (profiling, optimization, automation)Ensure compliance with Terms of Service, copyright, robots.Txt, and data protection regulationsPromote best practices (security, versioning, CI/CD, documentation)Conduct unit and integration testing; validate data qualityImplement observability: structured logging, metrics, and tracesCreate technical documentation and participate in code reviewsCollaborate with stakeholders and clients (Spanish/English)RequirementsMust Have:Web scraping tools: Requests/HTTPX, Playwright/Selenium, BeautifulSoup/lxml, CSS/XPath selectorsAPI development with FastAPI (or equivalent), authentication (tokens, OAuth), paginationGit (PRs, code review) and CI/CD (GitHub Actions/GitLab CI)SQL (PostgreSQL/MySQL)



and NoSQL (MongoDB/Redis); modeling and performance optimizationDocker; orchestration and deployment concepts (K8s desirable)Observability (structured logging, metrics, traces, and alerts)Cloud fundamentals (AWS/Azure/GCP)Advanced English (B2+/C1, oral and written)Nice to Have:Messaging/queues (Kafka/RabbitMQ), scheduled tasks (Celery/Arq)Data lakes/warehouses (S3/BigQuery/Snowflake) and dbt/Airbyte/PrefectVector DBs/embeddings (FAISS/Pinecone) and LLM opsGo/Rust/Node.Js for high-performance componentsSecurity (secrets management, rate limiting, OWASP)Lunch bonus: $500,000 COPUber between office and home (specific hours)$2,000 USD for training after the first yearContinuous learning opportunitiesGoogle Cloud certifications paid by the companyCorporate travel opportunitiesLaptop and tools providedFlexible schedule with team coordinationWhat We're Looking ForProactivity and sense of urgency; extreme ownershipAnalytical thinking and creative problem-solvingAdaptability to frequent changes and effective prioritizationClear communication with technical and non-technical audiencesCollaborative work and willingness to share knowledgeCustomer orientation and quality focusContinuous learning and curiosity about new tools/technologiesSuccess Metrics (First 90 Days)Success rate / availability (%)Lead time of changes / throughput−30% in 90 daysAPI reliabilitySLA/SLO (e.G., 99.5%)Monthly compliance30-60-90 Day Plan30 hours (1 week)Technical onboarding (repos, pipelines, standards). Take ownership of 1 scraper and 1 API. Map failure points and quick wins.Success: 1 stable release; 5+ PRs; basic documentation60 hours (2 weeks)Fix active scrapers. Performance improvements (profiling/optimization). Automate key tests and observability.90 hours (3 weeks)Deliver architecture/orchestration improvements. Reduce change lead time. Propose quarterly roadmap.Success: −30% lead time; SLOs met; roadmap approvedIn ScopeDevelopment of robust scrapers (proxy rotation, CAPTCHA handling, authentication, DOM change tolerance)Design/implementation of REST APIs with authentication, pagination, and security best practicesPipeline orchestration and monitoring (scheduling, retries, alerts, logging, metrics)Data normalization and transformation (ETL/ELT) and storage schema designUnit and integration testing; data quality validationObservability: structured logging, metrics, and tracesTechnical documentation and code reviews; collaboration with stakeholders (ES/EN)Out of ScopeUI/UX/front-end design beyond API endpoints and contractsDirect sales/commercial negotiation (technical support when required)On-premise physical infrastructure operation (cloud-based work)Data Ethics & ComplianceResponsibility to comply with Terms of Service, copyright, robots.Txt policies, and applicable data protection regulations (e.G., Law 1581 of 2012 in Colombia, GDPR if applicable). Escalate legal doubts before proceeding with new data sources.#J-18808-Ljbffr

📌 Data Solutions & Automation Engineer (Cundinamarca)
🏢 Bright Data
📍 Cundinamarca