Anyone AI is recruiting experienced Software Engineers for a specialized project focused on reviewing and evaluating real-world software engineering tasks derived from open-source repositories.The work involves assessing whether coding tasks based on real GitHub issues and pull requests are technically sound, reproducible, appropriately tested, and representative of the kinds of problems professional software engineers solve every day.What You’ll Work OnReal-world bug fixes and feature implementationsOpen-source repositories and pull requestsUnit tests and test coverageRepository setup and dependency managementReproducibility and environment configurationTask difficulty and complexityMulti-file and cross-module code changesTechnical feedback and quality assessmentYou’ll determine whether tasks are clearly specified, technically solvable, supported by sufficient tests, and free from issues such as flaky tests, missing dependencies, ambiguous requirements, or environment-specific behavior.What We’re Looking For3+ years of professional software engineering experienceStrong experience working with large, multi-file codebasesExperience reviewing pull requests, debugging issues, and maintaining production codeStrong understanding of unit testing and test coverageAbility to evaluate whether tests correctly validate a solution without unnecessarily restricting implementation approachesExperience with dependency management, environment setup,



and reproducibilityStrong understanding of Git and GitHub-based development workflowsAbility to analyze complex technical problems and provide clear written feedbackNice to HaveContributions to or maintenance of open-source projectsExperience with SWE-Bench, SWE-Bench Verified, or similar coding benchmarksExperience with major Python open-source projects such as Django, Flask, scikit-learn, SymPy, matplotlib, requests, or pytestExperience with Docker, CI/CD, pip, conda, or dependency pinningKnowledge of test fixtures, test isolation, or property-based testingExperience designing technical assessments or reviewing coding challengesExperience with AI/ML evaluation, data curation, RLHF, or benchmark developmentWhat You’ll Be Responsible ForReviewing coding tasks derived from real GitHub issues and pull requestsAssessing whether problem statements and success criteria are clear and completeEvaluating unit tests for correctness, coverage, and robustnessIdentifying flaky tests, missing dependencies, version conflicts, and environment issuesDetermining whether tasks can be reliably reproduced across environmentsAssessing the real-world difficulty and complexity of each taskProviding clear recommendations on whether tasks should be accepted, improved, or excludedEngagementWork Type: RemoteEngagement: Part-time, project-based consultingFocus: Software engineering, open-source code review, testing, and technical evaluationThis role is a strong fit for experienced engineers who enjoy debugging complex codebases, reviewing pull requests, working with open-source software, and evaluating what makes a software engineering problem well designed.#J-18808-Ljbffr

📌 Senior Software Engineer – Open Source & Swe-Bench Evaluation (Colombia)
🏢 Anyone AI
📍 Colombia