Postúlate en Kit Empleo: kitempleo.com.co/empleo/1at3u4

About Us
Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high‑performing teams by matching them with professionals who truly fit their needs.
Role Overview
We are looking for an AI Evaluation Engineer specialized in planning and operations to design and build benchmark tasks that simulate real‑world scenarios such as scheduling, logistics, and resource allocation.
This role focuses on planning, scheduling, and operational optimization problems , where multiple agents must collaborate to solve constraint‑rich scenarios involving resources, timelines, and dependencies.
Commitments Required: 8 hours per day with an overlap of 4 hours with PST.
Employment type: Contractor assignment (no medical/paid leave).
Duration of contract: 4 weeks+.
Location: Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Indonesia, Kenya, Nigeria, Turkey, Vietnam.
Interview: take home assessment (60min) + short interview.
Responsibilities




- Design and build multi‑agent benchmark tasks involving:
- Planning, scheduling, and resource allocation
- Operational decision‑making (logistics, project planning, incident response, capacity planning)
- Create constraint‑rich problem statements with multiple interacting variables.
- Develop verification scripts to evaluate:
- Feasibility (all constraints satisfied)
- Completeness (all requirements met)
- Optimality (efficiency of solutions)
- Define task decomposition strategies across specialized sub‑agents (e.g., resource allocation, constraint resolution, optimization).
- Model realistic operational systems with dependencies, timelines, and constraints.
- Implement validation logic and evaluation pipelines using Python.
- Work with Docker environments for reproducibility and execution.
- Collaborate with internal teams to improve task quality, coverage, and evaluation rigor.

Requirements
- 5+

Postúlate en Kit Empleo: kitempleo.com.co/empleo/1at3u4

📌 AI Evaluation Engineer (Colombia)
🏢 Gramian Consulting
📍 Colombia