Environments

TaskData is one example. Register a task provider with @env.tasks. Dataset is a versioned collection of traces for later training or comparison.

from enroute import Dataset, Environment, TaskData

env = Environment(name="arithmetic", version="0.1.0")

@env.tasks
def tasks():
    return [
        TaskData(task_id="q1", input="What is 2+2?", expected="4"),
        TaskData(task_id="q2", input="Name a primary color", expected="red"),
    ]

print([task.task_id for task in env.iter_tasks()])
dataset = Dataset.from_traces("arithmetic-rollouts", traces=[])

The Environments screen in this product will isolate keys and traffic the same way. Until that ships, define environments in code and keep hosted keys in the workspace Account section.