Core Concepts
Classifications
Agent-to-human outreach to collect or confirm structured data labels
Overview
Classifications enable agents to collect structured data labels from humans. Common use cases:
- Content moderation
- Sentiment analysis
- Support ticket triage
- Data validation
- Training data collection
Basic Example
Synchronous vs Async
Synchronous (Default)
Agent waits for human input:
Asynchronous
Agent continues while waiting for classification:
Best Practices
-
Clear Options
- Use mutually exclusive categories
- Include specific examples in descriptions
- Set clear criteria for each option
-
Structured Feedback
- Use
promptFill
for consistent responses - Guide humans to provide specific details
- Collect rationale for important decisions
- Use
-
Quality Control
- Consider multiple reviewers for critical data
- Track inter-rater agreement
- Use
runId
to group related classifications
-
Efficient Workflows
- Batch similar items together
- Use async mode for large volumes
- Provide sufficient context in prompts
Next Steps
Private Beta
Classifications are currently in private beta. To request access, please contact us at contact@humanlayer.dev.