Design a Data Labeling and Annotation Platform — System Design Interview Practice
Design a data labeling platform that supports image, text, and audio annotation, manages labeling workflows, ensures label quality, and provides active learning-based sample selection. Work through the requirements, architecture trade-offs, and an interactive design review.
Requirements and concepts to consider
- Support image bounding box, polygon, and segmentation annotationsRequirement
- Handle text classification, NER, and relation labelingRequirement
- Manage labeling task assignment and workflowsRequirement
- Implement inter-annotator agreement metricsRequirement
- Provide pre-labeling with model-assisted suggestionsRequirement
- Support consensus labeling and adjudicationRequirement
- Integrate with ML training pipelinesRequirement
- Track labeling throughput and quality metricsRequirement
- mlopsConcept to explore
- data labelingConcept to explore
- annotationConcept to explore
- active learningConcept to explore
- data qualityConcept to explore
- crowdsourcingConcept to explore