Design a Natural Language Processing Pipeline — System Design Interview Practice
Design an NLP pipeline that processes unstructured text through entity extraction, relation extraction, sentiment analysis, topic modeling, and text classification at scale. Work through the requirements, architecture trade-offs, and an interactive design review.
Requirements and concepts to consider
- Extract named entities (people, organizations, locations)Requirement
- Identify relationships between entitiesRequirement
- Classify documents into categoriesRequirement
- Perform sentiment and opinion analysisRequirement
- Extract key phrases and topicsRequirement
- Handle multiple languagesRequirement
- Process structured and unstructured textRequirement
- Provide API for batch and real-time processingRequirement
- mlConcept to explore
- nlpConcept to explore
- nerConcept to explore
- text classificationConcept to explore
- sentiment analysisConcept to explore
- transformersConcept to explore