Build a Scalable Data Lake Architecture — System Design Interview Practice
Design a data lake architecture that ingests data from multiple sources, catalogs and organizes data, enables SQL queries, and provides data visualization capabilities. Work through the requirements, architecture trade-offs, and an interactive design review.
Requirements and concepts to consider
- Store structured and unstructured dataRequirement
- Catalog and discover datasetsRequirement
- Query data without moving itRequirement
- Transform and prepare dataRequirement
- Implement data governanceRequirement
- Control access to sensitive dataRequirement
- Visualize data insightsRequirement
- Support petabyte-scale dataRequirement
- awsConcept to explore
- data lakeConcept to explore
- s3Concept to explore
- glueConcept to explore
- athenaConcept to explore
- analyticsConcept to explore