Each test is 5 questions with varying difficulty.
AI Prep covers AI Agents, Generative AI, ML Fundamentals, NLP & LLMs and a lot more, with adaptive tests and daily challenges. Fully offline on Android. Free to try, one-time unlock for lifetime access.
Medallion Architecture is a data design pattern used to logically organize data in a lakehouse, facilitating a structured progression of data quality and usability. As of 2026, it is the industry-standard framework for building scalable, reliable data platforms on top of storage layers like Delta Lake or Apache Iceberg. The architecture organizes data into three distinct layers: Bronze (raw), Silver (cleansed/integrated), and Gold (business-ready). Data engineers, analytics engineers, and platform architects are frequently tested on this topic because it reveals their ability to balance raw data ingestion with downstream consumption requirements. Junior candidates are expected to understand the purpose of each layer and the basic flow of data. Senior candidates must demonstrate expertise in schema evolution, handling late-arriving data, managing stateful transformations between layers, and optimizing storage costs while maintaining high performance for BI and ML workloads.
Medallion Architecture is critical because it solves the 'data swamp' problem inherent in unorganized data lakes. By enforcing a tiered structure, organizations can ensure that raw data is preserved for auditability (Bronze), while refined, deduplicated, and enriched data is available for high-performance analytics (Silver) and executive dashboards (Gold). In 2026, with the rise of complex AI pipelines, the Silver layer often serves as the primary source for feature stores and training datasets, making its integrity paramount. This topic is a high-signal interview subject because it forces candidates to explain the trade-offs between storage costs, compute overhead, and data latency. A strong answer demonstrates a deep understanding of how to build idempotent pipelines, manage schema evolution without breaking downstream consumers, and implement effective data governance. Weak answers often fail to distinguish between the physical storage layer and the logical transformation logic, or they treat the architecture as a rigid rule rather than a flexible framework for data lifecycle management.
The architecture follows a linear progression of data refinement where each layer acts as a source for the next. Data flows from external systems into the Bronze layer, undergoes transformation into the Silver layer, and is finally aggregated into the Gold layer.
[Source Systems]
↓
[Bronze Layer]
↓
[Silver Layer]
↓ ↓
[BI Tools] [Gold Layer]
↓
[ML Models]Using MERGE INTO statements to update existing records in Silver/Gold layers while inserting new ones.
Trade-offs: Ensures idempotency but can be slower than append-only operations.
Defining strict schemas on write to ensure Silver layer data matches expected types.
Trade-offs: Prevents bad data from entering but can cause pipeline failures on source changes.
Using watermarks or change logs to process only new data since the last pipeline run.
Trade-offs: Reduces compute cost significantly but adds complexity to pipeline state management.
| Reliability | Use ACID-compliant storage formats to ensure atomic writes and rollback capabilities during failures. |
| Scalability | Partition data by date or business unit to optimize read/write performance in large-scale datasets. |
| Performance | Use Z-Ordering or data skipping indexes to speed up queries on large Gold layer tables. |
| Cost | Implement lifecycle policies to move older Bronze data to cheaper cold storage (e.g., S3 Glacier). |
| Security | Apply fine-grained access control at the table or row level to restrict data access by layer. |
| Monitoring | Track data freshness, row counts, and schema drift alerts using observability tools. |
No. While popularized by Databricks, it is a logical design pattern that can be implemented on any data lakehouse platform using formats like Apache Iceberg, Hudi, or Delta Lake on AWS S3, Azure Data Lake, or Google Cloud Storage.
A Landing zone is typically a temporary, transient area where raw files arrive. The Bronze layer is the first persistent, queryable state of that data, usually stored in a structured format like Delta, which provides ACID guarantees.
While tempting, skipping the Silver layer is discouraged. It serves as a crucial abstraction layer that decouples raw ingestion from business-specific logic, ensuring that if business rules change, you don't have to re-ingest raw data.
It uses schema evolution features provided by the storage format (e.g., Delta Lake). By enabling auto-merge or explicit schema evolution, the pipeline can adapt to new columns without failing, provided the changes are additive.
Not necessarily. While star schemas are common for BI, the Gold layer can also contain flattened wide tables for ML or specific aggregates for real-time dashboards, depending on the business use case.
Silver is generally normalized and at the grain of the source system, focusing on cleaning and integration. Gold is denormalized and aggregated, focusing on performance and ease of use for business reporting.
Idempotency is achieved by using 'MERGE' or 'UPSERT' operations instead of 'INSERT'. By defining a unique business key, you ensure that re-running a pipeline updates existing records rather than creating duplicates.
dbt is primarily used to manage the transformation logic between the Silver and Gold layers. It provides version control, testing, and documentation, which are essential for maintaining a high-quality Gold layer.
Yes, it can, as data is stored in multiple layers. However, this is usually offset by the performance gains, improved data quality, and the ability to implement lifecycle policies to move older data to cheaper storage.
Medallion is a manifestation of the ELT (Extract, Load, Transform) pattern. You extract and load raw data into the Bronze layer, then transform it in-place through the Silver and Gold layers using the lakehouse's compute engine.
AI Prep covers AI Agents, Generative AI, ML Fundamentals, NLP & LLMs and a lot more, with adaptive tests and daily challenges. Fully offline on Android. Free to try, one-time unlock for lifetime access.