-
Explain modern data architecture strategies
Assessment Strategies
Skill demonstration in lab, written assessment, and semester-long project
Criteria
Describe the differences between and applications of OLTP and OLAP systems effectively
Describe strengths and weaknesses of data warehouses, data lakes, and data lakehouses
Distinguish ETL from ELT and choose an appropriate approach for a given use case
Explain how and why data models differ for BI reporting, machine learning, and AI-oriented use
-
Create data products for analytical use
Assessment Strategies
Skill demonstration in lab, written assessment, and semester-long project
Criteria
Design star schemas with clearly defined fact and dimension tables with effective grain and keys
Shape data models that are effective for BI tools and easy for analysts to understand
Write documentation that effectively describes the use and purpose of curated datasets
-
Identify effective data storage formats
Assessment Strategies
Skill demonstration in lab, written assessment, and semester-long project
Criteria
Choose effective file forms (CSV, JSON, Parquet, etc.) for different stages of a data pipeline
Justify the choice of format based on performance, schema evolution, and interoperability criteria
Incorporate open table formats, employing table-centric thinking
Manage data ingestion and transformation on a professional-grade data platform, such as Databricks
-
Apply governance and security principles
Assessment Strategies
Skill demonstration in lab, written assessment, and semester-long project
Criteria
Organize governed data using catalog, schema, and table abstractions in a tool such as Unity Catalog
Apply basic permissioning models to control access to data, with particular attention to PII
Utilize data lineage information to explain where data comes from and who depends on it
-
Create end-to-end data pipelines
Assessment Strategies
Skill demonstration in lab, written assessment, and semester-long project
Criteria
Design end-to-end pipelines that move data from raw sources through curated outputs
Describe and define DAGs of tasks that include job triggers, dependencies, and schedules
Structure data flows using a medallion (bronze-silver-gold) architecture
Explain the purpose of each layer in terms of cleanliness, usability, and trust
-
Utilize multiple data processing approaches
Assessment Strategies
Skill demonstration in lab, written assessment, and semester-long project
Criteria
Differentiate between batch and streaming processing, selecting the best pattern for different scenarios
Minimize full data reloads through incremental processing that leverages watermarks and CDC
-
Create reliable, high-quality data products
Assessment Strategies
Skill demonstration in lab, written assessment, and semester-long project
Criteria
Apply ACID-related concepts to data pipelines and storage
Design pipelines and transformations that are idempotent, resilient to reruns and partial failures
Assess the quality of data by implementing data quality checks
Describe multiple types of data quality checks and how they contribute to data quality
-
Utilize engineering principles (“Data as Code”)
Assessment Strategies
Skill demonstration in lab, written assessment, and semester-long project
Criteria
Maintain a sharable, version-controlled repository of artifacts using Git and GitHub
Ensure that all artifacts – transformations, quality checks, etc. are versioned, reviewable, and reproducible
Demonstrate good engineering practices of clear naming, modular code, simple tests, and documentation
-
Describe DataOps concepts
Assessment Strategies
Skill demonstration in lab and written assessment
Criteria
Describe continuous integration and continuous delivery (CI/CD) in the context of data pipelines
Explain how automated checks and repeatable deployment contribute to analytical system reliability and safety
-
Utilize curated data for insight and AI readiness
Assessment Strategies
Skill demonstration in lab, written assessment, and semester-long project
Criteria
Answer business questions using known tools and techniques that attach to gold-layer, curated data
Describe how a data product supports downstream workflows under different types of analytic scenarios
Communicate technical designs and decisions in clear language to nontechnical stakeholders