A production-grade SQL data warehouse built on Medallion Architecture — consolidating fragmented data sources into a single, reliable foundation for analytics and reporting.
This project implements a modern SQL-based data warehouse using Microsoft SQL Server, following the Medallion Architecture (Bronze → Silver → Gold) to unify ERP and CRM data sources into a clean, analytics-ready repository.
Key capabilities include:
- Automated ETL pipelines for ingestion, transformation, and loading
- Star schema data modeling optimized for high-performance analytical queries
- SQL-based reporting for actionable business insights
- Built with data governance, quality, and compliance best practices at every layer
The warehouse is structured across three layers of the Medallion Architecture:
| Layer | Purpose |
|---|---|
| 🥉 Bronze | Raw data ingested as-is from CSV source files into SQL Server — no transformations applied |
| 🥈 Silver | Cleansed, standardized, and normalized data — prepared and validated for analytical use |
| 🥇 Gold | Business-ready data modeled into a star schema — optimized for reporting and dashboards |
This project covers the full data engineering lifecycle:
- Data Architecture — Medallion-based warehouse design with clearly separated concerns per layer
- ETL Pipelines — Extraction, transformation, and loading scripts from raw sources to analytical models
- Data Modeling — Fact and dimension tables structured for efficient querying and reporting
- Analytics & Reporting — SQL-driven insights ready for BI tools and stakeholder dashboards
Develop a modern SQL Server data warehouse to consolidate sales data from multiple systems, enabling reliable analytical reporting and data-informed decision-making.
| Area | Details |
|---|---|
| Data Sources | ERP and CRM systems provided as CSV files |
| Data Quality | All quality issues identified and resolved prior to analysis |
| Integration | Unified data model combining both sources for cross-functional analytics |
| Scope | Latest dataset snapshot; historical versioning out of scope |
| Documentation | Full data catalog, naming conventions, and model documentation included |
Everything used in this project is free and open source.
| Tool | Purpose |
|---|---|
| 📁 Datasets | Raw ERP & CRM CSV source files |
| 📋 Notion Project Board | Full project phases, tasks, and progress tracking |
| 🗄️ SQL Server Express | Lightweight SQL Server for local database hosting |
| 🖥️ SSMS | GUI for database management and query execution |
| ✏️ DrawIO | Architecture diagrams, data flow, and model design |
data-warehouse-project/
│
├── datasets/ # Raw source datasets (ERP and CRM CSV files)
│
├── docs/ # Architecture diagrams and project documentation
│ ├── data_architecture.drawio # Overall warehouse architecture diagram
│ ├── data_flow.drawio # End-to-end data flow across all layers
│ ├── data_models.drawio # Star schema and dimensional model design
│ ├── etl.drawio # ETL techniques and transformation methods
│ ├── data_catalog.md # Field descriptions, metadata, and data dictionary
│ └── naming-conventions.md # Naming standards for tables, columns, and files
│
├── scripts/ # SQL scripts organized by layer
│ ├── bronze/ # Raw data extraction and loading scripts
│ ├── silver/ # Data cleansing and transformation scripts
│ └── gold/ # Analytical model and star schema scripts
│
├── tests/ # Data quality checks and validation scripts
│
├── README.md # Project overview and setup guide
├── LICENSE # MIT License
├── .gitignore # Git ignored files and directories
└── requirements.txt # Project dependencies
This project is licensed under the MIT License. You are free to use, modify, and distribute it with proper attribution.
Hi there! I'm Ranjith Ankilla — a Data Engineer and IT professional with 5 years of experience building data systems that drive real business outcomes.
I specialize in designing, building, and optimizing large-scale Data Pipelines, ETL processes, SQL-based warehousing, and BI dashboards that transform raw data into decisions. I bring both technical depth and a sharp business mindset to every project.
