Databricks Projects
Welcome to my Databricks project portfolio. This space highlights my analytics engineering trajectory through a curated selection of projects focused on scalable architecture and clean code. I invite you to review my work and reach out to discuss potential future collaborations.
DISCLAIMER
The following case studies are representative scenarios created solely to demonstrate my technical skills, design thinking, and problem-solving approach.
They do not represent real client data, production systems, or proprietary architectures. Any resemblance to actual organizations, datasets, or implementations is purely coincidental.
Architected an automated data pipeline using Databricks Notebooks and PySpark to ingest and clean 100-home pilot sensor data by filtering negative values and extreme 'phantom spikes'. The solution optimized 'Smart Pricing' billing through conditional peak-hour logic and enforced strict data privacy compliance by applying SHA-256 hashing to anonymize customer PII within the Silver analytics layer.
A project on handling large CSV file by an intern named Simona where we guide her on the distributed computing process in Spark through Databricks notebook along with fulfilling certain ad-hoc requests.
What happens when a vital medical project is suddenly blocked by a GDPR crisis and a system-crashing "Shuffle Wall"? This case study follows Lead Data Engineer Chloé Laurent as she salvages the SantéFlux "Stress Spikes" project using salted hashing, Broadcast Joins, and advanced data cleansing for "Ghost Users." Explore how to navigate complex privacy mandates to deliver a high-performance "City Stress Matrix" under a mission-critical deadline.
Can your data architecture withstand a Black Friday surge when €150,000 is on the line every single hour? This case study tackles the "NordicStream AdTech Crisis," where a 5 TB pipeline froze at 99% due to extreme data skew and inefficient legacy code. By implementing advanced performance engineering—including salted joins for 1.2 billion skewed records, vectorized Pandas UDFs—this project demonstrates how to transform a catastrophic system failure into a high-performance production environment.
