The project focuses on building a modern cloud-based data processing platform, enabling efficient management and real-time analysis of large datasets. Key aspects include implementing ETL/ELT pipelines, optimizing data processing, and ensuring compliance with industry regulations. The solution leverages Azure, Apache Spark, and Data Lakehouse architecture to support strategic business decisions. […]
Shape the Data and Distribution architecture and technology stack within our new cloud-based datalake-house, being a hands-on contributor, senior lead in the big data and data lake space, capable of...