Big Data out of CFD

When you try to catch some yet unexplained flow physics regardless it is a real or numerical experiment... you will easily draw in data. Running a Computational Fluid Dynamics experiment of LES/DNS resolution (not to confuse with CFD modelling or RANS simulation) has a major advantage over a lab experiment. It allows extracting all the values of measured or derived parameters from any point of the volume of interest (domain) at any time without flow interruption. That of course yields in an ocean of... or better to call it... Big Data.

In this presentation, I want to show you not what I discovered (that was the topic of my presentation in December 2020), but how did I handle such data. What kind of technical challenges I met and how did I overcome them. Also, how did I avoid producing 12 PB of data and still keeping them valuable, proven and reliable for other researchers. Finally, why it was possible only at CIŚ cluster.