The EVOLVE project is funded by the European Union under Horizon 2020 research and innovation programme. It aimed to take concrete steps in bringing the big data, high-performance computing, and cloud worlds in a unique platform that increases the ability to extract value from massive and demanding datasets. EVOLVE is European Commission’s answer to the unprecedented data growth currently experienced.

Back in 2017, when EVOLVE was designed, proposing a software environment to ensure the convergence of Cloud, HPC and Big Data was already a complex task. But once a software layer is able to conceal the border between these worlds, nothing prevents it from hiding more, in EVOLVE’s specific case the hardware heterogeneity of the platform, from the computing elements to multiple storage tiers. The rise of virtualization and the need for convergence was easy to predict, but betting on the right technologies was much harder.

EVOLVE’s consortia has been supporting from the start containers, micro-services and notebooks as key enablers for convergence. The market has proven EVOLVE right. EVOLVE was an early adopter of the AI wave, and rode this wave quite continuously (which means rewriting some applications, in Tensorflow, then Pytorch, then TF2) during the last 3 years.

The platform is now a success. EVOLVE’s converged heterogeneous platform is on-line with innovative applications fully ported and the results at scale have been fully demonstrated through several pilots and use cases. Application end-users can now drive everything through a notebook interface, and thanks to an array of microservice they can orchestrate complex pipelines and visualise their results.

The project itself has led to more than 36 scientific contributions, which means 1 scientific paper (conference, workshop and journal) per month. Some of these papers were published in the most prestigious conferences or journals. Together with other projects funded by the same H2020 ICT call a book was written on the topic of “HPC, Big Data, AI Convergence Towards Exascale – Challenge and Vision.

Additionally, during these three years, EVOLVE has been an innovation incubator it was designed to foster innovation, with a state-of-art platform and a consortium made of a carefully balanced selection of industrial and academic partners. The project has delivered at least 12 innovations. Some are already on the market, such as Earth Observer, or are hosted by the Linux Foundation; and some are helping to address global challenges, for instance you can check the climate.gov website powered by the visualization technologies matured in EVOLVE. EVOLVE has achieved many things on-premise, within the data-centre, and now the new frontier is on the Edge and in the Fog. This trend is already addressed by some of EVOLVE's innovations, such as DataShim to transparently move a container jointly with its data set; or MemoPort to accelerate multi-cloud transfer. The whole list of innovations is available on EVOLVE’s website here.

With the project now ending in November, the project coordinator Jean-Thomas Acquaviva from DDN Storage France remarks: “We hope that EVOLVE innovations will bear the heritage of this great moment spent between multiple organizations across Europe... and get ready for the next one!”