Design and Setup of a Distributed Storage Repository for NetFlow Records

IP traffic traces are widely used by researchers for different purposes. On one hand, traces may be used to observe traffic characteristics in order to improve traffic analysis tools. On the other hand they may also be used to evaluate new algorithms based on real traffic traces. One important drawback when designing new algorithms for analysing IP traffic is the lack of access to real traffic based on which the new algorithm can be evaluated. Often, researchers use traffic traces available locally (such as those captured within a lab network or within a subnetwork of a university), but they rarely have access to traces collected by other parties.

The European project EMANICS within one of its activities aims to design a platform for collecting NetFlow records at multiple sites and make them available to the different project partners. In order to achieve this, the platform needs to offer well-defined interfaces that allow it to access NetFlow records stored locally by the different partners. Additionally, it shall offer granular access control to these data, as well as the possibility to anonymize the results.