Pervasive sensor systems offer unbounded possibilities for monitoring and tracking objects, machines, and spaces. To maximize the benefit from a sensor system, sensor data requires efficient preprocessing and analysis. Big data techniques make distributed processing of huge amounts of data fast and cost-effective, making them a practical necessity for sensor data. However, the real-time requirements and the sheer velocity and volume of data from large sensor systems require a dedicated approach to designing the data processing pipeline. This paper discusses viewpoints and requirements for designing a sensor data pipeline, with specific focus on data input, live preprocessing, and storage.

Jussi Ronkainen, Antti Iivari (VTT): Designing a data management pipeline for pervasive sensor communication systems

Presented at tThe 10th International Conference on Future Networks and Communications (FNC 2015) / The 12th International Conference on Mobile Systems and Pervasive Computing (MobiSPC 2015) Affiliated Workshops

http://www.sciencedirect.com/science/article/pii/S1877050915016749