In this paper, we introduce a new technique called Detecting Changes Using Data Cubes of Detection Classifiers or the DCDCDC Algorithm. More specifically, for each cell in a multidimensional data cube, we develop a separate baseline and corresponding change detection algorithm. For example, a three dimensional data cube might include dimensions for time, spatial location and weather conditions.
Our novelity is that we have developed a methodology so that we can quickly compute and updates thousands or tens of thousands such baselines, providing an effective means of detecting changes, even in very large amounts of multi-modal sensor data.
We developed a testbed containing: real time data from over 830 highway traffic sensors in the Chicago region, data about weather, and text data about events that might affect traffic. The goal was to detect in real time interesting changes in traffic conditions.
For this study, We built a separate baseline for each hour in the day, for each day in the week, and for every 2 or 3 traffic sensors, resulting in over 42,000 separate baseline models. We also built a baseline engine to build the necessary baselines automatically. We modified an open source scoring engine to process in real time each new sensor reading, update the appropriate feature vectors, score the updated feature vectors using the baseline models, and send out real time alerts to hande held devices when deviations from the baselines were detected.