Real-Time Monitoring and Surveillance using Data Stream Mining

by Philip S. Yu, IBM T.J. Watson Research Center, USA

With the advance of data gathering and communication technologies, it becomes increasingly possible to support real-time monitoring of large amount of information from diverse information sources. Examples include trade surveillance for security fraud and money laundering, network monitoring for intrusion detection, bio-surveillance for terrorist attacks, and various sensor network based monitoring applications. Data is viewed as a continuous stream in this kind of applications. Problems such as data mining which have been widely studied for traditional data sets cannot be easily applied to the data stream domain. This is because the large volume of data arriving in a stream renders most algorithms too inefficient as most mining algorithms require multiple scans of data which is unrealistic for stream data. More importantly, the characteristics of the data stream can change over time and the evolving pattern needs to be captured. In this talk, I'll provide an overview, discuss the issues and focus on how to mine evolving data streams.


This page has been accessed times since August 7, 2003.