Grand Challenges on the Road to Practical Data Mining Systems
by Usama Fayyad, DMX Group, USA
The past two decades has seen a huge wave of computational systems for
the "digitization" of business operations from ERP, to manufacturing,
to systems for customer interactions. These systems increased the
throughput and efficiency of conducting "transactions" and resulted in
an unprecedented build-up of data captured from these systems. The
paradoxical reality that most organizations face today is that they
have more data about every aspect of their operations and customers,
yet they find themselves with an ever diminishing understanding of
either. Data Mining has received much attention as a technology that
can possibly bridge the gap between data and knowledge. While some
interesting progress has been achieved over the past few years,
especially when it comes to techniques and scalable algorithms, very
few organizations have managed to benefit from the technology.
Despite the recent advances, some major hurdles exist on the road to
the needed evolution. Furthermore, most technical research work does
not appear to be directed at these challenges, nor does it appear to
be aware of their nature. This talk will cover these challenges and
present them in both the technical and the business context. The
exposition will cover deep technical research questions, practical
application considerations, and social/economic considerations. The
talk will draw on illustrative examples from scientific data analysis,
commercial applications of data mining in understanding customer
interaction data, and considerations of coupling data mining
technology within database management systems. Of particular interest
is the business challenges of how to make the technology really work
in practice. There are many unsolved deep technical research problems
in this field and we conclude by covering a sampling of these.
This page has been accessed 
times since August 28, 2003.