Stream Data Processing: A Quality of Service Perspective (eBook)

Modeling, Scheduling, Load Shedding, and Complex Event Processing

Sharma Chakravarthy, Qingchun Jiang (Autoren)

eBook Download: PDF

2009 | 2009
XXVI, 324 Seiten
Springer US (Verlag)
978-0-387-71003-7 (ISBN)

Infos

The systems used to process data streams and provide for the needs of stream-based applications are Data Stream Management Systems (DSMSs).

This book presents a new paradigm to meet the needs of these applications, including a detailed discussion of the techniques proposed. Ii includes important aspects of a QoS-driven DSMS (Data Stream Management System) and introduces applications where a DSMS can be used and discusses needs beyond the stream processing model. It also discusses in detail the design and implementation of MavStream. This volume is primarily intended as a reference book for researchers and advanced-level students in computer science. It is also appropriate for practitioners in industry who are interested in developing applications.

---

In recent years, a new class of applications has come to the forefront { p- marily due to the advancement in our ability to collect data from multitudes of devices, and process them e ciently. These include homeland security - plications, sensor/pervasive computing applications, various kinds of mo- toring applications, and even traditional applications belonging to nancial, computer network management, and telecommunication domains. These - plications need to process data continuously (and as long as data is available) from one or more sources. The sequence of data items continuously gen- ated by sources is termed a data stream. Because of the possible never-ending nature of a data stream, the amount of data to be processed is likely to be unbounded. In addition, timely detection of interesting changes or patterns or aggregations over incoming data is critical for many of these applications. Furthermore, the data arrival rates may uctuate over a period of time and may be bursty at times. For most of these applications, Quality of Service (or QoS) requirements, such as response time, memory usage, and throughput are extremely important. These application requirements make it infeasible to simply load the incoming data streams into a persistent store and process them e ectively using currently available database management techniques.

Inhalt

Preface 7
Objectives 8
Intended Audience 9
Acknowledgements 9
How to Use the Book 11
Contents 13
List of Figures 19
List of Tables 22
List of Algorithms 23
1 INTRODUCTION 24
1.1 Paradigm Shift 26
1.2 Data Stream Applications 28
1.3 Book Organization 29
2 OVERVIEW OF DATA STREAM PROCESSING 31
2.1 Data Stream Characteristics 31
2.2 Data Stream Application Characteristics 32
2.3 Continuous Queries 34
2.4 Data Stream Management System Architecture 41
3 DSMS CHALLENGES 44
3.1 QoS-Related Challenges 44
3.2 Concise Overview of Book Chapters 48
4 LITERATURE REVIEW 53
4.1 Data Stream Management Systems 53
4.2 QoS-Related Issues 58
4.3 Complex Event Processing 61
4.4 Commercial and Open Source Stream and CEP Systems 67
5 MODELING CONTINUOUS QUERIES OVER DATA STREAMS 69
5.1 Continuous Query Processing 70
5.2 Problem Denition 74
5.3 Modeling Relational Operators 77
5.4 Modeling Continuous Queries 89
5.5 Intuitive Observations 102
5.6 Experimental Validation 105
5.7 Summary of Chapter 5 113
6 SCHEDULING STRATEGIES FOR CQs 114
6.1 Scheduling Model and Terminology 115
6.2 Impact of Scheduling Strategies on QoS 122
6.3 Novel Scheduling Strategies for CQs 124
6.4 Experimental Validation 145
6.5 Summary of Chapter 6 155
7 LOAD SHEDDING IN DATA STREAM MANAGEMENT SYSTEMS 156
7.1 The Load Shedding Problem 157
7.2 Integrating Load Shedders 159
7.3 Load Shedding Framework 162
7.4 Experimental Validation 177
7.5 Summary of Chapter 7 184
8 N F M: AN INTER-DOMAIN NETWORK FAULT MANAGEMENT SYSTEM 186
8.1 Network Fault Management Problem 187
8.2 Data Processing Challenges for Fault Management 189
8.3 Stream- and Event-Based N F M Architecture 192
8.4 Three-Phase Processing Model for N F M 197
8.5 Transactional Needs of Network Management Applications 203
8.6 Summary of Chapter 8 205
9 INTEGRATING STREAM AND COMPLEX EVENT PROCESSING 206
9.1 Motivation 207
9.2 Event Processing Model 210
9.3 Complex Event Vs. Stream Processing 214
9.4 MavEStream: An Integrated Architecture 219
9.5 Stream-Side Extensions 222
9.6 Event-Side Extensions 226
9.7 Summary of Chapter 9 232
10 MavStream: DEVELOPMENT OF A DSMS PROTOTYPE 234
10.1 MavStream Architecture 235
10.2 Windows Types 239
10.3 Stream Operators and CQs 241
10.4 Buffers and Archiving 248
10.5 Run-time Optimizer 250
10.6 QoS-Delivery Mechanisms 262
10.7 System Evaluation 267
11 INTEGRATING CEP WITH A DSMS 280
11.1 MavStream: Integrated Issues 281
11.2 Design of the Integrated System 285
11.3 Implementation Details of Integration 294
11.4 Stream Modiers 300
11.5 Additional Benets of CEP Integration 303
11.6 Summary of Chapter 11 304
12 CONCLUSIONS AND FUTURE DIRECTIONS 305
12.1 Looking Ahead 305
12.2 Stream Processing 306
12.3 Integration of Stream and Event Processing 309
12.4 Epilogue 311
References 312
Index 332

Produkt-Details

• Erscheint lt. Verlag: 9.4.2009
• Reihe/Serie: Advances in Database Systems
• Reihe/Serie: Advances in Database Systems
• Zusatzinfo: XXVI, 324 p. 50 illus.
• Verlagsort: New York
• Sprache: englisch
• Themenwelt: Informatik ► Datenbanken ► Data Warehouse / Data Mining
• Themenwelt: Mathematik / Informatik ► Informatik ► Grafik / Design
• Themenwelt: Mathematik / Informatik ► Informatik ► Netzwerke
• Themenwelt: Informatik ► Theorie / Studium ► Algorithmen
• Themenwelt: Naturwissenschaften
• Schlagworte: Architecture • Chakravarthy • currentsmp • Database • Database Management • Data processing • Dom • Monitor • QoS • security • Solutions • Stream • Stream Data
• ISBN-10: 0-387-71003-5 / 0387710035
• ISBN-13: 978-0-387-71003-7 / 9780387710037