Event Processing (VL/UE)
Prof. Dr. Matthias Weidlich
Content
Sensing of data is a major trend these days. The number of devices that are connected to the Internet and continuously emit events is growing drastically. Event processing systems are a technology that helps to make sense of these events, by filtering event data, transforming events, and matching event query patterns against a set of incoming event streams. Yet, the increasing volume, velocity, variety and distribution of event sources imposes challenges for the design and implementation of event processing systems. To cope with these requirements, various competing approaches have been proposed in the literature, each taking particular design decisions.
Structure
In the first part of the course, lectures and recitations will focus on the fundamental models and algorithms of event processing systems. That includes common event models, languages for event processing, techniques to achieve robustness, and optimisations of event processing.
The second part of the course will be organised as a seminar. Each student will be asked to read a recent research paper on event processing (selection from a given list) and give a critical assessment of the approach presented in the paper in the form of a 45min presentation.
The course will be given in English. The first lecture will take place on Friday, 20th April, 2018.
Exam
There will be an oral exam at the end of the semester. To be eligible to take the exam, each student will be required to give a presentation (45min) on a research paper in the second part of the course.
Exam dates:
- Friday, August 3, 2018
- Monday, August 6, 2018
- Monday, August 13, 2018
- Friday, October 12, 2018
Please register for one of the slots (30min, some time between 9am and 3pm) available at each of these dates with Mrs Riemer (Rud 25, room 4.402).
Credit Points
The course counts for 5 LP and is open for: Informatik, Master of Science (M.Sc.) Informatik, Master of Education (M.Ed.) Wirtschaftsinformatik, Master of Science (M.Sc.). The related area of specialisation is "Daten- und Wissensmanagement".
Dates
VL | Fr 13-15 | RUD 26, Raum 1'307 |
UE | Fr 15-17 | RUD 26, Raum 1'307 |
Lecture Slides
- Lecture 0 - Organisation
- Lecture 1 - Introduction
- Lecture 2 - Pub/Sub
- Lecture 3 - Subscription Matching
- Lecture 4 - Notification Routing
- Lecture 5 - CQL
- Lecture 6 - CQL Joins
- Lecture 7 - Event Algebra
- Lecture 8 - Sequence Queries
- Lecture 9 - Optimisations (updated)
- Lecture 10 - Logic-based model
- Lecture 11 - Questions
Recitations
- Exercise 1
- Exercise 2 (updated link) (SimpleFireAlarm.zip)
- Exercise 3
- Exercise 4
- Exercise 5
- Exercise 6
- Exercise 7
- Exercise 8 (pda_ex8.zip, sorted100K.csv.gz)
Presentation Slots
Date | Topic and Presenters |
---|---|
June 29, 2018 |
1) Location-aware Pub/Sub (slides) 7) TRex |
July 06, 2018 |
9) Speculative Processing 12) Fault Tolerance 16) Handshake Join 17) Elastic Streaming |
July 13, 2018 |
4) Spatial Top-K Pub/Sub 10) Uncertain Events 11) Imprecise CEP 14) Semantic Optimisation |
|
Presentation Topics
The talks are scheduled for 29.06.2018, 06.07.2018, and 13.07.2018.
1) Location-aware Pub/Sub
Long Guo, Dongxiang Zhang, Guoliang Li, Kian-Lee Tan, Zhifeng Bao: Location-Aware Pub/Sub System: When Continuous Moving Queries Meet Dynamic Event Streams. SIGMOD Conference 2015: 843-857 paper link
2) Overlay Mending in Pub/Sub
Chen Chen, Roman Vitenberg, Hans-Arno Jacobsen: OMen: overlay mending for topic-based publish/subscribe systems under churn. DEBS 2016: 105-116 paper link
3) Pub/Sub via Gossiping
Pooya Salehi, Christoph Doblander, Hans-Arno Jacobsen: Highly-available content-based publish/subscribe via gossiping. DEBS 2016: 93-104 paper link
4) Spatial Top-K Pub/Sub
Xiang Wang, Wenjie Zhang, Ying Zhang, Xuemin Lin, Zengfeng Huang: Top-k spatial-keyword publish/subscribe over sliding window. VLDB J. 26(3): 301-326 (2017) paper link
5) ZStream
Yuan Mei and Samuel Madden. ZStream: A cost-based query processor for adaptively detecting composite events. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of data (SIGMOD ’09), pages 193–206, 2009. paper link
6) Secret
Nihal Dindar, Nesime Tatbul, Renee J. Miller, Laura M. Haas, and Irina Botan. Modeling the execution semantics of stream processing engines with secret. The VLDB Journal, 22(4):421–446, 2013. paper link
7) TRex
Gianpaolo Cugola and Alessandro Margara. Complex event processing with T-REX. J. Syst. Softw., 85(8):1709–1728, 2012. paper link
8) XML Streaming
Barzan Mozafari, Kai Zeng, Carlo Zaniolo: High-performance complex event processing over XML streams. SIGMOD Conference 2012: 253-264 paper link
9) Speculative Processing
Christopher Mutschler, Michael Philippsen: Adaptive Speculative Processing of Out-of-Order Event Streams. ACM Trans. Internet Techn. 14(1): 4:1-4:24 (2014) paper link
10) Uncertain Events
Segev Wasserkrug, Avigdor Gal, Opher Etzion, Yulia Turchin: Efficient Processing of Uncertain Events in Rule-Based Systems. IEEE Trans. Knowl. Data Eng. (TKDE) 24(1):45-58 (2012) paper link
11) Imprecise CEP
Haopeng Zhang, Yanlei Diao, Neil Immerman: Recognizing patterns in streams with imprecise timestamps. Inf. Syst. 38(8): 1187-1211 (2013) paper link
12) Fault Tolerance
Andre Martin, Thomas Knauth, Stephan Creutz, Diogo Becker de Brum, Stefan Weigert, Christof Fetzer, Andrey Brito: Low-Overhead Fault Tolerance for High-Throughput Data Processing Systems. ICDCS 2011: 689-699 paper link
13) Incremental Aggregation
K. Tangwongsan, M. Hirzel, S. Schneider, and K.-L. Wu. General Incremental Sliding-window Aggregation. Proc. VLDB Endow., 8(7):702–713, Feb. 2015 paper link
14) Semantic Optimisation
Luping Ding, Karen Works, Elke A. Rundensteiner: Semantic stream query optimization exploiting dynamic metadata. ICDE 2011:111-122 paper link
15) Pattern Sharing
Medhabi Ray, Chuan Lei, Elke A. Rundensteiner: Scalable Pattern Sharing on Event Streams. SIGMOD Conference 2016: 495-510 paper link
16) Handshake Join
Pratanu Roy, Jens Teubner, Rainer Gemulla: Low-Latency Handshake Join. PVLDB 7(9): 709-720 (2014) paper link
17) Elastic Streaming
Thomas Heinze, Mariam Zia, Robert Krahn, Zbigniew Jerzak, Christof Fetzer: An adaptive replication scheme for elastic data stream processing systems. DEBS 2015: 150-161 paper link
18) Saber
Alexandros Koliousis, Matthias Weidlich, Raul Castro Fernandez, Alexander L. Wolf, Paolo Costa, Peter R. Pietzuch: SABER: Window-Based Hybrid Stream Processing for Heterogeneous Architectures. SIGMOD Conference 2016: 555-569 paper link
See AGNES for further details: