Using Complex Event Processing (CEP) with Microsoft StreamInsight to Analyze Twitter Tweets 1: Introduction and Resources

Introduction and Resources

Welcome to my 7-post series on Complex Event Processing with Mircosoft StreamInsight. Enjoy!

Microsoft StreamInsight is a very lightweight, powerful and for .NET developers easy to use Complex Event Processing (CEP) engine. It uses .NET LINQ queries to write queries on data streams.

With minimal effort, StreamInsight allows you to answer questions such as “what is the one minute aggregate of my value X grouped by dimension D”. This series of blog posts explains the concepts behind CEP and StreamInsight and implements a sample application that can be used to analyze live data from Twitter.

Index of this blog series:

Part 1 (03.10.13): CEP with StreamInsight and Twitter 1:
Introduction and Resources

Part 2 (06.10.13): CEP with StreamInsight and Twitter 2:
What are CEP and StreamInsight

Part 3 (13.10.13): CEP with StreamInsight and Twitter 3:
StreamInsight Programming

Part 4 (20.10.13): CEP with StreamInsight and Twitter 4:
Reactive Programming

Part 5 (27.10.13): CEP with StreamInsight and Twitter 5:
The Sample Application 1

Part 6 (03.11.13): CEP with StreamInsight and Twitter 6:
The Sample Application 2

Part 7 (10.11.13): CEP with StreamInsight and Twitter 7:
The Sample Application 3

The best resources that I found:

Video: “What’s new in Complex Event Processing using StreamInsight for SqlServer 2.1” by Torsten Grabs, Principal Program Manager Lead, Microsoft

Video: “My Campus Walk: I walk, StreamInsight analyzes, Worldwide Telescope visualizes.” Interesting project about visualization of a walk path from geo-spatial data processed by StreamInsight.

Video: “Heat mapping H5N1 Outbreaks with StreamInsight and Binq Maps”. Really awesome demo application that feeds processed stream data to a Bing Maps application.


A tool I absolutely recommend, especially for StreamInsight Development,  is LINQPad ( LINQPad includes the following StreamInsight sample packages that can be executed and played with:

      • 101 StreamInsight Samples
      • Hitchhikers Guide to StreamInsight
      • Performance Counter Samples
      • Reactive Extensions
      • LINQ in Action

Important Note:
If you use the free version of LINQPad you will not get any intellisense or auto-complete functionality. In my opinion this makes it useless. As soon as I bought a personal license (about USD30), LINQPad turned into one of my favorite dev tools of all times. It is extremely lightweight and understands C#, VB, F#, SQL, LINQ, ADO.NET, StreamInsight, etc.


TwitterStream Sample: The application that we are going to develop in the following blog posts:


Hitchhikers Guide to StreamInsight Queries: Very comprehensive guide to writing queries in StreamInsight. Code is included in the codeplex download below. Samples are included in LINQPad.

StreamInsight Product Team Samples 2.1

StreamInsight Product Team Samples 2.0
Very important collection of samples such as: TextFileReaderAdapter, TextFileWriterAdapter, SqlAdapter, WcfAdapter and lots of applications.

JAhlen StreamInsight Example Adapters v2.1
Stock trading sample application.



Blog: Official Blog of the Microsoft StreamInsight Dev Team

Blog: StreamInsight explained to non-programmers

Blog: Getting Started with StreamInsight 2.1

Technet: Microsoft StreamInsight

MSDN: Microsoft StreamInsight

MSDN Forum: SQL Server –> Microsoft StreamInsight


Next Post:

Using Complex Event Processing (CEP) with Microsoft StreamInsight to Analyze Twitter Tweets 2: What are CEP and StreamInsight?

ToolTips: Unlocking Files with Sysinternals Process Monitor
Using Complex Event Processing (CEP) with Microsoft StreamInsight to Analyze Twitter Tweets 2: What are CEP and StreamInsight?

0 Comment


This site uses Akismet to reduce spam. Learn how your comment data is processed.

15 49.0138 8.38624 1 0 4000 1 300 0