StreamInsight Series

Over the next few weeks/months I am planning to do a series of blog-posts about Microsoft’s entry into the Stream Data and Complex Event Processing domain; StreamInsight.

I will update this page with links as I do publish the individual posts.

• Stream and Complex Event Processing from a Relational Guy’s Eye (published September 18, 2009)

StreamInsight Stuff

As you probably know, Microsoft’s implementation of Complex Event Processing (CEP) – called StreamInsight was released as a Community Technology Preview (CTP) a couple of days ago. The release was CTP2 (I have no idea when and if CTP1 was released). Anyway, I have a love for anything message based – and CEP is definitely message based, so I was obviously over this as a “rash”.

The last few days I have been playing around with researching this, and trying to come to grips with what it is all about. Part of the release are some samples and some help files in the shape of a chm file. It is definitely a lot to get your head around, and I will post more in the coming weeks.  However, here is a heads up if you cannot get some of the samples to work;

When I started with StreamInsight I opened up the samples, and actually read the README.txt files, as well as part of the chm help document (I cannot be a developer, reading the help …). I tried first the ObjectModel sample app. It compiled OK (cool, let’s ship), and when I tried to run it I actually got some output to the console. At first glance it looked OK, but looking at the output a bit more closely I saw I had some exceptions there, looking something like this:

Query Exception: Microsoft.ComplexEventProcessing.Engine.OperatorExecutionExcep
tion: The adapter 'sensorInput' of type 'CepSamples.InputAdapters.TextFileInterv
alInput', query 'TrafficSensorQuery', failed to start. ---> System.FormatExcepti
on: String was not recognized as a valid DateTime.
at System.DateTimeParse.Parse(String s, DateTimeFormatInfo dtfi, DateTimeStyl
es styles)
at System.Convert.ToDateTime(String value, IFormatProvider provider)
at CepSamples.InputAdapters.TextFileIntervalInput.CreateEventFromLine(String
line) in C:\Program Files\Microsoft StreamInsight CTP2\docs_samples\Samples\Inpu
tAdapters\TextFileInputAdapter\TextFileIntervalInput.cs:line 247


I looked into the TextFileIntervalInput. cs file at around the line number mentioned above, and saw some code looking like this:

// set Start time
evt.StartTime = Convert.ToDateTime(split[0], CultureInfo.CurrentCulture);
evt.EndTime = Convert.ToDateTime(split[1], CultureInfo.CurrentCulture);


I then looked at the input data file for this sample, TrafficSensor.csv, and saw that it had some date data formatted according to “en-US” culture. However, I am running under “en-UK” culture. So when the Convert.ToDateTime code runs it will fail, because it cannot convert to my culture.

I fixed that by using DateTime.TryParse instead (I will leave it up to you – my two readers – to figure out the syntax), and that sorted that problem. I then had the same issue with a second input adapter file: TextFileEdgeInput.cs, and fixed it in the same way. Note, you may get the eception in the TextFileEdgeInput.cs exception before the exception in TextFileIntervalInput.cs.

So, if you run under non-US culture this is an issue you can run into, but it looks it is just for the ObjectModel sample. The other samples seem to run OK.

I am really intrigued by StreamInsight and will drill down deeper into it, and also try to blog my findings along the way. If you are interested in this, please leave a comment.

Oh, and if you want it from the “horses mouth”, the StreamInsight team at Microsoft has a blog here. There is also a support forum here.

SQL Server 2008 R2 August CTP

Yesterday I downloaded and installed the August CTP of SQL Server 2008 R2, and today I played around with it for a while. So, what are my impressions…

Well, from a perspective of being a relational dev and internals guy, my immediate response is … “yawn – where is the beef”. I.e, it is not much there, and I doubt we will see much more in coming releases. However, if I were a BI / reporting guy I’d be over the moon, and definitely look forward to future CTP’s! Even if I were a (wait for it …) DBA I would be fairly interested.

I will let you decide for yourself what is interestimg for you, but one thing that is not in the CTP at the moment but is promised (and keeps me interested) is StreamInsight (based on Complex Event Processing). This will be part of SQL Servr 2008 R2. Coming from the financial industry and dealing with message based applications (that’s why I love SQL Server Service Broker), this is something I am really interested in. So, even if you are a T-SQL / internals guy, do not despair – there may be something for us as well.