Inspired by the several sessions of Tech-Ed Europe I decided to spend some effort on Domain Specific Languages (DSL) and Guidance Automation Toolkit (GAT). Although I have done some reading on Software Factories the area of building a Software Factory is quite new for me. Therefore I thought it would be interesting (for me and my current employer) to see the capabilities of these new technologies that are first members of Microsofts Software Factories initiative. To make it a little more appealing than just creating some sort of “Hello World” example I decided to try to combine it with some of my “day to day” work (which is architecting systems). Because I am definitely no expert in DSL’s I also decided to share some of my thoughts and issues to hopefully get some feedback to push me in the right directions.

Here is the idea: build a Domain Specific Language (DSL) to model “services”, perform some code generation and use GAT to provide some context-specific menus and templates within Visual Studio.Net IDE to speed up development based on the DSL.

Doesn’t sound too difficult so let’s give it a try. One of the things I do know about building a Software Factory is that you have to define what (reusable) parts can be identified within the domain. You also have to make sure the complete implementation details for all of the parts you are trying to model in a DSL and automate with GAT are available (and proven!).

For now, let’s assume these conditions are met. I will describe some of the parts I think are members of the service domain. Because my focus is on getting familiar with DSL and GAT, I will not describe all parts (and details) within the domain. As soon as I am totally up to speed with DSL and GAT (and still find it interesting) I will complete the model to make it a little more useful.

One of the most obvious parts that can be identified within this domain is the actual “service” itself. When translating this to a DSL this can be the “root” of the DSL. The service is composed out of other parts that I will call “service components” (it’s just a name).

Some of the service components I identified for this example:

• Service Interface; operations that are exposed by the service (service contract).
• Service Operation; definition of a service operation.
• Transport Interface; the representation of the service interface targeted at a specific technology (web service, enterprise service, remoting.)
• Message; definition of the messages exposed by service operation
• Service implementation; the actual implementation of the business logic
• Service Adapter; layer between the transport interface and the service implementation.

The above service components are all in their most basic form and they might not even cover the whole service domain. For example, it is very likely that in reality the message contains a data definition that is implemented as a data class based on an XSD definition. For now (this example) I will forget about al of these details. In this example a message is just represented by a class with a name that can be used as either input or output for a service operation.

I think, I will soon find out what parts (or details) I am missing here when I start modelling my DSL. During the modelling phase I will add only the absolute necessary properties and relationships.

The next step will be to prepare the environment by installing the DSL Toolkit and the Visual Studio 2005 SDK for Beta 2 and than we are ready to start experimenting with DSL.

To be continued…

Today I visited one of my customers to finish some work for a system we build. During the day some discussion came up about the tons of reference data that the system contains. The past few years the systems functionality increased enormously and so was the need for reference data. Although the systems isn’t based on a real service oriented design, most of the systems is exposed as web services because the functionality (and data) is also used by other systems at the customer’s location. At first the discussion covered system related stuff but soon changed to a more general discussion about how to handle data in a service oriented system.

Two weeks ago I visited a presentation of Ron Jacobs (Product Manager Patterns & Practises) at Tech-Ed 2005 Europe that was about handling data in a service oriented world. For me it was very easy to use some of his content to keep the discussion alive. Below some parts of the discussion:

The purpose of reference data is mainly to fill out requests and interpret response messages between services. Reference data is maintained by the service itself. The service implements a separate interface that can be used by service consumers to retrieve the reference data for a service. Further, reference data must be versioned, and gets updated periodically.

The service that maintains the reference data must provide the service consumers with a mechanism to retrieve the latest version of the reference data. The publisher of the reference data might implement some sort of a notification mechanism to notify it’s subscribers of a new version of the data. A service consumer can store the latest version of the reference in a local format to optimize the communication with the service. Of course this has some consequences for the development of the service because some sort functionality for this (storage of reference data) has to be developed.

Interaction between services includes request, response and reference data

Data (message) is immutable and therefore need a unique identifier. This identifier might include a version number of the data. Retrieving the data by using a unique id should ALWAYS return the same data no matter when the data is retrieved.

The data outside the services is represented as messages and is understood by both the sender and receiver. Special attention must be given to the schema that describes the message to reduce to possibility of misinterpretation of the message (data).

The data inside the service can be in any format that is appropriate for the service and is (and stays) private to the service.

Whenever data is needed by many services, choose one owner of the data. The owner is the only one that can make changes in the data. Whenever other services need an update of the data they send a request for an update to the owner. If the owner decides to make the update it publishes the changes to all subscribers. In this case the version number of the (reference) data increases. Request and response messages should be archived for auditing purposes.

Of course there is a lot more that can be said about data and services but the above topics kept us busy for most of the afternoon.

I attended a Chalk & Talk session with Steve Cook this morning about Software Factories. Just some of the things that were discussed:

Today building software is mostly craftsmanship. Compared to other industries the software industry isn’t very successful. Poor reuse.

Only if you know all the stable and reusable parts or architectural components it makes sense to start building a Software Factory for it. Start bottom up and start small otherwise you will probably fail. You have to know all the details about the parts you like to build a Software Factory for.

One of the basic ideas behind Software Factories: define only the unique features of the system that needs to build. (Other parts can be reused).

Most of the things we currently do are a waste of energy. All of it is redundant work. Automate all the rote and menial tasks. Let people do the interesting work.

The idea is to get a general development environment (VS.NET for example) and load the Domain Specific stuff in to tailor it. The Software Factory knows were you are and what you need at that time and delivers the right assets in the right context.

The Software Factory Schema defines a set of viewpoints and their relationships. A viewpoint describes how to look at things a view is the things you see.

It looks like Microsoft is working hard on some samples of Software Factories. In this talk Steve showed us a little of a Powerpoint presentation that demonstrates how to build an 'online Petshop' by using a Software Factory. Like Steve said, it's only a presentation but I assume they didn't created all the screenshot on the 106 sheets by hand

Today I attended a great session of Arvindra Sehmi and Beat Schwegler. First Arvindra talked about the “new” way of modelling connected systems. He explained there is a need for a “Service Oriented Architecture Model” (SOAD)  that sits between the business model and the technical model. While the business model is used to describe the needed capabilities (the “what” part) the SOAD model is al about modelling the contract of the service. At the end we have the Technical model that covers the technical details, like service interface, service implementation, etc. Of course the story covered a lot more details than the above description and it sounded pretty good.

The second part of the session was covered by Beat. He talked about the technical model. Currently he is working together with Cristhian Weyer and Maarten Mullender on a series of articles (first one can be found here) that will provide us with a set of recommendation for designing and implementing connected system (services). In this session Beat showed us that he is already working on the actual implementation of the recommendations. He showed us a Software Factory that is suports building services (by using the GAT). Pretty cool stuff!!!

From what you could see from the demo this series of articles (and implementation) will really show you some of the best practices for building these kinds of systems. Beat defined 8 steps that you have to do to design a service:

1. Design the data
2. Create the data classes
3. Design the messages
4. Design the interfaces
5. Create the transport interfaces (for example web services)
6. Create the adapters
7. Implement the internals
8. Connect the internals to the adapters

The artifacts of the above steps all formed a separate Visual Studio project within 1 solution.

I will not cover all of the above steps but some of the things were pretty interesting:

Within the first step (design the data) we create the XSD schema. This is actually the “data on the wire”. Based on these schemas the data classes are created by using for example XSD.exe. This of course is nothing new, but what we see most of the time now is that the product of step 1 is referred to as the messages. Beat actually showed us that in step 3 we create the actual message class that has a public field (result of step 2) which is actually the message payload. This messages class uses some techniques that I wrote about before to support for versioning and flexibility.

Another interesting thing is that in the “technical model”, when talking about the “service interface” we actually mean the interface definition (public interface IOrderService {}). Besides this the service has some “transport interfaces”, these are the actual implementations of the interface for a specific transport type (web service, Enterprise Service, etc.). Maybe nothing new here but is a good thing that is “written down” as a piece of guidance.

The last thing that I will write about is the introduction of “adapters”. These are actually a “layer” between the service interface (transport) and service internals. By using this technique the service implementation “knows” nothing about the actual message which makes it easier to change the message without breaking the service implementation. Of course in that case the adapter needs to be changed.

All of the artifacts that were created in the above 8 steps were supported by the “Software Factory”. Although it was very early work that Beat was showing it looked really cool, so makes sure to follow the series of articles!

While Tech-ed Europe is on its way it becomes clear to me that a lot of people are currently re-thinking how to implement their architectural requirements. A lot of people that I spoke are (of course) evaluating Visual Studio 2005 Beta 2, Indigo Beta 1 and WSE 3.0 CTP. Some of them (like me) are currently using some sort of a framework or toolset to support their architectural and service oriented needs.

During the talks I had with some people one of the issues that cam up was:  do we need another framework on top of .NET 2.0, Indigo and WS-*? And after that: Is the guidance that we are using now (for the current technology) still valid when all the new stuff is available? And if not, what do we need and when! In my opinion all interesting issues that sooner or later become very valid questions for a lot of us.

Some of the people I spoke, planned to simply migrate their current framework to .NET 2.0 when that is available. This wouldn’t be my preferred strategy. I think there is a relatively big change that simply migrating existing stuff doesn’t use the full power of the new technology. On the other hand, if it takes very little effort to migrate it can be a valid option.

Others thought that simply wait for Indigo to ship is the best option. In the meantime try to decide what you need (framework, guidance, etc.) and try to get that ready as soon as possible. For new systems in the timeframe between .NET 2.0 arrival and Indigo, simply use .NET 2.0 and if needed use WSE 3.0 to solve the WS-* issues.

In my current set of frameworks, guidance, etc. there is special attention for “separating the cross cutting concerns from the actual business logic” and separating the service interface from the service implementation. This is enforced by using a framework. I’ am still thinking how to solve these issues in the new (.NET 2.0, Indigo) world. For separating the service interface from the service implementation I think you don’t really need the help of a framework. This is just a pattern that you can apply.

A little different might be solving the “separating the cross cutting concerns from the business logic issue”. I have some ideas about this but I’ am not sure yet. Unfortunately I haven’t met any people here at Tech-Ed that have some ideas about this, but I’ am sure they must be around here somewhere. Have any ideas about this and like to have a discussion? Let me know!!

In his session Gianpaolo Carraro discussed the so called five pillars of connected systems. Up to now, when talking about connected systems and service orientation most of the discussion is about how to design services, the four tenets,  best practises, etc. Of course these issues are essential when it comes to connected systems but there are other issues that comes in play when thinking of connected systems. This is where the “five pillars of connected systems” comes in.

The “messaging” pillar is the most fundamental and covers the connectivity possibilities of systems in a standardized way. This is actually the pillars that is discussed over and over already (and will definitely need some more discussion in the coming period).

The other pillars that can be identified are:

·         Identity and Access

·         Data

·         Interaction

·         Workflow

The thing that Gianpaolo tried to explain is that all connected systems that we will build in the future need (to some extend) attention on all of the five pillars.

After discussing the pillars briefly, he also presented his “6 rules for success when building connected systems”.  5 of the rules each apply to one of the 5 pillars and 1 rule just concerns some generic issues not related to any of the pillars in particular. It was actually this one (generic) rule that makes sense to me.

This rule is about “accommodate and anticipate for change”. Sooner or later the system you are building will change, so it’s best to design for that. Some of the techniques he mentioned for that is; make use of abstract factories, provider model, pipeline architecture, separation of concerns and inception points.

I had a little more difficulty with the other 5 rules for success so I will not quote them here. To me they didn’t seem to be “rules” but looked more like summing up some attention points. I think some more details are needed in this area to become real rules.

At the end of the presentation it became clear that all of the 5 pillars will get some extra attention in the near future and the so called rules will be supported by (new) features of some of the Microsoft tools or technologies.

More information about the 5 pillars of connected systems can be found on the blog of Gianpaolo. Not everything is there yet, but according to his blog will arrive soon!

Update: changed the name of the “ intergration pillar”  into “ interaction pillar”. These postings are written during the Tech Ed talks, not a lot of time to validate the content. Thanks Carlo for commenting on this!

One of the sessions that I attended today at Tech-Ed Europe is “Building and using Software Factories” with Steve Cook and Annie Matthewman.

I have been reading quite some stuff about Software Factories lately. I even started the book called “Software Factories” but haven’t finished it yet. Although there is quite some information available it kind of bothered me that most of it is purely theoretical, I haven’t seen any real implementation of a Software Factory yet (until now).

Steve first gave some basic information about Software Factories. One of the, maybe obvious, things he stated was that is it only useful to build a Software Factory if you build similar systems every time. If that is the case, the next thing you have to define is a “package” of reusable knowledge that eventually can form the Software Factory. So, in order to be able to build a Software Factory you first need to know what is available and how it can be reused.

The reusable items can vary from a piece of documentation, an architecture baseline, code snippets, wizards, etc.

Summarized, a Software Factory can be described as:

·         (customizable) structured collection of assets.

·         Software Schema that defines the context for the assets.

·         Explains people what to do.

·         Helps people do it.

One of the goals for Software Factories is: easy things should easy and difficult things should be possible.

The second part of the session covered a demo of a real implemented Software Factory. The Software Factory provided guidance for building a “wizard like” windows application communicating with web services.

To support building Software Factories, there are actually two new “technologies/toolkits”. One of them concerns Domain Specific Languages (DSL) and helps building DSL end the appropriate DSL designers. The other toolkit is the Guidance Automation Toolkit (GAT).

For the Software Factory that was demonstrated Steve and Annie actually build a “new” DSL for this problem domain and an appropriate designer for the new DSL. Building this wasn’t covered in this session, but will be covered in another session.

The GAT is a relatively new toolkit that is downloadable from the Microsoft website. The GAT can be best described of a Software Factory for building Software Factories.

From what I saw in the demo one of the pieces of GAT is  a (very) big XML file that describes the activities, actions and their contexts that together form the  (part of) Software Factory. When a Software Factory is build by using the GAT the end result all integrates very nice with Visual Studio.NET helping the developer building the piece of software that is supported by the Software Factory.

I haven’t actually spent time on evaluating the GAT yet but the demo looked pretty cool so I think I will give it a try soon.

This is my schedule for Tech-Ed Europe 2005. Looking forward to some (in my opinion) very interesting sessions and speakers. The sessions are a mix of the “connected systems“  and “architecture“ track. For most of the timeslots I choose an alternative session, just in case there are no seats left in the room when I arrive.

 If you happen to be in the same sessions, say hello, I’ am the one carrying a Tech-Ed bag J

Because I was assigned to project it wasn’t sure if I could make it to Tech Ed 2005 Europe. Luckily, we found a replacement for me at the project J. So I will be there!

Interested in more in-depth details about the implementation of the WS-I Basic security profile application? 

Go read the blog of Hernan de Lahitte who was member of the dev team. I a series of posts he will share his experiences in building this sample and talk about the issues that came up.