I found some spare cycles to continue my DSL experiment.

I wasn’t happy with the initial version of my domain model, so I decided to make some changes in it. Although the model isn’t the most important part of the experiment I figured that it should at least be possible to compile it to continue the experiment.

After making some changes in the domain model, I ran into several problems when I tried to get the designer file in synch with the domain model. As a result executing the “translate template” step caused a lot of errors that I was not able to solve.

Let me try to explain what happened. Suppose we have the following model:

When I used the “modelisoft add in” to get the designer file (.dsldd) in synch with the model I received the following warning:

Warning : in MelCollection of shapemap for 'ResponseMessage', Concept

> 'ServiceOperation' has a role 'ResponseMessage' but it is not

Aggregate

I figured that I made some fundamental errors in my model but couldn’t find anything about it in the accompanying DSL Tools walkthrough documents. After some time I decided to contact Alan Cameron Wills who is a member of the DSL Tools team and is very active on the DSL Tools forum. Within half an hour he responded and solved my problem. Thanks Alan!

He told me that in the current version of the DSL Tools there’s the following restriction:

 All concepts of all shapes that appear on the diagram must be embeddings of the root class.

In practise this means that models should be very flat. What I had to do is create an embedding relation (solid line in the diagram below) from “service” (root of the model) to “RequestMessage” and “ResponseMessage”. After that I had to change the right-hand roles “Min” attribute to 1 and change the “Accepts” attribute of the left-hand roles to “All”

(changes are within red lines)

Maybe a bit strange but it works!

After applying the suggested solution my model looks like:

Note: the model isn’t finished yet. This picture is just to explain this restriction!

Alan also mentioned that the future release of the DSL Tools will have a much better “modelling experience”, till then we have to use the above described solution to get around this restriction.

After installing the DSL toolkit it is relatively easy to create and maintain the “domain model file” by drag and drop items on the designer that comes with the toolkit. After that, however, you have to get the “designer model file” in synch with the “domain model file”. This file is used to generate C# code that makes it possible to use your language in the experimental build version of Visual Studio.Net.

Getting this designer file in synch, really is a pain in the ass! It is a lot of work and for people not very familiar with the designer file structure (yet) it’s very likely to make mistakes with it. I assume Microsoft didn't have the time to solve this issue for this release of the DSL Toolkit but I hope they will solve it for the final release!?

Fortunately I found this Visual Studio add-in that makes it real easy to keep the two files in synch. Modelisoft, thank you for making my life a little less painful!

According to this posting of Daniel Cazzulino, Visual Studio Add-ins are dead. This might be true but this add-in is definitely alive and kicking!

Found some time to continue with my little experiment today.

After executing the installation of the earlier mentioned toolkits I was ready to perform my first steps in this totally new world. Installation all went fine, so that a good thing. One of the things I started with was reading the so called walkthrough documents (at this moment four docs are available). Unfortunately these documents are the only documentation that I can find for the DSL Toolkit. This is probably because of the CTP status of the toolkit but I assume this will get better in time. Although the walkthrough docs provide some guidance I think it is a bit poor. The docs mainly tell you which button to “click” but do not provide you with real background information about the DSL concepts. Definitely some room for improvement in this area!

My first experiences with building the example stated in the first “walkthrough” document, were a struggle. Most of this was because I didn’t take the time to execute all of the steps stated in the document. (I guess this is just a normal “men” thing to first start trying it yourself and only read the manual when you f.cked up). Second time I followed the docs very carefully but I was left with some errors. Luckily I found this posting on “”Xenopz Blog” that helped me with some remaining issues and that couldn’t be resolved by just reading the manual.

After some time I decided to start with the “Blank Language” template and start building my own DSL. This template provide you with two classes (“ConceptA” and “ConceptB”) and some relationships between them.

I started building my own DSL by simply renaming the two classes in the model (this is suggested in the walkthrough documents). One of the things that I find a little inconvenient (other words came up in mind first) in this is that after you changed the name of the classes and relationships in the designer you also have to perform some “find and replace” actions in one of the (many) xml files to make the designer for your language work. 

After some time this is what the first version of my model looks like.

During the creation I didn’t make any attempt to execute the “translate template” step or build the solution. Thinking back I’ am not sure if this is the smartest thing I could do.

My next step will be executing the “template translation” (to get the designer definition file in sink with the above domain model) and build and debug the language.

Haven’t tried this with this model but I expect some troubles with it. Getting the designer definition file in sink includes making manual changes in the (complicated) xml file. More on that later.

For, now Just a short explanation of the model (language):

At the root of the language I created the “service” class. Just below that you can see the “servicecomponent” class. I figured that all the other service components (interface, operation, etc.) might share some common needs (properties) in the future. Therefore all service components inherit from the “servicecomponent” class. 

The relation between the “servicecomponent” and the “servicecomponent” class is there to facilitate creating relationships between the several service components. Further you can see the “serviceAdpater” that acts as the class that couples the “serviceinterface” and “serviceimplementation” class. The “serviceinterface” class groups one ore more “serviceoperation” classes and is referenced by a “transportinterface” class. Finally the “serviceoperation” class references two “message” class to support the modelling of an input and output request for a service operation.

All still very limited, but it gave me a nice introduction in some of the possibilities of the DSL Toolkit. The above language can now be used in a so called “experimental build” environment to actually try the language and model a service. I haven’t tried that yet, so I’ am not sure what to expect!

When writing this post I suddenly realize that the two references from the “service operation” to the message class might not make sense. Maybe it is better to extend the message class with an attribute that indicates the type of the message (input or output). By using an enumeration for this I can make sure only the correct types can be selected. After that I can delete one of the referenced message classes and change the remaining reference to accept one or two messages.

To be continued…

Inspired by the several sessions of Tech-Ed Europe I decided to spend some effort on Domain Specific Languages (DSL) and Guidance Automation Toolkit (GAT). Although I have done some reading on Software Factories the area of building a Software Factory is quite new for me. Therefore I thought it would be interesting (for me and my current employer) to see the capabilities of these new technologies that are first members of Microsofts Software Factories initiative. To make it a little more appealing than just creating some sort of “Hello World” example I decided to try to combine it with some of my “day to day” work (which is architecting systems). Because I am definitely no expert in DSL’s I also decided to share some of my thoughts and issues to hopefully get some feedback to push me in the right directions.

Here is the idea: build a Domain Specific Language (DSL) to model “services”, perform some code generation and use GAT to provide some context-specific menus and templates within Visual Studio.Net IDE to speed up development based on the DSL.

Doesn’t sound too difficult so let’s give it a try. One of the things I do know about building a Software Factory is that you have to define what (reusable) parts can be identified within the domain. You also have to make sure the complete implementation details for all of the parts you are trying to model in a DSL and automate with GAT are available (and proven!).

For now, let’s assume these conditions are met. I will describe some of the parts I think are members of the service domain. Because my focus is on getting familiar with DSL and GAT, I will not describe all parts (and details) within the domain. As soon as I am totally up to speed with DSL and GAT (and still find it interesting) I will complete the model to make it a little more useful.

One of the most obvious parts that can be identified within this domain is the actual “service” itself. When translating this to a DSL this can be the “root” of the DSL. The service is composed out of other parts that I will call “service components” (it’s just a name).

Some of the service components I identified for this example:

• Service Interface; operations that are exposed by the service (service contract).
• Service Operation; definition of a service operation.
• Transport Interface; the representation of the service interface targeted at a specific technology (web service, enterprise service, remoting.)
• Message; definition of the messages exposed by service operation
• Service implementation; the actual implementation of the business logic
• Service Adapter; layer between the transport interface and the service implementation.

The above service components are all in their most basic form and they might not even cover the whole service domain. For example, it is very likely that in reality the message contains a data definition that is implemented as a data class based on an XSD definition. For now (this example) I will forget about al of these details. In this example a message is just represented by a class with a name that can be used as either input or output for a service operation.

I think, I will soon find out what parts (or details) I am missing here when I start modelling my DSL. During the modelling phase I will add only the absolute necessary properties and relationships.

The next step will be to prepare the environment by installing the DSL Toolkit and the Visual Studio 2005 SDK for Beta 2 and than we are ready to start experimenting with DSL.

To be continued…

Today I visited one of my customers to finish some work for a system we build. During the day some discussion came up about the tons of reference data that the system contains. The past few years the systems functionality increased enormously and so was the need for reference data. Although the systems isn’t based on a real service oriented design, most of the systems is exposed as web services because the functionality (and data) is also used by other systems at the customer’s location. At first the discussion covered system related stuff but soon changed to a more general discussion about how to handle data in a service oriented system.

Two weeks ago I visited a presentation of Ron Jacobs (Product Manager Patterns & Practises) at Tech-Ed 2005 Europe that was about handling data in a service oriented world. For me it was very easy to use some of his content to keep the discussion alive. Below some parts of the discussion:

The purpose of reference data is mainly to fill out requests and interpret response messages between services. Reference data is maintained by the service itself. The service implements a separate interface that can be used by service consumers to retrieve the reference data for a service. Further, reference data must be versioned, and gets updated periodically.

The service that maintains the reference data must provide the service consumers with a mechanism to retrieve the latest version of the reference data. The publisher of the reference data might implement some sort of a notification mechanism to notify it’s subscribers of a new version of the data. A service consumer can store the latest version of the reference in a local format to optimize the communication with the service. Of course this has some consequences for the development of the service because some sort functionality for this (storage of reference data) has to be developed.

Interaction between services includes request, response and reference data

Data (message) is immutable and therefore need a unique identifier. This identifier might include a version number of the data. Retrieving the data by using a unique id should ALWAYS return the same data no matter when the data is retrieved.

The data outside the services is represented as messages and is understood by both the sender and receiver. Special attention must be given to the schema that describes the message to reduce to possibility of misinterpretation of the message (data).

The data inside the service can be in any format that is appropriate for the service and is (and stays) private to the service.

Whenever data is needed by many services, choose one owner of the data. The owner is the only one that can make changes in the data. Whenever other services need an update of the data they send a request for an update to the owner. If the owner decides to make the update it publishes the changes to all subscribers. In this case the version number of the (reference) data increases. Request and response messages should be archived for auditing purposes.

Of course there is a lot more that can be said about data and services but the above topics kept us busy for most of the afternoon.

I attended a Chalk & Talk session with Steve Cook this morning about Software Factories. Just some of the things that were discussed:

Today building software is mostly craftsmanship. Compared to other industries the software industry isn’t very successful. Poor reuse.

Only if you know all the stable and reusable parts or architectural components it makes sense to start building a Software Factory for it. Start bottom up and start small otherwise you will probably fail. You have to know all the details about the parts you like to build a Software Factory for.

One of the basic ideas behind Software Factories: define only the unique features of the system that needs to build. (Other parts can be reused).

Most of the things we currently do are a waste of energy. All of it is redundant work. Automate all the rote and menial tasks. Let people do the interesting work.

The idea is to get a general development environment (VS.NET for example) and load the Domain Specific stuff in to tailor it. The Software Factory knows were you are and what you need at that time and delivers the right assets in the right context.

The Software Factory Schema defines a set of viewpoints and their relationships. A viewpoint describes how to look at things a view is the things you see.

It looks like Microsoft is working hard on some samples of Software Factories. In this talk Steve showed us a little of a Powerpoint presentation that demonstrates how to build an 'online Petshop' by using a Software Factory. Like Steve said, it's only a presentation but I assume they didn't created all the screenshot on the 106 sheets by hand

Today I attended a great session of Arvindra Sehmi and Beat Schwegler. First Arvindra talked about the “new” way of modelling connected systems. He explained there is a need for a “Service Oriented Architecture Model” (SOAD)  that sits between the business model and the technical model. While the business model is used to describe the needed capabilities (the “what” part) the SOAD model is al about modelling the contract of the service. At the end we have the Technical model that covers the technical details, like service interface, service implementation, etc. Of course the story covered a lot more details than the above description and it sounded pretty good.

The second part of the session was covered by Beat. He talked about the technical model. Currently he is working together with Cristhian Weyer and Maarten Mullender on a series of articles (first one can be found here) that will provide us with a set of recommendation for designing and implementing connected system (services). In this session Beat showed us that he is already working on the actual implementation of the recommendations. He showed us a Software Factory that is suports building services (by using the GAT). Pretty cool stuff!!!

From what you could see from the demo this series of articles (and implementation) will really show you some of the best practices for building these kinds of systems. Beat defined 8 steps that you have to do to design a service:

1. Design the data
2. Create the data classes
3. Design the messages
4. Design the interfaces
5. Create the transport interfaces (for example web services)
6. Create the adapters
7. Implement the internals
8. Connect the internals to the adapters

The artifacts of the above steps all formed a separate Visual Studio project within 1 solution.

I will not cover all of the above steps but some of the things were pretty interesting:

Within the first step (design the data) we create the XSD schema. This is actually the “data on the wire”. Based on these schemas the data classes are created by using for example XSD.exe. This of course is nothing new, but what we see most of the time now is that the product of step 1 is referred to as the messages. Beat actually showed us that in step 3 we create the actual message class that has a public field (result of step 2) which is actually the message payload. This messages class uses some techniques that I wrote about before to support for versioning and flexibility.

Another interesting thing is that in the “technical model”, when talking about the “service interface” we actually mean the interface definition (public interface IOrderService {}). Besides this the service has some “transport interfaces”, these are the actual implementations of the interface for a specific transport type (web service, Enterprise Service, etc.). Maybe nothing new here but is a good thing that is “written down” as a piece of guidance.

The last thing that I will write about is the introduction of “adapters”. These are actually a “layer” between the service interface (transport) and service internals. By using this technique the service implementation “knows” nothing about the actual message which makes it easier to change the message without breaking the service implementation. Of course in that case the adapter needs to be changed.

All of the artifacts that were created in the above 8 steps were supported by the “Software Factory”. Although it was very early work that Beat was showing it looked really cool, so makes sure to follow the series of articles!

While Tech-ed Europe is on its way it becomes clear to me that a lot of people are currently re-thinking how to implement their architectural requirements. A lot of people that I spoke are (of course) evaluating Visual Studio 2005 Beta 2, Indigo Beta 1 and WSE 3.0 CTP. Some of them (like me) are currently using some sort of a framework or toolset to support their architectural and service oriented needs.

During the talks I had with some people one of the issues that cam up was:  do we need another framework on top of .NET 2.0, Indigo and WS-*? And after that: Is the guidance that we are using now (for the current technology) still valid when all the new stuff is available? And if not, what do we need and when! In my opinion all interesting issues that sooner or later become very valid questions for a lot of us.

Some of the people I spoke, planned to simply migrate their current framework to .NET 2.0 when that is available. This wouldn’t be my preferred strategy. I think there is a relatively big change that simply migrating existing stuff doesn’t use the full power of the new technology. On the other hand, if it takes very little effort to migrate it can be a valid option.

Others thought that simply wait for Indigo to ship is the best option. In the meantime try to decide what you need (framework, guidance, etc.) and try to get that ready as soon as possible. For new systems in the timeframe between .NET 2.0 arrival and Indigo, simply use .NET 2.0 and if needed use WSE 3.0 to solve the WS-* issues.

In my current set of frameworks, guidance, etc. there is special attention for “separating the cross cutting concerns from the actual business logic” and separating the service interface from the service implementation. This is enforced by using a framework. I’ am still thinking how to solve these issues in the new (.NET 2.0, Indigo) world. For separating the service interface from the service implementation I think you don’t really need the help of a framework. This is just a pattern that you can apply.

A little different might be solving the “separating the cross cutting concerns from the business logic issue”. I have some ideas about this but I’ am not sure yet. Unfortunately I haven’t met any people here at Tech-Ed that have some ideas about this, but I’ am sure they must be around here somewhere. Have any ideas about this and like to have a discussion? Let me know!!

In his session Gianpaolo Carraro discussed the so called five pillars of connected systems. Up to now, when talking about connected systems and service orientation most of the discussion is about how to design services, the four tenets,  best practises, etc. Of course these issues are essential when it comes to connected systems but there are other issues that comes in play when thinking of connected systems. This is where the “five pillars of connected systems” comes in.

The “messaging” pillar is the most fundamental and covers the connectivity possibilities of systems in a standardized way. This is actually the pillars that is discussed over and over already (and will definitely need some more discussion in the coming period).

The other pillars that can be identified are:

·         Identity and Access

·         Data

·         Interaction

·         Workflow

The thing that Gianpaolo tried to explain is that all connected systems that we will build in the future need (to some extend) attention on all of the five pillars.

After discussing the pillars briefly, he also presented his “6 rules for success when building connected systems”.  5 of the rules each apply to one of the 5 pillars and 1 rule just concerns some generic issues not related to any of the pillars in particular. It was actually this one (generic) rule that makes sense to me.

This rule is about “accommodate and anticipate for change”. Sooner or later the system you are building will change, so it’s best to design for that. Some of the techniques he mentioned for that is; make use of abstract factories, provider model, pipeline architecture, separation of concerns and inception points.

I had a little more difficulty with the other 5 rules for success so I will not quote them here. To me they didn’t seem to be “rules” but looked more like summing up some attention points. I think some more details are needed in this area to become real rules.

At the end of the presentation it became clear that all of the 5 pillars will get some extra attention in the near future and the so called rules will be supported by (new) features of some of the Microsoft tools or technologies.

More information about the 5 pillars of connected systems can be found on the blog of Gianpaolo. Not everything is there yet, but according to his blog will arrive soon!

Update: changed the name of the “ intergration pillar”  into “ interaction pillar”. These postings are written during the Tech Ed talks, not a lot of time to validate the content. Thanks Carlo for commenting on this!

One of the sessions that I attended today at Tech-Ed Europe is “Building and using Software Factories” with Steve Cook and Annie Matthewman.

I have been reading quite some stuff about Software Factories lately. I even started the book called “Software Factories” but haven’t finished it yet. Although there is quite some information available it kind of bothered me that most of it is purely theoretical, I haven’t seen any real implementation of a Software Factory yet (until now).

Steve first gave some basic information about Software Factories. One of the, maybe obvious, things he stated was that is it only useful to build a Software Factory if you build similar systems every time. If that is the case, the next thing you have to define is a “package” of reusable knowledge that eventually can form the Software Factory. So, in order to be able to build a Software Factory you first need to know what is available and how it can be reused.

The reusable items can vary from a piece of documentation, an architecture baseline, code snippets, wizards, etc.

Summarized, a Software Factory can be described as:

·         (customizable) structured collection of assets.

·         Software Schema that defines the context for the assets.

·         Explains people what to do.

·         Helps people do it.

One of the goals for Software Factories is: easy things should easy and difficult things should be possible.

The second part of the session covered a demo of a real implemented Software Factory. The Software Factory provided guidance for building a “wizard like” windows application communicating with web services.

To support building Software Factories, there are actually two new “technologies/toolkits”. One of them concerns Domain Specific Languages (DSL) and helps building DSL end the appropriate DSL designers. The other toolkit is the Guidance Automation Toolkit (GAT).

For the Software Factory that was demonstrated Steve and Annie actually build a “new” DSL for this problem domain and an appropriate designer for the new DSL. Building this wasn’t covered in this session, but will be covered in another session.

The GAT is a relatively new toolkit that is downloadable from the Microsoft website. The GAT can be best described of a Software Factory for building Software Factories.

From what I saw in the demo one of the pieces of GAT is  a (very) big XML file that describes the activities, actions and their contexts that together form the  (part of) Software Factory. When a Software Factory is build by using the GAT the end result all integrates very nice with Visual Studio.NET helping the developer building the piece of software that is supported by the Software Factory.

I haven’t actually spent time on evaluating the GAT yet but the demo looked pretty cool so I think I will give it a try soon.