It’s been a while since I wrote about the progress of my DSL experiment. Why does it take so long? Is it so hard to build a DSL? Well, actually sometimes I think it is. It takes some time to really understand what is going on in all the files that are part of the DSL solution. Most of this is because of the technology preview status of the toolkit and therefore the limited documentation that is available. In the current release some of the files have to be maintained by hand which can only be done if you have a good understanding of the xml structure of the files. Most of this manual work will disappear in the next release of the toolkit. Luckily there is an active forum and the DSL Toolkit team that is willing to help!

But, let’s continue on the status. Three weeks ago Christian Weyer of Thinktecture contacted me to let me know he liked the work I am doing in my DSL experiment. As you can read in this post on his blog Christian is thinking about using DSL’s for modelling service interfaces and data contracts for some time. We decided to discuss this “service modelling” domain via email and to join forces and build this language together.

So, the last few weeks we discussed the artifacts that can be identified within the service domain and how to model them in our DSL. As a result some of the names of the concepts that I modelled in an early version of the language have changed to make them more descriptive and some new concepts are introduced. Currently I am making the last changes in the model. After that we have to finish the designer for the language and finally we can do some code generation. In another post I will show a next version of the model and hopefully explain some of the ideas behind it.

Hopes this clears thinks up. The experiment is not dead. Expect to hear of it a lot more in the coming days, weeks, months.

To be continued…

Now that I am in the process of building my own DSL for modelling services I started thinking about the usage of DSL’s a little more. Suddenly I see all kind of opportunities for new DSL’s but also some issues so I thought why not share them with you and ask for your opinion.

Thinking about the “service DSL” I am experimenting with a little further, a possible next step would be to build a language that models the cross cutting concerns for services.

Suppose we have all (or at least some J) our crosscutting concerns modelled in a DSL and we are able to simply configure a service with the crosscutting concerns by using the DSL designer and “drop” the crosscutting concerns on the service area. (For now, let’s assume it is possible to attach the crosscutting concern to the service by only using a configuration file without the need to make any changes in code.)

A language like this would make it relatively easy to generate the necessary config file that actually attaches the crosscutting concerns to the services that are “hosted” in the WSE pipeline, Indigo Windows Communication Foundation or some other piece of infrastructure.

Although being able to attach crosscutting concerns to services would be great for creating the initial configuration file the next step is of course to also maintain these “settings” for production software. Unfortunately there are some burdens on our (my) way to make this happen. For example, the current release of the DSL tools only support “one-way” generation. This makes it impossible to generate a model based on an existing configuration file. I assume this will change in the future. Is it Microsoft? Similar functionality is already available in the VS.NET class designer.

A second issue might be the hosting of the DSL. Until now the DSL designers can only be hosted within the Visual Studio.Net IDE. If Microsoft isn’t going to change this in the future, IT Pro, responsible for the running the services in production, have to use the Visual Studio.Net environment to maintain their configuration files. Forcing the IT Pro department to use Visual Studio.NET for maintaining and supporting software (configuration) might sound strange but this might become reality sooner or later. If we believe Microsoft IT Pro will start using VS.Net to create and maintain their logical data centre models in the near future. On the other hand it might be good to have some sort of a light weight hosting environment for DSL’s.

A third thing that comes to mind is the “size of the DSL” and the possibility to share concepts between DSL’s. One of the goals of a DSL is to target a certain problem domain. In my example, we have a DSL for modelling services and one for attaching crosscutting concerns to services. Both languages share common issues and therefore in theory can be integrated in one language. But wait, this would mean that our IT Pro department that is only supposed to use the “crosscutting concern part” of the language also has the possibility to use the “modelling services” part of the language. Modelling (and building) services is definitely no responsibility of IT Pro and thus integrating the two different features (modelling and crosscutting concerns) into one language might seem reasonable from an domain perspective but doesn’t reflect the responsibilities of the IT Pro employee very well.

An obvious option is to keep the two features in separate languages. In this case however we have the “service concept” that is part of both languages. Of course we can model (and maintain) this concept in both languages but is that what we want?! The ideal situation would be to have the possibility to “import” concepts from one DSL into another DSL and have the capability to use some sort of access control on the details of a concept. In this example we could have one DSL targeting “services” but make sure the IT Pro department can only use the “service concept” for attaching crosscutting concerns to services and cannot view (and use) all details that the developer can use to model the service.

Of course the above issues are described by using the “service” domain I am experimenting with but I assume they can be translated to a more general case.

What do you think, should we “worry” about issues as “one way” generating, DSL hosting environments and sharing concepts between DSL’s or just focus on things we can do with the current DSL technology?

Just a quick update:

I received two emails of people asking me if my little DSL experiment has stopped. The answer is NO!

Because of some positive feedback I decided to spend more time on the design of the language. At first, the model was mostly meant to help me understand the DSL toolkit, now I’ am actually trying to build a usable language.  

That’s why it is a bit quite on the topic, just because I don’t think it is very interesting to write about every small change in the language design.

In my opinion the experiment is getting more interesting and definitely more useful.

To be continued…

Although my model isn’t quit finished yet, I decided to see how my language behaves in the Visual Studio experimental build environment. You can enter this environment by executing “Start without Debugging” from the VS.Net “Debug” menu in the VS.Net environment you are designing your language in.

Within the experimental build environment I created a new “serviceoperation” class (GetCustomerDetails). After that I created a new “responsemessage” (GetCustomerDetailsResponse) class. And last, a relation between the “serviceoperation” and the “responsemessage”. The model for this looked something like this (actually the little collapse picture in the right corners wasn’t there at first).

To be honest I wasn’t quite happy whit the “modelling experience”. Besides the poor colours of the classes (which can be easily changed) I wasn’t sure if this is the best way of using the designer for this part of the language. Therefore I had another look at the possibilities and found out about “compartment shapes”. A compartment shape can be defined as a shape (class) that can host another shape (class).

Here is what I did; I changed my original “serviceoperation” class into a compartment shape that is capable of “hosting” a “reguestmessage” shape and an “reponsemessage” shape. To create a compartment shape you have to make some changes in the designer xml file. Not that straight forward but I’ am sure this will change in the next release of the toolkit!?

As you can see in the picture below the “serviceoperation” class now hold to “placeholders”. One for the “requestmessage” class and one for the “responsemessage” class.

By right clicking the response message placeholder you actually get a menu option to create the “responsemessage” class (see below). Till now my messages classes in my language only have a “name” attribute. I have to test what happens if my messages classes contains a little more attributes. I’ am not sure how this is represented in within the designer and how easy it is to view or change example the message attributes. I assume only the name if the message will be displayed in the serviceoperation class and by double clicking you can change the attributes but a little more effort is needed on this. (In the image below I right clicked the RequestMessage placeholder to add an new “requestmessage“ class).

For now, imagine yourself designing (and generating the code) your services with a language like this. I think it’s pretty cool!

To be continued…

I found some spare cycles to continue my DSL experiment.

I wasn’t happy with the initial version of my domain model, so I decided to make some changes in it. Although the model isn’t the most important part of the experiment I figured that it should at least be possible to compile it to continue the experiment.

After making some changes in the domain model, I ran into several problems when I tried to get the designer file in synch with the domain model. As a result executing the “translate template” step caused a lot of errors that I was not able to solve.

Let me try to explain what happened. Suppose we have the following model:

When I used the “modelisoft add in” to get the designer file (.dsldd) in synch with the model I received the following warning:

Warning : in MelCollection of shapemap for 'ResponseMessage', Concept

> 'ServiceOperation' has a role 'ResponseMessage' but it is not

Aggregate

I figured that I made some fundamental errors in my model but couldn’t find anything about it in the accompanying DSL Tools walkthrough documents. After some time I decided to contact Alan Cameron Wills who is a member of the DSL Tools team and is very active on the DSL Tools forum. Within half an hour he responded and solved my problem. Thanks Alan!

He told me that in the current version of the DSL Tools there’s the following restriction:

 All concepts of all shapes that appear on the diagram must be embeddings of the root class.

In practise this means that models should be very flat. What I had to do is create an embedding relation (solid line in the diagram below) from “service” (root of the model) to “RequestMessage” and “ResponseMessage”. After that I had to change the right-hand roles “Min” attribute to 1 and change the “Accepts” attribute of the left-hand roles to “All”

(changes are within red lines)

Maybe a bit strange but it works!

After applying the suggested solution my model looks like:

Note: the model isn’t finished yet. This picture is just to explain this restriction!

Alan also mentioned that the future release of the DSL Tools will have a much better “modelling experience”, till then we have to use the above described solution to get around this restriction.

After installing the DSL toolkit it is relatively easy to create and maintain the “domain model file” by drag and drop items on the designer that comes with the toolkit. After that, however, you have to get the “designer model file” in synch with the “domain model file”. This file is used to generate C# code that makes it possible to use your language in the experimental build version of Visual Studio.Net.

Getting this designer file in synch, really is a pain in the ass! It is a lot of work and for people not very familiar with the designer file structure (yet) it’s very likely to make mistakes with it. I assume Microsoft didn't have the time to solve this issue for this release of the DSL Toolkit but I hope they will solve it for the final release!?

Fortunately I found this Visual Studio add-in that makes it real easy to keep the two files in synch. Modelisoft, thank you for making my life a little less painful!

According to this posting of Daniel Cazzulino, Visual Studio Add-ins are dead. This might be true but this add-in is definitely alive and kicking!

Found some time to continue with my little experiment today.

After executing the installation of the earlier mentioned toolkits I was ready to perform my first steps in this totally new world. Installation all went fine, so that a good thing. One of the things I started with was reading the so called walkthrough documents (at this moment four docs are available). Unfortunately these documents are the only documentation that I can find for the DSL Toolkit. This is probably because of the CTP status of the toolkit but I assume this will get better in time. Although the walkthrough docs provide some guidance I think it is a bit poor. The docs mainly tell you which button to “click” but do not provide you with real background information about the DSL concepts. Definitely some room for improvement in this area!

My first experiences with building the example stated in the first “walkthrough” document, were a struggle. Most of this was because I didn’t take the time to execute all of the steps stated in the document. (I guess this is just a normal “men” thing to first start trying it yourself and only read the manual when you f.cked up). Second time I followed the docs very carefully but I was left with some errors. Luckily I found this posting on “”Xenopz Blog” that helped me with some remaining issues and that couldn’t be resolved by just reading the manual.

After some time I decided to start with the “Blank Language” template and start building my own DSL. This template provide you with two classes (“ConceptA” and “ConceptB”) and some relationships between them.

I started building my own DSL by simply renaming the two classes in the model (this is suggested in the walkthrough documents). One of the things that I find a little inconvenient (other words came up in mind first) in this is that after you changed the name of the classes and relationships in the designer you also have to perform some “find and replace” actions in one of the (many) xml files to make the designer for your language work. 

After some time this is what the first version of my model looks like.

During the creation I didn’t make any attempt to execute the “translate template” step or build the solution. Thinking back I’ am not sure if this is the smartest thing I could do.

My next step will be executing the “template translation” (to get the designer definition file in sink with the above domain model) and build and debug the language.

Haven’t tried this with this model but I expect some troubles with it. Getting the designer definition file in sink includes making manual changes in the (complicated) xml file. More on that later.

For, now Just a short explanation of the model (language):

At the root of the language I created the “service” class. Just below that you can see the “servicecomponent” class. I figured that all the other service components (interface, operation, etc.) might share some common needs (properties) in the future. Therefore all service components inherit from the “servicecomponent” class. 

The relation between the “servicecomponent” and the “servicecomponent” class is there to facilitate creating relationships between the several service components. Further you can see the “serviceAdpater” that acts as the class that couples the “serviceinterface” and “serviceimplementation” class. The “serviceinterface” class groups one ore more “serviceoperation” classes and is referenced by a “transportinterface” class. Finally the “serviceoperation” class references two “message” class to support the modelling of an input and output request for a service operation.

All still very limited, but it gave me a nice introduction in some of the possibilities of the DSL Toolkit. The above language can now be used in a so called “experimental build” environment to actually try the language and model a service. I haven’t tried that yet, so I’ am not sure what to expect!

When writing this post I suddenly realize that the two references from the “service operation” to the message class might not make sense. Maybe it is better to extend the message class with an attribute that indicates the type of the message (input or output). By using an enumeration for this I can make sure only the correct types can be selected. After that I can delete one of the referenced message classes and change the remaining reference to accept one or two messages.

To be continued…

Inspired by the several sessions of Tech-Ed Europe I decided to spend some effort on Domain Specific Languages (DSL) and Guidance Automation Toolkit (GAT). Although I have done some reading on Software Factories the area of building a Software Factory is quite new for me. Therefore I thought it would be interesting (for me and my current employer) to see the capabilities of these new technologies that are first members of Microsofts Software Factories initiative. To make it a little more appealing than just creating some sort of “Hello World” example I decided to try to combine it with some of my “day to day” work (which is architecting systems). Because I am definitely no expert in DSL’s I also decided to share some of my thoughts and issues to hopefully get some feedback to push me in the right directions.

Here is the idea: build a Domain Specific Language (DSL) to model “services”, perform some code generation and use GAT to provide some context-specific menus and templates within Visual Studio.Net IDE to speed up development based on the DSL.

Doesn’t sound too difficult so let’s give it a try. One of the things I do know about building a Software Factory is that you have to define what (reusable) parts can be identified within the domain. You also have to make sure the complete implementation details for all of the parts you are trying to model in a DSL and automate with GAT are available (and proven!).

For now, let’s assume these conditions are met. I will describe some of the parts I think are members of the service domain. Because my focus is on getting familiar with DSL and GAT, I will not describe all parts (and details) within the domain. As soon as I am totally up to speed with DSL and GAT (and still find it interesting) I will complete the model to make it a little more useful.

One of the most obvious parts that can be identified within this domain is the actual “service” itself. When translating this to a DSL this can be the “root” of the DSL. The service is composed out of other parts that I will call “service components” (it’s just a name).

Some of the service components I identified for this example:

• Service Interface; operations that are exposed by the service (service contract).
• Service Operation; definition of a service operation.
• Transport Interface; the representation of the service interface targeted at a specific technology (web service, enterprise service, remoting.)
• Message; definition of the messages exposed by service operation
• Service implementation; the actual implementation of the business logic
• Service Adapter; layer between the transport interface and the service implementation.

The above service components are all in their most basic form and they might not even cover the whole service domain. For example, it is very likely that in reality the message contains a data definition that is implemented as a data class based on an XSD definition. For now (this example) I will forget about al of these details. In this example a message is just represented by a class with a name that can be used as either input or output for a service operation.

I think, I will soon find out what parts (or details) I am missing here when I start modelling my DSL. During the modelling phase I will add only the absolute necessary properties and relationships.

The next step will be to prepare the environment by installing the DSL Toolkit and the Visual Studio 2005 SDK for Beta 2 and than we are ready to start experimenting with DSL.

To be continued…

Today I visited one of my customers to finish some work for a system we build. During the day some discussion came up about the tons of reference data that the system contains. The past few years the systems functionality increased enormously and so was the need for reference data. Although the systems isn’t based on a real service oriented design, most of the systems is exposed as web services because the functionality (and data) is also used by other systems at the customer’s location. At first the discussion covered system related stuff but soon changed to a more general discussion about how to handle data in a service oriented system.

Two weeks ago I visited a presentation of Ron Jacobs (Product Manager Patterns & Practises) at Tech-Ed 2005 Europe that was about handling data in a service oriented world. For me it was very easy to use some of his content to keep the discussion alive. Below some parts of the discussion:

The purpose of reference data is mainly to fill out requests and interpret response messages between services. Reference data is maintained by the service itself. The service implements a separate interface that can be used by service consumers to retrieve the reference data for a service. Further, reference data must be versioned, and gets updated periodically.

The service that maintains the reference data must provide the service consumers with a mechanism to retrieve the latest version of the reference data. The publisher of the reference data might implement some sort of a notification mechanism to notify it’s subscribers of a new version of the data. A service consumer can store the latest version of the reference in a local format to optimize the communication with the service. Of course this has some consequences for the development of the service because some sort functionality for this (storage of reference data) has to be developed.

Interaction between services includes request, response and reference data

Data (message) is immutable and therefore need a unique identifier. This identifier might include a version number of the data. Retrieving the data by using a unique id should ALWAYS return the same data no matter when the data is retrieved.

The data outside the services is represented as messages and is understood by both the sender and receiver. Special attention must be given to the schema that describes the message to reduce to possibility of misinterpretation of the message (data).

The data inside the service can be in any format that is appropriate for the service and is (and stays) private to the service.

Whenever data is needed by many services, choose one owner of the data. The owner is the only one that can make changes in the data. Whenever other services need an update of the data they send a request for an update to the owner. If the owner decides to make the update it publishes the changes to all subscribers. In this case the version number of the (reference) data increases. Request and response messages should be archived for auditing purposes.

Of course there is a lot more that can be said about data and services but the above topics kept us busy for most of the afternoon.

I attended a Chalk & Talk session with Steve Cook this morning about Software Factories. Just some of the things that were discussed:

Today building software is mostly craftsmanship. Compared to other industries the software industry isn’t very successful. Poor reuse.

Only if you know all the stable and reusable parts or architectural components it makes sense to start building a Software Factory for it. Start bottom up and start small otherwise you will probably fail. You have to know all the details about the parts you like to build a Software Factory for.

One of the basic ideas behind Software Factories: define only the unique features of the system that needs to build. (Other parts can be reused).

Most of the things we currently do are a waste of energy. All of it is redundant work. Automate all the rote and menial tasks. Let people do the interesting work.

The idea is to get a general development environment (VS.NET for example) and load the Domain Specific stuff in to tailor it. The Software Factory knows were you are and what you need at that time and delivers the right assets in the right context.

The Software Factory Schema defines a set of viewpoints and their relationships. A viewpoint describes how to look at things a view is the things you see.

It looks like Microsoft is working hard on some samples of Software Factories. In this talk Steve showed us a little of a Powerpoint presentation that demonstrates how to build an 'online Petshop' by using a Software Factory. Like Steve said, it's only a presentation but I assume they didn't created all the screenshot on the 106 sheets by hand