In a previous post about “the next generation software factories” I wrote about the factory meta model and it’s role in providing a better user experience for the factory user. A more detailed explanation of the factory Meta model, or better, Factory Product Model (FPM) can be found in this post on Jezz’s blog. We can even see an experimental version of an authoring tool for the FPM in this post.

All of the above mentioned posts are using screenshots of the EFx Factory as an attempt to explain the Factory Product Model concept and it’s potential. However, because EFx isn’t publicly available at this moment we reckon that missing knowledge of EFx and its underlying domain might be debit to understanding the ideas behind the FPM.

So, to solve this issue we decided to take the first official drop of Service Factory from p&p and use that as a test case for discussing the Factory Product Model and possible future authoring tooling. Most people interested in software factories should at least be familiar with Service Factory as an early example so we think we can use this factory and the problem domain it is solving as a good example.

In the next couple of posts we will define the FPM for Service Factory by first using an experimental version of an FPM authoring tool. Along the way we will describe the important concepts of the FPM and demonstrate what Service Factory could have looked like if it leveraged a FPM. We will also describe the relationship of the Service Factory FPM to its underlying physical solution structure and how all of this related to a fictional Factory Schema for this factory.

We will not only use this series of posts to share information to help peoples understanding of these concepts, but also to validate and refine our current thinking in this space so any feedback is highly appreciated!

Let’s start. The first thing to do when authoring a factory is to make sure we fully understand the problem domain the factory is supposed to solve. For the Service Factory, this domain is already well defined and is reflected in the Service Factory solution structure. We will use the following concepts from Service Factory in our discussions:

• Service Contract
• Service Interface
• Service Operation
• Message Type
• Data Type
• Service Implementation
• Business Entities
• Business Logic
• Data Access Logic

Some of the above Service Factory concepts are directly realised in the Visual Studio solution structure. Others, for example Service Operations, are “hidden” somewhere in a C# code file and can only be viewed by opening the physical file.

We will use the above list of concepts together with their physical representation (projects, source files) in Service Factory to model a FPM for Service Factory. We will then use this higher level abstraction of the Service Factory to see if we can further improve the factory user experience by driving Service Factory from this abstraction.

Stay tuned…

Service Description Language meets Service Factory

I have been working for quite some time now with the DSL Tools and spend the last couple of months on Service Factory and related to that a little bit on the Guidance Automation Toolkit. Now, I decided to integrate the two and see how easy it is to make customizations in Service Factory and call GAT recipes from the DSL Tools.

Both our DSL (Service Description Language) and Service Factory cover the same problem domain of “designing services” so this seems to be a perfect fit ;)

I decide to start simple and added two new custom commands to the message shape in our DSL. As you can see below we now have the opportunity to call the “Generate DataTypes” and “Generate MessageType” recipe from the message shape.

My first concern was to just make it work so I didn’t spend any time yet on optimizing the Service Factory recipes to make them match better the DSL. I reckon this is necessary to make the user experience of the DSL integrated in Service Factory better.

If we have a look at (some of) the properties of the DataContract (picture below) that is modelled in the message, we can see that the DataContract is defined in the “RestaurantData.Xsd”.

After selecting the “Service Factory: Generate DataType” command on the message shape we can see that the XSD that describes the DataContract is added to the Service Factory “datatypes” project and that a datatype class is generated out of the XSD file.

Another thing we can do now is generate a Message Type for the message that is modelled. In the picture below we can see that we can execute the “Service Factory: Generate Message Type” command and that we can send the necessary data from the message in the model to the recipe in the Service Factory package.

As said, the Service Factory recipes and wizard need some changes to make them fit better with the DSL. For example in this case we don’t need the “Response Class Name” textbox on the wizard because we are starting the recipe from one message in the model. To solve this we can of course execute this recipe from the Service Operation in our DSL or customize the Service Factory recipes.

In a future post I will share some more technical details on how to call GAT recipes from a DSL, how share data between the DSL and GAT, etc.

For me it was good to see that it is possible to combine DSL Tools and GAT and that is relatively easy to make changes in the Service Factory package (not shown yet).

To be continued...

In the last couple of months I have seen the enormous power of these first examples of software factories but also saw some of pain points exhibited by them. I decided to write some them down also describe some of the (possible) features of the next generation software factories. At some points I am using the EFx factory and my discussions with Jezz as a reference for this post.

Where are we today

After the introduction of the Software Factories initiative and the publication of the “Software Factories” book it took quite some time before the first examples of Software Factories became available. Apparently the Software Factories concept was so totally new that people needed this time to get familiar to the concept and possible advantages.

Even now, when looking at the first examples of Software Factories, we can see that most of these examples are in their infancy. One of the reasons for this is the fact that not all Software Factories concepts, like for example Software Factory Schema and Software Factory Template, are not as well defined and understood as we would like them to be. As a result, there is very limited tool support for authoring factories.

Tools like the Guidance Automation Toolkit (GAT) and Domain Specific Languages Toolkit (DSL) have proven to be successful in these early examples of factories but their learning curve is steep and these assets only offer authoring of some of the work products which make up a factory, they offer not support of the factory overall as a cohesive development environment.

Early examples

When we have a close look at some of the characteristics of the first wave of factories from the patterns & practices team, for example ‘The Service Factory’, we can see that they are primarily based upon GAT recipes only. These ‘super’ guidance packages make extensive use of related recipes to create source artefacts in a solution. The factories are technically driven to automate the common tasks of architects and developers that would normally have to be hand crafted.

Whilst the recipes of these factories encapsulate valuable knowledge and experience of the solution domain, they act only as a means to write source code which has to then be subsequently tailored towards a specific solution. These factories make little or no use of modelling to describe the problem domain and offer little abstraction. Hence these factories still require expert knowledge of the solution domain in order to exercise them effectively.

As a result, the factory addresses a problem domain from a technical, Visual Studio related point of view instead of a more conceptual perspective that simplifies the problem domain itself.

Because of this tight coupling of these factories with the physical artefacts of a solution we cannot easily differentiate in views between the various types of factory users. This means all users experience the Factory in exactly the same way.

Another issue in the current wave of Software Factories is a missing, formalized way of describing software factories. An often heard concept in this space is ‘Software Factory Schema’ but till now nobody seems to know exactly how this is implemented. As a consequence the Software Factory Schema isn’t supported or integrated in any form of tooling in these early factories, and consequently we are still faced with a solution structure based approach in the early examples of these factories.

Software Factory User Experience

As stated previously, the user experience of today’s software factories is governed by the view of a solution in terms of its physical artefacts and therefore has a rather technical solution based focus. Let’s have a look at the two views that currently represent the factory in today’s examples.

Solution Structure view

The first view that can be identified is the “Solution Structure” view. This is a view we are all familiar with and has been around for a long time (as seen in the Solution Explorer tool window). It provides an overview of the Visual Studio solution structure and shows us all the different artefacts, like projects and source files that our software solution is composed of. Developers and architects familiar with Visual Studio know how to use this view and nobody really complains about it. Simply because it just represents the way we are developing software today.

Besides an overview of the solution file, project files, assembly references and code files it also reflects the architectural layering of the software component. It even provides us with build information, the naming of the individual components and the way the software will most likely be deployed. Basically, this view tells us a lot more about the physical infrastructure of the solution, the .NET framework and the build engine than it tells us about the purpose and concepts of the software we are developing.

In actual fact, the solution structure is often tailored by developers to describe the physical deployment view of a solution and the logical structure view of it at the same time. Often the result is a compromise of both these views, neither fully satisfying either, and hence every development team has opposing opinions on a best representation of these views.

This view is always the same for all project members. We have no way to filter the content of this view. The architect, the developer and the tester are all faced with the same view of all artefacts to represent their different concerns. Most of the time this isn’t necessary at all and it even makes software development more complex than needed.

Guidance Navigator view

The second, so called, “Guidance Navigator” view, is a new view driven by necessity by the Software Factory initiative. A first example of this view is available in the factory examples from the Microsoft patterns & practises team. This “Navigator” view provides us with a clear overview of all the activities (recipes) that can be executed in the solution.

Although we can also start these activities from the context menus on the individual items in the solution view it’s the “Guidance Navigator view” that summarises all the recipes and makes them easier to discover in one place. Another benefit of the “Guidance Navigator” view is that actions are tagged with additional help and background information to improve first time factory users understanding and improve the user experience. This is one example of the support for the “guidance in context” pillar of software factories.

However, an important aspect that is missing in this current implementation of the “Guidance Navigator view” is the lack of support for pre- and post conditions for activities. We need these conditions to control when activities become valid next steps for the factory user who is navigating the factory. Without this capability the user is still required to know which recipes to run in which order and with no way of telling the state of a completed activity as a whole. This view also currently lacks the capability to filter activities based upon the role of the factory user.

What we can learn from this “Guidance Navigator view” is that we can dramatically improve the usability of our software factory by just representing the same information in a different way. The “Navigator view” is a good start and it does provide us with a much better view on the factory activities but, we are not there yet! It still doesn’t provide us with a conceptual view on the underlying problem domain of the factory and thus the software we are developing. 

Software Factory Schema

One of the things we are currently missing a way to describe our software factory in an understandable and predictable way. The suggested means for this is the ‘Factory Schema’. This schema is often referred to as the “recipe” for the software component we are developing in the factory. It lists all the ingredients like, source files, models, frameworks and describes how to combine them to build the software component. It also describes the different views we have of the individual parts and the factory as a whole.

Although this description is quite clear it isn’t so clear what pieces of information should actually be part of the software factory schema. Even if we are able to identify all individual parts of the software factory schema most of us don’t know in what form we have to describe this to make it usable. Till now there is no formal, publicly available specification of the software factory schema that we can use to write our own schema. One of the things we do know is that a well described and implemented software factory schema might be the key to success for future software factory development and usage. The Software Factory Schema offers very important benefits for the software factory user.  

Let’s have a closer look at the software factory schema and try to describe the most important parts of it. The first concept in the schema is the “viewpoint”. Basically a viewpoint is a related set of activities, artefacts (work products), etc. that have a relationship. We can use the viewpoint to specify what elements of the software factory will be valuable and thus visible for a specific type of software factory user (stakeholder). It’s the viewpoint concept that gives us the opportunity to specify how different types of factory users will experience the factory. This might be based on the role of the factory user which makes role another concept in the software factory schema. Viewpoints can be related to each other by creating mappings between viewpoints.

Now we know viewpoints contain a set of activities we also know that activity is another important concept within the software factory schema.

An activity can be defined as a set of actions that we have execute in order to produce a work product or artefacts. Activities can be grouped together in a set of related activities.

A third concept in the Software Factory Schema is the workproduct. A workproduct can be best described as “anything” the software factory produces. It can be anything from one single xml file to a combination of code files or models of part thereof. Workproducts can be hierarchical grouped together. By grouping together workproducts we can define additional abstraction levels on top of the low level workproducts like for example source files. We will later see how we can use these abstractions to boost the usability of software factories. Activities use assets on their turn to produce workproducts. Think of assets as code generators, frameworks, domain specific languages, etc. that are used to create the workproducts. Workproducts have state that will change over time from, for example, created, partially ready to finished, by executing one or more activities.

(Have a look at this post from Jezz for more detailed information about the Software Factory Schema.)

Software Factory MetaModel

Another important concept in the Software Factories area is the Software Factory MetaModel. The Software Factory meta model helps us describe the software factory at a higher abstraction level. As we have seen earlier, today’s factories are mostly described and exposed by using the capabilities and terms available in the Visual Studio.NET infrastructure (Solutions, projects and files). Important concepts in our factory currently are reflected only in project names, names of source files, and layout of the projects and solutions etc. A concept that the factory defines which may include several source files cannot be easily represented in physical solution structure, and also many of the physical source artefacts created by Visual Studio tools have little meaning in the schema of the factory.

Therefore we need a Software Factory meta model – a model of the concepts of the factory that the user understands in terms of the problem domain. We can use this meta model to define abstractions that exist in the problem domain that the software factory deals with. In other words the meta model provides us with a mechanism we can use to describe our factory at a more conceptual level. We can define the so called software factory concepts in this meta model and relate them to each other. To make this concept a little more clear let’s have a look at the following example.

Think about a software factory for developing services. Thinking about the “service domain” we might identify concepts like “Service Interface”, “Service Operation” and “Message Contract”. When building a software factory for this domain by using the current tooling we will see that these concepts only reflect in documentation, project names and files names. The factory is driven by executing recipes that eventually add source files to solutions structures. There is no way to navigate through the factory by, for example, iterate through all the “Service Operations” that are available in the solution that is built with the factory. Simply because for the factory itself and Visual Studio.NET this concept doesn’t exist.

All that the first wave of factories have knowledge about at this moment is a set of files grouped together in a Visual Studio solution together with a set of recipes.

Figure 1 shows us how a service, built the Microsoft Service Factory, is represented to the factory user. Later on, we will see how we can change this representation by leveraging the factory meta model.

Figure 1. Representation of a service built in Service Factory.

The presence of a Software Factory meta model would enable us to describe the concepts that exist within the problem domain and thus are valid for the software factory. A populated meta model for the above described factory would provide us with a capability to navigate through the factory by using concepts like “Service Interface” and “Service Operations” instead of navigating through the projects and underlying files in the Visual Studio solution.

To demonstrate the power of the software factory meta model we will use EFx factory which is the first factory that leverages a factory meta model. EFx is an architectural guidance factory that is an offering of Microsoft Services. We will use EFx factory to introduce the “software factory view” that is built on top of the factory meta model.

Software factory view

The software factory view is the third view we can identify in the software factory area. In this view, we are displaying the factory concepts, defined in the factory meta model, to the factory user. The factory user isn’t confronted with physical artefacts like solution, projects and source files. The result is a view that allows the user to construct the products of the factory in terms of the problem domain. As we can see in figure 2 which represents the software factory view for EFx factory we are now dealing with concepts like Applications, Subsystems, Business Entities, etc.

Figure 2. Software factory view in on top of the EFx factory meta model.

The hierarchy in this view doesn’t reflect the underlying file system or solution explorer structure at all. Instead, we are looking at the conceptual structure of the software we are writing. We can execute our recipes on these concepts, and have them relate to one or more physical artefacts in the solution such as source files, diagrams and other models that extend the factory schema.

The advantages of this view are numerous. First of all, this view simplifies the problem we are dealing with during our development activities. Developers don’t need knowledge of all the underlying details like project structures, files, deployment scenarios anymore. All they are faced with are the conceptual pieces of the software they are developing together with the activities, implemented as recipes that are related to the concepts. They can execute activities to produce the associated workproducts without worrying where to put it in the Visual Studio solution structure or store it on the file system. With this software factory view in place we can even change the complete underlying Visual Studio solution structure, to for example meet specific architectural requirements and deployment scenarios, without developers even knowing about it.

Another advantage of this view is that we can leverage the viewpoints we defined in our software factory schema to hide specific concepts from different types of software factory users. This means we can target the factory user experience based on their role to better meet the requirements of the different types of users. We can even define a viewpoint that only displays the concepts that make sense to business analysts or end users so we can discuss the software development process with them.

A view like this requires that the factory maintain the state of the Meta Model, rather than creating physical artefacts using recipes. Instead recipes are used to configure the meta model, and physical artefacts are then generated at a later stage from the meta model.

Summarized we can say that we can use the software factory view to represent the factory concepts described in the meta model to improve the user experience of the factory.

In some future posts I will elaborate a little more about some of the above described concepts. In the meantime check out Jezz blog for a lot more interesting information on software factories and EFx factories in particular.

It’s been a long time since I wrote about the never ending story of the small demo Domain Specific Language we are building. In the meantime I received a few mails asking me for the status and Steven Kelly even mentions in this post that I probably gave up on the DSL Tools.

Well…, as we wrote in our recently published article (still only available for subscribers) we have plans to make it downloadable as soon as the V1 of the Microsoft DSL Tools hits the streets. To prepare a little for that I decided it was about time to install the latest June CTP of the DSL Tools and try if I was able to get this DSL working in the new DSL Tools.

I used the migration tool that comes with the June CTP to migrate my old domain model and designer definitions into the completely new format. Although this migration went quite smooth, I decided to start from scratch again, just to get familiar with this new release of the DSL Tools.

To be honest, that took some time. I considered myself an “experienced” DSL Tools user so decided it wasn’t necessary to read the (limited) documentation. I was wrong! Because of that I noticed far too late that there is a “DSL Details” window that helps you set some (important) properties for the domain model and designer. This was one of the reasons I didn’t get things working. Once I found the DSL Details window (View/Other Windows from the menu) things became much easier.

Another issue that took some time was that during debugging my DSL the shapes didn’t appear after dropping them on the design surface. They were added to the underlying model and did appear in the “Explorer” window. After some time I found out that I had to add my domain classes as “Element Merge Directives” of the root (in my DSL “Service”).

The good thing about the new DSL Tools is that you (almost) don’t need to manually edit the underlying XML anymore. I only needed that to make my compartment shapes work. Once you understand the concepts, the “DSL Explorer” makes it relatively easy to build a domain model and designer and it looks *nice*!

As far as I am concerned, the new June CTP is a very nice upgrade of the DSL Tools.

Now I only have to migrate all my custom code (if still needed), add some validations and include the WSDL generation code and I am done J.

Hopefully there will be no breaking changes for V1!

Great news! The July drop of Service Factory is available for download from the public workspace. This drop includes the ASMX package for Service Factory together with a totally new Data Access package. No need for me to go into any details because Tom already did a great job explaining the features of this July drop. One of the great features is the new "Navigator". This really improves the user experience of Service Factory!

Go download it, have fun and let us know what you think about it in the message board!

The article about Domain Specific Languages I wrote together with Christian Weyer for CoDe magazine is now online. It’s called “Introducing Domain Specific Languages” and briefly describes the Software Factories initiatives before it focuses on Domain Specific Languages in general and our DSL for the remainder of the article.

At this moment only a piece of the full article is available for non-subscribers. This means you either have to subscribe to CoDe magazine or save the link and get back in 2 months when the full article is available for all site visitors of the CoDe magazine.

 Hope you like it!

This Monday I will be travelling to Redmond to attend a Service Factory workshop that is organized by the Microsoft P&P team for a few of Microsoft’s customers.

I will be there as one of the members of the projects Advisory Board so I can hear the customer feedback firsthand and try to help them a during the implementation sessions. I am sure I can learn a lot from discussing their requirements, issues, ideas, etc. 

Hopefully the workshop schedule leaves some time to chat with Don, Jason, Tom, Pablo and a lot of others.

It's finally there! Download the first public drop of Service Factory and let us know what you think of it in the Service Factory community.

It started as project Batman, than it became Service BAT and now, the official name is just confirmed as Web Service Software Factory. A little long, so let's just call it Service Factory!

We can finally start talking about it! The last couple of days Jason started giving some hints about changing the way we are developing service oriented applications. Christian is even thinking a little further and dreams about a possible relation with this?? All a little confusing but definitely interesting. It’s time to shred some light and start talking about Service BAT!!

The last few months I have been participating in this very cool project of the Microsoft P&P team coded named Service BAT (Baseline Architecture Toolkit). I believe this delivery is likely to change the way you are building services for the .NET platform.

So what is it? Service BAT is a toolkit that provides architectural guidance, tools, patterns, wizards, etc. to help you designing and building services using Windows Communication Foundation and ASMX. The Guidance Automation Toolkit is used to integrate all of this very nicely into Visual Studio.NET 2005. One of the great things of Service BAT is the level of customization. Service BAT doesn’t force to do anything you don’t like. You can customize most parts of Service BAT to make it meet your requirements.

Please note that this is just a very limited description; it’s really much more than this but you have to find out for yourself.

In the next few days I (and hopefully some others) will provide some more details. There is a public community for Service BAT that you can check out. An early code drop will follow in a few days.

For now, just a few screenshots of (an early drop) of Service BAT. More details will follow but this is the (very) short story that belongs to the screenshots.

The Service BAT architecture is based on separation of concerns (responsibilities). This separation is also translated in the Visual Studio solution structure. You can base your service development on one of the templates that Service BAT provides.

You can choose for a full blown template that will provide you with a complete solution structure including a lot of projects or you can choose for an empty solution structure and decide for your self what projects (responsibilities) you need in your solution structure (service). You can than add specific types of projects to your solution. All project types correspond to a set of responsibilities. For example we have a project type that holds and provide guidance for datacontracts, one for service contracts (interface), etc.

Service BAT provides you with recipes (actions) that you can execute. The recipes that are available for a project type are based on the responsibilities of a project type. As you can see in the screenshot below the “datacontract” project type provides you with guidance and tools to create a datacontract. You choose between creating a datacontract  by using a wizard, Code Snippets or create one out of an existing XSD.

(Later you will see that you can add responsibilities to a project type as part of the customization of Service BAT.)

As said, just a very quick and incomplete explanation of some of the features of Service BAT. Hope this makes you interested enough to check out the Service BAT community. Go download the early drop as soon as it is available and let us know what you think in the community!