Previous posts in the Factory 201 series:
To continue the thread of actually creating your factory, in this post we will answer the question "With what would you build it?" We will discuss the software factory enabling technologies available today (on the Microsoft platform) and don’t speculate about possible future tooling. In our previous post “What would you build?” we discussed what we would build for our factory, and now we will see how that is mapped and addresses by specific technologies.
Let's have a look at some tooling and technologies and see how we can use that to build our software factories, starting today.
Automated Actions
Today's generations of software factories, like the ones available from Microsoft p&p, are primarily composed of a related set of automated activities (called recipes) that encapsulate a huge amount of best practices and domain expertise. The recipes often represent repetitive development and design tasks which result in generated source code or configuration files to help accelerate the development of our factory product. Let’s find out what tooling we can use to build these automated activities and how we can expose these, so called recipes, to our factory users.
Guidance Automation Toolkit/Extensions
The Guidance Automation Toolkit (GAT) is a toolkit that simplifies the development of automated actions into re-usable assets and provides mechanisms to make these assets available to architects and developers using Visual Studio. To accomplish this, GAT leverages the Guidance Automation Extensions (GAX) which is actually a runtime environment that expands the capabilities of Visual Studio and makes it easier to integrate and run automate tasks from the Visual Studio environment. So, we are actually using GAT and GAX together to build and run recipes and deploy them as guidance packages. ehhh...? Now what are recipes and guidance packages?
Recipes
Think of a recipe as set of automated instructions, called actions that together represent a specific automated activity or task. These actions are relatively small units of work and can be anything from creating a file, getting the value of a registry key or return the name of the currently selected project within a Visual Studio solution.
A recipe is defined in an XML file in a predefined structure that is interpreted by GAX at runtime. GAX supports invoking wizards for a recipe to gather user input that can be used as variables during the execution of the recipe to create variability in the artefacts the recipe outputs or configures. For example, a wizard can prompt the user to select a file location, ask for a name of a file that gets generated or ask the user to select items from a list. Recipes that don’t require user input can gather the variables from the environment to use in the same way. Recipes also can provide a convenient means of giving the user feedback about the actions being performed.
We can build recipes using GAT, it provides us with a special solution template in Visual Studio to author recipes, it comes with predefined solution structure, example actions and recipes and other utilities that give us a head start in building our own recipes and wizards.
Guidance Package
A Guidance Package is an installable Visual Studio package that contains a collection of recipes (and some their support classes like, converters, value providers, etc.) and is basically our deployment mechanism for the recipes that we built and want to expose in our software factory. In fact, the current generation of software factories as we see today are for the most part simply implemented as pure guidance packages.
That leaves us with another question: "how do we build a guidance package"? Again, this is where GAT comes in. Actually, GAT is a guidance package itself that helps architects and developers build guidance packages. Once we are done developing and testing our guidance package we can create a Windows installer package (MSI) and distribute our guidance package to our users.
Unfortunately, building and testing guidance packages (recipes) isn't as straightforward as we want it to be. For example, building recipes basically comes down to hacking in XML files and isn't supported by any graphical designers. Also the documentation is still limited. This makes the learning curve for developing guidance packages a little steep.
Artefact Templates & Generation
As said, some of the automated activities in our software factory will possibly result in one or more generated artefacts for our solution. These artefacts can be anything in the range from source code, XML files or a piece of documentation. To support this kind of artefact generation in our software factory we can use text templates (also referred to as "T4" templates) and the Text Templating Engine. This technology is included in GAT but also in the DSL Tools which we will discuss a little later in this post.
A text template is basically a text file that contains the (boilerplate) text you want to generate (i.e. C#/VB.NET code, XML, text etc.) that is mixed with control markers (<#, #>) and control blocks. The text between the markers is evaluated and provides the dynamic behaviour to the text template. Control blocks are used to control the processing flow of our templates. In other words they determine what text is output and how the Text Templating Engine navigates over a provided object model. This works very similar to how classic ASP work where you could combine server side directives with the raw HTML code.
Below an example of a T4 template that comes from Service Factory. The text between the markers (in bold) is evaluated and mixed with the static text in the template.
namespace <#= ServiceImplementationNamespace #>
{
[System.Web.Services.WebService(Namespace = "<#= ServiceContractNamespace #>", Name = "<#= ServiceImplementation #>")]
[System.Web.Services.WebServiceBindingAttribute(ConformsTo = System.Web.Services.WsiProfiles.BasicProfile1_1, EmitConformanceClaims = true)]
public class <#= ServiceImplementation #> : <#= ServiceContractInterfaceNamespace #>.<#= ServiceContractInterface #>
{
}
}
The most powerful feature of text templates is that they can be invoked from recipes and they can receive variables from our recipes. In this way we can build recipes that collect input from our users or factory environment and use that as input in our text templates to generate artefacts.
A major advantage of using text templates as a means to generate artefacts is that they can be customized very easily, typically to reformat the textual output or enhance it in some way (perhaps to suit an organisations coding standards for example). Text templates are deployed as plain text files so it is straight forward to customize them.
The fact that text templates are so easy to customise makes them perfectly suited for an iterative development approach. We can start with a “hard coded” copy of the desired output and we can identify the variability points in the code. We can then pass some arguments to the template and from that point we can add control blocks to parameterize it and hook into the underlying argument’s object models. One disadvantage of a text template is the lack of designer support like syntax colouring (we added the colouring in the example manually) and intellisense. This makes writing and maintaining large text templates a bit ’un-guided’.
Solution & Project Templates
As we saw in “What would you build?” we can use the solution structure to provide guidance on how to physically relate the artefacts that represent the product of our factory. Unfortunately, defining this (physical) structure is often challenging because of the struggle to define both the deployment model and the logical architectural model of the product in this single structure. Today, the solution structure is all we have (Solution Explorer), so let’s have a look how we can implement predefined solution structures in our factories to describe the physical structure (deployment) of the product.
[The logical architecture is best left to another separate view, which as we will see later we can provide with deep Visual Studio integration]
Solution structures are captured in so called Visual Studio Templates (*.vstemplate files) that are written in XML and are expanded by the Visual Studio template engine. Visual Studio 2005 now supports the creation and packaging of these templates. However, these templates are by default static, and require a separate mechanism to gather variables from the user or environment to enhance them with dynamic values. As it happens, GAT has just that mechanism, and can bind recipes and wizards to these templates.
Let's have a look how we can use this mechanism with GAT to ‘unfold’ certain types of templates:
- Solution templates. These templates unfold an initial Visual Studio solution file (*.sln) and structure that could represent the entire physical structure of the product of our factory.
- Project templates. These templates unfold a project (*.csproj, *.vbproj etc.) within an existing solution.
- Item templates. These templates unfold new project items such as a class files, XML files or any other valid project item.
One of the primary features of GAT is automating the creation of the solution structure. GAT accomplishes this by hooking into the VS template mechanism. This means recipes are allowed to participate in the template unfolding so they can, for example, collect input from the user to customize the names of the template artefacts and take additional actions to further configure the created solution items. For example, write some settings to a app.config file. A perfect example of this is demonstrated in Service Factory where the user is prompted to provide the name of the service, which is then used in the naming of the solution and projects within the solution. We have now discussed GAT and recipes, and learned they are perfectly suited for automating tasks and providing initial solution structures. Let’s look at providing abstractions to our factory and enhancing the design experience and productivity of automating solution assembly. Recipes with wizards often do form a visual abstraction of the underlying part of the product they are automating, but this abstraction is lost when the wizard completes. Let’s have a look at Domain Specific Languages and the DSL Tools and see how they can help enhance that experience and increase the level of abstraction represented in our factory.
Domain Specific Languages
A Domain Specific Language is a custom language that targets a specific domain and helps us describe the problem by using concepts that make sense for the target domain. But how would we build one? This is where the DSL Tools come to rescue. We can use The DSL Tools, (part of VS SDK), to define our domain model and easily represent that domain model with a graphical designer. Together, the domain model and the graphical designer form our DSL that we can integrate into our software factory.
[You can also provide a textual representation of your language instead of visually, but this is not supported currently out-of-the-box with the DSL Tools yet, unless you are prepared to define that language in XML.]
The DSL Tools also supports artefact generation using T4 templates which means we can write templates that generate (source code) artefacts directly from models we created in our DSL. The DSL Tools is just everything we need to build Domain Specific Languages today.
How would we use the DSL Tools?
So, how would we use the DSL Tools to build a DSL for our factory? Would we build one big DSL and designer that covers the complete product? The answer to this question is probably NO. Although our factory only targets one product (with several variants maybe!) it is likely that the scope of the domain is too broad to fit in just one designer (view). Therefore, we probably want to build a DSL that only targets one concern, component or aspect in our software factory. For example, we can use a DSL to model a piece of the architecture of the product that is build in our factory or use a DSL to model one of the work products (logical components) in our factory. In this case both DSL's help us presenting the right level of abstraction to our factory user for each of the sub-problems we identified in our factory.
But wait…, we have to make it very clear why we state to keep the scope of the DSL (domain model) limited to only one concern or piece of architecture. In fact, we can think of examples, like the factory schema, that are represented in a ‘larger’ domain model. Mostly, this is because off a limitation in the current technology. The current release of the DSL Tools only supports one view per domain model. In other words, the view that is exposed in our DSL is tightly coupled to the domain model. In future releases of the DSL Tools this limitation will disappear which means we can define multiple views with each view exposing only a part of a, possibly larger, domain model (like the factory schema). Please note that this doesn’t mean we have to build huge domain models in the future, but it will give us more flexibility in our domain model (DSL) design.
[You can find more information on how to use DSL’s in the context of software factories in the article ‘Bare Naked Languages or What Not To Model’ from Jack Greenfield]
Another important limitation of the current DSL tools is that there is no easy way to reference elements in one domain model with those in another, so stitching together domain models to fit larger domains is also challenging. However, there are DSL Power Toys soon to be released called ‘DSL Integration Service’ that enable that cross-referencing.
Another thing to keep in mind is that we don't have to start with a DSL in our factory from the very first beginning. It is absolutely possible that an asset that is implemented as a GAT recipe in a first iteration of our factory evolves in a DSL in one of the next iterations of our factory development cycle.
DSL Advantages
An important advantage of a DSL in our factory is providing a simplified view (abstraction) of the problem domain and to help us to provide architectural guidance in assembling that part. By using specific languages we can present our factory users with the right level of abstraction and use the designer as a way to provide context for our actions and the artefacts we are delivering from our factory. Recipes can also be applied to these designers to manipulate the underlying domain model, and automate the configuration and creating of the elements in these models.
The domain model that is introduced by the DSL (and stored on disk) can be validated and evaluated to determine the progress, validity and completeness of the product we are building. The domain model also provides us with a means to introduce “state” in our factory. We can generate artefacts over and over again for our populated domain model, something we can't do easily with just code generating recipes, where we have to populate the wizards with the necessary information on every run.
A very good example of the usage of a DSL, together with recipes is in the EFx Software Factory, which can be seen here.
DSL Challenges
So, if Domain Specific Languages can add value to our software factories, why haven't we seen them in many factories yet? We think the answer to this question is easy. The reason for that is the lack of "out of the box" integration between GAT and the DSL Tools. Currently, it is hard to use model information in our GAT recipes, without writing the custom code to support this ourselves. Luckily this scenario will be supported soon by means of a DSL Power Toy that becomes available.
[A more complete description of Domain Specific Languages, the Microsoft DSL Tools, together with an example of an 'real life' DSL can be found in this article]
Integrated Design Experiences
The technologies that we discussed so far are first class citizens in Microsoft's software factory offering. However, there are more technologies, which are not dedicated “software factory tools”, that can add enormous value to our software factories.
Be warned, these technologies are a little more difficult to use than those described earlier.
Tool-Windows and Custom Editors
One of the things we will notice when we are building software factories is that there will be situations where the decoupled GAT/GAX and DSL Tools features simply don't provide the experience we need. For example, we might want to provide a more integrated user experience for our factory users and decide that we need to provide the user with a larger view of the factory product.
To support this product view we might want to build custom tool-windows or specific editors. An example of this can be seen in the EFx Factory that hosts a custom tool-window that provides a ’Product Explorer’ view for the factory in addition to the default ‘Solution Explorer’ view.
We may want to enhance views with the use of customer editors that provide different abstractions and increase the productivity of modifying the view. An example of that can be seen here where we built a custom tool-window to maintain hierarchical data within a DSL.
It's good to know that we can use the VS SDK for all that. This SDK provides us with utilities and documentation that explains, for example, how to build a tool-window and hook that in Visual Studio. The bad part of this is that creating Visual Studio extensions like this isn't always as straight forward as we like it to be. Besides, building just the tool-window (or editor) isn't enough; we also have to make sure it integrates with GAX or the DSL Tools. Unfortunately this isn't something that is described in great detail anywhere so we are on our own in implementing this.
If you are interested in this type of integration you better make sure you have the VS SDK documentation ready and find your way to the GAT, DSL Tools and Visual Studio Extensibility forums!
Distributed Application and System Designers
One means of providing a high level view of your product is to leverage the ‘Distributed Application and Systems Designers’ that come with the ‘Visual Studio Team Edition for Software Architects’. These designers, although currently geared to deployment, provide a useful view of applications which can be composed into larger black box systems. If your factory targets the domain of applications represented on this design surface, this should be a natural choice for leveraging in describing your product, rather than creating DSL’s which duplicate this functionality.
Unfortunately, integrating with these designers today is very challenging, although it is possible to provide custom shapes using the SDM SDK (part of the VS SDK). Integrating recipes and solution templates into these designers is more of a commitment and currently not officially supported since this requires integration with a non-public API. However it is possible to a large degree, as demonstrated by the EFx Factory here. Where you can ‘drill into’ the applications on these diagrams and expose DSL’s containing the inner workings of the applications. You can also ‘Implement’ these shapes by using custom GAT project templates.
In the future, this type of integration will become a lot easier and integration between these designers and GAT and other DSL’s will be a better experience for factory builders. This will provide many factories the capability for top-down solution design.
Summary
In this post we have discussed the relevant technologies that we can use to build automation assets for real life software factories today. We haven't talked about technical details, issues, missing features, etc. which of course are there. The technologies are still in their early days and scenarios for software factory use are still maturing. The learning curve is steep now, and that will definitely become easier in the coming years. However, it is absolutely possible to build some very good factories with the above described technologies. So, if you are interested in factories you better start looking at these technologies right away!
Now that you know what to do, and what technologies to use for that, the next post in this series “How would you build it?” addresses how to put all these assets together in a process of assembling your factory.