Previous posts in the Factory 201 series:

• What are they (concretely)? (Edward)
• When would you build one? (Jezz)
• What do I tell my Manager? (Edward)
• How long will it take? (Jezz)
• What would you build? (Jezz)
• With what would you build it? (Edward)
• How would you build it? (Jezz)

---

After reading all of our previous posts (don’t tell me you skipped one!) we know what automated assets to build, which tools and technologies to use and how to build a factory. The final step here will focus on the quality attributes we have to keep in mind during the design and development of our factory in order to make it a stunning success! In other words, “What should we strive for?” when building our factories?

Abstraction

In a previous post “With what would you build it?” we already mentioned the power of using abstractions in our factory. Let’s dig in this a little further and see why abstraction is definitely the top thing we should strive for!

Increased Productivity

One of the advantages of using abstractions in our software factory is significantly increased productivity. We can think of an abstraction as a simplified representation of a specific problem. In fact, we are probably using abstractions in our job as a developer or architect every day. (i.e. 4G programming languages, class diagrams, API’s, class libraries, frameworks, DSL’s etc.)

Think about it, for example, if we build a helper class to perform a specific task (say: logging to a file) we are actually using abstraction. We create a class and some methods that simplify the details of logging, file management, rolling-over, naming, log details and formats etc. Once we have the helper class in place we can just re-use the helper class using some basic parameters without focussing on the details of how it works anymore. We solved our problem more simply by creating the helper class.

Now, let’s elaborate on this a little further. Suppose we only use this helper class in certain circumstances of a particular type of solution, let’s say only for certain components of a particular application. In this case, we don’t use or need all the possible parameters of the helper class (because we only use it in a certain way in this context), and let’s say we identify 5 different cases of use, each with different parameter values. Now we can build another higher-level helper class that wraps the original helper class, and defines only one parameter which describes one of our 5 cases – perhaps an enumeration. Our new helper class is now an abstraction of the older helper class – because it simplified its usage in this particular context. Typically, this new level of helper class becomes part of a framework for those types of applications, because we have made certain types of assumptions about how the helper class is to be used.

[To clarify this, using abstraction for simplifying a problem is something we see all time. For example, take ADO.NET that hides a lot of the low level details that are needed to communicate with a SQL Server database. On top of that we have Enterprise Library that adds another abstraction level to make data access even more straight forward. In the future, ADO.NET vNext and the Entity Framework will do an another attempt to raise the level of abstraction for “data programming”]

As we continue to build up the levels of abstraction, you can see that we can solve more specific problems at less level of detail, and also by using a higher level description or language. If we can get close to a point where we can provide these abstractions in terms of the specific problem we are solving, we are making it real easy to program our application (based on minimal information). This is where productivity is gained.

Abstraction Roadmap

When developing a solution you can think of many abstractions, when you add modelling and automation to this picture the possibilities for abstraction build upon each other vastly:

Platform API’s → Application API’s →Helper Functions →Class Libraries → Frameworks → Domain Models → Software Factories

Each of the abstractions denoted above increases the level of abstraction of the problem you are solving, and in each case also narrows the domain of the solution you are providing for – this is the simplification part. I am sure it’s obvious to you that using a domain model to configure an application is far quicker than writing programming API’s to describe the same thing.

The other key point here is that I can configure my application in terms of high level things that make sense to that application and not in the low level details of the API’s. This means that those using the domain model now don’t need to be programmers they can have higher level skill-sets like solution design, requirements gathering and interpretation etc. By raising the level of abstractions we are moving from a physical design (platform API’s) to a logical design (domain model) that exposes less details and is understandable for a much wider audience.

Automated Guidance

Another very important aspect we have to keep in mind during the development of our factory is the level of automated guidance we expose to our factory users. In our post “What would you build?” we have already discussed the different types of automation assets that we might want to use in our factory but we have make sure we use them in their most effective way.

Remove Mundane Tasks

A top quality attribute (from a developer’s perspective) for using our factory would be to provide automated assets that remove the rote and menial tasks from the factory user’s ToDo list. At best remove these types of duties completely from the developers ToDO list. We as solution developers, often have some predictable boring tasks that we have to do over and over again. By exposing automated assets for these types of tasks we save our factory users from completing these mundane and in-variable tasks and let them focus on the variable and more exciting design aspects of the solution they are building. Our factory users will definitely love us for that!

Transfer Domain Knowledge

The design of our automated assets must ensure they capture knowledge and experiences and can be used to make that knowledge available for others. Using the example of a DSLs’ domain model which holds the knowledge about the underlying solution domain and how it can be and should be manipulated. By using the DSL, the user is presented with all the knowledge of how to describe the domain, the parts and relationships of that domain, and the description of the pieces within it.

The important part of this is that the domain model helps the user to figure out the key parts (concepts) in the domain. The domain model (and designer in case of the DSL) guides the user through the process of modelling a solution for the problem by using concepts that make sense to him. In fact, the user learns the domain and the important concepts of it but also how to provide a solution by using these abstractions.

Enforce Compliance and Completeness

When we design the automated assets (and factory as a whole) we encapsulate the architecture of the factory product in the domain model that is used by the factory – this is the model of the product.

We can build automation assets to populate or configure the product model and in this way ensure compliance with the implicit architecture of the product. Because these models have configuration to address their variance, validation can be used explicitly to ensure the factory user has not configured parts of it incorrectly with respect to the whole solution.

Automation assets can also be used to examine the model, check for completeness and use that information to guide the factory user in his next steps he needs to execute in the factory to complete the product.

Full Automation vs. Flexibility

When designing our automated assets we have to keep in mind to keep them limited and focused on one particular concern. It doesn’t make sense to build a huge fully automated asset that covers multiple concerns and generates a lot of unrelated artefacts (i.e. source code and configuration files). It also doesn’t make sense to build a DSL that tries to model to world. If we expose a number of small, highly focussed automation assets we have a much more flexible factory because we can, for example, change the order in which we manipulate the assets, combine them or even decide not to use some automation assets at all to better meet the requirements for the product we have to build.

However, if we go too far with variability and refining our abstractions and automated assets to a more detailed level we might end up with a factory that is less productive because there are more “moving parts” that needs to be configured which also increases the complexity of using the factory.

Finding a balance between full automation and flexibility is absolutely key here!

Usability

Because we are so busy developing our software factory we might forget that building a fully fledged and super cool software factory that generates the perfect product isn’t enough to make everyone want to use it. In the end it is often the factory user that has a huge influence in the adoption of our software factory. This means we have to make it cool to use the thing and make it flexible and intuitive enough to use and improve the experience of building solutions in this domain with it!

You don’t want people to discard you factory because it too hard to use, or discover how to use effectively. Don’t assume they will read the manual!

Promote Adoption

Another thing we have to keep in mind during the development of our factory is the fact that the factory is likely to be used by other less-skilled and less-knowledgeable developers than the ones that building the factory. They might not even have to be developers! This means, we might have an issue here in convincing other development teams to adopt the factory and use that instead of their existing tools, frameworks and utilities they are so familiar with. Or worse, have built themselves!

To get our factory adopted by our audience we have to lower the bar for them to start using the factory. Of course there are multiple ways to achieve this and they will probably differ dependent on the organization structure we are working in. Suppose we are working in an organization where the factory is build by a dedicated team that is known by the several project teams responsible for delivering products. In this example, the factory team can support the project teams in the start up phase of their project to get familiar with the factory and makes sure the project team gets up to speed on using the factory real quick. Once the factory is adopted by the project team responsible for building the product, the factory authoring team can start collecting the feedback from this team to enhance the factory and increase the usability of the factory even better. You see this in the process described in “How would you build it?”

Role Sensitivity

A software factory, just like any other development tool, is potentially used by a lot of project team members (factory users). i.e. Developers, Architects, Testers, User Experience Engineers, Technical Writers etc. To get the full potential from our factory, and factory users, we must make sure the factory targets the audience that is supposed to use the factory in the way they want to use it. In other words before we start developing our factory we need to know what types of users we expect for our factory. Do we only target architects, developers, testers or is it a mix of these types of users that we are aiming at?

Why is it important to know the factory users? Well, each type of user is interested in a different set of concerns and views of those concerns (In factory terminology this is called a ViewPoint). For example, an architect might only be interested in the high level view of the system design while the developer is interested in much more technical details.

Once we know our audience, and those who we are building the factory for, we can attempt to provide them with the right tools, views and level of abstraction to support them do their job in the best possible way and be more productive when using the factory.

Customization

Software development is rapidly changing. New technologies, new paradigms happen almost every day. To be successful in software development we have to keep up with these changes, and therefore our software factory has to too!

In our post “How long will it take?” we have seen that our factory only start to pay back after we have used it for greater than ~5 times, so we better make sure we can customize our factory very easily to meet evolving requirements and additional product variants. This is exactly why we have an iterative development cycle as defined by “How would you build it?”

One of the key requirements of any development tool like a factory is to be flexible enough to address as many use scenarios and development environments as possible. Otherwise its’ use is limited. There are two aspects of customization worth noting here:

Factory Customization

Firstly, there is customization of the factory itself, usually to accommodate a change in architecture or change in variability of the existing architecture that the factory models. This is hard to accommodate if the factory itself presents a closed programming interface. However, this can be accommodated by defining good extensible infrastructures to the architecture so that those using the factory can easily customize it to add or remove features or variability that the factory presents. We see this most when ‘horizontal factories’ are customized to suit certain vertical scenarios, where variability in most cases wants to be predefined, and fixed, and therefore requires removal from the user experience of the factory.

As an example. Consider a (horizontal) factory that builds web services. If I want to ‘verticalise’ this factory and customise it to suit building certain types of financial web services, then it’s likely that for financial web services, I want to pre-define certain Operations Contracts and Data Contracts to use. The Operation and Data Contracts are one point of variability of the horizontal factory, and the horizontal factory provides assets to allow the user to configure them. Now that I want to customise this factory to make a new verticalised version that is financial services based, I want to be able to say that the Operation and Data Contracts are fixed to certain industry standards, and no longer configurable by the factory user.

Product Customization

Secondly, there is customization of the automation assets used to create the product itself. Typically, the scenario here is that you want to alter the automation assets used to transform the model of the product into a physical solution.

As an example. Consider a factory that generates source code as its final product. The source code is determined by the factory in code templates, which are used to transfer the models of the factory into a physical solution. So consider, that the generated code does not agree with you in some way. Often, this needs to be changed to suit an organisations’ coding standards, or include some organisations standard copyright header, or perhaps you want to change the code that is output because you have a more optimal way to write the solution.

Extensibility & Composability

One of the key issues of building factories is in providing the right guidance and solutions to varying requirements. Creating a factory for a large domain increases the chance that the guidance it offers is limited in addressing wider usage scenarios.

For example, consider a factory that builds web services. This factory is likely at some point to define a mapping of the web service’s logical architecture to source code artefacts. In doing that, the factory itself will have to make decisions about what patterns and technologies to use to do that. The assumptions it makes to do that may not be valid for all cases, and it is such cases that may prohibit the use of the factory at all, if the factory user cannot customize or influence how that is done. In smaller domains the chances of meeting this issue are smaller, because the possible solutions are smaller in number. However, even in smaller domains, there can be several solutions that apply to different circumstances. This means that larger domains need to be composed of smaller domains, where each smaller domain could be swapped out for a different implementation.

Composite Factories

In order to practically achieve this we need to be able to compose larger domains from smaller domains, and have the domains interact with each other. Furthermore we need to be able to potentially swap-out each domain with a different implementation.

In factories, this can be achieved by providing extensibility mechanisms that accommodate plugging in and out various other factories to provide the sub-domains. We call these sub-domain factories ‘factorettes’. These extensibility points are key for expanding the use of your factory, so others could provide alternative implementations for the pieces of you factory to suit their needs.

One such use of this is to provide specific technology implementations or strategies for implementation such as the ‘EFx Factory’ does, so that factory users can pick and chose which technology to use to implement the Service Contract or Data Mapping layers, for example. Of course you can choose many other aspects to make extensible in your factory apart from technology. Perhaps you want to allow other factories to provide better views of a part of a domain, or provide better implementations of it as well.

Summary

In this post we tried to list the important quality attributes for software factories. Building factories isn’t (only) about using the available technologies to build some automated stuff that is just cool. We have to realize that we need to take special care to ensure the factory is useful for our users, improves their productivity and adds value to help them deliver high quality products.

Previous posts in the Factory 201 series:

• What are they (concretely)? (Edward)
• When would you build one? (Jezz)
• What do I tell my Manager? (Edward)
• How long will it take? (Jezz)
• What would you build? (Jezz)

---

To continue the thread of actually creating your factory, in this post we will answer the question "With what would you build it?" We will discuss the software factory enabling technologies available today (on the Microsoft platform) and don’t speculate about possible future tooling. In our previous post “What would you build?” we discussed what we would build for our factory, and now we will see how that is mapped and addresses by specific technologies.

Let's have a look at some tooling and technologies and see how we can use that to build our software factories, starting today.   

Automated Actions

Today's generations of software factories, like the ones available from Microsoft p&p, are primarily composed of a related set of automated activities (called recipes) that encapsulate a huge amount of best practices and domain expertise. The recipes often represent repetitive development and design tasks which result in generated source code or configuration files to help accelerate the development of our factory product. Let’s find out what tooling we can use to build these automated activities and how we can expose these, so called recipes, to our factory users.

Guidance Automation Toolkit/Extensions

The Guidance Automation Toolkit (GAT) is a toolkit that simplifies the development of automated actions into re-usable assets and provides mechanisms to make these assets available to architects and developers using Visual Studio. To accomplish this, GAT leverages the Guidance Automation Extensions (GAX) which is actually a runtime environment that expands the capabilities of Visual Studio and makes it easier to integrate and run automate tasks from the Visual Studio environment. So, we are actually using GAT and GAX together to build and run recipes and deploy them as guidance packages. ehhh...? Now what are recipes and guidance packages?

Recipes

Think of a recipe as set of automated instructions, called actions that together represent a specific automated activity or task. These actions are relatively small units of work and can be anything from creating a file, getting the value of a registry key or return the name of the currently selected project within a Visual Studio solution.

A recipe is defined in an XML file in a predefined structure that is interpreted by GAX at runtime. GAX supports invoking wizards for a recipe to gather user input that can be used as variables during the execution of the recipe to create variability in the artefacts the recipe outputs or configures. For example, a wizard can prompt the user to select a file location, ask for a name of a file that gets generated or ask the user to select items from a list. Recipes that don’t require user input can gather the variables from the environment to use in the same way. Recipes also can provide a convenient means of giving the user feedback about the actions being performed.

We can build recipes using GAT, it provides us with a special solution template in Visual Studio to author recipes, it comes with predefined solution structure, example actions and recipes and other utilities that give us a head start in building our own recipes and wizards.

Guidance Package

A Guidance Package is an installable Visual Studio package that contains a collection of recipes (and some their support classes like, converters, value providers, etc.) and is basically our deployment mechanism for the recipes that we built and want to expose in our software factory. In fact, the current generation of software factories as we see today are for the most part simply implemented as pure guidance packages.

That leaves us with another question: "how do we build a guidance package"? Again, this is where GAT comes in. Actually, GAT is a guidance package itself that helps architects and developers build guidance packages. Once we are done developing and testing our guidance package we can create a Windows installer package (MSI) and distribute our guidance package to our users.

Unfortunately, building and testing guidance packages (recipes) isn't as straightforward as we want it to be. For example, building recipes basically comes down to hacking in XML files and isn't supported by any graphical designers. Also the documentation is still limited. This makes the learning curve for developing guidance packages a little steep.

Artefact Templates & Generation

As said, some of the automated activities in our software factory will possibly result in one or more generated artefacts for our solution. These artefacts can be anything in the range from source code, XML files or a piece of documentation. To support this kind of artefact generation in our software factory we can use text templates (also referred to as "T4" templates) and the Text Templating Engine. This technology is included in GAT but also in the DSL Tools which we will discuss a little later in this post.

A text template is basically a text file that contains the (boilerplate) text you want to generate (i.e. C#/VB.NET code, XML, text etc.) that is mixed with control markers (<#, #>) and control blocks. The text between the markers is evaluated and provides the dynamic behaviour to the text template. Control blocks are used to control the processing flow of our templates. In other words they determine what text is output and how the Text Templating Engine navigates over a provided object model. This works very similar to how classic ASP work where you could combine server side directives with the raw HTML code.

Below an example of a T4 template that comes from Service Factory. The text between the markers (in bold) is evaluated and mixed with the static text in the template.

namespace <#= ServiceImplementationNamespace #>

{

   [System.Web.Services.WebService(Namespace = "<#= ServiceContractNamespace #>", Name = "<#= ServiceImplementation #>")]

   [System.Web.Services.WebServiceBindingAttribute(ConformsTo = System.Web.Services.WsiProfiles.BasicProfile1_1, EmitConformanceClaims = true)]

   public class <#= ServiceImplementation #> : <#= ServiceContractInterfaceNamespace #>.<#= ServiceContractInterface #>

   {

   }

}

The most powerful feature of text templates is that they can be invoked from recipes and they can receive variables from our recipes. In this way we can build recipes that collect input from our users or factory environment and use that as input in our text templates to generate artefacts.

A major advantage of using text templates as a means to generate artefacts is that they can be customized very easily, typically to reformat the textual output or enhance it in some way (perhaps to suit an organisations coding standards for example). Text templates are deployed as plain text files so it is straight forward to customize them.

The fact that text templates are so easy to customise makes them perfectly suited for an iterative development approach. We can start with a “hard coded” copy of the desired output and we can identify the variability points in the code. We can then pass some arguments to the template and from that point we can add control blocks to parameterize it and hook into the underlying argument’s object models. One disadvantage of a text template is the lack of designer support like syntax colouring (we added the colouring in the example manually) and intellisense. This makes writing and maintaining large text templates a bit ’un-guided’.

Solution & Project Templates

As we saw in “What would you build?” we can use the solution structure to provide guidance on how to physically relate the artefacts that represent the product of our factory. Unfortunately, defining this (physical) structure is often challenging because of the struggle to define both the deployment model and the logical architectural model of the product in this single structure. Today, the solution structure is all we have (Solution Explorer), so let’s have a look how we can implement predefined solution structures in our factories to describe the physical structure (deployment) of the product.

[The logical architecture is best left to another separate view, which as we will see later we can provide with deep Visual Studio integration]

Solution structures are captured in so called Visual Studio Templates (*.vstemplate files) that are written in XML and are expanded by the Visual Studio template engine. Visual Studio 2005 now supports the creation and packaging of these templates. However, these templates are by default static, and require a separate mechanism to gather variables from the user or environment to enhance them with dynamic values. As it happens, GAT has just that mechanism, and can bind recipes and wizards to these templates.

Let's have a look how we can use this mechanism with GAT to ‘unfold’ certain types of templates:

• Solution templates. These templates unfold an initial Visual Studio solution file (*.sln) and structure that could represent the entire physical structure of the product of our factory.
• Project templates. These templates unfold a project (*.csproj, *.vbproj etc.) within an existing solution.
• Item templates. These templates unfold new project items such as a class files, XML files or any other valid project item.

One of the primary features of GAT is automating the creation of the solution structure. GAT accomplishes this by hooking into the VS template mechanism. This means recipes are allowed to participate in the template unfolding so they can, for example, collect input from the user to customize the names of the template artefacts and take additional actions to further configure the created solution items. For example, write some settings to a app.config file. A perfect example of this is demonstrated in Service Factory where the user is prompted to provide the name of the service, which is then used in the naming of the solution and projects within the solution. We have now discussed GAT and recipes, and learned they are perfectly suited for automating tasks and providing initial solution structures. Let’s look at providing abstractions to our factory and enhancing the design experience and productivity of automating solution assembly. Recipes with wizards often do form a visual abstraction of the underlying part of the product they are automating, but this abstraction is lost when the wizard completes. Let’s have a look at Domain Specific Languages and the DSL Tools and see how they can help enhance that experience and increase the level of abstraction represented in our factory.

Domain Specific Languages

A Domain Specific Language is a custom language that targets a specific domain and helps us describe the problem by using concepts that make sense for the target domain. But how would we build one? This is where the DSL Tools come to rescue. We can use The DSL Tools, (part of VS SDK), to define our domain model and easily represent that domain model with a graphical designer. Together, the domain model and the graphical designer form our DSL that we can integrate into our software factory.  

[You can also provide a textual representation of your language instead of visually, but this is not supported currently out-of-the-box with the DSL Tools yet, unless you are prepared to define that language in XML.]

The DSL Tools also supports artefact generation using T4 templates which means we can write templates that generate (source code) artefacts directly from models we created in our DSL. The DSL Tools is just everything we need to build Domain Specific Languages today.

How would we use the DSL Tools?

So, how would we use the DSL Tools to build a DSL for our factory? Would we build one big DSL and designer that covers the complete product? The answer to this question is probably NO. Although our factory only targets one product (with several variants maybe!) it is likely that the scope of the domain is too broad to fit in just one designer (view). Therefore, we probably want to build a DSL that only targets one concern, component or aspect in our software factory.  For example, we can use a DSL to model a piece of the architecture of the product that is build in our factory or use a DSL to model one of the work products (logical components) in our factory. In this case both DSL's help us presenting the right level of abstraction to our factory user for each of the sub-problems we identified in our factory.

But wait…, we have to make it very clear why we state to keep the scope of the DSL (domain model) limited to only one concern or piece of architecture. In fact, we can think of examples, like the factory schema, that are represented in a ‘larger’ domain model. Mostly, this is because off a limitation in the current technology. The current release of the DSL Tools only supports one view per domain model. In other words, the view that is exposed in our DSL is tightly coupled to the domain model. In future releases of the DSL Tools this limitation will disappear which means we can define multiple views with each view exposing only a part of a, possibly larger, domain model (like the factory schema). Please note that this doesn’t mean we have to build huge domain models in the future, but it will give us more flexibility in our domain model (DSL) design.

[You can find more information on how to use DSL’s in the context of software factories in the article ‘Bare Naked Languages or What Not To Model’ from Jack Greenfield]

Another important limitation of the current DSL tools is that there is no easy way to reference elements in one domain model with those in another, so stitching together domain models to fit larger domains is also challenging. However, there are DSL Power Toys soon to be released called ‘DSL Integration Service’ that enable that cross-referencing.

Another thing to keep in mind is that we don't have to start with a DSL in our factory from the very first beginning. It is absolutely possible that an asset that is implemented as a GAT recipe in a first iteration of our factory evolves in a DSL in one of the next iterations of our factory development cycle.

DSL Advantages

An important advantage of a DSL in our factory is providing a simplified view (abstraction) of the problem domain and to help us to provide architectural guidance in assembling that part. By using specific languages we can present our factory users with the right level of abstraction and use the designer as a way to provide context for our actions and the artefacts we are delivering from our factory. Recipes can also be applied to these designers to manipulate the underlying domain model, and automate the configuration and creating of the elements in these models.

The domain model that is introduced by the DSL (and stored on disk) can be validated and evaluated to determine the progress, validity and completeness of the product we are building. The domain model also provides us with a means to introduce “state” in our factory. We can generate artefacts over and over again for our populated domain model, something we can't do easily with just code generating recipes, where we have to populate the wizards with the necessary information on every run.

A very good example of the usage of a DSL, together with recipes is in the EFx Software Factory, which can be seen here.

DSL Challenges

So, if Domain Specific Languages can add value to our software factories, why haven't we seen them in many factories yet? We think the answer to this question is easy. The reason for that is the lack of "out of the box" integration between GAT and the DSL Tools. Currently, it is hard to use model information in our GAT recipes, without writing the custom code to support this ourselves. Luckily this scenario will be supported soon by means of a DSL Power Toy that becomes available.  

[A more complete description of Domain Specific Languages, the Microsoft DSL Tools, together with an example of an 'real life' DSL can be found in this article]

Integrated Design Experiences

The technologies that we discussed so far are first class citizens in Microsoft's software factory offering. However, there are more technologies, which are not dedicated “software factory tools”, that can add enormous value to our software factories.

Be warned, these technologies are a little more difficult to use than those described earlier.

Tool-Windows and Custom Editors

One of the things we will notice when we are building software factories is that there will be situations where the decoupled GAT/GAX and DSL Tools features simply don't provide the experience we need. For example, we might want to provide a more integrated user experience for our factory users and decide that we need to provide the user with a larger view of the factory product.

To support this product view we might want to build custom tool-windows or specific editors. An example of this can be seen in the EFx Factory that hosts a custom tool-window that provides a ’Product Explorer’ view for the factory in addition to the default ‘Solution Explorer’ view.

We may want to enhance views with the use of customer editors that provide different abstractions and increase the productivity of modifying the view. An example of that can be seen here where we built a custom tool-window to maintain hierarchical data within a DSL.

It's good to know that we can use the VS SDK for all that. This SDK provides us with utilities and documentation that explains, for example, how to build a tool-window and hook that in Visual Studio. The bad part of this is that creating Visual Studio extensions like this isn't always as straight forward as we like it to be. Besides, building just the tool-window (or editor) isn't enough; we also have to make sure it integrates with GAX or the DSL Tools. Unfortunately this isn't something that is described in great detail anywhere so we are on our own in implementing this. 

If you are interested in this type of integration you better make sure you have the VS SDK documentation ready and find your way to the GAT, DSL Tools and Visual Studio Extensibility forums!

Distributed Application and System Designers 

One means of providing a high level view of your product is to leverage the ‘Distributed Application and Systems Designers’ that come with the ‘Visual Studio Team Edition for Software Architects’. These designers, although currently geared to deployment, provide a useful view of applications which can be composed into larger black box systems. If your factory targets the domain of applications represented on this design surface, this should be a natural choice for leveraging in describing your product, rather than creating DSL’s which duplicate this functionality.

Unfortunately, integrating with these designers today is very challenging, although it is possible to provide custom shapes using the SDM SDK (part of the VS SDK). Integrating recipes and solution templates into these designers is more of a commitment and currently not officially supported since this requires integration with a non-public API. However it is possible to a large degree, as demonstrated by the EFx Factory here. Where you can ‘drill into’ the applications on these diagrams and expose DSL’s containing the inner workings of the applications. You can also ‘Implement’ these shapes by using custom GAT project templates.

In the future, this type of integration will become a lot easier and integration between these designers and GAT and other DSL’s will be a better experience for factory builders. This will provide many factories the capability for top-down solution design.

Summary

In this post we have discussed the relevant technologies that we can use to build automation assets for real life software factories today. We haven't talked about technical details, issues, missing features, etc. which of course are there. The technologies are still in their early days and scenarios for software factory use are still maturing. The learning curve is steep now, and that will definitely become easier in the coming years. However, it is absolutely possible to build some very good factories with the above described technologies. So, if you are interested in factories you better start looking at these technologies right away! 

Now that you know what to do, and what technologies to use for that, the next post in this series “How would you build it?” addresses how to put all these assets together in a process of assembling your factory.

Previous posts in the Factories 201 series:

What are they (concretely)? (Edward)

When would you build one? (Jezz)

---

To continue the factory 201 series let's discuss a next question: What do I tell my manager? Or maybe, how do we explain the benefits of software factories to the management to justify moving to a software factories approach.

Before we ask the management to move to a software factories approach we have to explain what the value proposition of a software factory is, and what is to be gained from a software factory approach. As we all know, most managers are not interested in technical details so we have to focus on the topics that are important for them! However, the value that we can get from this approach requires a change in the way we develop software today, and this requires a firm commitment from management to achieve success with this approach. Let’s have a look.

Please note that in this post I am using my own experiences from working for a System Integrator. These types of companies are building and using software factories in-house to deliver products for customers. Some of the topics I describe might be (slightly) different for other type of companies, for example for companies that build factories and sell them to their customers without using them in-house.

Value Proposition

The first step might be to tell them what a factory is, we can use the “What are they (concretely)?” post for that. The next step to get them really interested is to explain the benefits of software factories.

Increase Productivity

Probably the easiest benefit to explain is increase of productivity, decrease in cost. Factories automate parts of our development and design work and as a result of that the development team will be more productive during the development phase of the product, and that reduces development costs. But there is more!

Predictable, Consistent Results

Factories will also standardize the product that is developed and therefore we need less time for design and proof of concepts. Another benefit is the improved and consistent quality of the product that is built by the factory. After all, the assets for the factory are built by domain experts and remove the dependency on the very few highly skilled resources (sometimes referred to as ‘local heroes’) or over-resourced projects, to deliver the same high quality work.  A lot of companies are working on improving their software delivery process, reaching a certain CMMI levels and software factories can definitely help with that.

Re-use of: Skills, Resources, Value

Factories use abstractions to simplify the development of solutions. They encapsulate domain specialist experiences and knowledge into the assets behind these abstractions. As a result of that we don’t need to constantly staff each project with very best domain experts, since the assets themselves contain the required experience to solve the problem for us in the best way. Basically, the factory provides a collection of domain skills and experience we can use on each project. These assets represent the actual captured value of our skilled resources.

Furthermore, it’s possible we can re-sell these assets to other companies who may also find getting skilled resources a challenge.

Competitive Advantage

All of the above mentioned benefits are somewhat related to the product that is delivered from the factory, the resources we need for that and are mostly focused on the internal organization. Let’s have a look at some benefits that are recognized externally by investing in factories.

Software factories are hot and will probably become even hotter when they reach the larger community. This is obvious because the value proposition is so strong! Therefore companies can use the presence of software factories in marketing campaigns, and use them as a means to communicate their effectiveness and professionalism to their customers. They can display their innovation in this space and can use that in their recruitment activities to attract senior and skilled professionals.

Resource Attainment

Of course, in the end, our company can also leverage the factory approach to help solving its own resource issues. We can use the best resources to do the most challenging work and actually built the software factories. In this way we continuously challenge these resources, instead of making them participate in ‘boring’ project work and ordinary tools, to make it more likely they will stick with the company. The less experienced resources can become the factory users and be way more productive as they were before.

Commitments

Of course, after telling all of the above described benefits to our management they can't wait to adopt the software factories approach, but wait! There is more we have to tell them to make sure the adoption of software factories within our company has a fair chance to succeed.

Dedicated Resources

We already discussed the requirement for dedicated resources in a previous post (“When would you build one?”) but we cannot stress enough that you have to use the very best experienced, skilled architects and developers to build your factory. They have to be domain experts for the product the factory outputs but also have experience in software (tool) development in general.

However, we all know that the very best resources are the first resources to be pulled off a project and put on another. Since the value of factory assets (and therefore the factory as a whole) directly depend on these resources, it is critical they must not be removed from the factory development project. We all know, and so do the managers, that it’s our customers, market, and organizational culture who decide the allocation of our resources internally. To mitigate this we have to change these conditions to free ourselves from this powerful force to be able to keep these resources focused. The factory development project needs to be separated from the core business development projects, shipping ‘ordinary’ products, and requires a new organizational structure, cost structure, goals, culture and market to succeed. Management needs to support this.

Return on Investment

Return on investment is about profit and growth and thus more important than anything in the world for our management. We have seen the benefits that factories can bring us but before the management is willing to invest in software factories they probably want to know how fast they get their return on investment. After all, why would they want to invest in something that never pays back?

As we have already seen in our "When would you build one?" post, we probably need to build more than ~5 products with a software factory before it pays back so we better make sure we invest in a factory that is likely to be used more than once. We will discuss how and when this investment is returned in one of our future posts “How long will it take?”

Of course, the expectations on the ROI will be different than the established expectations for existing ordinary product development. So, there again, this calls for a separate organizational structure with its own values on what to expect from the product the factory delivers.

Not a Big-Bang Effort!

We have to make sure we keep the initial investment costs as low as possible. Therefore, trying to build a huge, fully featured factory from scratch in one attempt doesn't make sense, not only from a technical perspective but also from an investment perspective. We need to see a return on each iteration of the factory development.

To be successful, a progressive, iterative approach is preferred. The best thing you can do here is to start with small increments in automation of the product your factory creates. With each increment we identify the variable parts of the product and design those into the next iteration. This way, management can see an incremental return on every cycle.

To ensure your factory is applicable to more scenarios in its market, make sure management promote its use in many projects, even if the factory covers only a (relative) small part of the solution that is built. This way your factory can quickly identify many valuable scenarios of use, and support those in its evolving design.

Another option we have to minimize the initial investment costs is to re-use and customize an existing factory. For example, Microsoft patterns & practices released some (early examples of) software factories that are totally free and ready for use. It only takes us a few days to get familiar with these factories. After that we are ready to start using them in real life projects. Our project will benefit from the factory that we are using and in the meantime we will build experience with using software factories in our software development process. All of this with minimal investments but the experience we will gain from that will be invaluable for our future factory development activities. 

Measuring Success

One of the first questions we will have to answer when asking for an investment in factories is: "how do we measure the success of the factory?” In other words, we know the factory is supposed to increase our productivity, improve the quality and degree of standardization, but how do we notice and when will we see our first results? We have to make sure the management knows what to expect and therefore we have to specify this upfront.

For example, we can state the factory delivers a 5 % productivity increase during the development phase for projects larger than 2000 man hours, we can maybe save 80 hours during the design phase because we can use a standardized architecture documents and need less time for building a proof of concept. Further we have to specify when we can expect these results. For example, do we benefit from the factory in our first project or is it more likely this will happen in the second or even third project because the factory needs some improvements, people have to get familiar with it, etc. Of course, all of this depends on the state of the factory itself, the project, etc. but it is crucial that we use this kind of data to validate the effectiveness of the factory and the investment we are making in software factories.

Promote Use and Feedback

Factory development isn't something we do as a "stand alone" activity. After all, the factory is supposed to be re-used by a lot of projects and different project teams within our company. To make sure we don't suffer from the "not invented here" syndrome we have to make sure the factory authoring team, but also the management communicates openly about these initiatives. We have to tell our colleagues about our initiative, demo an early drop or even better, let them play with it and ask them to provide feedback, tips hints, etc.

Support Factory Adoption

Another thing we can do to increase the adoption of our factory is to help people use the factory on their project. Tell them a member of the factory authoring team is available at the start of their project to support them in using the factory. This will give them a head start, not only from the productivity and guidance they get from the factory itself but also from a domain expert that demonstrates them how to best use the factory in their situation. From experience we can tell this is something that is really appreciated by most projects. We can also ask projects that we supported to provide feedback to the management and tell them how the factory helped them in their project. This will definitely help us in marketing our factory approach within the company.

Summary

In this post I tried to list the benefits of software factories for our company. From reading this list it might be tempting to move to a software factory approach as soon as possible in your current everyday development. However, to make this move successful we have to make sure we get the full support and commitment from the management within the organization. They need to see the value and benefits of this approach and how they can minimize risk. We also discussed the need for their support for an organizational structure that is separated from the company’s core business with its own structure, culture, goals and values on what to expect from move to the software factory approach.