Previous posts in the Factory 201 series:

• What are they (concretely)? (Edward)
• When would you build one? (Jezz)
• What do I tell my Manager? (Edward)
• How long will it take? (Jezz)
• What would you build? (Jezz)

To continue the thread of actually creating your factory, in this post we will answer the question "With what would you build it?" We will discuss the software factory enabling technologies available today (on the Microsoft platform) and don’t speculate about possible future tooling. In our previous post “What would you build?” we discussed what we would build for our factory, and now we will see how that is mapped and addresses by specific technologies.

Let's have a look at some tooling and technologies and see how we can use that to build our software factories, starting today.   

Automated Actions

Today's generations of software factories, like the ones available from Microsoft p&p, are primarily composed of a related set of automated activities (called recipes) that encapsulate a huge amount of best practices and domain expertise. The recipes often represent repetitive development and design tasks which result in generated source code or configuration files to help accelerate the development of our factory product. Let’s find out what tooling we can use to build these automated activities and how we can expose these, so called recipes, to our factory users.

Guidance Automation Toolkit/Extensions

The Guidance Automation Toolkit (GAT) is a toolkit that simplifies the development of automated actions into re-usable assets and provides mechanisms to make these assets available to architects and developers using Visual Studio. To accomplish this, GAT leverages the Guidance Automation Extensions (GAX) which is actually a runtime environment that expands the capabilities of Visual Studio and makes it easier to integrate and run automate tasks from the Visual Studio environment. So, we are actually using GAT and GAX together to build and run recipes and deploy them as guidance packages. ehhh...? Now what are recipes and guidance packages?

Recipes

Think of a recipe as set of automated instructions, called actions that together represent a specific automated activity or task. These actions are relatively small units of work and can be anything from creating a file, getting the value of a registry key or return the name of the currently selected project within a Visual Studio solution.

A recipe is defined in an XML file in a predefined structure that is interpreted by GAX at runtime. GAX supports invoking wizards for a recipe to gather user input that can be used as variables during the execution of the recipe to create variability in the artefacts the recipe outputs or configures. For example, a wizard can prompt the user to select a file location, ask for a name of a file that gets generated or ask the user to select items from a list. Recipes that don’t require user input can gather the variables from the environment to use in the same way. Recipes also can provide a convenient means of giving the user feedback about the actions being performed.

We can build recipes using GAT, it provides us with a special solution template in Visual Studio to author recipes, it comes with predefined solution structure, example actions and recipes and other utilities that give us a head start in building our own recipes and wizards.

Guidance Package

A Guidance Package is an installable Visual Studio package that contains a collection of recipes (and some their support classes like, converters, value providers, etc.) and is basically our deployment mechanism for the recipes that we built and want to expose in our software factory. In fact, the current generation of software factories as we see today are for the most part simply implemented as pure guidance packages.

That leaves us with another question: "how do we build a guidance package"? Again, this is where GAT comes in. Actually, GAT is a guidance package itself that helps architects and developers build guidance packages. Once we are done developing and testing our guidance package we can create a Windows installer package (MSI) and distribute our guidance package to our users.

Unfortunately, building and testing guidance packages (recipes) isn't as straightforward as we want it to be. For example, building recipes basically comes down to hacking in XML files and isn't supported by any graphical designers. Also the documentation is still limited. This makes the learning curve for developing guidance packages a little steep.

Artefact Templates & Generation

As said, some of the automated activities in our software factory will possibly result in one or more generated artefacts for our solution. These artefacts can be anything in the range from source code, XML files or a piece of documentation. To support this kind of artefact generation in our software factory we can use text templates (also referred to as "T4" templates) and the Text Templating Engine. This technology is included in GAT but also in the DSL Tools which we will discuss a little later in this post.

A text template is basically a text file that contains the (boilerplate) text you want to generate (i.e. C#/VB.NET code, XML, text etc.) that is mixed with control markers (<#, #>) and control blocks. The text between the markers is evaluated and provides the dynamic behaviour to the text template. Control blocks are used to control the processing flow of our templates. In other words they determine what text is output and how the Text Templating Engine navigates over a provided object model. This works very similar to how classic ASP work where you could combine server side directives with the raw HTML code.

Below an example of a T4 template that comes from Service Factory. The text between the markers (in bold) is evaluated and mixed with the static text in the template.

namespace <#= ServiceImplementationNamespace #>

{

   [System.Web.Services.WebService(Namespace = "<#= ServiceContractNamespace #>", Name = "<#= ServiceImplementation #>")]

   [System.Web.Services.WebServiceBindingAttribute(ConformsTo = System.Web.Services.WsiProfiles.BasicProfile1_1, EmitConformanceClaims = true)]

   public class <#= ServiceImplementation #> : <#= ServiceContractInterfaceNamespace #>.<#= ServiceContractInterface #>

   {

   }

}

The most powerful feature of text templates is that they can be invoked from recipes and they can receive variables from our recipes. In this way we can build recipes that collect input from our users or factory environment and use that as input in our text templates to generate artefacts.

A major advantage of using text templates as a means to generate artefacts is that they can be customized very easily, typically to reformat the textual output or enhance it in some way (perhaps to suit an organisations coding standards for example). Text templates are deployed as plain text files so it is straight forward to customize them.

The fact that text templates are so easy to customise makes them perfectly suited for an iterative development approach. We can start with a “hard coded” copy of the desired output and we can identify the variability points in the code. We can then pass some arguments to the template and from that point we can add control blocks to parameterize it and hook into the underlying argument’s object models. One disadvantage of a text template is the lack of designer support like syntax colouring (we added the colouring in the example manually) and intellisense. This makes writing and maintaining large text templates a bit ’un-guided’.

Solution & Project Templates

As we saw in “What would you build?” we can use the solution structure to provide guidance on how to physically relate the artefacts that represent the product of our factory. Unfortunately, defining this (physical) structure is often challenging because of the struggle to define both the deployment model and the logical architectural model of the product in this single structure. Today, the solution structure is all we have (Solution Explorer), so let’s have a look how we can implement predefined solution structures in our factories to describe the physical structure (deployment) of the product.

[The logical architecture is best left to another separate view, which as we will see later we can provide with deep Visual Studio integration]

Solution structures are captured in so called Visual Studio Templates (*.vstemplate files) that are written in XML and are expanded by the Visual Studio template engine. Visual Studio 2005 now supports the creation and packaging of these templates. However, these templates are by default static, and require a separate mechanism to gather variables from the user or environment to enhance them with dynamic values. As it happens, GAT has just that mechanism, and can bind recipes and wizards to these templates.

Let's have a look how we can use this mechanism with GAT to ‘unfold’ certain types of templates:

• Solution templates. These templates unfold an initial Visual Studio solution file (*.sln) and structure that could represent the entire physical structure of the product of our factory.
• Project templates. These templates unfold a project (*.csproj, *.vbproj etc.) within an existing solution.
• Item templates. These templates unfold new project items such as a class files, XML files or any other valid project item.

One of the primary features of GAT is automating the creation of the solution structure. GAT accomplishes this by hooking into the VS template mechanism. This means recipes are allowed to participate in the template unfolding so they can, for example, collect input from the user to customize the names of the template artefacts and take additional actions to further configure the created solution items. For example, write some settings to a app.config file. A perfect example of this is demonstrated in Service Factory where the user is prompted to provide the name of the service, which is then used in the naming of the solution and projects within the solution. We have now discussed GAT and recipes, and learned they are perfectly suited for automating tasks and providing initial solution structures. Let’s look at providing abstractions to our factory and enhancing the design experience and productivity of automating solution assembly. Recipes with wizards often do form a visual abstraction of the underlying part of the product they are automating, but this abstraction is lost when the wizard completes. Let’s have a look at Domain Specific Languages and the DSL Tools and see how they can help enhance that experience and increase the level of abstraction represented in our factory.

Domain Specific Languages

A Domain Specific Language is a custom language that targets a specific domain and helps us describe the problem by using concepts that make sense for the target domain. But how would we build one? This is where the DSL Tools come to rescue. We can use The DSL Tools, (part of VS SDK), to define our domain model and easily represent that domain model with a graphical designer. Together, the domain model and the graphical designer form our DSL that we can integrate into our software factory.  

[You can also provide a textual representation of your language instead of visually, but this is not supported currently out-of-the-box with the DSL Tools yet, unless you are prepared to define that language in XML.]

The DSL Tools also supports artefact generation using T4 templates which means we can write templates that generate (source code) artefacts directly from models we created in our DSL. The DSL Tools is just everything we need to build Domain Specific Languages today.

How would we use the DSL Tools?

So, how would we use the DSL Tools to build a DSL for our factory? Would we build one big DSL and designer that covers the complete product? The answer to this question is probably NO. Although our factory only targets one product (with several variants maybe!) it is likely that the scope of the domain is too broad to fit in just one designer (view). Therefore, we probably want to build a DSL that only targets one concern, component or aspect in our software factory.  For example, we can use a DSL to model a piece of the architecture of the product that is build in our factory or use a DSL to model one of the work products (logical components) in our factory. In this case both DSL's help us presenting the right level of abstraction to our factory user for each of the sub-problems we identified in our factory.

But wait…, we have to make it very clear why we state to keep the scope of the DSL (domain model) limited to only one concern or piece of architecture. In fact, we can think of examples, like the factory schema, that are represented in a ‘larger’ domain model. Mostly, this is because off a limitation in the current technology. The current release of the DSL Tools only supports one view per domain model. In other words, the view that is exposed in our DSL is tightly coupled to the domain model. In future releases of the DSL Tools this limitation will disappear which means we can define multiple views with each view exposing only a part of a, possibly larger, domain model (like the factory schema). Please note that this doesn’t mean we have to build huge domain models in the future, but it will give us more flexibility in our domain model (DSL) design.

[You can find more information on how to use DSL’s in the context of software factories in the article ‘Bare Naked Languages or What Not To Model’ from Jack Greenfield]

Another important limitation of the current DSL tools is that there is no easy way to reference elements in one domain model with those in another, so stitching together domain models to fit larger domains is also challenging. However, there are DSL Power Toys soon to be released called ‘DSL Integration Service’ that enable that cross-referencing.

Another thing to keep in mind is that we don't have to start with a DSL in our factory from the very first beginning. It is absolutely possible that an asset that is implemented as a GAT recipe in a first iteration of our factory evolves in a DSL in one of the next iterations of our factory development cycle.

DSL Advantages

An important advantage of a DSL in our factory is providing a simplified view (abstraction) of the problem domain and to help us to provide architectural guidance in assembling that part. By using specific languages we can present our factory users with the right level of abstraction and use the designer as a way to provide context for our actions and the artefacts we are delivering from our factory. Recipes can also be applied to these designers to manipulate the underlying domain model, and automate the configuration and creating of the elements in these models.

The domain model that is introduced by the DSL (and stored on disk) can be validated and evaluated to determine the progress, validity and completeness of the product we are building. The domain model also provides us with a means to introduce “state” in our factory. We can generate artefacts over and over again for our populated domain model, something we can't do easily with just code generating recipes, where we have to populate the wizards with the necessary information on every run.

A very good example of the usage of a DSL, together with recipes is in the EFx Software Factory, which can be seen here.

DSL Challenges

So, if Domain Specific Languages can add value to our software factories, why haven't we seen them in many factories yet? We think the answer to this question is easy. The reason for that is the lack of "out of the box" integration between GAT and the DSL Tools. Currently, it is hard to use model information in our GAT recipes, without writing the custom code to support this ourselves. Luckily this scenario will be supported soon by means of a DSL Power Toy that becomes available.  

[A more complete description of Domain Specific Languages, the Microsoft DSL Tools, together with an example of an 'real life' DSL can be found in this article]

Integrated Design Experiences

The technologies that we discussed so far are first class citizens in Microsoft's software factory offering. However, there are more technologies, which are not dedicated “software factory tools”, that can add enormous value to our software factories.

Be warned, these technologies are a little more difficult to use than those described earlier.

Tool-Windows and Custom Editors

One of the things we will notice when we are building software factories is that there will be situations where the decoupled GAT/GAX and DSL Tools features simply don't provide the experience we need. For example, we might want to provide a more integrated user experience for our factory users and decide that we need to provide the user with a larger view of the factory product.

To support this product view we might want to build custom tool-windows or specific editors. An example of this can be seen in the EFx Factory that hosts a custom tool-window that provides a ’Product Explorer’ view for the factory in addition to the default ‘Solution Explorer’ view.

We may want to enhance views with the use of customer editors that provide different abstractions and increase the productivity of modifying the view. An example of that can be seen here where we built a custom tool-window to maintain hierarchical data within a DSL.

It's good to know that we can use the VS SDK for all that. This SDK provides us with utilities and documentation that explains, for example, how to build a tool-window and hook that in Visual Studio. The bad part of this is that creating Visual Studio extensions like this isn't always as straight forward as we like it to be. Besides, building just the tool-window (or editor) isn't enough; we also have to make sure it integrates with GAX or the DSL Tools. Unfortunately this isn't something that is described in great detail anywhere so we are on our own in implementing this. 

If you are interested in this type of integration you better make sure you have the VS SDK documentation ready and find your way to the GAT, DSL Tools and Visual Studio Extensibility forums!

Distributed Application and System Designers 

One means of providing a high level view of your product is to leverage the ‘Distributed Application and Systems Designers’ that come with the ‘Visual Studio Team Edition for Software Architects’. These designers, although currently geared to deployment, provide a useful view of applications which can be composed into larger black box systems. If your factory targets the domain of applications represented on this design surface, this should be a natural choice for leveraging in describing your product, rather than creating DSL’s which duplicate this functionality.

Unfortunately, integrating with these designers today is very challenging, although it is possible to provide custom shapes using the SDM SDK (part of the VS SDK). Integrating recipes and solution templates into these designers is more of a commitment and currently not officially supported since this requires integration with a non-public API. However it is possible to a large degree, as demonstrated by the EFx Factory here. Where you can ‘drill into’ the applications on these diagrams and expose DSL’s containing the inner workings of the applications. You can also ‘Implement’ these shapes by using custom GAT project templates.

In the future, this type of integration will become a lot easier and integration between these designers and GAT and other DSL’s will be a better experience for factory builders. This will provide many factories the capability for top-down solution design.

Summary

In this post we have discussed the relevant technologies that we can use to build automation assets for real life software factories today. We haven't talked about technical details, issues, missing features, etc. which of course are there. The technologies are still in their early days and scenarios for software factory use are still maturing. The learning curve is steep now, and that will definitely become easier in the coming years. However, it is absolutely possible to build some very good factories with the above described technologies. So, if you are interested in factories you better start looking at these technologies right away! 

Now that you know what to do, and what technologies to use for that, the next post in this series “How would you build it?” addresses how to put all these assets together in a process of assembling your factory.

Previous posts in the Factories 201 series:

What are they (concretely)? (Edward)

When would you build one? (Jezz)

To continue the factory 201 series let's discuss a next question: What do I tell my manager? Or maybe, how do we explain the benefits of software factories to the management to justify moving to a software factories approach.

Before we ask the management to move to a software factories approach we have to explain what the value proposition of a software factory is, and what is to be gained from a software factory approach. As we all know, most managers are not interested in technical details so we have to focus on the topics that are important for them! However, the value that we can get from this approach requires a change in the way we develop software today, and this requires a firm commitment from management to achieve success with this approach. Let’s have a look.

Please note that in this post I am using my own experiences from working for a System Integrator. These types of companies are building and using software factories in-house to deliver products for customers. Some of the topics I describe might be (slightly) different for other type of companies, for example for companies that build factories and sell them to their customers without using them in-house.

Value Proposition

The first step might be to tell them what a factory is, we can use the “What are they (concretely)?” post for that. The next step to get them really interested is to explain the benefits of software factories.

Increase Productivity

Probably the easiest benefit to explain is increase of productivity, decrease in cost. Factories automate parts of our development and design work and as a result of that the development team will be more productive during the development phase of the product, and that reduces development costs. But there is more!

Predictable, Consistent Results

Factories will also standardize the product that is developed and therefore we need less time for design and proof of concepts. Another benefit is the improved and consistent quality of the product that is built by the factory. After all, the assets for the factory are built by domain experts and remove the dependency on the very few highly skilled resources (sometimes referred to as ‘local heroes’) or over-resourced projects, to deliver the same high quality work.  A lot of companies are working on improving their software delivery process, reaching a certain CMMI levels and software factories can definitely help with that.

Re-use of: Skills, Resources, Value

Factories use abstractions to simplify the development of solutions. They encapsulate domain specialist experiences and knowledge into the assets behind these abstractions. As a result of that we don’t need to constantly staff each project with very best domain experts, since the assets themselves contain the required experience to solve the problem for us in the best way. Basically, the factory provides a collection of domain skills and experience we can use on each project. These assets represent the actual captured value of our skilled resources.

Furthermore, it’s possible we can re-sell these assets to other companies who may also find getting skilled resources a challenge.

Competitive Advantage

All of the above mentioned benefits are somewhat related to the product that is delivered from the factory, the resources we need for that and are mostly focused on the internal organization. Let’s have a look at some benefits that are recognized externally by investing in factories.

Software factories are hot and will probably become even hotter when they reach the larger community. This is obvious because the value proposition is so strong! Therefore companies can use the presence of software factories in marketing campaigns, and use them as a means to communicate their effectiveness and professionalism to their customers. They can display their innovation in this space and can use that in their recruitment activities to attract senior and skilled professionals.

Resource Attainment

Of course, in the end, our company can also leverage the factory approach to help solving its own resource issues. We can use the best resources to do the most challenging work and actually built the software factories. In this way we continuously challenge these resources, instead of making them participate in ‘boring’ project work and ordinary tools, to make it more likely they will stick with the company. The less experienced resources can become the factory users and be way more productive as they were before.

Commitments

Of course, after telling all of the above described benefits to our management they can't wait to adopt the software factories approach, but wait! There is more we have to tell them to make sure the adoption of software factories within our company has a fair chance to succeed.

Dedicated Resources

We already discussed the requirement for dedicated resources in a previous post (“When would you build one?”) but we cannot stress enough that you have to use the very best experienced, skilled architects and developers to build your factory. They have to be domain experts for the product the factory outputs but also have experience in software (tool) development in general.

However, we all know that the very best resources are the first resources to be pulled off a project and put on another. Since the value of factory assets (and therefore the factory as a whole) directly depend on these resources, it is critical they must not be removed from the factory development project. We all know, and so do the managers, that it’s our customers, market, and organizational culture who decide the allocation of our resources internally. To mitigate this we have to change these conditions to free ourselves from this powerful force to be able to keep these resources focused. The factory development project needs to be separated from the core business development projects, shipping ‘ordinary’ products, and requires a new organizational structure, cost structure, goals, culture and market to succeed. Management needs to support this.

Return on Investment

Return on investment is about profit and growth and thus more important than anything in the world for our management. We have seen the benefits that factories can bring us but before the management is willing to invest in software factories they probably want to know how fast they get their return on investment. After all, why would they want to invest in something that never pays back?

As we have already seen in our "When would you build one?" post, we probably need to build more than ~5 products with a software factory before it pays back so we better make sure we invest in a factory that is likely to be used more than once. We will discuss how and when this investment is returned in one of our future posts “How long will it take?”

Of course, the expectations on the ROI will be different than the established expectations for existing ordinary product development. So, there again, this calls for a separate organizational structure with its own values on what to expect from the product the factory delivers.

Not a Big-Bang Effort!

We have to make sure we keep the initial investment costs as low as possible. Therefore, trying to build a huge, fully featured factory from scratch in one attempt doesn't make sense, not only from a technical perspective but also from an investment perspective. We need to see a return on each iteration of the factory development.

To be successful, a progressive, iterative approach is preferred. The best thing you can do here is to start with small increments in automation of the product your factory creates. With each increment we identify the variable parts of the product and design those into the next iteration. This way, management can see an incremental return on every cycle.

To ensure your factory is applicable to more scenarios in its market, make sure management promote its use in many projects, even if the factory covers only a (relative) small part of the solution that is built. This way your factory can quickly identify many valuable scenarios of use, and support those in its evolving design.

Another option we have to minimize the initial investment costs is to re-use and customize an existing factory. For example, Microsoft patterns & practices released some (early examples of) software factories that are totally free and ready for use. It only takes us a few days to get familiar with these factories. After that we are ready to start using them in real life projects. Our project will benefit from the factory that we are using and in the meantime we will build experience with using software factories in our software development process. All of this with minimal investments but the experience we will gain from that will be invaluable for our future factory development activities. 

Measuring Success

One of the first questions we will have to answer when asking for an investment in factories is: "how do we measure the success of the factory?” In other words, we know the factory is supposed to increase our productivity, improve the quality and degree of standardization, but how do we notice and when will we see our first results? We have to make sure the management knows what to expect and therefore we have to specify this upfront.

For example, we can state the factory delivers a 5 % productivity increase during the development phase for projects larger than 2000 man hours, we can maybe save 80 hours during the design phase because we can use a standardized architecture documents and need less time for building a proof of concept. Further we have to specify when we can expect these results. For example, do we benefit from the factory in our first project or is it more likely this will happen in the second or even third project because the factory needs some improvements, people have to get familiar with it, etc. Of course, all of this depends on the state of the factory itself, the project, etc. but it is crucial that we use this kind of data to validate the effectiveness of the factory and the investment we are making in software factories.

Promote Use and Feedback

Factory development isn't something we do as a "stand alone" activity. After all, the factory is supposed to be re-used by a lot of projects and different project teams within our company. To make sure we don't suffer from the "not invented here" syndrome we have to make sure the factory authoring team, but also the management communicates openly about these initiatives. We have to tell our colleagues about our initiative, demo an early drop or even better, let them play with it and ask them to provide feedback, tips hints, etc.

Support Factory Adoption

Another thing we can do to increase the adoption of our factory is to help people use the factory on their project. Tell them a member of the factory authoring team is available at the start of their project to support them in using the factory. This will give them a head start, not only from the productivity and guidance they get from the factory itself but also from a domain expert that demonstrates them how to best use the factory in their situation. From experience we can tell this is something that is really appreciated by most projects. We can also ask projects that we supported to provide feedback to the management and tell them how the factory helped them in their project. This will definitely help us in marketing our factory approach within the company.

Summary

In my previous and first post in the "Factories 201" series I hinted at Jezz to join me in this initiative. As you can read from this post on his blog he agreed to participate. We had a little chat to talk about some of the firts topics we would like to discuss and I just noticed that Jezz  posted his first post "Factories 201 - When would you build one?". So, if you are interested in factories I suggest you subscribe to his blog (if not all ready) and follow us in this joined effort.

Enjoy!

In my previous post "Software Factories: Where to start" I tried to list some resources that might be interesting for new comers in the software factory space. I have been thinking about factories and also participating in this space for some time now and I recently realized that, although there is a lot of buzz around software factories it is still very difficult to enter the world of software factories.

It might sound simple; install GAT and the DSL Tools and build your first factory but unfortunately, from what I experienced, it isn't that easy (it might be me of course!). I had (and still have) a lot of questions like for example: "What is a software factory", "When would you build one", "How would you build one", "What tools would you use" and many, many more.

It's therefore that  I decided to share some of my thinking's about factories to hopefully provide some answers to people that are about to join the software factories space. I am not a very formal guy so will try to write about these topics from a (factory) developer or (factory) architect perspective and keep it practical and understandable.

To be honest I am not sure I have the answers for all questions just yet but we will see where this leads too. Regular readers of this blog might have noticed that I discuss a lot of my software factory related work with Jezz. He did a lot of interesting posts on factories lately so hopefully he reads this post and will comment, or even better, add some content when he sees me struggling in my attempts (don't feel forced Jezz, but know that I am counting on you to help me out! ).

Ok, now we know the plan, let's make sure the title of this post makes sense and start with the first question:

What are they (concretely)?

If you were looking for a formal definition of what a software factory is, you would find one in the Software Factories” book written by Jack Greenfield and Keith Short. In this book, a software factory is defined as:

“A Software Factory is a software product line that configures extensible tools, processes and content using a software factory template based on a software factory schema to automate the development and maintenance of variants of an archetypical product by adapting, assembling and configuring framework-based components.”

Fair enough, but do we, as a developer, know what to build by just reading this definition? Not me. The patterns & practices team came up with a more pragmatic definition, that is in fact more accurate for the factories being built today:

“A software factory is a structured collection of related software assets. When a software factory is installed in a development environment, it helps architects and developers predictably and efficiently create high-quality instances of specific types of applications.”

But even this definition, does not really tell you what the factory itself is in concrete terms, at least not in enough detail to help you get started in building one. So we are going to use a much more simplified, and concrete definition that is more to the point from a developers perspective, describing what it actually is and how it physically works. Then from this, you will get a basic understanding of how the definitions above describe more precisely the moving parts of the factory.

[Please note, that this definition is being used here for the purposes of helping you understand the context of this series on building factories, and is not a formal and general definition of what a software factory is when it is finally shipped. The following definition only focuses on the concrete automation part of a factory. Inside the box of a real factory you will find other physical guidance assets like documentation, samples, reference implementations, training media etc. and together these make up a complete software factory.]

It’s a Tool!

First of all, a software factory is a software development tool. This tool is used to automate the assembly and/or configuration of a software solution that is addressing a well known and described, specific problem domain. It’s here where we see our first difference between a software factory and another development tool like for example Visual Studio.NET. A software factory is targeted at only a small problem domain while we can use Visual Studio.NET to basically build anything we want (of course within limits).

Configures and runs within Visual Studio

Although there is a difference between software factories and Visual Studio.NET (for the Microsoft platform) in the size of the problem domain they target, they are also very much related. This is because a software factory runs within Visual Studio.NET and basically configures this environment to target the development of a specific problem domain. Things like menus, windows, editors, layout etc.

For example, think of the Microsoft p&p Service Factory that configures Visual Studio to develop web services. This (early generation) software factory adds some menus, wizards and templates to the standard Visual Studio environment (Solution Explorer) to help developers build services.

Uses Abstraction

Another characteristic of a software factory is that it provides abstractions to simplify the problem domain it targets. An abstraction is something that simplifies the thing it describes and only focuses upon on only the important characteristics or variable parts of it.

In factories you can provide abstractions using several techniques like frameworks, specialized editors, wizards, models and diagrams.

Although providing abstractions is one of the most important requirements of a software factory, we haven’t seen a ‘publically available’ factory that uses all of these techniques - but I am sure we will see more in the future.

If you are interested in examples of abstractions that are used in factories, you might want to read this MSDN article about the EFx factory. The DSL that is used in this factory is a perfect example of the usage of abstraction.

Creates a Product

Another very important aspect of factories is its output. A software factory outputs a generated solution for the problem domain it targets. The product of a factory is what it outputs. A software factory doesn’t necessarily output source code. For example, it’s also possible that a factory generates some configuration files that are used to configure an existing framework, some documents, or whatever makes sense to the problem domain it targets. They can also just configure other products which solve a particular domain. One thing to keep in mind is, that is likely that the factory output needs some manual work or customization before it is ready to use.

Customization

There is also another type of customization that is very applicable to software factories. This is, the customization of the software factory itself instead of the output it generates. This type of customization leads to an iterative approach where we start with a small factory that iteratively increases. It is this type of customization that we use to address a new concern in the factory or, solve a defect in the factory product, or customize our assets to suit your development requirements and corporate standards.

Summary

So, here we are. I tried to explain what a software factory really is by describing some off its key characteristics without using common software factory terms like “product line”, “schema”, “variants”, etc. Because of that I might be missing some important stuff but I am sure I can make up for that in a way that make sense in future posts when I cover some other, software factory related topics.

For those of you who can’t wait for that, and like to read a little bit more about a software factories definition in more software factory related jargon, I can recommend this post from Jezz.

If there are any software factory related questions that you might have and want me to answer please let me know by commenting on this post. I cannot guarantee I have the answers but we can at least discuss about it and try to find the answer together.

To be continued…

As it looks like now, 2007 will be the year where software factories get a lot of exposure and attention. We have seen quite a lot of activity in this space in the second half of 2006 but there is definitely more to come! This also means that there will be a lot of new people entering the software factory space in the next coming months. For those of you who are not that experienced yet with software factories I decided to create a short list with some interesting links to existing software factories, forums, blogs, etc. Hopefully, this overview gives you a head start in entering the exiting world of software factories.  

Available software factories

If you are a practical guy (or girl) and like to get your hands dirty as soon as possible you might want to get yourself an existing software factory and start experimenting with it right away. In that case, check out the following offerings from Microsoft P&P. Clicking the links below will bring you to the MSDN landing page for each of the factories were you will find detailed information, links to the community and links to the download pages.

1. Web Service Software Factory
2. Web Client Software Factory
3. Smart Client Software Factory
4. Mobile Client Software Factory

You may also want to subscribe to the blogs of some of the P&P Product Managers: Don Smith, Tom Hollander and Eugenio Pace to keep you up to date with the latest news and future directives of the P&P factories. If you are interested in an indication of the "release dates" of the (future) P&P deliverables, check out their brand new "Upcoming Releases" page. 

Software Factories Tools

After experimenting with these existing factories for some time you might want to modify them (to make them better suite your requirements), extend them with some Domain Specific Languages or completely build your own factories from scratch. In that case you have to check out the technologies that drive the current wave of software factories.

First of all, we have the Guidance Automation Toolkit which you can use to create “packages” that can be installed in Visual Studio.NET and basically represents software factories as we see them today. You can download it from here and you can ask your questions about it (which you will definitely have ;)) here.

Second, we have the DSL Tools that you can use to create Domain Specific Languages that you can include in your software factories. You can download the DSL Tools from here and ask your questions about it here.

If you are interested in the combination of the Guidance Automation Toolkit and the DSL Tools you might want to read this article that describes some of the possible integration scenarios. (the library that is used in this article isn’t available yet, but will be soon!).

If you plan to start building your own factories, based on the above mentioned technologies, you also might be interested in the Clarius Software Factories Toolkit. This toolkit adds some extra features to the existing technologies to make it easier to develop your own factories. 

Background information, theory and concepts

Of course, there are also people who are really interested in the theory and the concepts behind the software factories strategy. If that’s you, go get yourself the Software Factories book written by Jack Greenfield and Keith Short. Once you have read this book you realize that we are still at the very early beginning with software factories and there is a lot of improvement to be made in the coming years. Some time ago I have written a post myself about some of the limitations in the first wave of software factories. As you can read there today’s software factories lack a model, schema and a lots of other stuff.

If you are interested in the jargon that is often used in the software factories space, check out this “Software Factories ABC” that I co-authored with Jezz. You better subscribe to his blog if you are interested in software factories because Jezz is an active factory blogger and has some really interesting posts about models, schema and tools. If you are still hungry for more information you can also check the software factories landing page on MSDN for more information.

So, we are done with the list. Of course there is a lot more interesting information about software factories but I am sure you will find them when you start your journey by experimenting with the currently available factories and reading the information I linked too in this post. Hopefully this list can be off any help to "software factories newbies" to get familiar with this very interesting technology as soon as possible.

Happy factories!

Update: also check out the Software Factories Swicki as the online search enigne for all factory related stuff!

Service Factory V2, is there!  This release provides some great guidance for building service for WCF and ASP.NET (ASMX). Get it here and make sure to post any feedback and/or questions on the Service Factory community site!

Ok, I have been tagged by Erwyn so here is my list of 5 things you might not know about me.

1. When I got my first job back in 1995 I started programming in Microsoft Visual Foxpro 3.0. At that time I found it SO much better than all the other languages (dBbase III+, Pascal and even Cobol) I programmed in before and was really happy with this decision of my employer to use this. Visual Foxpro was a data centric, object-oriented environment and I continued programming in ( newer versions of) Visual Foxpro till the moment the first CTP of .NET came out. From that moment I started programming C# and never went back. When I had my first look at LINQ I was very surprised to see the “Visual Foxpro data centric approach” being integrated in .NET.
2. I am a KOI keeper! I spend a lot of my spare time on taking care of my 10 beautiful KOI. Because KOI require an excellent water quality I am using several filtration units, millions of Nitrosomonas, UV light and air pumps to keep my pond clear. (some examples of these beauties, unfortunately not mine)
3. I am married for almost 5 years now with my wife Miranda. I live in the Netherlands in a small town named Hoorn and together with my wife I have a beautiful daughter, named Emily, of almost 2 years old. 
4. In the evenings, when I am not busy with my KOI or family, I spend most of my time with my laptop on the    couch. I do not watch TV a lot (only some movies) and don't like to sit behind a desk when I am not at work.  So, all my blog reading, sending mail, MSN, etc. is done from my very comfortable couch.
5. At the age of 17 I started with fitness. Unfortunately I got really addicted to this and fitness slowly became bodybuilding! In that period I was in the gym 7 days a week, ate 3 kilos of cooked potatoes, 300 gram steamed vegetables, a grilled chicken breast fillet, 2 pieces of fruit, some protein shakes a day. At the top of my "bodybuilding carrier" I weight 84 kilogram (184.8 lbs) which is pretty my for a small guy (approximate 5 feet 6 inches) like me. After some years, when I got my first job at a large Dutch Bank, I was aked to wear suits at work. So, I went to the store to buy some new suits. It was at that moment, when I noticed I couldn't buy "normal suits" anymore, I finally realized my body wasn't in normal proportions any more. From that moment I started thinking about quitting fitness (bodybuilding) and luckily after some time I did. (I decided not to post any pictures to make this story not even more embarrassing )

I tag Jezz, Rene, Dennis, Carlo, Christian, Mark

Some time ago I started building a new Domain Specific Language for modeling message and data contracts. Building the DSL was pretty straight forward so it didn't take me too much time to build the beauty. However, during the debugging and testing phase of that DSL I noticed that the user experience of the DSL wasn't too good. For example, this particular DSL lets you model a data contract. Each data contract has a relation with one or more data members (implemented as compartment items) and for each data member you have to set its name, type, order and required properties. So, to add a data member to a data contract in my DSL I first had to do two right mouse clicks, after that I had to type the name of the data member, after that I had to switch to the properties windows and type the various values of the data members properties. This just didn't feel too good, especially when modeling a number of message and data contracts in a row. At that time I decided not to use the DSL, simply because of the, in my opinion, poor user experience.

Just recently, during some other Domain Specific Language related work I noticed again that I had some trouble getting the user experience right for modeling "relations" between model elements. Again, it was the same number of mouse clicks that bothered me (maybe it's just me?!). So, it was about time to try and find a solution for this "problem". I soon decided to build my own custom tool window specifically for my DSL but didn't know yet how to represent the "relational data" in that tool window. Luckily, in one of my talks with Jezz he came up with this brilliant idea to use the "VirtualTreeControl" (Microsoft.VisualStudio.VirtualTreeGrid.dll) for this purpose. He already used this control in the EFx Factory as you can see here. I have to be honest, this control is a real bitch to figure out, it isn't documented and I don't even know if it is officially supported but with some help of Jezz and a wrapper he built for it specifically for DSL relations, I got this working for my message-/data contracts DSL.

With this control you can show hierarchical domain relationships, and each row can have child rows, and you can display whatever columns of data you want from the relationships and domain classes.

As you can see in the picture below I can now maintain my "relational data", for my data contracts (in this case data members) in my custom tool window ("Data Contract Editor". I can easily add new data members and set their properties without having to switch between the mouse and keyboard over and over again. I think this control really rocks and I absolutely love its user experience.

The control is especially useful for exposing domain relations and domain classes which don’t have shapes or connectors on the diagram. For example, children of domain classes displayed in compartments. Therefore the control reduces shape ‘clutter’ on the diagram.

I like to consider tool windows and/or editing controls like this as "out of the box" features for future releases of the DSL Tools! In my opinion, it will dramatically improve the usage scenarios of DSL build with the DSL Tools.

For my current assignment, which is related to integrating existing and developing new Software Factories in our software development process, I decided to have a look at project GlidePath. According to the GlidePath website this project will deliver guidance on how to build software for Vista.  It delivers a web based repository of software factories that can loaded into the Visual Studio environment.  One of the very interesting features of GlidePath is the possibility to update the Software Factories over an RSS feed. It’s actually this feature that made me investigate GlidePath to see if we can use this or a similar approach in our own development process.

Once installed, you can use the Glidepath guidance package to create a Visual Studio solution structure. Instead of generating a full blown solution structure, like for example Service Factory does, Glidepath provides you with a rather empty solution.  At that moment the solution doesn’t reflect the actual product at all but rather provides you with the possibility to decide what factory to use in this solution structure (and thus what product to build?).  In this release of Glidepath you can only select one factory (MicrosISV factory) but I assume this will change in the future. Once I had added the MicroISV factory to my Glidepath solution I was surprised to see a lot of viewpoints popup in my solution structure (at this moment represented as .txt files).  From what I understood from Glidepath the idea behind these viewpoints is that you can select and configure them to specify how to build your product in the factory.  Glidepath will create new projects or enable packages (factories) in your current solution based on the viewpoints you configure for your product. Currently, Glidepath populates the MicroISV factory with viewpoints like “DataAccess”, “Deployment “, “Blogging”, “Legal”, etc.

Figure 1. Glidepath solution structure

After playing with Glidepath for a while I suddenly realized that I didn’t had a clue what kind of product I was actually building in my factory. I could see that my solution was populated with a project for “blogging” and project for “CSharp-WinForms-UX” because I selected these viewpoints for this solution.  However, from looking at the solution structure I wasn’t able to tell whether I was building a “blogging service”, a “blogging site” a “windows client for my blogging service” or whatever. Of course I understand this is related to this very early release of Glidepath and therefore lack of implemented packages, but I realized this was caused by the fact that the Glidepath factory doesn’t  have any notice (yet) about the product itself and therefore couldn’t tell me either what product I was building.

I was interested by this and tried to understand what this would mean for me if I had to implement (build a guidance package) one of the viewpoints for MicroISV. How would I know about the underlying product my viewpoint will be used on? Is it even necessary to have knowledge about that or can we generalize viewpoint implementations so they can be used on all products we are building? To be honest, I don’t have the answers for that at this moment.  My current thinking is that it would be helpful to have a formal description of the product I am implementing my viewpoint for but as said, I don’t have the answers yet.

A related question that interests me is: “what comes first?” Let’s say I was asked to build a factory for product X. Would I start with defining the viewpoints for my factory (security, deployment, etc.) or would I start by describing the basics of product X first? Or would I do this simultaneously?

Logically I would say, I first need a full picture of the product (product X) I am building the factory for.  Ideally, I would describe this product at a conceptual level (product model) in a way that makes this knowledge easily accessible and understandable for other people.  Once I know what I am talking about, and thus have understanding of the problem domain, I can decide what viewpoints are important, how to provide automated guidance for that, etc.

If we have captured the knowledge about the product we cannot only use this to author the factory but also to guide the factory user through the product at runtime. The factory can use the product model to communicate with the factory user in terms that make sense to the factory user and are related to the problem domain he is developing software for.

Currently Jezz and I are working on an experimental software factory authoring approach using a tool called the Factory Product Model (FPM).  I will not explain this approach or tool in detail in this post but if you are interested in more details you can read this, this, this and this post.

In this experiment, we are trying to validate whether a formal description (model) of the product the factory is building is a valid starting point for defining the factory and what it builds.

As far as I understand now, viewpoints will definitely play an important part in factory authoring and usage but what I am trying to find out is what impact they have and how they possibly relate to something like FPM.

I think specifying viewpoints is all about (some of) the requirements of the product we are building in the factory and therefore impact the type of guidance that is needed in the factory.

After all a viewpoint is what you get when you associate a work product, to an assets and a set of activities. Therefore viewpoints and work products are tightly related. It is just a question of which one do you define first?

I would be very interested in any ideas about this topic. What do you think? Do we need a product first approach (an FPM)? How does this relate to the Glidepath viewpoints approach I described? Do we need both? Do we need them in a specific order?

Let me know!