One of the things that can help keeping service interfaces stable are well defined messages. However, it is very likely that at some point in the lifecycle of a service, adjustments in the messages are needed to support new functionality. These versioning problems in messages can be handled very easily by making use of the [XmlAnyElement] and [XmlAnyAttribute] in the messages.

When we have a look at the XML schema standard we can see that it defines the “<any/>” item for arbitrary xml content and the “<anyattribute />” item for undefined attributes inside an xml document. We can use these two items to define XML schemas that are highly extensible. By using the .NET [XmlAnyElement] and [XmlAnyAttribute] attributes we can deliver this extensibility feature of the xml schema in the messages used by our services.

Using the above mentioned attributes is very simple. We can add the attributes to a field in our message. The “NormalMessageContent” field on the “SampleMessage” class below is the only field that is a fixed part of the message.The other two field are there to hold de “custom xml” (FlexibleMessageContent) and the “custom attributes” (FlexibleMessageAttributes).  

[Serializable]

public class SampleMessage

{

 

  public string NormalMessageContent;

 

 [XmlAnyAttribute]

  public XmlAttribute[] FlexibleMessageAttributes;    

 

 [XmlAnyElement]

  public XmlElement[] FlexibleMessageContent;

}

The XmlSerializer (deserialize) will store all unmapped (unrecognized) XML elements in the “FlexibleMessageContent” field. All unrecognized attributes are stored in the “FlexibleMessageAttributes” field. Basically any content in an xml document other than the “NormalMessageContent” element will be available in our service through the two “flexible” fields.

So, By adding two fields to our message and tag them with the above mentioned attributes we can create messages that support versioning very well. Besides using this technique for versioning we can also use this to standardize messages. By using the above approach it is also possible to create a message with a few standard fields (messageId, timestamp, etc.) and one field that is used for the actual message content. In theory this message can be send to every service. It’s up to the service to decide if it understands the message that is stored in the message content field of the message. Creating service interfaces with these kinds of standardized messages might be a good way to keep the services interface stable!

Besides working with EDRA I am also very interested in technologies (or products) like Indigo and FABRIQ. Both of course different products, but with some concepts in common. For me it’s important to understand the concepts of these new technologies so I can use this knowledge in my current work when it comes to making architectural/design decisions that are as future proof as possible. To get familiar with the concepts in both Indigo and FABRIQ I decided to try to introduce some of these concepts on top of EDRA.

For the record: this is all just for my own fun! I have no intention to use the result of this in my current work that I am doing with EDRA.

It is my intention to change as little as possible in EDRA (preferable nothing), so all new functionality will be on top of EDRA. The ultimate goal is to build an (small) application that is using concepts like messages and ports similar as they are available in FABRIQ and Indigo but using EDRA ‘behind the scenes’. All of course in a very limited version. Just enough for me to understand the implementation consequences of these concepts within an architecture.

One of the first things I noticed when looking at FABRIQ and Indigo is the similarity in the concept of the ‘message’. Both products have a well defined message class. This is something that is not available in the current release of EDRA. In EDRA a message is based on a XSD, each message is of its ‘own’ type and is not derived from, for example, ‘EDRAMessage’ class. Because I think a well defined and consistent message structure has some advantages compared to the way messages are handled in EDRA, I decided to first introduce the concept of a message (like FABRIQ and Indigo) in my ‘study project’.

If we look at the Indigo message class (FABRIQ message looks very similar) we can see that this is actually a ‘wrapper’ around a SOAP envelope. The message basically consists of two different parts, the message content and a collection of message headers. The message content part is used for sending application specific data from one service to another. If we take a better look at the message content, we can see that this is some sort of a ‘free format’. This means it is an unspecified structure. This is no problem because, as long as the message sender and the message receiver understand the message, everybody is happy. The infrastructure doesn’t need to understand the message content!

In indigo, the actual message content is reachable by using the ‘reader’ property of the ‘content’ object of the message class (Message.Content.Reader). This property in fact returns an XMLReader positioned at the beginning of the ‘soap:body’ part (= actual message content) . Using an XMLReader for accessing the content of a message has some advantages. When Indigo (and FABRIQ) creates a message it uses an XMLReader to first read the ‘soap:header’ part of the message. As soon as it finished reading the header information it stops reading and leaves the XMLReader positioned at the beginning of the message content (soap: body). The message content is then available by using the reader property as shown above. Treating a message like this, saves an enormous amount of time (especially for large messages) because the whole message doesn’t need to be loaded in some sort of a DOM before the infrastructure can do its magic to process the message. In fact, the infrastructure doesn’t need the message content at all.

When we look at the message headers, the story is a bit different. Message headers are needed by the infrastructure to process the message. The infrastructure can even change headers if needed. Because headers are used by the infrastructure most of them are system defined. The message headers are reachable (Indigo) by using the ‘Headers’ property on the message class. Message headers are added (or even changed) to the message by the infrastructure and NOT by the sending or receiving application. Headers can be used by the infrastructure for, routing, security, etc. One very obvious header is the ‘to-header’. This header holds the URI of the receiving application (service) and is used by the infrastructure to deliver the message at the right spot. Preferably the ‘to-header’ isn’t populated by the sending application but by a part of the infrastructure (directory service). This guarantees location transparency between services or applications. Another very common message header is the ‘action-header’. This header isn’t used by the infrastructure but is used by the receiving service of the message. This ‘action-header’ can be used by the receiver of the message to internally map the message to the right  place in the service or application that is capable of handling that specific ‘type’ of message.

So, where are we? I used the above to organize some of my thoughts and learning’s about messages in Indigo and FABRIQ. Although the concepts of messages and headers are available in EDRA they are implemented in a very different way. My next step will be defining a (very basic) message class en probably two header classes (to- and action- header). The message class must be capable of wrapping an EDRA message (XSD typed message) and supports storing the headers. The implementation of the message class and headers must be just enough to support the concepts described above. After that I will dive into the concepts of ports and channels and implement these concepts in a very basic way. When this is finished I will try to map these things to the EDRA pipeline and see if I can get all off these things to work together….

Of course the big question is: is it worth it to spend time on the things I planned to do?

Well…..maybe not, but at least it helps me understand new concepts and gives me some fun!

During the development of a small prototype we discovered an issue when using EDRA with the Remoting interface. We had to build this prototype to demonstrate the changes we made to EDRA and prove that the way we want to make use off EDRA really works. The prototype contained 3 services: 1 user interface service, 1 business process service and 1 business information service. The application flow is very simple: the user interface service does an EDRA call to the business process services and the business process service does an EDRA call to the business information service. This is a little different as for example the Global Bank implementation where only 1 EDRA call is made (from the user interface to the service).

The above means that the business action of the business process service does an EDRA call to the business information service. When executing this with the remoting interface transport and ‘inProc’ dispatching we got the following error: ‘tcp channel already registered’. After some investigation we noticed the following line in the ‘RemotingInterfaceListenerController.cs’ file

Transport = new TcpServerChannel(props, bsfsp);

Because in this statement, no name is supplied for the TcpServerChannel that is created (not part of ‘props’), the remoting runtime uses the default name ‘tcp’. When this piece of code is executed twice, this is causing the exception ‘tcp channel already registered’.

Because during development of our services we like to use remoting interface and ‘inproc’ dispatching (easy debugging), we had to solve this issue.

The most easy way to solve this issue is supplying an empty string (“”) to the constructor of the TcpServerChannel as part of the IDictionary ‘props’.

This should look something like

props [“port”] = _port;

props [“name”] = “”;

Transport = new TcpServerChannel(props, bsfsp);

But… maybe it’s better to get the name out of the configuration file and decide, based on the information returned by the ChannelServices.RegisteredChannels property, if the channel is already registered!

Ever asked yourself the question: “Am I (still) doing the right thing?” Well, I did!
I was triggered by someone that asked me why we are making all these changes to EDRA and why we are making all those guidelines about “how to use EDRA”. Isn’t EDRA the perfect SOA framework without all that changes, he asked. Well…. To be honest, I said, I don’t think so. Then of course the next question was: why should we use it? This made me think, are we doing the right thing?

Of course it is much easier to continue talking about SOA for a couple of months or even years, dreaming about al the good thinks SOA will bring us in the future. But he… we have to start somewhere, so why not start now!
There are tons of articles or writings about SOA on the internet, however there are only a few successful SOA implementations in real life projects (that I know off). In my opinion implementing a perfect SOA architecture cannot be done by just selecting the right framework, or tool (at least not at this moment). Implementing SOA needs a different way of thinking! For most of us, getting familiar with this “new way” of thinking takes time. To reduce this time as much as possible, adopting a framework might be the right thing to do. While using a framework, we will encounter all kind off issues that we wouldn’t have found when we only continued reading about the perfect world of SOA.

Whether to adopt a specific framework or not isn’t a decision we can make in one minute. We have to make sure that the framework meets you own architectural requirements. If some (or more) of our requirements are not covered by the framework, we have to find out how easy it is to introduce these requirements in the framework.
This is exactly what we are doing at the moment. We have our own set of requirements and we are mapping these requirements to EDRA features. Requirements that we think are important and are not available in EDRA will be implemented by our self. Of course we have to make sure that our own requirements are up to date with the latest ideas about SOA. This is why requirements are not static but can change in time, based on articles that we read or changes in technology and tools that we use.

For me, Adopting EDRA wasn’t based on its complete set of SOA features. In fact I don’t even think that Microsoft is positioning EDRA as a SOA framework. In my opinion the most important features of EDRA are: supplying a way to separate the business logic from the cross-cutting concerns (logging, authorization, etc.) and separate the business logic from the underlying transport that is used. I think these features give you a good starting point for building distributed systems. SOA features that we think are missing in EDRA (some of them I wrote about in previous posts) will be added to EDRA by our self.

One very important thing I try to keep in mind is to “abstract” the usage of EDRA away as much as possible. For me EDRA is just a piece of infrastructure that is used in our architecture. In the end it must be possible to replace EDRA by any other technology or framework that comes available without having to change all your services within the architecture (or at least as little as possible). So, it’s not only a question IF you want to use EDRA but also HOW you will use EDRA. Spending some time on writing design guidelines for services (interfaces) build against EDRA might give you more flexibility in the near future when switching to another version of EDRA or even a totally new framework or technology.

After spending a considerable amount of time thinking about the above issues, I think using EDRA as a starting point is a good thing to do (for our situation) and resources needed for introducing new (SOA) features will not be a waste of our time. By introducing some good design guidelines for our service we keep enough flexibility for changes in framework or technology in the near future.
So, in my opinion: Yes we are still doing the right thing!!

Today we finished a first version of our “deployment strategy” for services build with EDRA. Because we decided some time earlier to host all our services within 1 instance of EDRA (even if they are built for different projects/applications) we encountered some issues when using the “deployment strategy” that come with EDRA.

We noticed the first issue when trying to install two services on our development server created by two different developers on their local machines. Both developers had an EDRA installation on their machine. On both machines the EDRA assemblies had a different “PublicKeyToken”, of course this caused all kind of problems when deploying these services to our central instance of EDRA. The services were both compiled against a different set of EDRA assemblies. Installing these two services on our development server hosting one instance of EDRA caused all kind of compatibility issues between the different “PublicKeyTokens” in the EDRA assemblies and publicKeyTokens in the configuration files.

To solve this issue we build our “own” release set of EDRA assemblies. These assemblies are the ones that will be used for every project that is using EDRA. Also we made some changes in the ‘CreateNewTemplate.swf’ script and the “ApplicationTemplate.sln” that is used in this script. We deleted the ‘ReferenceArchitecure’ enterprise template from the solution to make sure no EDRA code files are available for our developers. Further we made some changes in the “Deploy.swf” and “SetupHelpers.swf” script to support only using the compiled assemblies within the Visual Studio solution created by our new “CreateNewTemplate.swf” script.

All this is included in the image that we created for setting up a .NET development machine. After installing this image on a machine, the developer ends up with machine containing all necessary Microsoft development tools, all EDRA assemblies installed in the GAC and a shortcut from the start menu to our modified “CreateNewTemplate” script. So now our developers are ready to build their EDRA based services without first installing EDRA.

After defining how to use the EDRA framework we had to define a strategy for our development, test and production servers. We decided to go for the “two server” option. This means hosting the service interface and service implementation on different servers.
We created a list with all files necessary for the two different server types. So only the necessary assemblies are installed on the machine (GAC). The EDRA configuration file is installed on both machines. On the server hosting the service interface we tested the WebService and Remoting transport. On the server hosting the service implementation we tested the WebService and DCOM dispatching transport. All combinations worked perfectly fine.

We think that the work described above brings us a relatively simple way of deploying an EDRA based service on either our development, test or production server. All the developer has to deliver is the “message”, “business action”and “interface transport” assemblies. And of course some EDRA configuration settings that will be included in the EDRA configuration files on the servers by our server administrator.

Today we finished version 1.0 of our directory service. This service is the first in a range of changes/additions we will make to the EDRA framework. All functionality as described in my previous posts is implemented in this first release. Below some implementation details for this service:
As a datastore for this ‘directory service’ we decided to use Active Directory. Of course using UDDI as a source for this is also a very (maybe even more) logical choice , but some requirements of our client made us decide to use Active Directory.
Because it is not that easy to implement a schema change in a company Active Directory, we used ADAM (Active Directory Application Mode) during development so we didn't have to wait before the schema change is implemented in the Active Directory.

After defining and implementing the schema in ADAM on our development server, the directory services was build using the default EDRA structure, created by the ‘CreateNewTemplate’ script. A simple caching mechanism is implemented within the service to reduce the relatively low performance of querying ADAM. The actual communication with ADAM is implemented in a ‘AdamProvider’ class that is registered in the directory service configuration file. This makes it relatively easy to implement an ‘UDDIProvider’ class in the (near) future.
The directory service is available through remoting and webservice interface. Because every service consumer within the architecture is allowed to use the directory service, none of the EDRA handlers for authentication or authorization are used for this service. During development no issues were found in using EDRA.
During the ‘deployment’ phase of our service however, we did encounter some (potential) issues. We think some adjustment need to be made in de setup and deployment scripts that come with EDRA. Especially in the situation that a ‘new service’ needs to be installed on the server that is already running an EDRA instance (and some services) we see some issues.

More on these deployment issues (and hopefully our solutions!) in a later post.

So this is the moment, I am starting my own blog. Working as an architect on several .NET projects my main interest is SOA. After some time of evaluation the Microsoft EDRA (aka Shadowfax) initiative I am about to start using EDRA in real life. This made me decide to share my thoughts, issues (and frustration??) in this process. Besides EDRA I also keep an eye on another 'new product' called FABRIQ. So maybe you will find some writting about this too.