Last week we decided to do an upgrade of EDRA 1.0 to EDRA 1.1 on our development machines. According to Microsoft, version 1.1 is the final version of EDRA. Microsoft indicates that in the beginning of next year they will start a similar project to take on the latest distributed design challenges using Microsoft’s BizTalk and "Indigo" technologies.

After making the change described my previous posting in the Microsoft EDRA 1.1 code, we compiled the EDRA 1.1 code using our own public key. After that we compiled our ‘EDRA extensions’ against the new EDRA 1.1 assemblies. Only one code change was needed to make all our code compile (Microsoft decided to change the name of one of the Handler Interfaces). Further the following changes were needed to make our test services work:

·         Remove ‘ra’ namespace prefix in ServiceReferenceArchitecture.config.

·         Move Pipeline configuration from ServiceReferenceArchitecture.config to the new ‘TemplateServices.config’ file.

·         Add two new configuration sections to specify the location of the ServiceReferenceArchitecture.config and the TemplateServices.config file.

Based on the results of our NUnit test, everything worked as expected (after the changes described above).

To make absolutely sure services implemented using EDRA 1.1 behave the same as services implemented using EDRA 1.0 we also did some manual testing against our test services. After that we had enough confidence to advice a project that is using EDRA 1.0 at this moment to do the upgrade to EDRA 1.1.

All the project has to do now is: install the new EDRA 1.1 assemblies; install the new version of our ‘EDRA extensions’; make the necessary changes in the configuration files; start system testing!

Currently, I am working myself through the WS-Policy specification. When doing this I found this very interesting article of Aaron Skonnard in which he explains how to use WS-Policy for webservice validation. The technique Aaron describes makes it possible to completely decouple validation logic from the webservice implementation, which of course improves service maintainability. After reading this article I decided to not only study the theoretical part of WS-Policy but also do some coding with it.

As I am doing some work with Microsoft EDRA at this moment, I decided to try to integrate WS-Policy into EDRA. Out of the box EDRA only supports schema (xsd) validation, so the technique Aaron describes can be very useful.

I haven’t done any testing with EDRA and WSE 2.0 yet, but according to Microsoft it must be possible. I actually do see some issues in integrating WS-Policy (WSE 2.0) and EDRA. These are not really technical issues. I think, when using WSE 2.0 and EDRA together, the WSE pipeline gets executed first. I am also almost sure that this isn’t easily changed (haven’t tested this yet). This means when using WS-Policy for doing some business validation and using EDRA for authorization, validation is executed before authorization is done. As I said, this is not a technical problem but can definitely be an issue. Of course using WSE for security is a very valid option to solve this issue.

I am very much aware that WS-Policy might change in the future and WSE doesn’t solve all related WS-Policy issues yet. One of the issues with WSE for example is the lack of policy discovery. WSE doesn’t provide a way to discover the policy statements of a service, so a solution must be found for this. In our situation, a possible solution for this is to store the URL for the service policy file together with all other service related info in our directory service. In this scenario it is at least clear where to find a service policy file if it exists.  

Although I see some issues, I think using WS-Policy in EDRA might be useful to get rid of business validations within the service implementation. I also think integrating it in EDRA doesn’t need a lot of work, so I will give it a try in the next couple of days.

Stay tuned!

Currently there is a bit of a discussion going on within our team whether to use the Remoting interface for services or not. When using EDRA, Remoting is one of the possibilities for creating a service interface, the question is: is Remoting the best option to use?

One of the ideas is to use a Remoting interface for ‘business information’ services and a webservice interface for the ‘business process’ services. I think this decision is primarily driven by a better performance that Remoting is supposed to bring us.

Personally I am not sure if performance is the best thing to base a decision on in a SOA environment. Using services implies crossing boundaries. Crossing boundaries is expensive by default! Of course Remoting can give some performance gain in this case in comparison to webservices but this shouldn’t be an issue in my opinion. If performance is what you want, don’t cross boundaries, don’t use services.

Another disadvantage of using Remoting for services is the lack of security. Of course using one of the EDRA handlers (or a custom handler) can provide some sort of a ‘security solution’ but security isn’t embedded in the design of Remoting.

It’s not that I don’t linke remoting because I do. Remoting can be very useful when using it internally in an application; in some scenarios it can be much more appropriate than using webservices. I only think Remoting shouldn’t be used across application (service) trust boundaries.

In my search for information about designing a service interface I came across some interesting postings of Aaron Skonnard about “contract first approach” for developing services. It looks like some guys are having some very interesting ideas about this. As I can read from there postings there isn’t yet very good tool support for this way of developing service contracts. Christian Weyer is having some thoughts about tool support for this. I haven’t had the time to dive into this, but certainly will very soon!

Although there are tons of articles covering SOA related issues on the internet, there is very little info about ‘defining a service interface’. During some support work that I am doing for a project at the moment I noticed that it can be hard for a software designer to define its ‘first’ service interface. Probably this is because defining an interface for a service is a little different than designing an interface for a component. I tried to find some guidance on this in the form of an article or document on the internet but couldn’t find any of this. Of course there is information about this but all in very small pieces in several articles. I tried to collect some of these pieces of guidance into one document so this can be used when defining a service interface.

Below some (maybe) helpful tips:

Define “business actions” on your service interface and not CRUD actions: Try to think of a service action as something that performs a real business task and not only an insert or delete in a database table. An “add” service action on the “Order” service implies that it only does an insert in the Order table, but maybe in reality it also checks the creditability of the customer that is placing the order. In this case a service action called “Add” doesn’t reflect the real business action.

Define the service interface as explicit as possible:  a service with an explicit service interface (contract) is much easier to consume than a service with a more “general” interface. It must be clear from the interface what kind of action is executed by a service action(s). This means, a service with an “execute” service action might not be the best thing to do. (I know I wrote some about a “generic” service interface in a previous posting but this was only to demonstrate the “message flexibility” that can be achieved with “[XmlAnyElement]”).

Avoid leaking internals: try not to leak service internals to the outside world. So, don’t expose for instance a .NET dataset on the service interface. Define messages based on an Xml schema (XSD). Also make sure not to include “technical attributes” in your messages. One way to make sure you don’t leak internal in your messages is to define your messages before you design your service implementation. When doing this, you cannot leak for instance technical attributes because you simply don’t have them defined yet!

Define coarse grained service interfaces: in general, coarse grained services have more business value and are easier to consume. I think trying to expose business actions instead of crud actions helps defining “coarse grained” services. Personally I think “coarse grained” a bit of a vague statement. I think coarse grained depends on the situation. Let’s call it right grained!

Above some of the things I try to keep in mind when defining a service interface. If you have any comments or additions, please let me know!

One of the things that can help keeping service interfaces stable are well defined messages. However, it is very likely that at some point in the lifecycle of a service, adjustments in the messages are needed to support new functionality. These versioning problems in messages can be handled very easily by making use of the [XmlAnyElement] and [XmlAnyAttribute] in the messages.

When we have a look at the XML schema standard we can see that it defines the “<any/>” item for arbitrary xml content and the “<anyattribute />” item for undefined attributes inside an xml document. We can use these two items to define XML schemas that are highly extensible. By using the .NET [XmlAnyElement] and [XmlAnyAttribute] attributes we can deliver this extensibility feature of the xml schema in the messages used by our services.

Using the above mentioned attributes is very simple. We can add the attributes to a field in our message. The “NormalMessageContent” field on the “SampleMessage” class below is the only field that is a fixed part of the message.The other two field are there to hold de “custom xml” (FlexibleMessageContent) and the “custom attributes” (FlexibleMessageAttributes).  

[Serializable]

public class SampleMessage

{

 

  public string NormalMessageContent;

 

 [XmlAnyAttribute]

  public XmlAttribute[] FlexibleMessageAttributes;    

 

 [XmlAnyElement]

  public XmlElement[] FlexibleMessageContent;

}

The XmlSerializer (deserialize) will store all unmapped (unrecognized) XML elements in the “FlexibleMessageContent” field. All unrecognized attributes are stored in the “FlexibleMessageAttributes” field. Basically any content in an xml document other than the “NormalMessageContent” element will be available in our service through the two “flexible” fields.

So, By adding two fields to our message and tag them with the above mentioned attributes we can create messages that support versioning very well. Besides using this technique for versioning we can also use this to standardize messages. By using the above approach it is also possible to create a message with a few standard fields (messageId, timestamp, etc.) and one field that is used for the actual message content. In theory this message can be send to every service. It’s up to the service to decide if it understands the message that is stored in the message content field of the message. Creating service interfaces with these kinds of standardized messages might be a good way to keep the services interface stable!

Besides working with EDRA I am also very interested in technologies (or products) like Indigo and FABRIQ. Both of course different products, but with some concepts in common. For me it’s important to understand the concepts of these new technologies so I can use this knowledge in my current work when it comes to making architectural/design decisions that are as future proof as possible. To get familiar with the concepts in both Indigo and FABRIQ I decided to try to introduce some of these concepts on top of EDRA.

For the record: this is all just for my own fun! I have no intention to use the result of this in my current work that I am doing with EDRA.

It is my intention to change as little as possible in EDRA (preferable nothing), so all new functionality will be on top of EDRA. The ultimate goal is to build an (small) application that is using concepts like messages and ports similar as they are available in FABRIQ and Indigo but using EDRA ‘behind the scenes’. All of course in a very limited version. Just enough for me to understand the implementation consequences of these concepts within an architecture.

One of the first things I noticed when looking at FABRIQ and Indigo is the similarity in the concept of the ‘message’. Both products have a well defined message class. This is something that is not available in the current release of EDRA. In EDRA a message is based on a XSD, each message is of its ‘own’ type and is not derived from, for example, ‘EDRAMessage’ class. Because I think a well defined and consistent message structure has some advantages compared to the way messages are handled in EDRA, I decided to first introduce the concept of a message (like FABRIQ and Indigo) in my ‘study project’.

If we look at the Indigo message class (FABRIQ message looks very similar) we can see that this is actually a ‘wrapper’ around a SOAP envelope. The message basically consists of two different parts, the message content and a collection of message headers. The message content part is used for sending application specific data from one service to another. If we take a better look at the message content, we can see that this is some sort of a ‘free format’. This means it is an unspecified structure. This is no problem because, as long as the message sender and the message receiver understand the message, everybody is happy. The infrastructure doesn’t need to understand the message content!

In indigo, the actual message content is reachable by using the ‘reader’ property of the ‘content’ object of the message class (Message.Content.Reader). This property in fact returns an XMLReader positioned at the beginning of the ‘soap:body’ part (= actual message content) . Using an XMLReader for accessing the content of a message has some advantages. When Indigo (and FABRIQ) creates a message it uses an XMLReader to first read the ‘soap:header’ part of the message. As soon as it finished reading the header information it stops reading and leaves the XMLReader positioned at the beginning of the message content (soap: body). The message content is then available by using the reader property as shown above. Treating a message like this, saves an enormous amount of time (especially for large messages) because the whole message doesn’t need to be loaded in some sort of a DOM before the infrastructure can do its magic to process the message. In fact, the infrastructure doesn’t need the message content at all.

When we look at the message headers, the story is a bit different. Message headers are needed by the infrastructure to process the message. The infrastructure can even change headers if needed. Because headers are used by the infrastructure most of them are system defined. The message headers are reachable (Indigo) by using the ‘Headers’ property on the message class. Message headers are added (or even changed) to the message by the infrastructure and NOT by the sending or receiving application. Headers can be used by the infrastructure for, routing, security, etc. One very obvious header is the ‘to-header’. This header holds the URI of the receiving application (service) and is used by the infrastructure to deliver the message at the right spot. Preferably the ‘to-header’ isn’t populated by the sending application but by a part of the infrastructure (directory service). This guarantees location transparency between services or applications. Another very common message header is the ‘action-header’. This header isn’t used by the infrastructure but is used by the receiving service of the message. This ‘action-header’ can be used by the receiver of the message to internally map the message to the right  place in the service or application that is capable of handling that specific ‘type’ of message.

So, where are we? I used the above to organize some of my thoughts and learning’s about messages in Indigo and FABRIQ. Although the concepts of messages and headers are available in EDRA they are implemented in a very different way. My next step will be defining a (very basic) message class en probably two header classes (to- and action- header). The message class must be capable of wrapping an EDRA message (XSD typed message) and supports storing the headers. The implementation of the message class and headers must be just enough to support the concepts described above. After that I will dive into the concepts of ports and channels and implement these concepts in a very basic way. When this is finished I will try to map these things to the EDRA pipeline and see if I can get all off these things to work together….

Of course the big question is: is it worth it to spend time on the things I planned to do?

Well…..maybe not, but at least it helps me understand new concepts and gives me some fun!

During the development of a small prototype we discovered an issue when using EDRA with the Remoting interface. We had to build this prototype to demonstrate the changes we made to EDRA and prove that the way we want to make use off EDRA really works. The prototype contained 3 services: 1 user interface service, 1 business process service and 1 business information service. The application flow is very simple: the user interface service does an EDRA call to the business process services and the business process service does an EDRA call to the business information service. This is a little different as for example the Global Bank implementation where only 1 EDRA call is made (from the user interface to the service).

The above means that the business action of the business process service does an EDRA call to the business information service. When executing this with the remoting interface transport and ‘inProc’ dispatching we got the following error: ‘tcp channel already registered’. After some investigation we noticed the following line in the ‘RemotingInterfaceListenerController.cs’ file

Transport = new TcpServerChannel(props, bsfsp);

Because in this statement, no name is supplied for the TcpServerChannel that is created (not part of ‘props’), the remoting runtime uses the default name ‘tcp’. When this piece of code is executed twice, this is causing the exception ‘tcp channel already registered’.

Because during development of our services we like to use remoting interface and ‘inproc’ dispatching (easy debugging), we had to solve this issue.

The most easy way to solve this issue is supplying an empty string (“”) to the constructor of the TcpServerChannel as part of the IDictionary ‘props’.

This should look something like

props [“port”] = _port;

props [“name”] = “”;

Transport = new TcpServerChannel(props, bsfsp);

But… maybe it’s better to get the name out of the configuration file and decide, based on the information returned by the ChannelServices.RegisteredChannels property, if the channel is already registered!

Ever asked yourself the question: “Am I (still) doing the right thing?” Well, I did!
I was triggered by someone that asked me why we are making all these changes to EDRA and why we are making all those guidelines about “how to use EDRA”. Isn’t EDRA the perfect SOA framework without all that changes, he asked. Well…. To be honest, I said, I don’t think so. Then of course the next question was: why should we use it? This made me think, are we doing the right thing?

Of course it is much easier to continue talking about SOA for a couple of months or even years, dreaming about al the good thinks SOA will bring us in the future. But he… we have to start somewhere, so why not start now!
There are tons of articles or writings about SOA on the internet, however there are only a few successful SOA implementations in real life projects (that I know off). In my opinion implementing a perfect SOA architecture cannot be done by just selecting the right framework, or tool (at least not at this moment). Implementing SOA needs a different way of thinking! For most of us, getting familiar with this “new way” of thinking takes time. To reduce this time as much as possible, adopting a framework might be the right thing to do. While using a framework, we will encounter all kind off issues that we wouldn’t have found when we only continued reading about the perfect world of SOA.

Whether to adopt a specific framework or not isn’t a decision we can make in one minute. We have to make sure that the framework meets you own architectural requirements. If some (or more) of our requirements are not covered by the framework, we have to find out how easy it is to introduce these requirements in the framework.
This is exactly what we are doing at the moment. We have our own set of requirements and we are mapping these requirements to EDRA features. Requirements that we think are important and are not available in EDRA will be implemented by our self. Of course we have to make sure that our own requirements are up to date with the latest ideas about SOA. This is why requirements are not static but can change in time, based on articles that we read or changes in technology and tools that we use.

For me, Adopting EDRA wasn’t based on its complete set of SOA features. In fact I don’t even think that Microsoft is positioning EDRA as a SOA framework. In my opinion the most important features of EDRA are: supplying a way to separate the business logic from the cross-cutting concerns (logging, authorization, etc.) and separate the business logic from the underlying transport that is used. I think these features give you a good starting point for building distributed systems. SOA features that we think are missing in EDRA (some of them I wrote about in previous posts) will be added to EDRA by our self.

One very important thing I try to keep in mind is to “abstract” the usage of EDRA away as much as possible. For me EDRA is just a piece of infrastructure that is used in our architecture. In the end it must be possible to replace EDRA by any other technology or framework that comes available without having to change all your services within the architecture (or at least as little as possible). So, it’s not only a question IF you want to use EDRA but also HOW you will use EDRA. Spending some time on writing design guidelines for services (interfaces) build against EDRA might give you more flexibility in the near future when switching to another version of EDRA or even a totally new framework or technology.

After spending a considerable amount of time thinking about the above issues, I think using EDRA as a starting point is a good thing to do (for our situation) and resources needed for introducing new (SOA) features will not be a waste of our time. By introducing some good design guidelines for our service we keep enough flexibility for changes in framework or technology in the near future.
So, in my opinion: Yes we are still doing the right thing!!

Today we finished a first version of our “deployment strategy” for services build with EDRA. Because we decided some time earlier to host all our services within 1 instance of EDRA (even if they are built for different projects/applications) we encountered some issues when using the “deployment strategy” that come with EDRA.

We noticed the first issue when trying to install two services on our development server created by two different developers on their local machines. Both developers had an EDRA installation on their machine. On both machines the EDRA assemblies had a different “PublicKeyToken”, of course this caused all kind of problems when deploying these services to our central instance of EDRA. The services were both compiled against a different set of EDRA assemblies. Installing these two services on our development server hosting one instance of EDRA caused all kind of compatibility issues between the different “PublicKeyTokens” in the EDRA assemblies and publicKeyTokens in the configuration files.

To solve this issue we build our “own” release set of EDRA assemblies. These assemblies are the ones that will be used for every project that is using EDRA. Also we made some changes in the ‘CreateNewTemplate.swf’ script and the “ApplicationTemplate.sln” that is used in this script. We deleted the ‘ReferenceArchitecure’ enterprise template from the solution to make sure no EDRA code files are available for our developers. Further we made some changes in the “Deploy.swf” and “SetupHelpers.swf” script to support only using the compiled assemblies within the Visual Studio solution created by our new “CreateNewTemplate.swf” script.

All this is included in the image that we created for setting up a .NET development machine. After installing this image on a machine, the developer ends up with machine containing all necessary Microsoft development tools, all EDRA assemblies installed in the GAC and a shortcut from the start menu to our modified “CreateNewTemplate” script. So now our developers are ready to build their EDRA based services without first installing EDRA.

After defining how to use the EDRA framework we had to define a strategy for our development, test and production servers. We decided to go for the “two server” option. This means hosting the service interface and service implementation on different servers.
We created a list with all files necessary for the two different server types. So only the necessary assemblies are installed on the machine (GAC). The EDRA configuration file is installed on both machines. On the server hosting the service interface we tested the WebService and Remoting transport. On the server hosting the service implementation we tested the WebService and DCOM dispatching transport. All combinations worked perfectly fine.

We think that the work described above brings us a relatively simple way of deploying an EDRA based service on either our development, test or production server. All the developer has to deliver is the “message”, “business action”and “interface transport” assemblies. And of course some EDRA configuration settings that will be included in the EDRA configuration files on the servers by our server administrator.