Friday, October 15, 2004

During the development of a small prototype we discovered an issue when using EDRA with the Remoting interface. We had to build this prototype to demonstrate the changes we made to EDRA and prove that the way we want to make use off EDRA really works. The prototype contained 3 services: 1 user interface service, 1 business process service and 1 business information service. The application flow is very simple: the user interface service does an EDRA call to the business process services and the business process service does an EDRA call to the business information service. This is a little different as for example the Global Bank implementation where only 1 EDRA call is made (from the user interface to the service).

 

The above means that the business action of the business process service does an EDRA call to the business information service. When executing this with the remoting interface transport and ‘inProc’ dispatching we got the following error: ‘tcp channel already registered’. After some investigation we noticed the following line in the ‘RemotingInterfaceListenerController.cs’ file

 

Transport = new TcpServerChannel(props, bsfsp);

 

Because in this statement, no name is supplied for the TcpServerChannel that is created (not part of ‘props’), the remoting runtime uses the default name ‘tcp’. When this piece of code is executed twice, this is causing the exception ‘tcp channel already registered’.

 

Because during development of our services we like to use remoting interface and ‘inproc’ dispatching (easy debugging), we had to solve this issue.

The most easy way to solve this issue is supplying an empty string (“”) to the constructor of the TcpServerChannel as part of the IDictionary ‘props’.

 

This should look something like

 

props [“port”] = _port;

props [“name”] = “”;

Transport = new TcpServerChannel(props, bsfsp);

 

But… maybe it’s better to get the name out of the configuration file and decide, based on the information returned by the ChannelServices.RegisteredChannels property, if the channel is already registered!

 

 

posted on 10/15/2004 12:35:49 PM UTC  #   
 Friday, October 08, 2004

Ever asked yourself the question: “Am I (still) doing the right thing?” Well, I did!
I was triggered by someone that asked me why we are making all these changes to EDRA and why we are making all those guidelines about “how to use EDRA”. Isn’t EDRA the perfect SOA framework without all that changes, he asked. Well…. To be honest, I said, I don’t think so. Then of course the next question was: why should we use it? This made me think, are we doing the right thing?

Of course it is much easier to continue talking about SOA for a couple of months or even years, dreaming about al the good thinks SOA will bring us in the future. But he… we have to start somewhere, so why not start now!
There are tons of articles or writings about SOA on the internet, however there are only a few successful SOA implementations in real life projects (that I know off). In my opinion implementing a perfect SOA architecture cannot be done by just selecting the right framework, or tool (at least not at this moment). Implementing SOA needs a different way of thinking! For most of us, getting familiar with this “new way” of thinking takes time. To reduce this time as much as possible, adopting a framework might be the right thing to do. While using a framework, we will encounter all kind off issues that we wouldn’t have found when we only continued reading about the perfect world of SOA.

Whether to adopt a specific framework or not isn’t a decision we can make in one minute. We have to make sure that the framework meets you own architectural requirements. If some (or more) of our requirements are not covered by the framework, we have to find out how easy it is to introduce these requirements in the framework.
This is exactly what we are doing at the moment. We have our own set of requirements and we are mapping these requirements to EDRA features. Requirements that we think are important and are not available in EDRA will be implemented by our self. Of course we have to make sure that our own requirements are up to date with the latest ideas about SOA. This is why requirements are not static but can change in time, based on articles that we read or changes in technology and tools that we use.

For me, Adopting EDRA wasn’t based on its complete set of SOA features. In fact I don’t even think that Microsoft is positioning EDRA as a SOA framework. In my opinion the most important features of EDRA are: supplying a way to separate the business logic from the cross-cutting concerns (logging, authorization, etc.) and separate the business logic from the underlying transport that is used. I think these features give you a good starting point for building distributed systems. SOA features that we think are missing in EDRA (some of them I wrote about in previous posts) will be added to EDRA by our self.

One very important thing I try to keep in mind is to “abstract” the usage of EDRA away as much as possible. For me EDRA is just a piece of infrastructure that is used in our architecture. In the end it must be possible to replace EDRA by any other technology or framework that comes available without having to change all your services within the architecture (or at least as little as possible). So, it’s not only a question IF you want to use EDRA but also HOW you will use EDRA. Spending some time on writing design guidelines for services (interfaces) build against EDRA might give you more flexibility in the near future when switching to another version of EDRA or even a totally new framework or technology.

After spending a considerable amount of time thinking about the above issues, I think using EDRA as a starting point is a good thing to do (for our situation) and resources needed for introducing new (SOA) features will not be a waste of our time. By introducing some good design guidelines for our service we keep enough flexibility for changes in framework or technology in the near future.
So, in my opinion: Yes we are still doing the right thing!!

posted on 10/8/2004 10:33:43 PM UTC  #   

Today we finished a first version of our “deployment strategy” for services build with EDRA. Because we decided some time earlier to host all our services within 1 instance of EDRA (even if they are built for different projects/applications) we encountered some issues when using the “deployment strategy” that come with EDRA.

We noticed the first issue when trying to install two services on our development server created by two different developers on their local machines. Both developers had an EDRA installation on their machine. On both machines the EDRA assemblies had a different “PublicKeyToken”, of course this caused all kind of problems when deploying these services to our central instance of EDRA. The services were both compiled against a different set of EDRA assemblies. Installing these two services on our development server hosting one instance of EDRA caused all kind of compatibility issues between the different “PublicKeyTokens” in the EDRA assemblies and publicKeyTokens in the configuration files.

To solve this issue we build our “own” release set of EDRA assemblies. These assemblies are the ones that will be used for every project that is using EDRA. Also we made some changes in the ‘CreateNewTemplate.swf’ script and the “ApplicationTemplate.sln” that is used in this script. We deleted the ‘ReferenceArchitecure’ enterprise template from the solution to make sure no EDRA code files are available for our developers. Further we made some changes in the “Deploy.swf” and “SetupHelpers.swf” script to support only using the compiled assemblies within the Visual Studio solution created by our new “CreateNewTemplate.swf” script.

All this is included in the image that we created for setting up a .NET development machine. After installing this image on a machine, the developer ends up with machine containing all necessary Microsoft development tools, all EDRA assemblies installed in the GAC and a shortcut from the start menu to our modified “CreateNewTemplate” script. So now our developers are ready to build their EDRA based services without first installing EDRA.

After defining how to use the EDRA framework we had to define a strategy for our development, test and production servers. We decided to go for the “two server” option. This means hosting the service interface and service implementation on different servers.
We created a list with all files necessary for the two different server types. So only the necessary assemblies are installed on the machine (GAC). The EDRA configuration file is installed on both machines. On the server hosting the service interface we tested the WebService and Remoting transport. On the server hosting the service implementation we tested the WebService and DCOM dispatching transport. All combinations worked perfectly fine.

We think that the work described above brings us a relatively simple way of deploying an EDRA based service on either our development, test or production server. All the developer has to deliver is the “message”, “business action”and “interface transport” assemblies. And of course some EDRA configuration settings that will be included in the EDRA configuration files on the servers by our server administrator.

posted on 10/8/2004 10:31:49 PM UTC  #   

Today we finished version 1.0 of our directory service. This service is the first in a range of changes/additions we will make to the EDRA framework. All functionality as described in my previous posts is implemented in this first release. Below some implementation details for this service:
As a datastore for this ‘directory service’ we decided to use Active Directory. Of course using UDDI as a source for this is also a very (maybe even more) logical choice , but some requirements of our client made us decide to use Active Directory.
Because it is not that easy to implement a schema change in a company Active Directory, we used ADAM (Active Directory Application Mode) during development so we didn't have to wait before the schema change is implemented in the Active Directory.

After defining and implementing the schema in ADAM on our development server, the directory services was build using the default EDRA structure, created by the ‘CreateNewTemplate’ script. A simple caching mechanism is implemented within the service to reduce the relatively low performance of querying ADAM. The actual communication with ADAM is implemented in a ‘AdamProvider’ class that is registered in the directory service configuration file. This makes it relatively easy to implement an ‘UDDIProvider’ class in the (near) future.
The directory service is available through remoting and webservice interface. Because every service consumer within the architecture is allowed to use the directory service, none of the EDRA handlers for authentication or authorization are used for this service. During development no issues were found in using EDRA.
During the ‘deployment’ phase of our service however, we did encounter some (potential) issues. We think some adjustment need to be made in de setup and deployment scripts that come with EDRA. Especially in the situation that a ‘new service’ needs to be installed on the server that is already running an EDRA instance (and some services) we see some issues.

More on these deployment issues (and hopefully our solutions!) in a later post.

posted on 10/8/2004 10:30:23 PM UTC  #   

Currently we are in the process of adopting EDRA in our software development process. In this process we are trying to map our own architectural requirements to EDRA functionality. For some of our requirements this means creating new EDRA handlers, or even creating new services (like the directory service described earlier). Today another requirement came up…

‘Within our systems it must be possible to monitor the complete flow of a message and all messages that are send as a result of this message’

… In our architecture we recognize 3 different kind of services: user interface services, business process services and business information services. These kind of services will be build as separate services according to the ‘EDRA specs’. So a user interface service (for example an ASP.NET page) is using the EDRA infrastructure to communicate with a business process service. This business process service is using the EDRA infrastructure to communicate with an business information service. All services in this example are completely separated from each other. This is a bit different from the Global Bank example (in my opinion) in which for example the process is implemented in a page controller in the front- end. In the Global Bank situation there is only 1 EDRA call to a service action (from user interface all the way to the business action). In our preferred way, there will be a least two EDRA calls, one from to the user interface to the business process service AND one from the business process service to the business information service.

Our logging requirement means that we would like to ‘see’ that the request message send from the user interface to the business process service “belongs” to the request message that is send from the business process service to the business information service.

At this moment I haven’t found the best solution for this requirement. These are my thoughts at the moment:

  • Implement a new logging handler that simply logs the message in a database. I think we can use the logging handler of Global Bank as a starting point for this
  • Define a set of default fields that are available on EVERY message that is send in the architecture. I am thinking of ‘timestamp’ and some sort of an ‘request id’
  • If a message is send from the user interface service to the business process service, supply an ‘request id’ in the message. (this message is logged in the database by the log handler). In the business process service use the ‘request id’ in the message from the user interface to store in the ‘request id’ field of the message that is send to the business information service (also this message is logged in the database by the log handler)
  • Now there is a way to ‘link’ (using the ‘request id’) the two log entries in the database to each other.
  • Make some sort of overview that shows the relation between the different messages. Now it is possible to see that a call to service ‘A’ means that also messages are sent to service ‘B’ and ‘C’

Any better ideas? .... let me know!

posted on 10/8/2004 10:26:12 PM UTC  #   
During a session to determine the requirements for our ‘directory service’ we decided to store some additional ‘role’ information for the services in our service repository. Within our architecture we would like to have the possibility to let our services react differently based on the ‘functional role’ off the service consumer. This means that a service can perform additional behavior based on the functional role that the service consumer has within an application. If no role information is supplied in the service request, the service performs its default behavior. If a valid role is submitted within the service request the service can, for example return some extra confidential data about an employee in case the functional role of the service consumer is ‘manager’.

So what does this mean for our directory service? The first version of this service will hold the following information for each service within the service repository: the logical service name, the roles the service supports (zero or more entries) and url information for every EDRA transport this service supports. At this moment our services only supports the default EDRA transports (webservice, remoting and MQ). Initially the service gets 1 service action to retrieve the information based on the logical name (and optional the transport) of the service. A simple caching mechanism will be used to make sure the service meets the performance requirements that apply to a service that will be used that much within the architecture.

One issue that has to be resolved is to determine what is the best way to deliver the functional role of the service consumer to the actual service. 'Forcing' the service consumer to supply this information as part of the request message isn't my favorite option. to be continued....

For the development, the VS.NET solution structure will be used that is created by the ‘CreateNewTemplate.wsf’ script that comes with EDRA.

Time to start developing!
posted on 10/8/2004 10:24:00 PM UTC  #   
 Thursday, August 19, 2004
Last week I completed a survey of the EDRA team for new functionality in the 1.1 release of EDRA. Today the results of this survey were published on the EDRA ‘Got Dot Net’ workspace. Based on the ranking that is provided on the workspace as a result of the survey I can tell that: WIZARDS is what we want! Further improved performance is high on the ranking list.

My own request for new features had nothing to do with wizards or performance. One of them concerned the introduction of some sort of a ‘service repository’. This repository contains endpoint (url) information of all services available within the architecture. In my opinion ‘service discovery’ is an important issue in SOA. In the current release of EDRA this ‘service repository’ feature is implemented by using one or more configuration files. In this case the configuration file is used in the user interface to get the url information for the services required in the user interface. This works fine when using EDRA (and services) for only one application, but what if we are creating services that are used within an enterprise by more than one application? Any change in the url of one service in the enterprise architecture needs a change in the configuration file of all the user interfaces that are using that service. Not the most ideal situation!

So what do we need? We need a ‘directory service’ that delivers access to the service repository. Whenever a user interface (or another service) wants to communicated with a service within the enterprise architecture, a call to the directory service is made, supplying the logical name of the requested service, to collect endpoint information of that service. This endpoint information is then used to make the actual service call to the requested service.

The service repository could even store endpoint information for more than 1 protocol (webservice and remoting). In that case it is even possible to decide at runtime what transporttype to use for calling a service. Unfortunately this is not possible when using the default EDRA version 1.0 features. To make this work we also have to abstract away the knowledge of the transporttype to use for calling a service within the EDRA world.

I think to use EDRA within an enterprise architecture we need to make some small adjustments like the one above. Now I know 'my wishes' are not on the EDRA version 1.1 feature list, there is only one solution: start building this features myself……
posted on 8/19/2004 9:42:12 PM UTC  #   
 Saturday, August 14, 2004

So this is the moment, I am starting my own blog. Working as an architect on several .NET projects my main interest is SOA. After some time of evaluation the Microsoft EDRA (aka Shadowfax) initiative I am about to start using EDRA in real life. This made me decide to share my thoughts, issues (and frustration??) in this process. Besides EDRA I also keep an eye on another 'new product' called FABRIQ. So maybe you will find some writting about this too.

posted on 8/14/2004 9:35:46 PM UTC  #