This provides some info and background information needed to configure and operate Weblogic messaging (partly using SAF).
We start with a discussion of how Weblogic JMS and SAF (Store-And-Forward) will be used to enable this traffic, including its features and limitations. Then the specific configuration steps are given, including things to note and watch out for which were learnt during the Proof Of Concept phase. After that, there is some information about how to operate and troubleshoot the connection, plus whatever extra information might be useful.
I have put the actual configuration steps which need to be executed via the Weblogic console (as opposed to the discussion and background information etc.) in these green boxes.
NB 1: each update on the Weblogic console is enclosed within “Lock & Edit” and “Activate Changes”, to ensure that each one is applied successfully before proceeding.
NB 2: be careful as some of the resource names are long and split up over two lines in this document!
NB 3: bear in mind a useful and comprehensible naming convention for the Weblogic JMS/SAF resources! e.g.
-ComponentName
(This is because if they all start with the same ComponentName, they kind of look the same, if you happen to only have browser width only able to show the first few characters.)
The aim here is to get a Weblogic domain talking to another Weblogic domain for the purposes of exchanging JMS traffic for messaging requirements. This involves setting up Weblogic JMS and SAF (Store-And-Forward) configurations to provide access to JMS Destinations on each domain. If you are using OSB/ALSB, ALSB components (proxy and business services) will also have to be written which will make use of the JMS/SAF configuration, so that needs to be in place before the ALSB components will work.
The capability to send JMS messages between the two domains will be provided using Weblogic JMS and its SAF (Store And Forward) feature. This feature allows JMS messages to be sent from one domain to another, being queued in the originating domain should the destination not be available. This is achieved by configuring a real Weblogic JMS queue in the destination domain, and “importing” that destination into the originating domain using the SAF resources. The Message Producing component is therefore sending to an imported representation of a real, remote queue. Message Consumers will consume from the “real” JMS destination and be unaware of it being anything to do with SAF.
In order to exchange messages via SAF, the two domains involved will need to “trust” each other. This means they need to share the same domain credential. We will need to agree with the other side the domain credential to use, and both sides will need to set it on their domain. See this page for explanation: http://download.oracle.com/docs/cd/E13222_01/wls/docs92/ConsoleHelp/taskhelp/security/EnableTrustBetweenDomains.html
Extract from this page reproduced here:
“Trust between domains is established so that principals in a Subject from one WebLogic Server domain are accepted as principals in another domain. When this feature is enabled, identity is passed between WebLogic Server domains over an RMI connection without requiring authentication in the second domain. When inter-domain trust is enabled, transactions can commit across domains. A trust relationship is established when the Domain Credential for one domain matches the Domain Credential for another domain.
By default, the Domain Credential is randomly generated and therefore, no two domains will have the same Domain Credential. If you want two WebLogic Server domains to interoperate, you need to replace the generated credential with a credential you select, and set the same credential in each of the domains.”
1. In Weblogic console, go to: Home -> Domain -> Security (tab)
2. Click “Lock & Edit” button
3. Click “Advanced”
4. Update “Credential” and “Confirm Credential” to be the agreed value
5. Click “Activate Changes” button
6. Restart the entire Weblogic domain (all servers, inluding the Admin server)
Do we actually need any extra JMS Servers? A suitable JMS server or servers (for the cluster) may already be available. This implies that suitable File Stores are also already set up, which are used by default.
This JMS Module groups all the configuration related to the SAF communication into one place. Is a good idea to group these configs together or you will lose stuff.
1. In Weblogic console, go to: Home -> Messaging -> JMS Modules
2. Click “Lock & Edit” button
3. Add new Module “JMSModule-ThisComponent”
4. Target the module to the local cluster
5. Save
6. Click “Activate Changes” button
This is to group together the components which need to be deployed together
1. In Weblogic console, go to: Home -> Messaging -> JMS Modules -> JMSModule-ThisComponent
2. Click on “Subdeployments” tab
3. Click “Lock & Edit” button
4. Add new Subdeployment “JMSSubDep-ThisComponent”
5. Target the subdeployment to the local cluster
6. Click “Activate Changes” button
Define the connection factories needed to support the message producers/consumers – or use the default ones...
- One to support the outbound producer
- One to support the inbound consumer
1. In Weblogic console, go to: Home -> Messaging -> JMS Modules -> JMSModule-ThisComponent
2. Click “Lock & Edit” button
3. Add new Connection Factory:
choose a sensible name!
Set JNDI name the same as the resource name
Accept all defaults and set Subdeployment to “JMSSubDep-ThisComponent”
Save
4. Add new Connection Factory:
Choose a sensible name!
Set JNDI name the same as the resource name
Accept all defaults and set Subdeployment to “JMSSubDep-ThisComponent”
Save
5. Add new Connection Factory:
Choose a sensible name!
Set JNDI name the same as the resource name
Accept all defaults (except below) and set Subdeployment to “JMSSubDep-ThisComponent”
Go to “Default Delivery” tab. Set “Default Time-to-Deliver” to 600000 (this would be a separate connection factory in case you wanted to have a retry facility which would put messages back onto the queue, but with a delay, to stop an immediate retry...)
Save
6. Click “Activate Changes” button
This defines the JMS Queue that will support incoming messages. Note that we don’t define a queue for the outbound messages, this will be handled via an imported SAF Destination, which we’ll come to later.
The queue is a Uniform Distributed Queue and has a Forward Delay option set to 10 seconds, to ensure that messages which land on a member queue which doesn’t have a consumer get automatically forwarded to another member queue which does have a consumer, if possible.
2. In Weblogic console, go to: Home -> Messaging -> JMS Modules -> JMSModule-ThisComponent
3. Click “Lock & Edit” button
4. Add new Distributed Queue:
ThisComponent.QueueName
Set JNDI name the same as the resource name
Accept all defaults
Click on “Advanced Targeting” and set Subdeployment to “JMSSubDep-ThisComponent”.
Save
Re-enter the newly created Queue. On the Configuration General tab, set “Forward Delay” to 10.
5. Click “Activate Changes” button
This contains the configuration which is needed to support the cross-domain JMS traffic for the outbound messages. Note that we don’t configure here anything to do with the inbound messages. They are dealt with in the previous section, and the idea is that the other side will have a corresponding imported SAF Destination on their side to support it.
To configure SAF, you first need a Store-and-Forward agent deployed on the cluster. Then you can import destinations which reside on remote Weblogic instances. This involves adding a “SAF Remote Context” which will instruct SAF to point to the remote Weblogic cluster at the URL provided by the other side. A “SAF Error Handling” resource is also added which determines what to do when problems occur. Then a “SAF Imported Destinations” resource is added to specify which queues should be imported from the Remote Context. All these resources are bundled up in the JMS Module we created earlier.
1. See Implementation Script document for definitive version of these steps!
2. In Weblogic console, go to: Home -> Messaging -> Store-and-Forward Agents
3. Click “Lock & Edit” button
4. Add new Store-and-Forward Agent “SAFAgent-ThisComponent”
Accept all defaults
Target to the local cluster
Save
5. Go to: Home -> Messaging -> JMS Modules -> JMSModule-ThisComponent so that you can add new resources to the module
6. Add new “SAF Error Handling” resource named “ErrorHandling-ThisComponent”
Change Message Handling Policy to “Log”
Save
7. Add new “Remote SAF Context” resource named “RemoteSAFContext-ThisComponent”
Set URL to that of the appropriate remote system for this environment (e.g. “t3://remotesystem:7001”).
Set the remote credentials (User Name and Password) which will be used
Accept other defaults
Save
8. Add new “SAF Imported Destinations” resource named “SAFImportedDestinations-ThisComponent”
set Remote SAF Context to “RemoteSAFContext-ThisComponent”
set SAF Error Handling to “ErrorHandling-ThisComponent”
Target to the “JMSSubDep-ThisComponent” subdeployment
Save
9. Re-enter the newly created “SAFImportedDestinations-ThisComponent” resource
Click on Configuration tab, Queues sub-tab
Add new SAF Queue with name of the real remote queue where we should be putting the outbound messages. See table below.
Remote JNDI name should be the same
Finish
10. Click “Activate Changes” button
11. It may be advisable to restart the domain to ensure everything is reinitialized with the new configuration and there aren’t any old bits and pieces hanging around. Weblogic does sometimes throw confusing errors which miraculously go away after a restart.
After the configuration above is done, and everything’s up and running, we should be in a position to see messages flowing across the domains, assuming you’ve got some way of creating test messages and JMS producers onto the SAF queue... (or consumers onto the real inbound queue).
If you’re going via OSB/ALSB you could use a Proxy service to create test messages...Create a test message with the proxy and check that it goes out and appears in the safagent logs (assuming you’ve switched on the message logging for the SAF destinations).
For the inbound messages, we need to have some sort of harness which simulates the remote systems (unless we have connectivity to a real one), and puts a suitable message onto an imported SAF queue, which will forward it to the REAL queue which has been added (as part of the above script) to our domain, and in turn will get picked up by whatever is listening/consuming that queue.
There is a bit of test Java code (QueueEchoer/QueueSender) from the POC stage to help with simulating the message flow, which could be set up to talk to whatever domains we have available. There are plenty of sample JMS Message Producer/Consumer java classes to try out on t’internet.
To point a producer to a SAF queue, you need to point it at the cluster address that the Destination (assuming it’s a Distributed Destination) is on. You can get the cluster address from the Weblogic console: Log into the Weblogic console, and navigate to “Clusters”. For the active cluster, copy the “Cluster Address” This could be something like “host1:7001,host2:7001,host3:7001” if there are multiple Weblogic instances that it’s deployed to...
You will probably want to set up configurable properties for things like:
- the remote system URL
- the destination cluster address (if applicable)
- the user name and password for the SAF context
- etc.
A random collection of notes concerning possible pitfalls and things learnt about Weblogic JMS/SAF...
If you see this in the logs it may mean that the domains don’t trust each other. See the above section on enabling domain trust.
If SAF fails to connect, one thing to make sure is that any necessary ports are opened on the firewall(s).
To debug SAF, use the Debug tab in the Server page and enable the jms.saf.DebugJMSSAF (+DebugMessaging) options. Also you need to ensure the debug options in the server(s) Logging tabs are set to Debug at least. Then restart the server(s). Log items referring to “JMSSAF” and other JMS-related items should start appearing in the server logs (suggestion: tail –f logname.log | grep “JMSSAF”).
It may also be useful to switch on Message Logging in the JMS Queue resources themselves. These get written to a jms.message.log file in the server logs directory. SAF messages can also be logged, these appear in files in the safagents subdirectory of the server logs directory.
There have been various rumours of known problems with SAF on Weblogic 9, although we haven’t encountered any - yet. According to registry.xml in the BEA Weblogic folder on our Dev system, we are at the following patch level of Weblogic 9.2:
level=9.2 ServicePackLevel=1 PatchLevel=0
Ensure “Default Targeting Enabled” isn’t ticked, and the items are deployed on the correct server, targeted to the right subdeployment, etc. Check they appear in the JNDI tree (any JNDI tree for the cluster should do, as cluster-available resources will appear in all the server-level JNDI trees).
This setting is connected with load balancing and has been identified as something which needs to be switched on in the connection factories. It should be on by default, add something in the scripts to check that this is definitely the case when the connection factories are created.
Messages/SAF Agent hangs?
Apparently can happen, especially if patches aren’t in. Apart from the list of possible patches which could help with this, check the following: need to ensure the JMS destination for the business service, which will actually send to the SAF queue, has got all the cluster hosts/ports mentioned. If you just have one it maybe a problem? E.g. check outbound message producer – is it pointing to an endpoint something like “t3://host1:7210,host2:7210/ConnexFactory/SAFQueueName”...
