Previous: Up: Deployment GuideNext:

Apache James Deployment

James is an open source Java based mail and news server supported by the Apache software foundation. It is implemented using a modular pattern and can extended by adding new modules and components. It can run either as a completely stand alone email system, or can be integrated into a more complex mail deployment acting as either the final mail destination or a smart relay.

This section is not intended to be a full fledged James configuration and deployment guide, but to explain how the Direct Project gateway module integrates with the James platform.

NOTE All James instructions in the following sections are specific to James 2.3.2. James 3 at the time of writing is still under development in beata relase and radically changes the structure of the James deployment.

NHINDSecurityAndTrustMailet

As stated earlier, James is modular platform that facilitates easy integration of custom processing modules into the SMTP stack. The modular constructs of James are the Mailet and Matcher interfaces. In fact the default James configuration consists mainly of pre-configured Mailets and Matchers. The Direct Project gateway module provides the NHINDSecurityAndTrustMailet as protocol bridge for James. This custom mailet intercepts messages as they pass through the James transport processor, extracts the sender and recipients from the SMTP envelope, and calls the SMTPAgent to process the messages according the security and trust policy. After processing, the bridge either allows the message to continue through the James stack or calls appropriate error handling routines if the message cannot be processed correctly.

Mailet installation

All custom modules are deployed in jar files and placed in the James lib directory:

%jamesInstallRoot%/apps/james/SAR-INF/lib

All jars and transient dependencies must be placed in this directory.

NOTE The reference implementation contains a pre-built James configuration in the direct-project-stock project that contains all of the necessary jars in the lib folder as well as a pre-configured mailet.

Mailet Configuration

The NHINDSecurityAndTrustMailet is added to the James processing stack by adding specific lines into the James configuration file:

%jamesInstallRoot%/apps/james/SAR-INF/config.xml

All configuration examples in this section are located in the config.xml file.

All custom Matchers and Mailet packages must be registered with James in the <mailetpackages> and <matcherpackages> elements.

   <mailetpackages>
      <mailetpackage>org.apache.james.transport.mailets</mailetpackage>
      <mailetpackage>org.apache.james.transport.mailets.smime</mailetpackage>
      <mailetpackage>org.nhindirect.gateway.smtp.james.mailet</mailetpackage>
   </mailetpackages>
   <matcherpackages>
      <matcherpackage>org.apache.james.transport.matchers</matcherpackage>
      <matcherpackage>org.apache.james.transport.matchers.smime</matcherpackage>
      <matcherpackage>org.nhindirect.gateway.smtp.james.matcher</matcherpackage>
   </matcherpackages>

To install the security and trust mailet into the James stack, you need to a mailet element under the transport processor. NOTE James executes mailets in the order that they are listed in the configuration file, so place your mailet appropriately if you have other custom processing that needs to be executed before or after the security and trust mailet.

Example:

   <mailet match="All" class="NHINDSecurityAndTrustMailet">
       <ConfigURL>http://localhost:8081/config-service/ConfigurationService</ConfigURL>   
       <MessageMonitoringServiceURL>http://localhost:8081/msg-monitor</MessageMonitoringServiceURL>
       <AutoDSNFailueCreation>General,ReliableAndTimely</AutoDSNFailueCreation>            
   </mailet>

Mailet

Attributes:

AttributeDescription
matchThe matcher used to decide if the message should be processed by the mailet.
classThe class name of the mailet. James searches through all the mailet packages to find a fully qualified class name.

Elements:

ElementTypeDescription
ConfigURLURLA URL to the location of the SMTPAgent configuration information. This information is forwarded to the SmtpAgentFactory to locate and parse the configuration
MessageMonitoringServiceURLURLA URL to the location of the message monitoring service for quality of service purposes. If this parameter is present, the mailet creates default NoOp TxService implementation.
AutoDSNFailueCreationComma Delimited StringDetermines if DSN bounce messages should be automatically created for failed security and trust recipients. Defaults to ReliableAndTimely if this parameter is not present.
UseOutgoingPolicyForIncomingNotificationsIndicates if the outbound trust policy should be applied to incoming notification messages. Defaults to false if this parameter is not present.

The above example tell James to send all messages to the agent using the All matcher. More on matchers later.

The ConfigURL must be a valid and well formed URL that is addressable by the James application. The other settings are optional.

Matchers

Every mailet is required to provide a matcher. A matcher is a thin piece of logic that determines if the message should be processed by the mailet. The result of matcher is a list of recipients that should be processed by the mailet. James provides a stock set of out of the box matchers, so may use any of them for the security and trust mailet if they suit your needs.

Depending on your SMTP deployment, you may want to use simple local delivery for outgoing messages when the recipients are in the same domain as your HISP. In this case you will not want the messages to be processed by the security and trust agent unless your HISP configuration allows users to not trust other users inside the same domain (i.e. user1@sample.com does not trust user2@sample.com). The gateway module provides a custom matcher called RecipAndSenderIsNotLocal which takes one parameter: a list of domains managed by your HISP. NOTE Multiple domains are comma delimited.

Example:

   <mailet match="RecipAndSenderIsNotLocal=securehealthemail.com,cerner.com" class="NHINDSecurityAndTrustMailet">
       <ConfigURL>http://localhost:8081/config/ConfigurationService</ConfigURL>       
   </mailet>

When James reaches this mailet, the matcher determines if the message is incoming from an external domain or an outgoing message. If the message is incoming, all recipients are sent to the mailet. If the message is outgoing, all external recipients are processed by the agent while all local recipients are not processed and remain unencrypted for local delivery. NOTE When deploying James in multi domain environment, unless all domains within a HISP mutually trust each other, all messages and recipients should be processed by the agent.

Configuration URL

The configuration URL can either reference an XML file that contains configuration that adheres the XML SMTP Agent configuration specification or and http(s) address that reference the configuration service.

Examples:

  • File: file:///home/ubuntu/james-agent-v2/james-2.3.2/apps/james/SAR-INF/STConfig.xml
  • HTTP: http://configServer/config/ConfigurationService

When using the HTTP based configuration service, it is assumed the service has been deployed and settings such as domains, anchors, and certificates have been configured.

When running in an development environment, you can launch the configuration service and configuration UI directly from the command line using maven. From the nhin-d build directory, open two command or terminal windows and run the mvn jetty:run command from the config-service directory first, then the config-ui service. The services use an embedded database called Derby to persist all settings. The configuration ui can be accessed from a browser at http://localhost:8080/config-ui using nhind as the username as password. The configuration service URL for the mailet configuration will be at http://localhost:8081/config/ConfigurationService.

Message Monitoring and Delivery Notification

Version 2.0 of the gateway introduced not only support for the delivery notification implementation guide, but better support for delivery quality of service in general. Increased support has been added in three sections:

The default behavior of the gateway has been designed and configured to be as passive as possible to older releases.

Rejected Security and Trust Messages

Previously the NHINDSecurityAndTrustMailet did not take any action for outgoing messages that were rejected for security and trust reasons. The AutoDSNFailueCreation settings configures the mailet to automatically generate DSN bounce messages for rejected recipients. The setting accepts a comma delimited list of out bound message scenarios that should auto generate bounce messages.

  • ReliableAndTimely - Messages that have been indicated they wish to enable the timely and reliable delivery notification option.
  • General - All other message.

NOTE: The settings are only applicable for out bound message and do not apply to encrypted, DNS, or MDN messages.

Message Monitoring

Messages that are not trusted by a receiving HISP do not receive any type of positive notification. To provide feedback to the sender, the NHINDSecurityAndTrustMailet can send outbound message information to a monitoring service that keeps track of all out bound messages and received notifications. The monitoring service is responsible for generating DSN bounce messages for messages that have not received notification messages within a given time period. The time period is configured in the monitoring service.

Delivery Failure

The gateway overrides the default James LocalDelivery mailet with the TimelyAndReliableLocalDelivery mailet to generate DSN bounce messages if the message cannot be delivered to the local users mailbox. It also adds the DirectBounce mailet to the 'local-address-error' processor to generate a DSN bounce if the local mail account does not exist.

Incoming Notification Policy

Trust can be configured to be directional meaning trust can be configured to allow either outbound only or inbound only messages to be trusted. If directional trust is set for outbound only, issues can occur with message monitoring because the incoming MDN and DSN notification messages will be marked as not trusted. To allow message monitoring and QoS in general to operate properly, notification (and only notifications messages) must be allowed to flow through the system as trusted messages. However the agent needs to ensure that only notification messages that are generated from trusted outbound destinations are allowed to marked as trusted.

The security and trust mailet supports the setting UseOutgoingPolicyForIncomingNotifications to allow incoming notification messages to be marked as trusted. The following is sample configuration:

   <mailet match="RecipAndSenderIsNotLocal=securehealthemail.com,cerner.com" class="NHINDSecurityAndTrustMailet">
       <ConfigURL>http://localhost:8081/config/ConfigurationService</ConfigURL> 
       <UseOutgoingPolicyForIncomingNotifications>true</UseOutgoingPolicyForIncomingNotifications>      
   </mailet>

For backward behavioural passivity, the mailet defaults this parameter to false if it does not exist in the mailet configuration. If you system will be configured with outbound only directional trust, it is recommended that this parameter be present and set to true.

SuppressAndTrackAggregate Mailet

There are two scenarios where MDN notification messages should be suppressed from being delivered to the sender's edge client:

The SuppressAndTrackAggregate mailet is responsible for sending inbound notification messages to the monitoring service and determining if the notification message should be suppressed. The following is an example configuration:

    <mailet match="IsNotification" class="SuppressAndTrackAggregate">
       <MessageMonitoringServiceURL>http://localhost:8081/msg-monitor</MessageMonitoringServiceURL>
       <ConsumeMDNProcessed>true</ConsumeMDNProcessed>            
    </mailet>

The following mailet elements are used:

ElementTypeDescription
MessageMonitoringServiceURLURLA URL to the location of the message monitoring service for quality of service purposes. If this parameter is present, the mailet creates default NoOp TxService implementation.
ConsumeMDNProcessedBoolean Indicates if all MDN processed messages should be suppressed. It is recommended that this be set to true. Setting to false will allow all MDN processed message to be sent to the sender's edge client. This setting is mainly here to enable passivity from older versions of the agent (setting to false enables the passive behavior).

Prebuilt Assembly Configuration

A full stock assembly and configuration is available that contains the minimum set of components to run the Direct Project in a single domain stand alone James deployment. See the stock project site for installation and configuration details.

Deployment Configuration Scenarios

The following scenarios are not at all an exhaustive list of deployment configurations, but some identified use cases and suggested best practice deployments.

Development Testing Configuration

This use case describes a scenario where the HISP is only used for development testing. See the Bare Metal Project Java source wiki for details.

Single Domain Standalone

This use case describes a scenario where the HISP consists of only one domain, and a deployed instance of James comprises the entire SMTP server stack.

Mailet configuration in this use case is fairly simple.

Example:

   <mailet match="RecipAndSenderIsNotLocal=mydomain.com" class="NHINDSecurityAndTrustMailet">
       <ConfigURL>file:///home/ubuntu/james-agent-v2/james-2.3.2/apps/james/SAR-INF/STConfig.xml</ConfigURL>       
   </mailet>

Because all messages sent from within the domain are delivered locally, the matcher is necessary to ensure local outgoing messages are not processed by the agent.

Single Domain Standalone With EMAIL Gateway

This use case describes a scenario where the HSIP consists of only one domain, mail is stored and retrieve using James, but all incoming and outgoing messages going to and coming from the backbone are handled by an email gateway.

This model assumes that the gateway is configured to send all incoming messages to James' configured incoming port (default 25). The mailet configuration is still fairly simple.

Example:

   <mailet match="RecipAndSenderIsNotLocal=mydomain.com" class="NHINDSecurityAndTrustMailet">
       <ConfigURL>file:///home/ubuntu/james-agent-v2/james-2.3.2/apps/james/SAR-INF/STConfig.xml</ConfigURL>       
   </mailet>

As with the previous configuration, all local mail is delivered locally. However all outgoing mail needs to be forwarded to the email gateway. The RemoteDelivery mailet handles forwarding out bound messages to a configured location. In this scenario we want local messages to stay local and all externally bound messages to be forwarded to the gateway. The mailet can be configured immediately following the security and trust mailet if no other processing is necessary in the transport processor.

Example:

         <mailet match="RecipientIsRegex=(?!mydomain.com)"  class="RemoteDelivery">
            <gateway>mygatewayhostname:25</gateway>
         </mailet>

Note the use of negative look arounds in the regex. James does not provide a matcher for remote recipients, but a negative look around regular expression can be used to mimic the desired affect. The RecipAndSenderIsNotLocal should not be used because it would send all incoming messages to the gateway.

Single Domain James as Processing Relay

This use case describes a scenario where the HISP has an existing stand alone email solution such as Postfix. The email server may not have the ability to inject custom processing modules into the server, but can relay messages to other servers for custom processing. After the custom email server finishes processing the message, the custom server must send the message back to the original email server (typically on port separate from the main incoming email port). In this scenario, James is the custom email server. The standalone email server may or may not be configured to handle local delivery before the custom processor is executed, but an identical mailet configuration can be used for both cases.

Example:

   <mailet match="RecipAndSenderIsNotLocal=mydomain.com" class="NHINDSecurityAndTrustMailet">
       <ConfigURL>file:///home/ubuntu/james-agent-v2/james-2.3.2/apps/james/SAR-INF/STConfig.xml</ConfigURL>       
   </mailet>

The mailet will still want to ensure that local outbound messages remain unprocessed because the standalone email server will still deliver these messages locally. Howerver, the relay matcher will need to match all messages so all messages are sent back to the standalone email server.

         <mailet match="All"  class="RemoteDelivery">
            <gateway>mystandaloneemailserver:10026</gateway>
         </mailet>

Note the custom port on the remote delivery email server. This is because the email server will more than likely accept processed messages on a separte port than incoming messages.

If the standalone email server and James are deployed on the same OS instance, you will probably need to change James' incoming port.

Example:

   <smtpserver enabled="true">
      <port>10025</port>

   .
   .
   .
   <smtpserver>

This assumes that the standalone email server is configured to forward messages to James on port 10025 for message processing.

Single Domain James as Injected Into Relay/Gateway Flow

This use case describes a scenario where the HISP already uses a relay or email gateway for incoming and outgoing messages. James is injected between the email server and gateway. The email server is configured to send all outgoing messages to James incoming port, and James RemoteDelivery mailet is configured to send all messages to the relay/gateway.

Example:

   <mailet match="RecipAndSenderIsNotLocal=mydomain.com" class="NHINDSecurityAndTrustMailet">
       <ConfigURL>file:///home/ubuntu/james-outgoing/james-2.3.2/apps/james/SAR-INF/STConfig.xml</ConfigURL>       
   </mailet>

   <mailet match="All"  class="RemoteDelivery">
      <gateway>mygateway:25</gateway>
   </mailet>

As with the previous use case, James' incoming SMTP port may need to be changed if the email server and James are deployed on the same OS instance.

What about incoming messages from the email gateway that need to go the server? How do you configue the RemoteDelivery mailet to deliver outgoing messages to the gateway and incoming messages to the email server. The answer is you can't. That's not entirely true. The problem is that there is not matcher that can determine if the messages are outbound or inbound. If they did exist (and someone is entirely free to write them), the configuration could look like this.

   <mailet match="RecipAndSenderIsNotLocal=mydomain.com" class="NHINDSecurityAndTrustMailet">
       <ConfigURL>file:///home/ubuntu/james-handle-it-all/james-2.3.2/apps/james/SAR-INF/STConfig.xml</ConfigURL>       
   </mailet>

   <mailet match="MesssageIsOutbound"  class="RemoteDelivery">
      <gateway>mygateway:25</gateway>
   </mailet>
   
   <mailet match="MesssageIsInbound"  class="RemoteDelivery">
      <gateway>mygateway:25</gateway>
   </mailet>

Something missing in this configuration is that the RemoteDelivery mailet requires a different mail repository URL for each mailet instance.

Another option is to deploy to sepearte James instances. One to handle incoming messages and the other to handle outgoing messages. The outgoing instance would be configured similar to the first example in this use case. The incoming instance would be configured similar to the following:

Example:

   <mailet match="All" class="NHINDSecurityAndTrustMailet">
       <ConfigURL>file:///home/ubuntu/james-incoming/james-2.3.2/apps/james/SAR-INF/STConfig.xml</ConfigURL>       
   </mailet>

   <mailet match="All"  class="RemoteDelivery">
      <gateway>myemailserver:25</gateway>
   </mailet>

The gateway would forward all incoming email to the inbound instance's SMTP port (needs to be different than the outgoing instance's port) and James forwards all messages on to the email server.

Multi Domain James as Injected Into Relay/Gateway Flow

This use case describes a scenario where the HISP already uses a relay or email gateway for incoming and outgoing messages and supports multiple domains on the its email server. James is injected between the email server and gateway. The email server is configured to send all outgoing messages to James incoming port, and James RemoteDelivery mailet is configured to send all messages to the relay/gateway.

In this scenario, it is suggested that the email server not use local delivery for messages coming from email clients. The reason is that each domain may have a sepearate trust policy and local delivered messages would never be sent to the agent for trust policy processing. If all domains in the HISP mutually trust each other, then local is an acceptable configuration option.

The James and mailet configurations are exactly the same as the previous scenario with one exception. The outgoing James instance should list all domains managed by the HISP in its matcher.

Example:

   <mailet match="RecipAndSenderIsNotLocal=mydomain.com,myotherdomain.com" class="NHINDSecurityAndTrustMailet">
       <ConfigURL>file:///home/ubuntu/james-outgoing/james-2.3.2/apps/james/SAR-INF/STConfig.xml</ConfigURL>       
   </mailet>

   <mailet match="All"  class="RemoteDelivery">
      <gateway>mygateway:25</gateway>
   </mailet>

Logging

James uses the Avalon logging system and Direct Project components use apache-commons-logging (JCL). Generally JCL will try to discover an application's configured logging mechanism, so you may be OK with the James logging configuration found in %jamesInstallRoot%/apps/james/SAR-INF/environment.xml. However you are free to change this and set custom log levels and formats. You can also override JCL to use a different logging sub system such as log4j or JDK 1.4. Details for configuring JCL can be found on the JCL guide.

A quick way to turn on TRACE logging for debugging purposes is to add the following lines to the phoenix.sh shell script and run James interactively with run.sh.

Search the phoenix.sh file for the following text:

JVM_OPTS="-Djava.ext.dirs=$JVM_EXT_DIRS" 

and change it to:

JVM_OPTS="-Djava.ext.dirs=$JVM_EXT_DIRS -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.SimpleLog -Dorg.apache.commons.logging.simplelog.defaultlog=trace"

This will configure JCL to output all logging to the console and sets the threshold to trace.


Previous: Up: Deployment GuideNext: