How to generate and send emails using Spring and Velocity

2008/03/05

Testing applications that communicate using email is more challenging than e.g. testing database access. Thanks to GMail and Spring, it can be done pretty easily.

Here is the scoop: I need to generate email notifying users about business events and after sending the email, store the content in the database. In real environment you will use corporate SMTP server and real email addresses of real people.
For development we can avoid bothering and spamming our customers by few simple tricks.

First step is to get new GMail account. Name it e.g. Company.Notifier or something easily distinguishable. In Spring, configure the sender bean:

<bean id="mailSenderGMail" class="org.springframework.mail.javamail.JavaMailSenderImpl">
           <property name="host">
                  <value>smtp.gmail.com</value>
           </property>
           <property name="javaMailProperties">
                  <props>
                          <prop key="mail.smtp.auth">true</prop>
                          <!-- this is important, otherwise you will get the
                                 exception: 530 5.7.0 Must issue a STARTTLS command -->
                          <prop key="mail.smtp.starttls.enable">true</prop>
                          <prop key="mail.smtp.timeout">25000</prop>
                </props>
            </property>
            <property name="username">
                   <value>company.notifier</value>
            </property>
            <property name="password">
                   <value>secret</value>
            </property>
            <property name="port" value="587"/>
            <!--   you don’t need to set the port number, 25 is default -->
     </bean>

The port can have value 25 (default) – but if you are using ISP provider such as Rogers, chances are the default port is blocked for outgoing connections – you can use port 587 instead.Second piece is component that actually uses the mailSender: notificationService.

    <bean id="notificationService" class="com.thinknostic.APP.service.NotificationServiceImpl">
           <property name="mailSender" ref="mailSenderGMail"/>
           <property name="velocityEngine" ref="velocityEngine"/>
           <property name="emailsToFile" value="true"/>
       <property name="alwaysCCList">
           <list>
               <value>miro.adamy+alwaysCC@thinknostic.com</value>
           </list>
       </property>

    .... deleted ...

   </bean>

Note the velocityEngine bean that is used to generate the body of the email from the template. The ‘alwaysCCList’ property is using nice little trick available with GMail: if you send email to YOURNAME+anything@gmail.com, the ‘+anything’ will be ignored but kept with email and it will arrive as if the address were just YOURNAME@gmail.com. You can use the postfix to find or autotag the emails.The code that sends email is actually pretty simple (the method of the notificationService)

public EmailContent sendNotificationEmail(EmailParameters params, EmailContent _content, boolean isDryRun) {

    final EmailContent c = mergeTemplate(params, _content);

    MimeMessagePreparator preparator = new MimeMessagePreparator() {
         public void prepare(MimeMessage mimeMessage) throws Exception {
            MimeMessageHelper message = new MimeMessageHelper(mimeMessage);
            message.setTo(c.getEmailTo());
            message.setFrom("DO-NOT-REPLY@company.com"); // could be parameterized...

            message.setText(c.getEmailBody(), true);
            message.setSubject(c.getEmailSubject());

            if (alwaysCCList != null && alwaysCCList.size() > 0) {
                message.setCc(alwaysCCList.toArray(new String[0]));
                c.setEmailCcFromList(alwaysCCList);
            }
            ...
         }
      };

      if (isDryRun || emailsToFile)
      {
              // save to file
            ...
      }

      if (!isDryRun)
          this.mailSender.send(preparator);
      return c;
}

The class EmailContent is container for email address, subject, body, CC list.
It gets created as empty class with only recipient name and email address passed from business method as well as name of the Velocity template that is used to render email body. The method mergeTemplate loads the Velocity template and renders the actual email body, using the parameters (which is more or less) a hash map, cotaining e.g. URL’s or information that needs to be inserted to email. The generated content is stored back to EmailContent, which will be after successful delivery written to database for audit and archiving purposes.If you are JavaScript or Ruby programmer, you will immediately recognize the ‘functor’ pattern: preparator is set to instance of anonymous inner class and is used with configured preparator.The actual rendering of the content using Velocity can be done like this:


private EmailContent mergeTemplate( EmailParameters params, EmailContent content) {
    Map<String, Object> model = new HashMap<String, Object>();
    model.put("param", params);
    model.put("content", content);

    String text = "MISSING TEXT";
    String subject = "Notification email";
    String template = content.getTemplateId();
    try {

        // get subject line

        if (template_names.containsKey(template)) {
            subject = template_names.get(template);
            VelocityContext velocityContext = new VelocityContext(model);
             StringWriter writer = new StringWriter (  ) ;
             PrintWriter out = new PrintWriter ( writer ) ; 

             Velocity.evaluate(velocityContext, out, "subject", new StringReader(subject));
             subject = writer.toString();

             model.put("subjectLine", subject);
            // now the body
             text = VelocityEngineUtils.mergeTemplateIntoString(velocityEngine,
                    "com/thinknostic/APP/service/email/"+template, model);

        }
        else {
            // TODO: log configuration error - template not found
        }

        content.setEmailBody(text);
        content.setEmailSubject(subject);
        return content;

    } catch (VelocityException e) {
        // back to untranslated
        // TODO: error report
        // subject = params.subject;
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

    return null;
}

In the above, template id is actual file name with body of the HTML email and the hashmap template_names (configured in Spring XML) maps the id to another string, which is then used as subject line. Both body and subject can contain macros.
Note: if you get conflict on ${} syntax, see this entry.

Now, finally – how do we test this ? It is quite easy, thanks to JUnit support in Spring.


@ContextConfiguration(locations={"/com/thinknostic/APP/test/test-context.xml",
"/com/thinknostic/APP/test/service/NotificationTests-context.xml"})
public class NotificationTests extends AbstractTransactionalJUnit4SpringContextTests {

    @Autowired
    NotificationServiceImpl notificationService;
    public NotificationServiceImpl getNotificationService() {
        return notificationService;
    }
    public void setNotificationService(NotificationServiceImpl notificationService) {
        this.notificationService = notificationService;
    }

    @Autowired
    ApprovalDAO        approvalDao;
    public ApprovalDAO getApprovalDao() {
        return approvalDao;
    }
    public void setApprovalDao(ApprovalDAO approvalDao) {
        this.approvalDao = approvalDao;
    }

      Document doc;
      EmailContent cont;
      Approval app;
      ApprovalStep step1;
      ApprovalStep step2;

      @Before
      public void createMockApproval() {
          this.logger.info("Before");
          // controlls whether to use emails
          notificationService.setReallySend(true);

          doc = (Document)DocumentInfo.createMockDI("103", 1);

          // create the approval and insert it into
          app = Approval.createMockApproval(1L, doc);
          approvalDao.insert(app);

          step1 = ApprovalStep.createMockStep(app.getApprId(), 1);
          approvalDao.insert(step1);
          step2 = ApprovalStep.createMockStep(app.getApprId(), 2);
          approvalDao.insert(step2);

          cont = EmailContent.createMock(app.getApprId(), step1.getStep(), "miro adamy", "miro_adamy@rogers.com");

      }

      @After
      public void cleanup() {
          this.logger.info("After");
          // reset
          notificationService.setReallySend(false);
          doc = null;
          cont = null;
          app = null;
          step1 = null;
          step2 = null;
      }

      private void assertEverything()
      {
          assertTrue("The email service is empty", notificationService != null);
        assertTrue("The email parameteres not available", emailParameters != null);
        assertTrue("The document does not exist", doc != null);
      }

    @Test
    public void approval_all_cancelled_test() {
        doc.setFilename("approval_all_cancelled");

        notificationService.notifyApprovalCancelledAll(doc, cont);
    }

    @Test
    public void approval_cancelled_test() {
        assertEverything();        

        doc.setFilename("approval_cancelled");
        EmailContent ui = EmailContent.createMock(app.getApprId(), step1.getStep(), "miro adamy", "Miro.Adamy@gmail.com");

        notificationService.notifyApprovalCancelled(doc, ui);
    }

    // and so on - many more tests
    ....
}    

Assumed that your Spring context configuration files defined in annotations are OK, all you have to do is run the suite. The XML defines beans with names corresponding to the test properties and autowiring will take care of the rest.Also notice that @Before method (which runs before each test) actually DOES modify the database and inserts records with same primary key over and over. This works thanks to transaction magic of Spring, which rolls back the inserts at the end of the test.Last thing to mention is using of “mockXXX” static methods to generate the instance of object. IT not really a mock object we are creating here, but a well defined instance of business object with some properties parametrized. You can even define it as a bean in Spring context XML, but that is IMHO an overkill – keeping everything in test Java class makes things easier to understand.

This method does belong to tests, but it is very convenient just to keep it with domain object. I usually create the ‘createMock’ method right after I add a new domain object

Advertisements

Spring PropertyPlaceholderConfigurer with Velocity

2008/02/25

Part of the configuration magic is (one of several) Spring post-processors, that allows load the property file or files and refer to the values in these files inside XML configuration files. The rationale behind these is that property files are much easier to edit that possibly huge XML config file and using property values prevents repeating same value over and over again. With this approach, the following configuration:

 	<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
		<property name="username" value="some_user"/>
		<property name="password" value="secret"/>
	</bean>

can be written as

 	<bean id="propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
		<property name="locations">
			<list>
				<value>WEB-INF/mail.properties</value>
				<value>WEB-INF/jdbc.properties</value>
			</list>
		</property>
	</bean>

	
 	<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
		<property name="username" value="${jdbc.username}"/>
		<property name="password" value="${jdbc.password}"/>
	</bean>

 

and (in jdbc.properties)

jdbc.username=some_user
jdbc.password=secret

If you are using in the same project Velocity and want to utilize the VTL macros inside the configured values, the approach above will not work, because PropertyPlaceholderConfigurer will try to replace ALL macros in form ${something} with a value from property file – and throw exception if not found.The solution is easy: you need to switch the prefix of the PropertyPlaceholderConfigurer to something else than the default ${ and avoid conflict wih Velocity. The modified configuration could look like:

 
	
	<bean id="propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
		<property name="placeholderPrefix" value="#{"/>
		<property name="locations">
			<list>
				<value>WEB-INF/mail.properties</value>
				<value>WEB-INF/jdbc.properties</value>
			</list>
		</property>
	</bean>
 	
	<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
		<property name="username" value="#{jdbc.username}"/>
		<property name="password" value="#{jdbc.password}"/>
	</bean>

 

Oracle SQLDeveloper and mystery of the missing host

2008/02/23

If you work with Oracle database, a very good tool you should not ignore is the Oracle SQL Developer. It gives you much more power when it comes to manipulating data or managing database. I use most of the time MyEclipseIDE plugins, but when I need e.g. export data, the SQLDeveloper is way to go.

Today I tried it for first time on the Mac. I started to configure and database connection and was completely confused by the error message “Port must contain numerical value”. Here was the dialog box:

picture-6.png

I typed the hostname:1521 but the error message kept coming back. There was no obvious way how to put 3 values – host, port and SID into two field. After checking the documentation, I remembered that the Windows version did have 3 text fields instead of two.

What was the problem ? The layout manager in Java did unfortunately completely hide one of the 3 fields. After resizing the dialog, all fields would be visible:

picture-4.png

After this, everything worked OK.

Obviously, Mac version of the SQLDeveloper could use some minor improvement. For example, the menu looks everything but like a finished Mac application:

picture-5.png


Book recommendation: Java Concurrency in Practice

2008/02/06

After long time, I stumbled upon something that is both educational and practical. Since Doug Lea’s classic, this is very readworthy work on concurrency in Java world. It focuses on Java 5 features and addresses both Web applications as well as GUI apps – each group having their own share of concurrency related problems

jcp.jpg

Here is the Amazon page for the book – check the reviews. There is also a sample chapter available on the book’s website. If you have Safari subscription, you can read it there as well.

Recommended.


Spring Framework – the biggest missing piece

2008/02/04

I am having the pleasure working with Spring 2.x since about October last year. It is my return back to Spring, after first encounter in 2005 (counting only larger projects). Same as in previous project, it has been so much fun :-). So much fun in fact, that I was spending my evening playing with code and reading up what is new in 2.5, rather that blogging 😉

I am still amazed about how nice is the whole thing designed and how many ways it can be used. It’s flexibility is simply breathtaking. The price for the flexibility is complexity – sometimes it is not obvious what of many options should be chosen and what are all these options anyway :-).

Spring has very good documentation, compared to most of the open source projects I have looked at (and it was pretty large number over last 10 years). There are also many books on Spring, several of them pretty good. All this makes the fairly steep learning curve more accessible. But what is still missing are good examples.

But wait a moment, you may say – Spring source distribution DOES have several examples – for example JPetstore (offering two alternative implementations of UI – once using Spring MVC, once using Struts), Petclinic, and several others. The source code is nicely readable and well commented. So what is the big deal ?

The big deal is the ratio of options and possibilities offered by the framework to the options actually presented and demonstrated by the examples. Let’s take only the Spring MVC, which is certainly less than 30 % of the area covered by framework. Considering the possible combination of components would easily justify 20 examples just on that topic, not 2 or 3. The examples select one or two of possible URL mappers, provide some examples of controllers, view resolution and then the actual view. What is missing are alternatives and explanation of the differences. How would using SimpleUrlHandlerMapping instead of BeanNameUrlHandlerMapping impact the application, is explained pretty well in Chapter 13 of the documentation. For an experienced developer this (and a source code itself) is usually enough. If you try to explain Spring based application to somebody without prior IoC / DI experience, you may find out that it is not as obvious and that having an example of working code that would show the actual different implementation and letting him/her to compare and experiment, turning one into another would probably be the easiest way how to get started.

I realize that creating enough examples could be possibly a code maintenance nightmare – at least if the pace of Spring development will remain the same. In many cases, more documentation on the samples, explaining how it works at the high level and discussing the alternatives would be almost as helpful as the full source. The otherwise very good documentation is quite brief on the Samples side – the Chapters 26 has about two screens, and that for all the samples.

And because this is definitely NOT a rant, but an attempt to make a constructive suggestions, I plan to do something about the situation myself and will post the notes from “dissecting the Spring App” that I used for explaining how things works in Spring MVC and around. As soon as I get them to presentable form ;-). Stay tuned.


Spring 2 MVC and ResourceBundleViewResolver problem

2008/01/24

This one took some time to figure out – it is one of those problems that are obvious after you know the solution but hard to see beforehand.

I had the ResourceBundleViewResolver configured with Tiles 2, like this:

<bean id="viewResolver" class="org.springframework.web.servlet.view.ResourceBundleViewResolver">
<property name="order" value="0"/>
<property name="basename" value="views"/>
<property name="defaultParentView" value="parent-view"/>
</bean>

and the view.properties was:

parent-view.(class)=org.springframework.web.servlet.view.JstlView
parent-view.(abstract)=true

about.url=/WEB-INF/jsp/about.jsp
form.url=/WEB-INF/jsp/form.jsp
list.url=/WEB-INF/jsp/list.jsp

It kept givin exception saying that /WEB-INF/jsp/list.jsp does not exist. Which it did. I tried couple of things, no result.

The problem was in trailing space AFTER list.jsp, therefore the path to JSP file searched for was “/WEB-INF/jsp/list.jsp “, not “/WEB-INF/jsp/list.jsp”.

Not sure if this is bug or feature – but removing trailing spaces should really be a sensible thing to do. As I found afterwards, the problem exists since 2005 – see this and this.


Using iBatis with Abator

2008/01/20

I am back to Java and on mission of re-discovering Spring coolness – and in awe how much was added since 1.2 to current 2.5.1. Thanks, SpringSource and hundreds of contributors.

Data layer of the current project is based on iBatis and Spring DAO. We have decided to try out the Abator tool from Apache (part of the iBatis suite) to generate initial version of data mapping (as we inherited the database schema).

Only necessary part is abator.jar (there are no other dependencies).

The abator is driven by the XML file that defines to which database to connect and which tables must be be processed.

Running abator is simple:

java -cp abator.jar org.apache.ibatis.abator.api.AbatorRunner PROJECT.xml true

Parameter ‘true’ defines whether generator will overwrite files.The generated files are stored in location from xml file (in our case ‘generated/src’). This directory MUST exist – Abator will not create it.The project file contains information about your database driver + connection the used generator, packages for generated Java and XML files. The Java5 generator is very useful, because it produces code using generics. Abator can also understand Spring based iBatis templates and produce code directly usable in Spring projects.Two “gotchas” I have encounter (so far):

If you generate maps with many tables, the Abator assumes that you will be using namespaces. The examples from Spring do not however use namespaces, so if you use the existing code as blueprint for your project, make sure you have the namespaces enabled in the sqlmap

<sqlMapConfig>
<settings useStatementNamespaces="true" />
...

Second is related to timestamp columns (e.g. CREATED_AT and similar columns).In Java, they are (most of the time) represented as java.util.Date object, which contains both date and time portion. In SQL, there are multiple data types available, depending on your database. We are using Oracle 10g and the generated map was using DATE as SQL type. This caused cutting of the time portion of the value and setting all times to 0:00:00.The solution was to use proper, explicit types for both SQL and Java. You may need to edit the map in multiple locations, replacing:

<result column="CREATE_DATE" property="createDate"
jdbcType="DATE" />

with


<result column="CREATE_DATE" property="createDate"
 jdbcType="DATETIME" javaType="java.util.Date" />

and the


CREATE_DATE = #createDate:DATE#,

with


CREATE_DATE = #createDate:DATETIME#,

After this, everything works fine.So far, the only minor unresolved annoyance is the abatorgenerated_ prefix of all SQL id’s – I cannot find the way how to switch it off.It is sometimes very useful – if you want to re-run the tool, it will not touch SQL without this prefix. In our case, where the generated code is just an initial step to speed us in getting the initial codebase out fast., we would actually prefer to avoid it and keep the SQL fragments id’s nice and tidy.