Here’s a something we learned when working on the game engine for the Citerus coding challenge CrazySnake (in Swedish). Maybe some of our findings about class loading in Clojure and usage of ServiceLoader will be useful for someone out there, so I thought I’d share them.

The competing teams provided their implementations of the Brain interface from our API and packaged the implementation in a jar-file.

The brain was injected into a snake, that moved around autonomously on a game field, eating fruits and avoiding to crash into other snakes or walls. Remember the old game called Snake that used to be on several mobile phones? Something like that, but autonomously and multi-player.

Since we wanted to load the implementations at runtime, we needed dynamic class loading.

The most commonly used approach to dynamic class loading has been simply use of the Class.forName(). However, this only supports loading of classes using their name, and not their type. Since we didn’t want to restrict the names of the implementing classes we simply wanted to load all implementations of the Brain interface that were provided. The ServiceLoader has been around since Java SE 6 and supports dynamic class loading based on a type. The ServiceLoader reads files in the META-INF/services/ directory on the classpath and tries to lazily instantiate classes based on the fully-qualified class-names in these files. The name of the file should correspond to the fully-qualified name of the service (interface) that the classes implement (in our case this was se.citerus.crazysnake.Brain). Look in the examples-projects here to see how it was set up.

We used java.util.ServiceLoader to load each team’s implementation by creating a URLClassLoader that was fed into the ServiceLoader.load(). This looked something like this:

  URL[] urls = {file.toURI().toURL()};
  ClassLoader classLoader = new URLClassLoader(urls);
  ServiceLoader<Brain> services = ServiceLoader.load(Brain.class, classLoader);

The approach we first used (seen above) worked fine for our initial test-brains that were implemented in Java and we were feeling quite happy with the results. However, we also wanted it to be possible to implement the brains in more JVM-supported languages than Java and Clojure was an obvious candidate. The solution worked fine with Groovy and Scala but when trying to implement a Brain in Clojure using :gen-class we stepped into some trouble. Below is the code for a stupid brain implemented in Clojure:

(ns se.citerus.crazysnake.brain.clojure-brain
  (:import
    [se.citerus.crazysnake Movement])
  (:gen-class
    :name se.citerus.crazysnake.brain.ClojureBrain
    :implements [se.citerus.crazysnake.Brain]
    :main false))

;-- Implementation of Brain interface

(defn -init [this participants meta])

(defn -getName [this]
  "ClojureBrain")

(defn -getNextMove [this state]
  Movement/FORWARD)

When we tried to load a Clojure-implementation the ServiceLoader threw an ServiceConfigurationError when accessing its iterator. Some research led us to the knowledge that Clojure uses the current Thread’s context class loader by default. Since the context classloader has no knowledge of our Clojure code (only our created classloader knows this), this didn’t work. This made us understand that we had to set the Thread’s context classloader before trying to call ServiceLoader.load(). The following code made the whole shebang work like a charm:

File brainImplementingJarFile = new File(pathToJar);
URL[] urls = {file.toURI().toURL()};
ClassLoader contextLoader = Thread.currentThread().getContextClassLoader();
ClassLoader classLoader = new URLClassLoader(urls, contextLoader);
Thread.currentThread().setContextClassLoader(classLoader);
ServiceLoader<Brain> services = ServiceLoader.load(Brain.class, classLoader);

// Reset the context classloader
Thread.currentThread().setContextClassLoader(contextLoader);

Hope someone will find this useful when working with dynamically loading Clojure classes!

I suspect this post will be quite massive so I’ll skip the screenshots in this one. Instead I decided to create a sample project structure and put it up on my github account. See the readme for information on how to set up the example projects.

Overall structure and idea

The parent project (with .parent suffix) is the mother of the build. Practically all configuration of the build is handled by its pom.xml file. We strive to keep the sub-projects as small as possible to minimize the duplication of information in the development environment (Eclipse in this case) and the headless build system (Tycho/Maven).

The information that we’re required to provide in the sub-projects’ pom.xml files are the version of the project (which is verified against the version provided in the Manifest.MF) and the type of project (feature/bundle/test bundle/update site).

I will describe the things that I reckon is most important in the example projects so that you (hopefully) can follow what’s happening when building the projects.

Features and update sites

The sample project contains a feature project and an update site project. These two projects are intended to give some extra help from within the Eclipse IDE. When the plugins are built by the CI server (I assume you have one!?), the server can also push the update site (containing the feature) to an HTTP server (or similar) that is available for developers that want to use the plugins as target platform in their development. This makes the build quite modular and nice to work with.

Tycho makes the automation for building for your features and update sites quite a simple task. You need to create a pom.xml for your feature looking somewhat like this:

<project>
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>se.mattiasholmqvist</groupId>
    <artifactId>examples</artifactId>
    <version>0.0.1-SNAPSHOT</version>
  </parent>
  <groupId>se.mattiasholmqvist</groupId>
  <artifactId>examples.feature</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>eclipse-feature</packaging>
</project>

Note: See the sub-project with .feature-suffix for the provided example.

This pom file is very similar to the bundle pom file. The only difference is the <packaging> tag that tells Tycho to build the project as a feature. The nice thing about these tiny poms is that the dependency management becomes quite easy. You can more or less work without caring about the Maven configuration since Tycho uses the Manifest and feature.xml to setup the correct build dependencies.

Of course, you would also need to specify a proper target platform. In my previous posts, I’ve shown how to do this with the -Dtycho.targetPlatform switch, which works but there is a cooler and more flexible way that also works great with Tycho.

Target platform definitions

Ok, we have our build set up nicely and all seems to be working fine. However, the great success of your project has made it grow significantly and it seems that not all developers that join the project are able (or willing) to setup their target platforms (in Eclipse) in accordance with the Tycho build since it requires quite a lot of understanding about Tycho/OSGi and perhaps Eclipse P2 as well. This causes disturbance to your project and you do your best to help your teammates out, but you reckon there must be a better/simpler way of tackling this issue than to manually manage the target platform installation for each developer.

A short tale about the target definition file

The target definition file is quite a newcomer to the Eclipse development environment. It is a nice feature that lets you specify the target platform in a structured fashion (although the previously supported directory-based platforms are still supported in Eclipse) which I highly recommend. Another even nicer thing is that Tycho also supports resolving the target platform from the Eclipse target definition file. This means that setting up the development environment for a new developer in simply a matter of locating the target definition file and click on “Set as target platform”, and the target platform is now consistent with the platform that Tycho uses. This is probably not a perfect description but you can find some more info from the Eclipse help and some more info on using them in practice can be found here.

The first step – creating the .target file

I like to keep my target platform in a separate project, just for structure and separation. My guess is that you can make it work by putting it anywhere you want, but in my example I’ve put it in the project called se.mattiasholmqvist.tychoexamples.targetplatform.

I’ve created a target definition file in Eclipse (it’s found in the New-> menu) and I’ve called this file example_target.target and placed it directly in the target platform project directory. I selected a few Eclipse dependencies that I needed for my bundle and saved the file.

We also need to provide some configuration to this project. I’m not sure if this is the only way to do this at the time of writing but I’m sure that it works for me. This is a snippet from the pom.xml that I use for the target-platform project:

<build>
  <plugins>
    <plugin>
    <groupId>org.codehaus.mojo</groupId>
      <artifactId>build-helper-maven-plugin</artifactId>
      <version>1.3</version>
      <executions>
        <execution>
          <id>attach-artifacts</id>
          <phase>package</phase>
          <goals>
            <goal>attach-artifact</goal>
          </goals>
          <configuration>
            <artifacts>
              <artifact>
                <file>example_target.target</file>
                <type>target</type>
                <classifier>example_target</classifier>
              </artifact>
            </artifacts>
          </configuration>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

Testing

Tycho supports unit-testing (JUnit) of code within bundles by default. This has long been a hazzle (for me atleast), when working with Ant + PDE  build so this support is extremely welcome. Tycho tests bundles via the integration-test Maven goal, which is different from the regular test goal. It also makes life easier if you know a bit about the conventions Tycho uses for finding tests (this might however be subject to change in the future).

Start by creating a simple bundle that will contain the classes of your application. Second, we will create a separate bundle that will contain our tests. The convention here is to use the same same as the bundle with your application classes and add the prefix .tests to the test bundle name. I provide a (really stupid) test in the project with .tests-suffix in the github repo.

Note: The Test suffix is important if you want to use the default configuration of Tycho, or it won’t find your test class. Below is a snippet from the configuration of the test plugin for my example projects which enables unit testing for the project:

<packaging>eclipse-test-plugin</packaging>

I also made the .tests project a fragment to the bundle it belongs to. This makes it easier to locate classes to depend on. I think this setup works well within Eclipse/OSGi development.

Code coverage using EclEmma

Using code coverage in an OSGi environment is special. There are however some Eclipse plugins that support code coverage and it is possible to integrate these with Tycho. I want to give credit to the guys at Sonatype for helping me out with this. They recently posted some information on their website related to code coverage, so my example project might serve as a good complement to this article. I thought it might be a good idea to wrap our discussions up in an example that is perhaps a bit more accessible than the posts on the mailing list. My example is however a bit more simplistic than the example in the article above. I do not copy resources from multiple test bundles, but instead generate a report for each bundle. Check out the article if you need that feature.

In large, the code coverage is handled by a Maven profile so you need to pass a -Pcoverage flag to enable it.

To run the test suite (1 test in the example project) with code coverage and reporting:

mvn integration-test verify -Pcoverage

In my examples project I’ve put together a small bundle project that contains Eclemma Equinox runtime. It would probably be possible just to deploy it to a locally accessible Maven repository but for ease of use of the examples I’ve made it a part of the build instead. When you run mvn integration-test with the coverage profile in the parent project directory, this bundle will do code coverage of all bundles/packages that matches the configuration in the parent pom.xml. See the <profiles> configuration in the parent- and test project pom.xml files for exactly how this is done.

Hope this post can help someone out there with use Tycho which is a great tool for your OSGi development.

Checkout Sonatype’s Tycho website to keep up-to-date with the development of Tycho.

Previously, PDE build have been the way to build RCP applications in a headless build environment (such as Hudson, for example), but all those Ant scripts and configuration options in the build.properties file are not that appealing to me.

Tycho provides support for building RCP applications from product configuration files. As we will see, this makes the headless build for RCP applications a lot less confusing.

One nice thing that I forgot to mention about Tycho in my previous post is that is has great support for Manifest-first development. This means that the development team does not need to synchronize their Manifest files with the pom.xml that Tycho uses, it is handled automatically for them. The developers can use “Imported packages” and “Required plug-ins” as usual and Tycho will use that information to resolve the compilation target for each bundle. I find this workflow easier and more appealing than pom-first development (for example BND-plugin with a separate .bnd file for configuring exported/imported packages). I guess you have to figure out which way works for you.

This might not be as appealing for some of you who are accustomed to specifying Maven dependencies in the pom.xml but Tycho supports pom-first development as well. However, since I like to work Manifest-first I will use it for my examples here. For info on pom-first development, check out the Tycho users mailing list or the official website.

This will be a quick intro on how to get started with a small Eclipse RCP application in Tycho. The first steps involves creating the setup of one plugin/feature/product so you who are Eclipse-wise don’t need to follow these steps thoroughly.

Note: This post will assume Tycho 0.7.0 and Eclipse 3.5.2. Some of the issues in this post will probably be fixed in upcoming releases of Tycho. Hopefully the main workflow will remain somewhat steady. I will also only provide examples for the explicit target platform resolver. The implicit p2 resolver should work just as fine, check out the previous post to see how to set it up. I will also not include all configuration options that are available for Tycho, but please comment if you miss something and I might add it later!

Let’s build an application!

Creating the application plug-in

Start by creating a plug-in that defines our RCP-application. There are some templates ready for creating example RCP application in the Eclipse distributions. I used the “Hello RCP” template which contains a simple application saying “Hello RCP” in the title bar. See the screenshots below for how I created my example plugin.

Creating the application feature

Products in Eclipse can either be plugin-based or feature-based.  We will build a feature-based application so we start by creating a project that will contain the main feature for our RCP application. The feature contains a feature.xml, which is used to define which plug-ins constitutes the feature. When asked for plug-ins to include in the feature, reference the plug-in that we created in the first step.

The feature will be referenced by the product definition, which tells Tycho to use it when constituting the product.

Configuring the product definition

We need to create the product definition that constitute our product. This will include the feature project that we created previously and the Eclipse RCP feature. Pay attention to the naming of the product definition file since Tycho uses the convention that the file name should match the maven module with a .product suffix.

Start by creating a new project (I prefer keeping the product file in a separate project).

Name the project so that it matches your product in whatever way you find appropriate. I named my project se.mattiasholmqvist.rcpexample. Tycho now assumes that my product definition file is name se.mattiasholmqvist.rcpexample.product

We create our product configuration file and add it to our product project.

Choose an ID and a name for the product and make sure that you set your product includes native launcher artifacts.

We need to create a new product definition. This definition refers to the application extension that our application plug-in defines (this comes from the RCP template).

You need to specify which features constitutes the product. Since we build an Eclipse RCP product you need to include the RCP feature (org.eclipse.rcp) and the feature that we created earlier which contains our application-defining plug-in. Make sure that the definition specifies that the product is build using features.

Setting up the build files

From our workspace location, we can now generate a parent pom.xml together with a pom-file for the plug-in project we just created. Fire up your command line and generate the pom-files.

mvn3 org.sonatype.tycho:maven-tycho-plugin:generate-poms -DgroupId=se.mattiasholmqvist -Dtycho.targetPlatform=/Users/mattias/dev/eclipse-3.5.2-delta/

This goal is typically run only once since you don’t want to replace any changes you’ve made to the parent pom. I use only the parent pom to add plug-ins to the build. I’m not sure if that’s the way to go but it seems like a good idea that the individual plug-in projects has minimum-sized pom files that we don’t need to change that often.

You don’t actually need to run <code>generate-poms</code>. This maven goal is only used to bootstrap the build by creating the initial pom files so you don’t need to create these files yourself initially. If you have a parent project with many plug-in projects this might be quite useful. If you want more control over what’s happening, just copy & paste the pom.xml files I created (shown below) and modify them to your need.

This renders a parent pom.xml that looks something like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <modelVersion>4.0.0</modelVersion>
  <groupId>se.mattiasholmqvist</groupId>
  <artifactId>rcpexample-parent</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>pom</packaging>
  <build>
    <plugins>
      <plugin>
        <groupId>org.sonatype.tycho</groupId>
        <artifactId>tycho-maven-plugin</artifactId>
        <version>0.7.0</version>
        <extensions>true</extensions>
      </plugin>
    </plugins>
  </build>
  <modules>
    <module>se.mattiasholmqvist.rcpexample.feature</module>
    <module>se.mattiasholmqvist.rcpexample.plugin</module>
  </modules>
</project>

As we can see this does not contain a module for the product project that we created earlier. Tycho cannot understand that our product project defines a product, but that’s a minor thing. We should add the product project to the parent pom.xml so that the modules section looks like this:

<modules>
  <module>se.mattiasholmqvist.rcpexample.feature</module>
  <module>se.mattiasholmqvist.rcpexample.plugin</module>
  <module>se.mattiasholmqvist.rcpexample</module>
</modules>

We also need to create a pom.xml in the product project that looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<project>
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>se.mattiasholmqvist</groupId>
    <artifactId>rcpexample-parent</artifactId>
    <version>0.0.1-SNAPSHOT</version>
  </parent>
  <groupId>se.mattiasholmqvist</groupId>
  <artifactId>se.mattiasholmqvist.rcpexample</artifactId>
  <version>1.0.0</version>
  <packaging>eclipse-application</packaging>
</project>

Note the eclipse-application packaging in the pom. This tells Tycho that the project is actually an RCP application and Tycho starts looking for a product definition file during the build.

Installing the delta pack in the target platform

To build applications that depend on platform-specific parts of the Eclipse SDK (such as SWT), we need to provide a target platform that contains the Eclipse Delta pack. There are multiple sources on the web that describes how to install the delta pack so look at for example this site or this one.

Building the product

We are now ready to build our product.

mvn3 package -Dtycho.targetPlatform=/Users/mattias/dev/eclipse-3.5.2-delta/

This should build your product and place an executable application under /target directory in your product project. There are however currently some issues that you might run into depending on your build environment/target platform.

The import org.eclipse.swt.graphics.Point cannot be resolved.

You might run into something like this:

[ERROR] Failed to execute goal org.sonatype.tycho:maven-osgi-compiler-plugin:0.7.0:compile (default-compile) on project se.mattiasholmqvist.rcpexample.plugin: Compilation failure: Compilation failure:
/Users/mattias/dev/projects/tychoexamples/clean/se.mattiasholmqvist.rcpexample.plugin/src/se/mattiasholmqvist/rcpexample/plugin/ApplicationWorkbenchWindowAdvisor.java (at line 1):[-1,-1]
	package se.mattiasholmqvist.rcpexample.plugin;
	^
The type org.eclipse.swt.graphics.Point cannot be resolved. It is indirectly referenced from required .class files

/Users/mattias/dev/projects/tychoexamples/clean/se.mattiasholmqvist.rcpexample.plugin/src/se/mattiasholmqvist/rcpexample/plugin/ApplicationWorkbenchWindowAdvisor.java (at line 3):[-1,-1]
	import org.eclipse.swt.graphics.Point;
	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...
...

This is a problem with Tycho not knowing which native environment to build for. Tycho needs this information to find the correct SWT fragment in the target platform. The expected default behaviour would be that Tycho looks for a fragment that is supported by the system running the build but currently there are some issues with MacOS X (which I use). However, this issue will be fixed in release 0.8.0 of Tycho.

So, we need to provide this information to Tycho somehow. In my case, I want to build for MacOS X, Cocoa windowing system and the x86 architecture. We have three options (that I know). We can provide these properties on the command line, in the Maven settings.xml or in the pom.xml directly.

On the command line it looks like this:

mvn3 package -Dtycho.targetPlatform=/Users/mattias/dev/eclipse-3.5.2-delta/ -Dosgi.os=macosx -Dosgi.ws=cocoa -Dosgi.arch=x86

In the pom.xml (or the settings.xml for that matter):

<properties>
  <osgi.os>macosx</osgi.os>
  <osgi.ws>cocoa</osgi.ws>
  <osgi.arch>x86</osgi.arch>
</properties>

Missing target environment
This might also be a problem:
[ERROR] Failed to execute goal org.sonatype.tycho:maven-osgi-packaging-plugin:0.7.0:product-export (default-product-export) on project se.mattiasholmqvist.rcpexample: Product includes native launcher but no target environment was specified -> [Help 1]

Which is due to the fact that the Tycho packaging plugin needs configuration to know which target environment to build the product for. We can solve this by adding the following to the plugins section of the parent pom.xml:

<plugin>
  <groupId>org.sonatype.tycho</groupId>
  <artifactId>maven-osgi-packaging-plugin</artifactId>
  <version>0.7.0</version>
  <configuration>
    <environments>
      <environment>
        <os>macosx</os>
        <ws>cocoa</ws>
        <arch>x86</arch>
      </environment>
    </environments>
  </configuration>
</plugin>

We’re done, let’s start the application!

Open the target directory of your product project (in my case se.mattiasholmqvist.rcpexample/target) and you should find an executable RCP application. In my case (since I’m on a Mac) I found a rcpexample.app in my target directory. Behind this file is this beautifully built, and extremely useful application:

That’s it for building a simple Eclipse RCP app with Tycho. Good luck!

As a reminder, please check out the Tycho users mailing list. Help the people at Sonatype out with feedback on their project.

The Tycho project provides plugins to Maven which will make it possible (and rather simple) to build OSGi bundles with Maven . There is currently a project overview which provides a good starting point (although some of the documentation is quite outdated). The best source for help is the Tycho users mailing list.

This post will be the first in a series of posts covering building OSGi bundles/Eclipse RCP applications with Tycho.

It is important to remember that Tycho relies on new features of Maven which is only currently available in the 3.0 version of Maven which has not yet been released, but it is possible to try this out by downloading an alpha version of Maven 3.0.

Environment setup

I downloaded the alpha-6 from the Maven download site. I also set up my environment to use mvn3 instead of mvn for Maven 3 since I didn’t want to break my Maven 2 installation.

Since Tycho has been pushed to Maven central since version its last couple of revisions (0.5.0 i think), we don’t need to download anything else or setup any additional Maven repositories.

Building OSGi bundles

Fire up Eclipse (I use 3.5.1) and create a new project. Choose plug-in project and choose the target to be Equinox runtime. This creates a simple bundle without any plugin.xml or other Eclipse-specific dependencies.

I also chose to create an Activator to have some code to compile.

I recommend that you keep your source code in a different directory from the workspace. That’s my personal preference but it keeps Eclipse IDE from cluttering the source directory too much.

Open up your command line and go to the directory that contains your bundle project. In my case it looks like this:

mvn3 org.sonatype.tycho:maven-tycho-plugin:generate-poms -DgroupId=se.mattiasholmqvist -Dtycho.targetPlatform=/Users/mattias/dev/eclipse-3.5.1-delta/

Tycho is downloaded and it generates a pom.xml for the current directory (parent project) and another pom.xml for the bundle project. It should look something like:

So, Tycho has generated two poms for us. Let’s get ready to build our OSGi bundle! Try:

mvn3 clean install

and watch the magic!?

What’s the problem here? Maven tells us that it cannot resolve the project due to a missing OSGi dependency to org.osgi.framework. It considers missing OSGi dependencies as build errors. This means that we must set up a proper target platform for Tycho, and tell it to use that target platform when building. This is required for any proper build environment that builds OSGi bundles.

Currently (version 0.7.0) Tycho supports two different ways of specifying a target platform.

Building with an explicit target platform

We can expliticly tell Tycho to use an Eclipse installation using:

-Dtycho.targetPlatform=/path/to/target

I downloaded the Eclipse SDK (3.5.1) and untarred it into /Users/mattias/dev/eclipse-3.5.1/. I can now build my simple bundle from the command line with:

mvn3 clean install -Dtycho.targetPlatform=/Users/mattias/dev/eclipse-3.5.1

Using an implicit target platform

If you don’t want to keep track of the target platform locally, it is possible to build the OSGi bundle with an implicit target platform. To do this, we need to help Tycho along by specifying that it should try to resolve the dependencies using a resolver. We also need to help Tycho by specifying locations (repositories) where it might find artifacts that matches our needs.

We’ll start by telling Tycho that it should use a p2 resolver (we’ll use Equinox from Eclipse.org as our target platform). We’ll add the following to the build section pom.xml for our parent project:

 <plugin>
  <groupid>org.sonatype.tycho</groupid>
  <artifactid>target-platform-configuration</artifactid>
  <version>${tycho-version}</version>
  <configuration>
    <resolver>p2</resolver>
  </configuration>
</plugin>

This tells Tycho to use an implicit resolver that supports p2 repositories. We are now one step from being able to build with an implicit target platform.

The ${tycho.version} is a Maven property that I use to switch between versions of Tycho. This is set to 0.7.0 in my case. If you want to use just this version, replace the property with the version literals. The version of this plugin should be the same as for the tycho-maven-plugin entry that was auto-generated to the pom.xml from the generate-poms target used earlier.

We need to setup the repositories that Tycho should use at build time. This is accomplished by normal repository configuration Maven-style. Tycho enhances these configuration options by supporting p2 repositories as well. Open your settings configuration (in my case ~/.m2/settings.xml) and add the following:

<settings>
  <profiles>
    <profile>
      <id>tycho</id>
      <repositories>
        <repository>
		  <id>galileo</id>
		  <layout>p2</layout>
		  <url>http://download.eclipse.org/releases/galileo/</url>
        </repository>
      </repositories>
    </profile>
  </profiles>
</settings>

It is now possible to build the simple bundle project by using:

mvn3 clean install

That’s it for now. Hope this helps with your headless OSGi builds! Keep your eyes open for the next part in this series of posts.

Some interesting links:

• Tycho Users mailing list
• Getting started with p2
• Sonatype’s Tycho GitHub repository

There is a method for numeric integration called Simpson’s rule which can be used to approximate the integral of a given function in some interval . I decided to try some Clojure on this.

The rule states that this approximation can be found by calculating:

where and .

First off, we can make simplify the problem a bit. We can break out the first and last element in the series (by calculating for the 0′th and n’th index) and also realize that the odd and even elements in the series have common factors. This yields:

If we break this down into separate problems, we realize that we first need to find the indices for the first and second series, respectively. Once we have found the indices, we can apply the function to the value of for each index.

Finding the indices turns out to be quite simple. We can generate a complete sequence (1-n) and then filter out all interesting values and return that sequence. The range function generates a sequence and the filter function filters the sequence and returns all values that satisfies a supplied predicate function. Luckily for us, Clojure provides the predicate functions odd? and even?, which answers to whether or not numbers are odd or even. As an example from the REPL:

(filter odd? (range 1 10))
(1 3 5 7 9)

We can use this method in the function sub-series which will be responsible for generating our sums:

(defn sub-series [n index-filter]
  (map #(f (+ a (* h %))) (filter index-filter (range 1 n))))

The sub-series function takes as arguments the function for which we’re approximating the integral, the start of the integration interval , the granularity of the integral , the value as defined in the problem statement and the predicate function which is used to determine which indices to include in the series.

This function will apply the function for each value in the series (from 1-n) that is satisfied by the given predicate function. Once the function has been applied to all those values, it will return the calculated values as a sequence.

(defn sub-series [f a h n index-filter]
  (map #(f (+ a (* h %))) (filter index-filter (range 1 n))))

I’m using a few Clojure API functions (map and filter) but I will not explain these further here. For more info about these functions, see the Clojure API.

Since the first series we want to generate only include even indexes, we pass the even? predicate function to our sub-series function. For the second series we pass the odd? predicate function. We also multiply all values in each sequence with the proper factor (2 and 4, respectively). We use reduce to add together all values in the generated sequences. As a final step we also add and and multiply the whole sum with :

(defn simpson [f a b n]
  (def h (/ (- b a ) n))
  (* (/ h 3) (+
    (reduce + (map #(* % 2) (sub-series f a h n even?)))
    (reduce + (map #(* % 4) (sub-series f a h n odd?)))
    (f a)
    (f b))))

Phew. Let’s try this out. First let’s try it on a simple function such as . We can use the identity function from the Clojure API for this:

(simpson identity 0.0 1.0 100)
0.5

So far so good…

Second, we’ll try a more sophisticated polynomial function, . We’ll define the polynomial:

(defn poly [x]
  (- (+ (* 7 x) (* 3 (cube x))) (* 8.5 (square x))))

And approximate the integral in :

(simpson poly 0.0 2.0 100)
3.3333333333333335

which is correct.

To clean the whole thing up I’ll letting the sub-series function be a local function. This yields the final version of our Simpson-approximation:

(defn simpson [f a b n]
  (let [h (/ (- b a ) n)]
  (letfn [
      (sub-series [index-filter]
        (map #(f (+ a (* h %))) (filter index-filter (range 1 n))))]
    (* (/ h 3) (+
    (reduce + (map #(* % 2) (sub-series even?)))
    (reduce + (map #(* % 4) (sub-series odd?)))
    (f a)
    (f b))))))

I think the end result of 10 LOC is quite neat. The rich API with filter, map, reduce etc is really fun to work with.

If you ever need to make an integral calculation, you can just paste this code and you have your area!

Simplified solution

As pointed out by Chris B, we don’t need to use map in this case, since we only multiply the sub-series values with constants (2 and 4, respectively) we can simply multiply the reduced sub-series with these values. If the sub-series values should be multiplied with a function that depends on the current value in the sub-series, map would be a good approach. This simplifies the solution a bit:

(defn simpson [f a b n]
  (let [h (/ (- b a ) n)]
  (letfn [
      (sub-series [index-filter]
        (map #(f (+ a (* h %))) (filter index-filter (range 1 n))))]
    (* (/ h 3) (+
    (* 2 (reduce + (sub-series even?)))
    (* 4 (reduce + (sub-series odd?)))
    (f a)
    (f b))))))

Clojure seemed like a nice fit since it is a Lisp and also runs on the JVM which makes the learning curve slightly less steep (for me atleast).

Square root calculation using Newton’s method

One of the examples in the book covers Newton’s method for calculating the square root of a number. The idea is that if we pick a guess for the square root, we’ll get a better guess by dividing our initial guess with the square rooted number. That is, if is a guess for the square root and is the number for which we’re calculating the square root, we can find:

Averaging this fraction with the guess helps with the convergence of the algorithm and is needed for Newton’s method to converge. The theory behind this is beyond me, but simple calculations can show that this average is actually the same as the original fraction and only affects the algorithm convergence behavior. This method can then be applied iteratively by using the answer as input to the next iteration. So we’ll use this instead:

Enough mumbling about stuff I don’t know that much about and let’s write some code!

I first define the square, average and abs help functions.

(defn square [x]
  (* x x))
(defn average [x y]
  (/ (+ x y) 2))
(defn abs [x]
  (if (< x 0)
    (- x)
    x))

We also need a stop criteria to be able to tell when we’ve found a good enough guess. For that, we’ll use our good-enough? predicate function. This function checks that the difference between the squared guess and the value we’re computing the square, is within our tolerance level. This will mean that we have found an acceptably good guess.

(defn good-enough? [guess x tolerance]
  (< (abs (- (square guess) x))
     tolerance))

Now, we can use a help function to iterate to a good enough guess. The next-guess function is responsible for calculating our next guess, given the previous guess and squared number as arguments. sqrt-iter is an recursive function that stops when the stop criteria (good-enough?) is satisfied. The sqrt function initiates the calculations and provides the recursive sqrt-iter with an initial guess.

(defn next-guess [guess x] (average guess (/ x guess)))

(defn sqrt-iter [guess x tolerance]
  (if (good-enough? guess x tolerance) guess
    (sqrt-iter (next-guess guess x) x tolerance)))

(defn sqrt [x initial-guess tolerance]
  (sqrt-iter initial-guess x tolerance))

We can now make approximate calculations of the square root of a number for a specific tolerance. For example, the square root of 2 with the tolerance 0.0001 can be calculated with:

(sqrt 2.0 1.0 0.0001)
=> 1.4142156862745097

I’m using 1.0 as the initial guess which I thought was reasonably good. This is all nice and dandy, but it doesn’t really show any huge advantages from using a non-functional programming language such as Java. Implementing this solution in Java is left as an exercise.

Refactoring to fixed-point

The power of functional programming and Clojure comes into play when we can find a generic pattern for many problems and use functions as first-order citizens. In this case, it turns out that the Newton approximation is a special case of a fixed-point calculation. We can refactor the solution by breaking out the responsibility of finding a fixed-point for a function into a separate function:

(defn fixed-point [func first-guess tolerance]
  (letfn [
    (good-enough? [first second tolerance]
      (< (abs (- first second)) tolerance))
    (eval-guess [guess]
      (let [next-guess (func guess)]
        (if (good-enough? guess next-guess tolerance)
          guess
          (eval-guess next-guess))))]
  (eval-guess first-guess)))

The fixed-point function takes a function as its first argument. The fixed-point function will evaluate the given function with the current guess. The result of this evaluation is used to determine if the function has converged to within the tolerance level. If so, the current guess will be returned, else the fixed-point will continue with the new guess as input. This means that we must provide a function that converge under these conditions (fixed-point). As we've seen, Newton's method includes such a function for calculating the square root of a number (the shown earlier). The fixed-point function also has a local function good-enough? that checks if the current guess is within our tolerance level (same as good-enough? in the previous solution).

I've used some more Clojure API support here, such as letfn and let, which let's me define local functions and variables. It makes the code a bit more concise and we don't get access to the local functions from outside the fixed-point function. Check out the Clojure API for more details on this.

We can now wrap up the refactoring with the following small function that uses the generic fixed-point to calculate the square root:

(defn sqrt [x tolerance]
  (fixed-point #(average % (/ x %)) 1.0 tolerance))

Here I'm using the # symbol, which is a short-hand notation for an anonymous function. The % within the function represents the first argument to the function. This means I'm actually passing an implementation of to the fixed-point function.

To clean this up a little bit more I'm breaking out the responsibility of average damping in a separate function:

(defn average-damp [func]
  #(average % (func %)))

The average-damp function applies the average damping to the provided function and returns a new function that is calculated with applied damping. This means that we can write our square root calculation function as:

(defn sqrt [x tolerance]
  (fixed-point (average-damp #(/ x %)) 1.0 tolerance))

We also get another payoff. We can now reuse the fixed-point method for approximating fixed points of other functions as well. For example, the golden ratio is a value that satisfies:

To calculate this using our fixed-point function we need to reduce it to the form:

.

We can pass this reduced form to our fixed-point function:

(defn golden [tolerance]
  (fixed-point (average-damp #(+ 1 (/ 1 %))) 1.0 tolerance))

To approximate the golden ratio, we can now call our golden method and pass a reasonable tolerance level (0.0001 in this case):

(golden 0.0001)
=> 1.6179380934832117

It turns out that this algorithm actually does not need the average-damping to converge. Since we have separated the function for finding the fixed-point from that of average-damping we can try:

(defn golden [tolerance]
  (fixed-point #(+ 1 (/ 1 %)) 1.0 tolerance))

Which yields the same results as the average-damped version:

(golden 0.0001)
=> 1.6179380934832117

Hopefully this post shows some examples on how you can use higher-order functions to modularize responsibilities. The original examples (in Scheme) in this post are available in the book and credit for this should go to Abelson, Sussman and Sussman. The book is also released under Creative Commons Unported licence 3.0.