Scaling JavaScript Apps – Part III: Ant Build Process

A multi-part look at how modern JavaScript development techniques and best practices are driving the Rich Internet Applications of tomorrow. Project structure, writing specifications, build processes, automated testing, templating and applying “separation of concerns” to the world of JavaScript are all to be covered. The IDE used will be Eclipse.

The build process extracts your most mundane and repetitive tasks from an iterative development loop and bundles them into one neat little script to be used and abused as often as you see fit. The use of such a step in JavaScript development hasn’t caught on terribly well just yet, but with increasing complexity in architecture and an ever expanding list of helpful utilities, it won’t take long for it to become a staple of all web app production.

There are a number of tools which will run your script – make, cake, rake, _ake are a few I’ve come across, but for ease of integration with Eclipse, the builder we’ll focus on is Apache Ant. Help yourself to the user manual to garner details on the tasks at your disposal, and how you can malleate them.

It all begins with build.xml.

<?xml version="1.0"?>
<project name="tux" default="build" basedir="../">

	<target name="build">
		<antcall target="lint-src" />
		<antcall target="lint-test" />
		<antcall target="test" />
		<antcall target="build-modules" />
	</target>

	<!-- ... -->

</project>

The structure above, which we’ll flesh out in a bit, gives a high-level look at what our build process will do. The top element — project — defines the name of the build process, and the default target to execute (if none is given). Think of a target as a single type of task or function, such as concatenation or compression, and configurable through the use of attributes and nested elements. Nested elements can even call other ant targets referenced by name, as witnessed. Given that I’ve stored all build-related files in a build directory hanging from the root folder, adding basedir="../" to the project tag will ensure all future path references will be relative to the root.

Lint

The first thing we should do is lint both the source, and test spec files. Verifying your code is clean and syntactically correct will be essential to running tasks further on in the process. Whether to use JSLint or JSHint is a matter of personal preference, but keep in mind there may come a time when you need to bend the rules in your favour and you’ll find JSLint much less friendly in this respect.

We’ll use Mozilla’s Rhino JavaScript environment to do this. Add the latest versions of rhino.jar, jshint.js and the adaptor jshint-rhino.js to your build directory before updating the XML. Script variables (“properties” in the Ant vocabulary) should be placed at the top of the file, or even in a separate file for improved maintenance. Add the option flags and predefined variables like so…

<project name="tux" default="build" basedir="../">

	<property name="jshint.flags" value="browser=true,maxerr=25,undef=true,curly=true,debug=true,eqeqeq=true,immed=true,newcap=true,onevar=true,plusplus=true,strict=true" />
	<property name="jshint.predef" value="console,$,namespace,noop,tux,Backbone,Store,_,format,parse" />
	<property name="jshint.predef.test" value="${jshint.predef},describe,xdescribe,xit,it,beforeEach,afterEach,expect,sinon,jasmine,loadFixtures,setFixtures,loadTemplate,fillForm" />

	<!-- ... -->

These are the parameters which will be passed to Rhino, the JSHint-Rhino adaptor will receive and parse before sending to JSHint. Confused? Open up the adaptor source code and you’ll find it’s nowhere near as daunting as it sounds. Notice that the jshint.predef property is extended by jshint.predef.test – due to additional global variables being made available via the testing libraries. These are functions and objects which would not normally be available to the production code. Right, here is how the linting has been carried out.

	<!-- lint source -->
	<target name="lint-src">
		<antcall target="lint">
			<param name="dir" value="src" />
			<param name="predef" value="${jshint.predef}" />
		</antcall>
	</target>

	<!-- lint tests -->
	<target name="lint-tests">
		<antcall target="lint">
			<param name="dir" value="specs" />
			<param name="predef" value="${jshint.predef.test}" />
		</antcall>
	</target>

	<!-- lint -->
	<target name="lint">
		<apply dir="build" executable="java">
			<fileset dir="${dir}" includes="**/*.js" />
			<arg line="-jar rhino.jar jshint-rhino.js" />
			<srcfile />
			<arg value="${jshint.flags}" />
			<arg value="${predef}" />
		</apply>
		<echo>${dir} JSHint Passed</echo>
	</target>

The two lint subjects (src and specs) differ only in the subject directory name, and predefined variables. The similarities have been abstracted into a target, which takes these two parameters, before running JSHint and the script file through the Rhino engine. Notice the use of the target parameters and properties defined earlier through the ${variable.name} syntax.

Test

The build process is run many a time during development, so we’ll safely assume that JS Test Driver previously explained is already running. With that in mind, all you need to do is define the following task to run your test suite in all captured browsers:

	<!-- run unit tests -->
	<target name="test">
		<java failonerror="true" dir="build" jar="build/JsTestDriver-1.3.2.jar" fork="true">
			<arg line="--reset --tests all --basePath ${basedir}" />
		</java>
		<echo>Jasmine Specs Passed</echo>
	</target>

All we’re doing here is running the JS Test Driver Java Archive and setting the execution context to /build, where it will find the jsTestDriver.conf listing all files to be loaded and in what order. Note – you can glob all files inside a directory, but not recursively.

server: http://localhost:9876

load:
  - lib/jasmine.js
  - lib/JasmineAdapter.js
  - lib/underscore.js
  - lib/jquery-1.6.1.js
  - lib/backbone.js
  - lib/*.js

  - src/util.js
  - src/core/*.js
  - src/accounts/*.js
  - src/tags/*.js
  - src/ledger/*.js
  - src/forms/*.js
  - src/schedule/*.js
  - src/reports/*.js

  - specs/specs-helper.js
  - specs/util.spec.js
  - specs/core/*.js
  - specs/accounts/*.js
  - specs/tags/*.js
  - specs/ledger/*.js
  - specs/forms/*.js
  - specs/schedule/*.js
  - specs/reports/*.js

During this process, you should see the output from the test suite with a ‘.’ to mark a passed test, and an ‘F’ for those that failed. Lastly, when the test suite has completed, a summary of pass/failures will appear. You can set the build process to fail and halt completely with failonerror="true". Otherwise, the build process is free to carry on to the next task.

Concatenate & Minify

Now that you’re coding like a boss, you’ll have developed the habit of breaking your source files into tiny, distinct units of functionality. On the flipside, you’ll immediately notice the pain of having to stitch each of these scripts into your page individually. How you structure your project is up to you, but when concatenating these script files you should aim to produce a single file for each top-level module. Here’s something I prepared earlier:

	<!-- build each module -->
	<target name="build-modules">
		<copy file="src/util.js" tofile="scripts/util.js" />
		<subant target="build-module" genericantfile="build/build.xml">
			<dirset dir="src" includes="*" />
		</subant>
		<echo>All modules built</echo>
	</target>
	
	<target name="build-module">
		<basename file="${basedir}" property="module" />
		<property name="modulefile" value="../../scripts/${module}.js" />
		
		<!-- concat src js -->
		<concat destfile="${modulefile}">
			<fileset dir="." includes="*.js" />
		</concat>

		<!-- build compressed version -->
		<java jar="../../build/compiler.jar" fork="true" dir="../../scripts">
			<arg line="--js ${module}.js --js_output_file ${module}.min.js" />
		</java>
		
		<echo>${module} module build successful</echo>
	</target>

Minification above is tasked to Google’s Closure Compiler. After this final task in the build process, the file will be readily available to include in your page, a la:

<script type="text/javascript" src="/scripts/module.min.js"></script>

Clean, no? Lose the .min while developing to use the pre-minification, debug-friendly code.

Automate

Most Eclipse packages come with support for Ant build files. If not, install the Eclipse Java EE Developer Tools. With your build process defined, you’ll want easy access to each of the targets from within the IDE. Right-click on your build file in the Project Explorer, and select Run As > Ant Build…. This should invoke a new configuration window, allowing you to save the build task. Tick and possibly reorder the targets you’d like to kick off, then hit save. Rinse, repeat, and voilà!

The final piece of Eclipse integration involves dosing the iterative TDD cycle with steroids, by having the test suite run whenever you save a new test case or update the source code. Hit up Project > Properties > Builders and Import… your previously defined “Run Tests” task. Edit the newly created builder and head to the Targets tab. Add the test target to the Auto Build list, if not already present, and save. Now whenever the project is updated, your test suite will automatically be executed for immediate TDD feedback. This can be easily toggled via Project > Build Automatically.

In the next and final part of the series, we’ll be looking at how to clean up script files by providing a dedicated file structure for template markup.

Scaling JavaScript Apps – Part I: File Structure

A multi-part look at how modern JavaScript development techniques and best practices are driving the Rich Internet Applications of tomorrow. Project structure, writing specifications, build processes, automated testing, templating and applying “separation of concerns” to the world of JavaScript are all to be covered. The IDE used will be Eclipse.

File structure.

Making folders, nesting directories, arranging files.

It’s certainly not the blockbuster blog opener I had intended. Household chores elicit higher levels of adrenaline. Yet — as a method of collecting thoughts and giving each piece of work meaning — getting it right can have far greater benefits than most realise. A well designed structure can deliver you right to where you need to be in an instant, while those poorly designed will have you spending hours each week in an infernal expand-collapse nightmare of your own making.

Where do we begin?

Folders stemming from the root are usually the first port of call. They provide an immediate overview into how both the app and the development process itself are deconstructed. Obviously this will vary based on the purpose, context and individual needs of the project, but the vanilla-flavoured tree remains essentially the same. With this in mind, let’s dive into the layout of the example JavaScript project we’ll be building upon through the course of this series.

  • /lib – external libraries, such as Underscore, Backbone and jQuery
  • /src – source files and templates – more details further on
  • /specs – specifications for automagically testing your source
  • /build – all tools and scripts required to lint, test and compile your project
  • /scripts – the end game – compiled JavaScript files for production use

Whether other types of resources such as stylesheets and images deserve their own root folder, or whether they are bundled together with JavaScript in the src directory is up to you. After learning more about the build process which brings the entire show together you should have a clear idea of what approach is better tailored for your team.

Once libraries have been selected and the build process has been nailed, all of your sweet time will be spent navigating through the src and specs. This is where you should look at the app as a whole and begin drawing clearly defined lines around each component. The goal here is to conjure up a list of these modules, and sub-modules where necessary, to use when populating these directories. Aim to keep the number at any level of nesting below a comfortable maximum – if you’ve grouped more than ten folders together, try to plant them under another level of grouping. It’s always beneficial to keep in mind how you think the application will grow, so as to avoid excess file renaming and updates of path references later on.

Only leaf folder nodes will contain files, a la namespacing. You should always avoid placing files in a folder that already contains child folders – a folder should be dedicated to either grouping sub-folders, or holding the source files that make up a single building block, not both. The exception to this rule are folders that group resource types, such as JavaScript templates, stylesheets and images.

The specs folder should simply mirror the src – wherever you find a specifications file, you should be able to find the source file it tests sitting in an identical path within the source structure.

I’ll take this opportunity to give an example while shamelessly plugging tux, an open-source personal finance app I’m currently building. Beneath the source folder I’ve created a folder for each of the functions served – accounts, ledger, schedule, budget, reports, goals, and so forth. These parts are all underpinned by an additional core module and all split the templates out into a jst resource folder.

One potentially helpful feature in Eclipse when dealing with large file structures is Working Sets. Each defined set can group related resources, and when activated will display only those resources in the Project Explorer panel. This is a fantastic way of de-cluttering the explorer for when you know the work to be undertaken relates to just one or a few parts of the app.

So that’s enough of that. In the next part, we’ll take a look at writing the specifications that define each unit before any production code is written, and we’ll begin to look at how we can use a build process to neatly run these tests without leaving the editor.