On Monday 23rd June, I was privileged to attend the first AngularJS Melbourne Meetup as a guest speaker. Thanks to Alex for organising the event. It was a great night!

You can view the slides below (no animation), or download the animated presentation here: Presentation (MS PowerPoint, 1.9MB)

The accompanying demo files:

The Problem

You want to keep your code DRY and/or you want to keep data separate form your code. So you pull the data out of your code into a separate data file (in an XML or JSON format). Sweet! Now you have appropriate syntax highlighting for your code and data, and you have separated your concerns. But how do you use the data from your code?

Well you can either:

  • Load the data at runtime
  • OR, recombine the code and data in a compilation step

There are pros and cons to both approaches, but lets say that the main priority is you want to avoid an extra HTTP request at runtime. That means that combining the files during compilation is the best option.

There are number of GruntJS plugins available which purport to offer the required functionality. Let’s take a look at the main features of 4 plugins:

  1. grunt-bake
  2. grunt-include-replace-cwd
  3. grunt-includes
  4. grunt-include-replace

Key requirements

Include any kind of file

Obviously, the plugin must allow you to include one file from another file:

// index.html:
//@@include('indexTemplate.html')

// indexTemplate.html:
<html><body>
Hello world
</body></html>

This should produce the following index.html file:

<html><body>
Hello world
</body></html>

Some plugins are adept and doing file-includes for only certain types of files, usually HTML. This is fine if you only need HTML file-includes. But an ideal solution shouldn’t care what kind of files you want to include. For example, you should be able to do this:

// routeLoader.js:
...
function getRoutes() {
	var routeTable = [];
	//@@include('routeData.jsonp')
	return routeTable;
}
...

// routeData.jsonp:
routeTable = ['really', 'big', 'list', 'of', 'things', ...];

This should produce the following routeLoader.js file:

...
function getRoutes() {
	var routeTable = [];
	routeTable = ['really', 'big', 'list', 'of', 'things', ...];
	return routeTable;
}
...

Templates and attribute passing

This is a really important feature which extends-on from the idea of being able to insert one file into another: the ability to insert one-or-more values into one-or-more places in an included file. For example:

// indexDev.html:
//@@include('indexTemplate.html', {environment: "Dev"})

// indexProd.html:
//@@include('indexTemplate.html', {envirionment: "Prod"})

// indexTemplate.html:
<html><body>
Hello world, you're in //@@environment now!
</body></html>

This produces indexDev.html…

<html><body>
Hello world, you're in Dev now!
</body></html>

…and indexProd.html

<html><body>
Hello world, you're in Prod now!
</body></html>

This capability provides a compile-time template engine! Very powerful.

Template and file-include configurable syntax

The ideal file-include plugin would allow you to set the syntax to use when defining the file-include command and when defining what a template-value looks like. Naïve implementations often use a syntax that conflicts with run-time template engines, or server-side template engines.

For example, if you use AngularJS, it uses {{}} notation for binding to data from within HTML fragments:

<html><body>
Hello world, you're in {{environment}} now!
</body></html>

If the file-include plugin uses the same syntax, you’re in trouble because the plugin will replace all the {{}} templates it finds before Angular gets to run! Most of the newer plugins allow you to configure the syntax for both templates and file-include commands:

includereplace: {
  options: {
    prefix: '//@@', // This works for HTML and JS replacements
    suffix: ''
  }
}

Support for Grunt 0.4+ file globbing conventions

Depending on when the plugin was written, the plugin will support different syntaxes for file specifications. You don’t want to have to specify every single file that has an include in it (unless you have a really-small project… size DOES matter).

For example, the following sub-tasks should be supported by your plugin:

include-plugin: {
  // Specify a list of JS files as inputs, and output them into the output/js directory
  js: {
    files: [
      { expand: true, cwd: '<%= env.environment.outputDir %>/js', src: ['*.js'], dest: '<%= env.environment.outputDir %>/js/' },
    ]
  },
  
  // Find all JS files in the test directory, at any depth, and output them to the output/test directory
  test: {
    files: [
      { expand: true, cwd: '<%= env.environment.outputDir %>/test', src: ['**/*.js'], dest: '<%= env.environment.outputDir %>/test/' }
    ]
  },
  
  // Find all html files in the output directory, except for ones ending in '...Template.html' and output them to the output directory
  // Note that we are over-writing the original files, and that's ok!
  web: {
    files: [
      { expand: true, cwd: '<%= env.environment.outputDir %>', src: ['*.html', '!*Template.html'], dest: '<%= env.environment.outputDir %>/' }
    ]
  }
}

Note that in all cases, none of the matching source files are expected to be concatenated into a single file by the plugin.

Other features

Some plugins support conditional logic in their templating, which can be useful if you are heavily using the templating features. In my case, I don’t need this functionality.

Plugins

grunt-bake

When I was first looking for a file-include plugin, grunt-bake was the first plugin I could get working. After using it a bit more and finding that it didn’t support specifying a directory as a destination, I created a local patch to do it. This patch had to be installed before any other Grunt tasks were run, so it became a glaring sign that something wasn’t right. In the end, the need for the patch file drove me to explore other file-include plugins further.

Features:

  • File include capability
  • Inline attributes for included files
  • Attributes via Gruntfile.js
  • Configurable template replacement pattern
  • Conditional logic (if, for)

Missing Features:

  • Specifying a directory as a destination for processed files
  • Configurable file-include syntax pattern

grunt-include-replace-cwd

I didn’t investigate grunt-include-replace-cwd in much depth due to it’s age, lack of examples and the sense that it was not being maintained.

Features:

  • File include capability
  • Configurable template replacement pattern
  • Configurable file-include syntax pattern
  • Specifying a directory as a destination for processed files
  • Inline attributes for included files
  • Attributes via Gruntfile.js

Missing Features:

  • Conditional logic (if, for)

grunt-includes

grunt-includes supports basic file-include features, but does not support attribute-passing or templates.

Features:

  • File include capability
  • Configurable template replacement pattern
  • Configurable file-include syntax pattern
  • Specifying a directory as a destination for processed files

Missing Features:

  • Inline attributes for included files
  • Attributes via Gruntfile.js
  • Conditional logic (if, for)

grunt-include-replace

include-replace is a great plugin which met my main requirements.

Features:

  • File include capability
  • Inline attributes for included files
  • Attributes via Gruntfile.js
  • Configurable template replacement pattern
  • Specifying a directory as a destination for processed files
  • Configurable file-include syntax pattern

Missing Features:

  • Conditional logic (if, for)

Conclusion

After evaluating these plugins, I decided on include-replace for the following reasons:

  • It supports normal Grunt 0.4+ file source/destination conventions. You don’t want to have to specify a destination file pattern for each source file-pattern, which is what some plugins forced you to do. Alternatively, they interpreted a directory specification as “concatenate all source files into the specified directory”!
  • It supports configurable file and template syntax which means that the plugin won’t interfere with any other template syntax that is in use (such as AngularJS, Handlebars, JSPs). It also allows me to choose a syntax that is ignored by lint-checkers for JS and HTML files.
  • It supports passing attributes to included-files globally (via Gruntfile.js) and inline.

After using it for a month, it performs reliably and does what I need it to.

Using this article as a guide and this one:

Setup

Basically what we will do is commit the source code into a source branch and have the compiled code on the master branch:

# rename current master to something else
git branch -m master master-old
# Create source branch from the old master and check it out
git checkout master-old && git checkout -b source

Publishing Steps

  1. Commit your new content to the source branch!!!
  2. Build your site from the source branch into /web (your output folder). I’m using grunt to do this step.
  3. Checkout the master branch
  4. Remove the existing content files from master
  5. Copy the new site content into the root project directory
  6. Add the new site content to git & push to origin

Here’s a script to do Step 3 onwards, which I call from grunt after doing step 1:

git status

# Checkout master branch
git checkout master

# Rename the /web directory to .web
mv web .web
mv node_modules .node_modules
 
# Remove all non . files
rm -rf *

# Copy everything from .web to current dir
rsync -a .web/ ./
 
# Remove .web 
rm -rf .web

# Add everything
git add .

# Commit using the same commit message as from the 'source' branch
git commit -C source
git push origin master

# Remove everything again
rm -rf *

# Re-add node_modules
mv .node_modules node_modules

# checkout source again
git checkout source

# undo any pending changes to any files
git checkout -- .
git status

I have been developing a website for a top-tier client in Melbourne. The site is built using AngularJS (v1.1.5 as of time of writing), SASS, Compass and GruntJs to pull it all together. The development team consisted of myself as lead dev, Mario Skouros, Peter Mescalchin and Steven Yip, with assistance from Michael Black and Vlado Grancaric. Well done guys!

Design

The visual and interface design was by George Metaxas, who is part of the client’s design team.

Under the hood

The site is part application, part CMS-driven website. It was technically challenging to get the client’s CMS to produce the content in a format and structure that AngularJS could consuem, but we found a solution that was workable.

The site is build for modern browsers (Android 2.3+, i-devices, Blackberry, …) and should work with desktop browsers as well (recent versions of Chrome, Safari and Firefox). We elected early on not to support IE because, well, IE is not very standards-compliant. But in our case, there are not many IE-users of the old mobile site, so it wasn’t worth investing in at this point in time.

Engine & Chassis - AngularJS

We used AngularJS 1.1.5 - which added nice animation support and the ability to cancel timeouts. Using Angular was fantastic, but there were difficulties getting the Apache re-write rules to work for when the site was at the root and when it was a sub-directory site. Overall, I’d highly recommend using it for web applications and websites (up to N-hundred pages).

Bolts - GruntJS

GruntJS is a fantastic tool for front-end devs. It allowed us to integrate font-icon-building, image optimization and source-code minimisation into our build process. Combined with Apache’s mod_deflate, we were able to get the home page download from 1.2MB to less than 300kB (75% reduction).

Paint - HTML5 & CSS3

The site design was implemented using SASS & Compass (with HTML). This kept the CSS from growing too big and the variable names allow us to maintain things more easily going forward. The CSS3 animation support in the Compass 0.13.alpha.4 helped us to do some nice CSS animations (like the loading spinner).

This post explains how I got live-reloading to work.

Idea

The basic idea is you use grunt-contrib-connect and [grunt-contrib-watch](https://github.com/gruntjs/grunt-contrib-watch) together to reload your site automagically in your browser(s). So when you change a file that’s being watched, your browser will update. Nice!

Lesson Learnt

  • Update your NPM packages
  • Make sure watch tasks are not overriding or recursing into each other

Problems

Be careful with your watch task. If it is watching lots of files (like **/*) then you may run into this error:

2013-10-30 08:49 node[49991] (CarbonCore.framework) FSEventStreamStart: register_with_server: ERROR: f2d_register_rpc() => (null) (-21)
2013-10-30 08:49 node[49991] (CarbonCore.framework) FSEventStreamStart: register_with_server: ERROR: f2d_register_rpc() => (null) (-21)
2013-10-30 08:49 node[49991] (CarbonCore.framework) FSEventStreamStart: register_with_server: ERROR: f2d_register_rpc() => (null) (-21)
...

The solution I found for this was to be more specific about the kind of files to watch. In my case, I was watching **/*.html, which meant I was watching source HTML files as well as generated HTML files. But also had a watch:dev task which was watching the generated files - including the HTML files!

watch: {
...
	dev: {
		files: ['dev/**/*.html'],
		tasks: ['<%= env.environment.envName %>'],
		options: {
			livereload: true
		}
	}
}

More problems

The next issue I found was that I needed to add the livereload.js file to your HTML files. That is no problem - just add it to a template or include file before the closing </body> tag:

<script src="//localhost:35729/livereload.js"></script>

But I noticed in Chrome that I was getting a 404 error loading livereload.js. So I tried using a different port - no improvment. After some googling, I came across this article which suggested that the version of grunt-contrib-watch might be out-of date.

So, I updated the package.json file to tell NPM to update grunt-contrib-watch to the latest version:

{
  ...
  "devDependencies": {
    ...
    //"grunt-contrib-watch": "~0.3.2",
    "grunt-contrib-watch": "*",  
   ...
  }
}

Then run npm:

sudo npm update --save-dev

This brought the package version to 0.5.3. Now, running the task again, I can see that livereload.js is loaded by Chrome correctly, and the page refreshes whenever I change a HTML or CSS page that is published into the dev folder - yay!

Final GruntFile.js

'use strict';
module.exports = function(grunt) {

	// Load tasks
	require('matchdep').filterDev('grunt-*').forEach(grunt.loadNpmTasks);
	
	grunt.initConfig({
		pkg: grunt.file.readJSON('package.json'),
		environments: {
			dev: {...	},
			prod: {	...	}
		},
		...
		watch: {
			...
			dev: {
				files: ['dev/**/*.html', 'dev/**/*.css'],
				tasks: ['<%= env.environment.envName %>'],
				options: {
					livereload: true
				}
			}
		},
		...
		exec: {
			build: {
				cmd: 'jekyll build'
			}
		},
		
		copy: {
			env: {
				files: [{expand: true, flatten: false, cwd:'_site', src: '**/*', dest: '<%= env.environment.outputDir %>'}]
			}
		},
		
		connect: {
			dev: {
				options: {
					port: 4000,
					base: 'dev'
				}
			},
			prod: {
				options: {
					port: 4000,
					base: 'dist'
				}
			}
		}
	});
	
	
	grunt.registerTask('env', 'Set environment variables for use by other tasks.', function(environment) {
		if (environment === null || environment === undefined) {
			grunt.warn('You must specify the environment as either DEV or PROD.');
			return false;
		}
		// Store the environment definition in 'env.environment'
		var envName = '' + environment.toLowerCase();
		var envDefinition = grunt.config.get('environments.' + envName);
		
		grunt.config.set('env.environment', envDefinition);
		
		var envName = grunt.config.get('env.environment.name');
		
		grunt.log.writeln(envName + ' environment <<<--------------------------------------');
	});
	
	grunt.registerTask('dev', ['env:dev', 'clean:env', 'exec:build', 'compass:build', 'copy:env', 'connect:dev', 'watch']);
};