Using modules and dependency management in JavaScript applications with AMD - require.js

June 2nd, 2013    by sigman    2224
  Tutorials   programming, javascript

requirejs logoOne of the first things I had to learn when starting developing JavaScript applications was how to overcome lack of dependency management and concept of modules and namespaces. Unfortunately there are few JS books touching on these topics. When building large scale application, implementing hundreds of <script> tags becomes really hard to manage, especially when we need to keep them in specific order. Also the more script tags are included in the html files, the more requests a browser needs to do - this may cause a significant impact to the overall user experience - loading times, UI freezes... As JS lately became a language of choice for developing web applications targeting not only desktops but also mobile devices, developers needed to find a solution for that. Instead of waiting for changes to the JS specification (http://wiki.ecmascript.org/doku.php?id=harmony:modules) and for implementing that consequently to the popular browsers, they agreed for a standard that would be used both on server and client side and they named it CommonJS (http://wiki.commonjs.org/wiki/CommonJS). The solution works great with JS on the server side but due to the fact that files are loaded synchronously, it doesn't work well with browsers as this would block user interface during script load process. So the client side solution was modified for asynchronous loading by wrapping modules in files into callback functions. This is known as AMD - Asynchronous Module Definition
(https://github.com/amdjs/amdjs-api/wiki/AMD, http://requirejs.org/docs/whyamd.html). One of the most popular AMD loader solutions is the require.js and in this article we will look how to use it.

Article imported from http://sierakowski.eu/javascript-tips/25-javascript-tips/118-using-modules-and-dependency-management-in-javascript-applications-with-amd-requirejs.html

Single-file car dealer shop application

As the oversimplified sample application we are going to build a small car/bike dealer application. The dealer sells two type of vehicles, cars and bikes. As an owner of the dealer's shop we are able to buy a new item and put in on the stock or we can sell it and earn some money. All transactions are reflected in the bank account, when buying things we need to pay for them if we have enough money, when selling our earnings are deposited.

For demonstration I decided to try to implement the functional inheritance pattern demonstrated in the "JavaScript: The Good Parts" book. We need items that we will be selling in the shop, so I created a base object called 'item' and both 'bike' and 'car' objects are extending it. Next we have the bank object that has a private variable named 'balance' and public methods that allow to deposit and withdraw our money. The initial deposit is set when creating a new bank object. The 'shop' object allows us to buy and sell items by adding or removing items from the stock and deducting or adding money to the bank. Finally at the end of the script file we initiate everything and buy and sell some items. Also to make it more complex, there is a dependency on the underscore.js library as we are going to use here and there.

scripts/app.js:

var item = function (spec) {
    var that = {};
 
    spec.itemType = 'item';
    spec.id = -1;
 
    that.getItemType = function () {
        return spec.itemType;
    };
 
    that.getId = function () {
        return spec.id;
    };
 
    that.setId = function (id) {
        spec.id = id;
    };
 
    that.getColor = function () {
        return spec.color;
    };
 
    that.getMake = function () {
        return spec.make;
    };
 
    that.getSellingPrice = function () {
        return spec.sellingPrice;
    };
 
    that.getPurchasePrice = function () {
        return spec.purchasePrice;
    };
 
    return that;
};
 
var car = function (spec) {
    var that = item(spec);
 
    spec.itemType = 'car';
    spec.mileage = spec.mileage || 'n/a';
    spec.engineType = spec.engineType || 'n/a';
    spec.model = spec.model || 'n/a';
 
    that.getMileage = function () {
        return spec.mileage;
    };
 
    that.getEngineType = function () {
        return spec.engineType;
    };
 
    that.getModel = function () {
        return spec.model;
    };
 
    return that;
};
 
var bike = function (spec) {
    var that = item(spec);
 
    spec.itemType = 'bike';
    spec.type = spec.type || 'n/a';
 
    return that;
};
 
var bank = function (spec) {
    var that = {};
 
    spec.balance = spec.balance || 0;
 
    that.getBalance = function () {
        return spec.balance;
    };
 
    that.withdraw = function (amount) {
        if (spec.balance - amount > 0) {
            spec.balance -= amount;
            console.log('Bank: balance after withdrawal of', 
                                amount, 'is', spec.balance);
            return true;
        }
        console.log('Bank: unsuccesful withdrawal of', amount, ', 
                               current balance is', spec.balance);
        return false;
    };
 
    that.deposit = function (amount) {
        spec.balance += amount;
        console.log('Bank: balance after deposit of', amount, 'is', spec.balance);
    };
 
    return that;
};
 
var shop = function (myBank) {
    var that = {}, stock = [], itemId = -1;
 
    that.getItemsOfType = function (itemType) {
        return _.filter(stock, function (item) {
            return item.getItemType() === itemType;
        });
    };
 
    that.buyItem = function (item) {
        if (myBank.withdraw(item.getPurchasePrice())) {
            item.setId(++itemId);
            stock.push(item);
            console.log('Item added to the stock (id:', item.getId() + ')');
            return item.getId();
        }
 
        console.log('Not enough savings to buy this item');
        return null;
    };
 
    that.sellItem = function (id) {
        var item = _.find(stock, function (item) {
            return item.getId() === id;
        });
 
        if (item) {
            stock.splice(stock.indexOf(item), 1);
            myBank.deposit(item.getSellingPrice());
            console.log('Item with id ', id, ' sold');
            return id;
        }
 
        console.log('Item with id ', id, 'not in stock');
        return null;
    };
 
    return that;
};
 
/* ----------------------- */
 
var myBank = bank({balance: 10000}),
    myShop = shop(myBank),
    itemIds = [];
 
// add something to the shop's stock
itemIds[0] = myShop.buyItem(car({
    sellingPrice    : 2300,
    purchasePrice   : 2000,
    make            : 'VW',
    color           : 'silver',
    mileage         : 100000,
    engineType      : '1.9 diesel',
    model           : 'Passat'
}));
 
itemIds[1] = myShop.buyItem(car({
    sellingPrice    : 8000,
    purchasePrice   : 6500,
    make            : 'Audi',
    color           : 'black',
    mileage         : 55000,
    engineType      : '2.5 petrol',
    model           : 'A4'
}));
 
itemIds[2] = myShop.buyItem(bike({
    sellingPrice    : 800,
    purchasePrice   : 750,
    make            : 'romet',
    color           : 'yellow',
    type            : 'mountain'
}));
 
itemIds[3] = myShop.buyItem(car({
    sellingPrice    : 800,
    purchasePrice   : 700,
    make            : 'Fiat',
    color           : 'red',
    mileage         : 170000,
    engineType      : '1.2 petrol',
    model           : 'Punto'
}));
 
myShop.buyItem(bike({
    sellingPrice    : 500,
    purchasePrice   : 450,
    make            : 'noname',
    color           : 'green',
    type            : 'city'
}));
 
// what is my current balance?
console.log('My current bank account balance: ', myBank.getBalance());
 
// time to sell something and start earning money!
myShop.sellItem(itemIds[1]);
myShop.sellItem(itemIds[2]);
 
// what is my current balance?
console.log('My current bank account balance: ', myBank.getBalance());

 

And index.html:

<script src="scripts/underscore.js"></script>
<script src="scripts/app.js"></script>

 

Here is a screenshot of the executed script.

phase0_browser.png

Breaking modules down into individual script files

The first improvement that we can make is to break the modules into individual js files, so item module would go to the items/item.js, the same for items/car.js, items/bike.js, app/shop.js and app/bank.js. The main.js file has the code running the shop that is taken from the end of the app.js file. We don't need the app.js anymore.

The folder structure now looks like this:

phase1_filestructure.png

We need to make a change to the index.html file:

<script src="scripts/libs/underscore.js"></script>
<script src="scripts/items/item.js"></script>
<script src="scripts/items/car.js"></script>
<script src="scripts/items/bike.js"></script>
<script src="scripts/app/shop.js"></script>
<script src="scripts/app/bank.js"></script>
<script src="scripts/main.js"></script>

 

If we test it again, it should work the same as it was working before with just a single script file.

phase1_browser_ok.png

But what would happen if we changed the order of the script tags and moved main.js one line up above the bank.js?

phase1_browser_notok.png

We broke the dependency order, because the main.js references the bank.js that wasn't loaded yet, the uncaught reference error is thrown. Of course in simple applications these dependencies are quite easy to manage, but imagine applications that consist of tens if not hundreds of different dependencies and modules.

 

Wrapping modules into require.js AMD type modules

With AMD we can avoid all of that. The first thing that we need to do is to wrap our modules in the define function callbacks and the main file using these modules in the require wrapper.

This is the item.js after wrapping in the define function, the content is literary copied and pasted inside of the callback function that is the first and only parameter to the define and the item object is returned from it. This structure is required to let other modules using it.

scripts/items/item.js:

define(function () {
    var item = function (spec) {
        var that = {};
 
        spec.itemType = 'item';
        spec.id = -1;
 
        that.getItemType = function () {
            return spec.itemType;
        };
 
        that.getId = function () {
            return spec.id;
        };
 
        that.setId = function (id) {
            spec.id = id;
        };
 
        that.getColor = function () {
            return spec.color;
        };
 
        that.getMake = function () {
            return spec.make;
        };
 
        that.getSellingPrice = function () {
            return spec.sellingPrice;
        };
 
        that.getPurchasePrice = function () {
            return spec.purchasePrice;
        };
 
        return that;
    };
    return item;
});

 

Because the bike.js and the car.js depend on the item.js, in the define method we additionally add that 'item' dependency. Note that an array with a string representing a name of the dependency is passed as the first parameter to the define method and then again as a parameter to the callback function that now became the second parameter of the define. Here is the bike.js example, the car.js is done in the similar way:

scripts/items/bike.js:

define(['item'], function (item) {
    var bike = function (spec) {
        var that = item(spec);
 
        spec.itemType = 'bike';
        spec.type = spec.type || 'n/a';
 
        return that;
    };
    return bike;
});

 

The bank.js would be wrapped in the same way as the item.js as it doesn't depend on any other module. But the shop.js is the module that has more dependencies as it depends on the car, the bike and the underscore library.

scripts/app/shop.js:

define(['underscore', 'car', 'bike'], function (_, car, bike) {
    var shop = function (myBank) {
        var that = {}, stock = [], itemId = -1;
 
        that.getItemsOfType = function (itemType) {
            return _.filter(stock, function (item) {
                return item.getItemType() === itemType;
            });
        };
 
        that.buyItem = function (item) {
            if (myBank.withdraw(item.getPurchasePrice())) {
                item.setId(++itemId);
                stock.push(item);
                console.log('Item added to the stock (id:', item.getId() + ')');
                return item.getId();
            }
 
            console.log('Not enough savings to buy this item');
            return null;
        };
 
        that.sellItem = function (id) {
            var item = _.find(stock, function (item) {
                return item.getId() === id;
            });
 
            if (item) {
                stock.splice(stock.indexOf(item), 1);
                myBank.deposit(item.getSellingPrice());
                console.log('Item with id ', id, ' sold');
                return id;
            }
 
            console.log('Item with id ', id, 'not in stock');
            return null;
        };
 
        return that;
    };
    return shop;
});

 

Now let's have a look at the main.js. This is our main application file that launches the application. Instead of wrapping in the define function, we wrap it into the require. We also add a config definition to specify the base folder and paths to different modules. Because the underscore library is not wrapped in the AMD compatible wrapper, we need to use a workaround to import it correctly (shim). The require function is similar in structure to define, as the first parameter we pass an array of dependencies and the second parameter is a callback that takes module names as the parameters. We do not return anything from the require callback.

scripts/main.js:

require.config({
    baseUrl: 'scripts',
 
    paths: {
        item: 'items/item',
        bike: 'items/bike',
        car:  'items/car',
 
        bank: 'app/bank',
        shop: 'app/shop',
 
        underscore: 'libs/underscore'
    },
 
    shim: {
        underscore: {
            exports: '_'
        }
    }
});
 
require(['shop', 'bank', 'car', 'bike'], function (shop, bank, car, bike) {
 
    var myBank = bank({balance: 10000}),
        myShop = shop(myBank),
        itemIds = [];
 
    // add something to the shop's stock
    itemIds[0] = myShop.buyItem(car({
        sellingPrice    : 2300,
        purchasePrice   : 2000,
        make            : 'VW',
        color           : 'silver',
        mileage         : 100000,
        engineType      : '1.9 diesel',
        model           : 'Passat'
    }));
 
    /*
 
    ...
 
    */
 
});

 

The last thing to get our application working again is to modify the index.html file, there is no need to load all these different js script files anymore. We don't need to remember about order in which we need to load scripts as well - all this is managed for us automatically. The only script tag links to the require.js itself but it also has data-main attribute that points the require to the main script, note that there is no need to put the "js" extension at the end.

index.html:

<script data-main="scripts/main" src="scripts/libs/require.js"></script>

 

The files structure looks like this now:

phase1_filestructure_require.png

 

And the application should execute in the same way as before we wrapped it into the AMD modules with requuire.js.

phase1_browser_require.png

<phase1.zip>

 

Optimising with r.js

Our application is broken into modules so it is easier for us developers to maintain and extend it. But having many different separate module files isn't a good thing for a user, browsers will need to make multiple network request to load all the modules. This isn't very problematic with a small application, but it may become an overhead with large scale applications depending on hundreds of different modules. The solution is to merge our modules into a single file. This is good for us as during the development we would still use separate files, that then would be merged during the build process (locally or on the continuous integration server) and from the end user's perspective there will be only one js file loaded with a single request from a browser.

For combining modules into one file we will use the r.js "adapter" that also includes the optimiser - all that is developed by the same team that works on the require.js project. Apart from just combining, the optimiser can also minify the script with UglifyJS. Full documentation of using the require optimizer is available here: http://requirejs.org/docs/optimization.html.

The r.js script can be run in a browser but I prefer to use it with Node.js as a part of the build process. To prepare our application for building, let's create a new build configuration file build.js. It is not required as we could pass build properties as options when executing the build process in the command line but it is certainly easier to manage it this way. In the build.js we specify the location of the run-time configuration file (that we will just create in a moment), name of the main application file (main.js in our case), location for the output (processed) file that will be in the "build" folder and finally I set the optimisation parameter to none for now as we want to see first how the combined modules look like without "minificating" it. Let's call this build file "build-dev" as it will not be optimised for production - not minified/uglified and all code comments will be kept there.

build-dev.js:

({
    baseUrl: 'scripts/',
    mainConfigFile: 'scripts/config.js',
    name: "main",
    out: "build/dealerapp-dev.js",
    optimize: 'none'
})

 

I also moved the require.config from the main.js to a separate file config.js (this file is called the runtime config file).

scripts/config.js:

require.config({
    baseUrl: 'scripts',
 
    paths: {
        item: 'items/item',
        bike: 'items/bike',
        car:  'items/car',
 
        bank: 'app/bank',
        shop: 'app/shop',
 
        underscore: 'libs/underscore'
    },
 
    shim: {
        underscore: {
            exports: '_'
        }
    }
});

 

This is how the current code structure looks like at the moment:
phase2_filestructure.png

 

So we are ready to run the script. In the terminal, in the root of our project folder, run this command:

node r.js -o build-dev.js

 

If everything is in place correctly you should be able to see a result like below:

phase2_terminal_norequire.png

But what is most important, the build file is placed in the build folder, so let's have a look at it. As you can see this is just pure merge of our modules, the underscore lib comes first and the other modules follow it.

 

<phase2_1/dealerapp-dev.js>

 

So let's link our index.html directly to this file and see how it works:

<script src="build/dealerapp-dev.js"></script>

 

Of course you should see an error like this:

phase2_browser_defineerror.png

 

We didn't add the require.js library as one of the dependencies! Let's update the build-dev.js file by adding a path to the require.js script file (property is named "requireLib" as "require" word is already reserved):

build-dev.js:

({
    baseUrl: 'scripts/',
    mainConfigFile: 'scripts/config.js',
 
    paths: {
        requireLib: 'libs/require'
    },
 
    include: ['requireLib'],
 
    name: "main",
    out: "build/dealerapp-dev.js",
    optimize: 'none'
})

 

Now when you run the build script again, you will notice that require.js is added at the top. If you refresh your browser, this time the page should launch correctly.

phase2_terminal_require.png

phase2_browser_withdefine.png

Once we learned how to combine multiple module files and how to get the output script running, let's have a look how to optimise it with the r.js "adapter". Let's create another build script - build.js and copy the entire content of the build-dev.js excluding the line optimize: false and change the out property to export the file to dealerapp.js.

build.js:

({
    baseUrl: 'scripts/',
    mainConfigFile: 'scripts/config.js',
 
    paths: {
        requireLib: 'libs/require'
    },
 
    include: ['requireLib'],
 
    name: "main",
    out: "build/dealerapp.js"
})

 

When we run our script we should see an additional line about uglifying dealerapp.js.

phase2_terminal_require_optimisation.png

phase2_browser_optimised.png

The main script file became now a 'one liner', from 101 KB dealerapp-dev.js we managed to get 33 KB in our production code and still our application works perfectly.

 

<phase2.zip>

 

Almond

The majority of the content of the combined file is now the require.js script. It contains functionality that we don't need, we just want to be able to load and run our AMD modules. Luckily there is another AMD loader that is a lot smaller and contains only necessary functionality - almond.js https://github.com/jrburke/almond, a library from James Burke - the same guy that stay behind the require.js library. Download it and place in the libs folder in place of the require.js file.

We need to slightly modify our build files, the build-dev.js and build.js, first remove references to the require.js and then instead of having 'main' in the name property we add a path to almond.js (without 'js') and the reference to 'main' will go to the 'include' property.

build-dev.js:

({
    baseUrl: 'scripts/',
    mainConfigFile: 'scripts/config.js',
 
    name: "libs/almond",
    include: "main",
    out: "build/dealerapp-dev.js",
    optimize: 'none'
})

 

We follow the same approach for the build.js file:

({
    baseUrl: 'scripts/',
    mainConfigFile: 'scripts/config.js',
 
    name: "libs/almond",
    include: "main",
    out: "build/dealerapp.js"
})

 

So let's run the r.js optimisation tool in terminal:

phase3_almondoptimisation.png

If we run the application in a browser, it runs in exactly the same way as before with the require.js. Let's compare file sizes after optimising with the require.js loader and the almond.js loader:

Require.js:

dealerapp-dev.js 99 KB
dealerapp.js 32 KB

 

Almond.js:

dealerapp-dev.js 32 KB
dealerapp.js 18 KB

 

<phase3.zip>

 

 

TBD

 

...

Comments imported from the original article @sierakowski.eu

 
+- 0 #2 Vladimir 2013-07-19 18:20
Good stuff.
Quote
 
 
 
+- 0 #1 szczypior 2013-07-11 21:14
Thanks for this tutorial. There aren't many examples on the internet on how to use require practically. Require.js documentations is comprehensive but not easy to understand at first.
Quote