Friday, March 13, 2015

Developing without the backend: Working with 3rd Party Scripts

In my last post Developing without the backend. Simple Web Socket development., I talked about how to work offline with a simple Web Socket server so you can write your front-end in peace.  That is cool, but what if I have to deal with a 3rd party JS library that requires access to the Internet, login info to their site, actually running on their site, etc.  Essentially anything that requires you to run outside of your development environment.

So how do you deal with these 3rd party JavaScript API's using angular and work without their backend?  What if you need the cloud and you are offline?  What do you do?
Not that kind of cloud.  
Angular has the built in capability to use this pattern when necessary to add, remove, or completely replace the functionality of a provider by using the $provide.decorator function.

Angular has 2 major phases.  A run phase where it is actually executing the code, and a config phase where it wires up your application.  The decorator function comes in handy during the config phase that runs before the run phase.  In this phase you can access the $provider service, as well as built-in providers for your services, factories, etc.  So first you want to create a service wrapper for the 3rd party library that you are using.  Whether or not you just return the 3rd party object or create wrappers for all the functions, I'll leave up to you, but this will work for either method.  Next, you can do is create a config for your module (don't worry you can have multiple config calls) that uses the $provide.decorator function to overwrite or completely replace the service we just created.  For more information on these take a look at these posts on angular dependency injection and extending the $q method.  They are excellent reads.

I created a sample on jsFiddle to show you how it will work.  In this example I will use the underscore library in my application and then use the function above to overwrite one of the functions.  Outside of this tutorial, the next step would be to update your grunt (or whatever build) tasks to remove the mock config file from your project during the build process so you would actually use the real library.

To start I will create a simple service to wrap the underscore library, so I can write better unit tests and develop disconnected.

factory('underscoreService', function(){
    return _;
})

No magic there.  Next I will refer to that service in a controller object and call a couple of methods on it.

controller('ctrl' ,['$scope', 'underscoreService', function ($scope, underscoreService){
    $scope.list = ['Mercury', 'Venus', 'Earth', 'Mars', 
                   'Jupiter', 'Saturn', 'Uranus', 
                   'Neptune', 'Pluto'];
    
    $scope.sampleList = underscoreService.sample($scope.list, 3);   
    $scope.shuffleList = underscoreService.shuffle($scope.list);

}])

No magic here.  I have a list that creates 2 additional lists with method calls from underscore.  Now I want to swap out the logic from the service before it ever gets used.  This is where the $provider.decorator comes in. I will overwrite the logic for shuffle with a function that just returns the array passed in.

config(['$provide', function($provide){
    // override the function. you can uncomment it if you just want it to shuffle
    $provide.decorator('underscoreService', function($delegate){
        $delegate.shuffle = function(list){return list};
        return $delegate;
    });
}]

In the decorator function you receive a parameter called "$delegate" which is the return value from the service, factory, etc, before it gets injected anywhere.  So you can either overwrite the methods there, or you could return your own object instead!!!1

Here is the end result:


Just remember to update your build process to remove the config file!
Cheers!!

Monday, March 9, 2015

Developing without the backend. Simple Web Socket development.


While working at Motorola, the JavaScript code I wrote did a lot of communication with embedded devices via Web Sockets.  Because the work by the embedded side was done in parallel to my work, I typically didn’t have access to the latest version, let alone a physical device.  So I needed to be able to test, demo, and seamlessly integrate my work when we were ready.  One of the major lessons I took from this is that, heavy front-end development should be agnostic to the back end.

This is where I love working with a tool like Node.js.  You can use and develop using simple tools without having a backend/database connection and test all of your scenarios, and give a good demo. Likewise you can mock out your data so that it tests all of your edge cases and fail conditions easily.  The best part is that it is self contained, and can be passed from developer to developer.

So what I want to show is how to create a super simple Node.js web socket test server that is then easily portable to the actual production back-end.  I started my project using the yeoman Angular generator, however it isn’t necessary to use either for what I am going to show.  What is necessary is Node.js, grunt, grunt-contrib-connect (version 0.8 at least), and a web socket node package (I used ws).   


If you just want to get the code and work with it then you can download it here.

1. Setup

Starting with the angular-generator, and making sure that I added the 2 packages that referenced above, I modified my project structure just a bit by adding a factories folder in my app/scripts directory to separate simple web socket factory my data access services, and a server folder in my root project directory.  In this folder we are going to put all of our test server code.  

2. Front-end development

In the factory folder I created a simple angular factory module that pretty much just returns a Web Socket object.  The reason I do this mostly has to do with Unit testing.  It creates one simple place to grab a web socket instance whenever it is necessary, but most importantly it creates an easy place to use jasmine spies to mock the WebSocket object that comes back in any unit tests that I write. 

From there I create a service that will get an instance of the web socket and basically abstracts the communication to and from the WebSocket from the rest of the app.  So to the app I expose a way to connect, disconnect, and individual methods for each method type I want to send.
angular.module('blogApp')
    .factory('socketService', ['socketFactory', '$rootScope', function (socketFactory, $rootScope) {

    var _socket = null,
        _transactionCounter = 0,
        _messageHandlers = {},
        Types = {
            Connect: "CONNECT",
            Disconnect: "DISCONNECT",
            SetServerTimer: "SET_TIMER",
            BroadcastMessage: "BROADCAST_MESSAGE",
            AsyncTimer: "ASYNC_TIMER",
            GenericResponse: "GENERIC_RESPONSE"
        };

    function isConnected(){
      return _socket !== null && _socket.readyState === WebSocket.OPEN;
    }

    function disconnect(callback){
        if (isConnected()){
            _socket.close();
            if (callback){
                callback();
            }
        }
    }

    function setServerTimer(timeout, times, callback){
        sendMessage({type: Types.SetServerTimer, data: {timeout: timeout, times: times} });
    }

    function sendBroadcastMessage(user, message){
        sendMessage({type: Types.BroadcastMessage, data: {user: user, message: message}});
    }

    function connect(callback){
        if (!isConnected()){
            _socket = socketFactory.createSocket();

            _socket.onopen = function onOpen(){
                if (isConnected()){
                    sendMessage({type: Types.Connect, data: "Hello World"}, callback);
                }
            };

            _socket.onerror = function onError(){
                console.log("Error");
            };

            _socket.onmessage = onMessageRecieved;

            _socket.onclose = function onClose(){
                //goodnight socket
                _socket = null;
                $rootScope.$broadcast(Types.Disconnect);
            };
        }
    }

    function sendMessage(message, callback){
        if (!isConnected()){
            return false;
        }

        message.transactionId = _transactionCounter++;
        if (callback != null){
            _messageHandlers[message.transactionId] = callback;
        }

        _socket.send(angular.toJson(message));

        return message.transactionId;
    }

    function onMessageRecieved(evt){
        var message =  angular.fromJson(evt.data);

        $rootScope.$apply(function(){
            if (message.transactionId != null && _messageHandlers[message.transactionId] != null){
                _messageHandlers[message.transactionId](message);
                _messageHandlers[message.transactionId] = null;
            }

            $rootScope.$broadcast(message.type, message);
        });
        
    }

    // Public API here
    return {
      isConnected: isConnected,
      disconnect: disconnect,
      connect: connect,
      setServerTimer: setServerTimer,
      sendBroadcastMessage: sendBroadcastMessage,
      Types: Types
    };
}]);

Each message to the service is stringified and parsed as JSON, but you can use pretty much any format.  We used protobuf at Motorola which has an open source implementation.  Each message also has a transaction Id and the ability to register a callback from it.  So if you have a traditional send & receive message this will work for you.  The server will send the appropriate response as necessary with the same transaction ID so it will know what callback to use.  


You may notice that it also has a reference to the $rootScope.  The reason why is when data comes back for the response above it is wrapped in an $apply call.  The $apply call notifies angular to check the watches and to update the UI as appropriate.  Most NG-events or methods such as ng-click or $http do this for you already, but there isn’t anything for web sockets.

Also we want to have a way that any $scope in the system can listen for a specific message in case there is an asynchronous message from the server or some event that is initiated from the server (such as a message from a different logged on user).  To do this I an utilizing the angular $broadcast method that will notify any listeners for the new data and let them handle it as they need to.  

Finally I am going to add a simple controller that has a reference to this service and some buttons to perform the appropriate actions (I won't paste that all here).  

 3. Test Server

So now that I have this front-end code I need something to actually hit.  I recommend writing unit tests for most of your code, and the socketFactory will help you with that, but it doesn’t help you see an interactive version of the code.  To get that we will need an actual server.


In my server folder that I created earlier I am now creating a socketServer.js class.  I want to handle the connection/disconnecting from the socket and mock out the data being sent and any responses necessary. 

Using ws I will create a socket server on the port I specified earlier. So I create message handlers for connection, close and message events.  
var ws = require('ws');
var SOCKET_PORT= 55800,
    Types = {
        Connect: "CONNECT",
        Disconnect: "DISCONNECT",
        SetServerTimer: "SET_TIMER",
        BroadcastMessage: "BROADCAST_MESSAGE",
        AsyncTimer: "ASYNC_TIMER",
        GenericResponse: "GENERIC_RESPONSE"
    };

var sockets = [];

exports.startServer = function(server, connect, options){

    // open a websocket with said server. 
    var commandSocket = new ws.Server({port:SOCKET_PORT});

    commandSocket.on("connection", function(socket){

        var _socket = socket;
        sockets.push(socket);
        console.log("Socket Connected");

        _socket.on("close", function(){
            console.log("Socket Closed");

            //remove from the list of sockets
            for(var i = 0; i < sockets.length; i++){
                if (_socket == sockets[i]){
                    sockets.splice(i, 1);
                    break;
                }
            }

            _socket = null;
        });

        _socket.on("message", function(data, flags){
            var message = JSON.parse(data);
            console.log("Socket message recieved: " + message.type);

            switch(message.type){
                case Types.Connect: 
                    handleConnectMessage(message);
                    break;
                case Types.SetServerTimer:
                    handleSetServerTimer(message);
                    break;
                case Types.BroadcastMessage:
                    handleBroadcastMessage(message);
                    break;
            }

        });
    });
});

In the connection event I will receive an instance of the socket object, so I will hold onto it and use closure to allow everything else have access to it.  Likewise I will add it to a running list of open connections that I have going so I can send any messages of a certain type to all active sockets.  


In the close event I will set the socket to null, and remove it from the active socket array.  Nothing crazy here.


The message event is where we will do the bulk of the work.  You can make this part as complicated or simple as possible.  I recommend that you keep it really simple for a few reasons.  First, you typically don’t really care about the data being sent back or any actual saving of the data.  Your front end typically cares that the message got sent correctly, and any responses that happen, do happen.  Second, you want to be able to change it easily.  The more you add to it the more complicated it will get.  You should be able to change the parameters or the message sequences easily, or it defeats the purpose of a quick and dirty server.  Finally, your end product is most likely just the front end.  Since that is the case you don’t want to spend more time than you need to, when you just want to test the scenarios or demo it to your client or PM.  

Finally we need to actually call our mock server.  That is where the specific version of gunt-contrib-connect comes in.  In the options for the server there is a callback function you can hook into when the server starts called onCreateServer.  That callback will pass you the server among with other information so you can hook into it.  In our case we just want to call our socketServer.js class and pass in the server.

      connect: {
      options: {
        port: 9000,
        // Change this to '0.0.0.0' to access the server from outside.
        hostname: 'localhost',
        livereload: 35729,
        onCreateServer: function(server, connect, options){
            var socketServer = require('./server/socketServer.js');
            
            socketServer.startServer(server, connect, options);
        }
      },

Before I ended this post, I wanted to just reiterate that the point of this is not to recreate all of the logic of the server, but to make one simple enough that it is easy to change and just big enough to work.

The code is available on Github. Feel free to download it and start playing. Just npm install, bower install, and grunt server and you are good to go.