Using HTML5 Canvas to Create a Transparent Image.

I recently have been working on learning some threejs stuff and trying to understand 3d concepts in general.  One of my favorite things is looking at 3d renderings of planets and other cosmic things.  So I found this post that talks about how he created the planets.  He uses the textures from planetpixelemporium for his planets.  Beyond the simple textures there are textures that define the bump map for rocky planets, textures for specular which defines the reflectiveness for each area, textures for clouds, and even textures for rings.

While I was going through his blogpost and his code, trying to make sure I understood everything he was saying, I came across this:

We build canvasCloud and use it as texture. It is based on the jpg images you see above: one for the color and the other for the transparency. We do that because jpg doesn’t handle an alpha channel. So you need to make the code to build the texture based on those images. 

I was super confused, and I apparently wasn't the only one based on his comments.  So armed with just his code, and google I set out to see what the deal is with multiple jpegs and transparency.  I found all kinds of interesting things out there and why people would write code doing this instead of using a png or another format that supports transparency like was done for this android app, was done for a game in JS, and this interesting trick that XORs two images to get a transparent one.

So armed with reasons why I wanted to visualize what this actually looks like when we go through the data and merge the two jpegs into an image with alpha.   I originally wanted to do this all in jsfiddle like I normally put my examples, but unfortunately jsfiddle doesn't support hosted images, I didn't want to pay one of the other online sandboxes to host them, and you can't look at the image data in a canvas that is from a different domain.  So I put this on a website I have been playing around with for a bit, so you too can see it in action.  The source is on github here.

To get this to work we will use 3 canvases (these don't have to be in the DOM to work).  One for the source image, one for the alpha (transparency) image which is all greyscale, and one for the destination.  We then load the two images we want by creating Image elements for each of the images, setting the src to the appropriate image, and then handling the onload event of each image.  The code looks like this:

    function loadImages(sourceImageUrl, alphaImageUrl){
        sourceImg = new Image();
        sourceImg.addEventListener("load", function(){
            sourceCanvas.height = sourceImg.height;
            sourceCanvas.width = sourceImg.width;
            sourceContext.drawImage(sourceImg, 0,0);            
            alphaImg = new Image();
            alphaImg.addEventListener('load', function(){
                alphaCanvas.height = alphaImg.height;
                alphaCanvas.width = alphaImg.width;
                alphaContext.drawImage(alphaImg, 0, 0);                
                startOverlay();
            });
            alphaImg.src = alphaImageUrl;
        });
        sourceImg.src = sourceImageUrl;
    }

Once that is done and we have the images loaded on the two canvases we will need to loop through the image data and copy over the RGB components from the source canvas and compute the alpha based on the brightness of the greyscale with white (255) being completely transparent, and black (0) being completely opaque (you can make this up however you want realistically).  Here is the code for that:

function startOverlay(){
        destinationContext.clearRect(0,0, destinationCanvas.height, destinationCanvas.width);
        
        var sourceData = sourceContext.getImageData(0, 0, sourceCanvas.width, sourceCanvas.height),
            alphaData = alphaContext.getImageData(0,0, alphaCanvas.width, alphaCanvas.height),
            destinationData = destinationContext.getImageData(0,0, destinationCanvas.width, destinationCanvas.height);

        var mergeProperties = $scope.mergeProperties;
        mergeProperties.x = 0;
        mergeProperties.y = 0;
        mergeProperties.offset = 0;
        
        function mergeImages(){

            for(var i = 0; i < mergeProperties.speed; i++){

                destinationData.data[mergeProperties.offset+0]  = sourceData.data[mergeProperties.offset+0];
                destinationData.data[mergeProperties.offset+1]  = sourceData.data[mergeProperties.offset+1];
                destinationData.data[mergeProperties.offset+2]  = sourceData.data[mergeProperties.offset+2];
                destinationData.data[mergeProperties.offset+3]  = 255 -  alphaData.data[mergeProperties.offset+0];

                mergeProperties.x++;
                mergeProperties.offset+=4;
                if (mergeProperties.x >= sourceImg.width){
                    mergeProperties.x = 0;
                    mergeProperties.y++;
                }
                if (mergeProperties.y >= sourceImg.height){
                    destinationContext.putImageData(destinationData,0,0);    
                    return;
                }
            }

            destinationContext.putImageData(destinationData,0,0);    
            $timeout(mergeImages, 10);

        }
        
        mergeImages();
    }

The Timeout code is there just to slow it down for my demo. For actual production you would not have the timeout and the speed. You would just loop until you hit the imageHeight.

So take a look at it, it is pretty cool visualization.
Cheers!

Developing without the backend. Simple Web Socket development.

While working at Motorola, the JavaScript code I wrote did a lot of communication with embedded devices via Web Sockets.  Because the work by the embedded side was done in parallel to my work, I typically didn’t have access to the latest version, let alone a physical device.  So I needed to be able to test, demo, and seamlessly integrate my work when we were ready.  One of the major lessons I took from this is that, heavy front-end development should be agnostic to the back end.

This is where I love working with a tool like Node.js.  You can use and develop using simple tools without having a backend/database connection and test all of your scenarios, and give a good demo. Likewise you can mock out your data so that it tests all of your edge cases and fail conditions easily.  The best part is that it is self contained, and can be passed from developer to developer.

So what I want to show is how to create a super simple Node.js web socket test server that is then easily portable to the actual production back-end.  I started my project using the yeoman Angular generator, however it isn’t necessary to use either for what I am going to show.  What is necessary is Node.js, grunt, grunt-contrib-connect (version 0.8 at least), and a web socket node package (I used ws).   

If you just want to get the code and work with it then you can download it here.

1. Setup

Starting with the angular-generator, and making sure that I added the 2 packages that referenced above, I modified my project structure just a bit by adding a factories folder in my app/scripts directory to separate simple web socket factory my data access services, and a server folder in my root project directory.  In this folder we are going to put all of our test server code.  

2. Front-end development

In the factory folder I created a simple angular factory module that pretty much just returns a Web Socket object.  The reason I do this mostly has to do with Unit testing.  It creates one simple place to grab a web socket instance whenever it is necessary, but most importantly it creates an easy place to use jasmine spies to mock the WebSocket object that comes back in any unit tests that I write. 

From there I create a service that will get an instance of the web socket and basically abstracts the communication to and from the WebSocket from the rest of the app.  So to the app I expose a way to connect, disconnect, and individual methods for each method type I want to send.

angular.module('blogApp')
    .factory('socketService', ['socketFactory', '$rootScope', function (socketFactory, $rootScope) {

    var _socket = null,
        _transactionCounter = 0,
        _messageHandlers = {},
        Types = {
            Connect: "CONNECT",
            Disconnect: "DISCONNECT",
            SetServerTimer: "SET_TIMER",
            BroadcastMessage: "BROADCAST_MESSAGE",
            AsyncTimer: "ASYNC_TIMER",
            GenericResponse: "GENERIC_RESPONSE"
        };

    function isConnected(){
      return _socket !== null && _socket.readyState === WebSocket.OPEN;
    }

    function disconnect(callback){
        if (isConnected()){
            _socket.close();
            if (callback){
                callback();
            }
        }
    }

    function setServerTimer(timeout, times, callback){
        sendMessage({type: Types.SetServerTimer, data: {timeout: timeout, times: times} });
    }

    function sendBroadcastMessage(user, message){
        sendMessage({type: Types.BroadcastMessage, data: {user: user, message: message}});
    }

    function connect(callback){
        if (!isConnected()){
            _socket = socketFactory.createSocket();

            _socket.onopen = function onOpen(){
                if (isConnected()){
                    sendMessage({type: Types.Connect, data: "Hello World"}, callback);
                }
            };

            _socket.onerror = function onError(){
                console.log("Error");
            };

            _socket.onmessage = onMessageRecieved;

            _socket.onclose = function onClose(){
                //goodnight socket
                _socket = null;
                $rootScope.$broadcast(Types.Disconnect);
            };
        }
    }

    function sendMessage(message, callback){
        if (!isConnected()){
            return false;
        }

        message.transactionId = _transactionCounter++;
        if (callback != null){
            _messageHandlers[message.transactionId] = callback;
        }

        _socket.send(angular.toJson(message));

        return message.transactionId;
    }

    function onMessageRecieved(evt){
        var message =  angular.fromJson(evt.data);

        $rootScope.$apply(function(){
            if (message.transactionId != null && _messageHandlers[message.transactionId] != null){
                _messageHandlers[message.transactionId](message);
                _messageHandlers[message.transactionId] = null;
            }

            $rootScope.$broadcast(message.type, message);
        });
        
    }

    // Public API here
    return {
      isConnected: isConnected,
      disconnect: disconnect,
      connect: connect,
      setServerTimer: setServerTimer,
      sendBroadcastMessage: sendBroadcastMessage,
      Types: Types
    };
}]);

Each message to the service is stringified and parsed as JSON, but you can use pretty much any format.  We used protobuf at Motorola which has an open source implementation.  Each message also has a transaction Id and the ability to register a callback from it.  So if you have a traditional send & receive message this will work for you.  The server will send the appropriate response as necessary with the same transaction ID so it will know what callback to use.  

You may notice that it also has a reference to the $rootScope.  The reason why is when data comes back for the response above it is wrapped in an $apply call.  The $apply call notifies angular to check the watches and to update the UI as appropriate.  Most NG-events or methods such as ng-click or $http do this for you already, but there isn’t anything for web sockets.

Also we want to have a way that any $scope in the system can listen for a specific message in case there is an asynchronous message from the server or some event that is initiated from the server (such as a message from a different logged on user).  To do this I an utilizing the angular $broadcast method that will notify any listeners for the new data and let them handle it as they need to.  

Finally I am going to add a simple controller that has a reference to this service and some buttons to perform the appropriate actions (I won't paste that all here).  

 3. Test Server

So now that I have this front-end code I need something to actually hit.  I recommend writing unit tests for most of your code, and the socketFactory will help you with that, but it doesn’t help you see an interactive version of the code.  To get that we will need an actual server.

In my server folder that I created earlier I am now creating a socketServer.js class.  I want to handle the connection/disconnecting from the socket and mock out the data being sent and any responses necessary. 

Using ws I will create a socket server on the port I specified earlier. So I create message handlers for connection, close and message events.  

var ws = require('ws');
var SOCKET_PORT= 55800,
    Types = {
        Connect: "CONNECT",
        Disconnect: "DISCONNECT",
        SetServerTimer: "SET_TIMER",
        BroadcastMessage: "BROADCAST_MESSAGE",
        AsyncTimer: "ASYNC_TIMER",
        GenericResponse: "GENERIC_RESPONSE"
    };

var sockets = [];

exports.startServer = function(server, connect, options){

    // open a websocket with said server. 
    var commandSocket = new ws.Server({port:SOCKET_PORT});

    commandSocket.on("connection", function(socket){

        var _socket = socket;
        sockets.push(socket);
        console.log("Socket Connected");

        _socket.on("close", function(){
            console.log("Socket Closed");

            //remove from the list of sockets
            for(var i = 0; i < sockets.length; i++){
                if (_socket == sockets[i]){
                    sockets.splice(i, 1);
                    break;
                }
            }

            _socket = null;
        });

        _socket.on("message", function(data, flags){
            var message = JSON.parse(data);
            console.log("Socket message recieved: " + message.type);

            switch(message.type){
                case Types.Connect: 
                    handleConnectMessage(message);
                    break;
                case Types.SetServerTimer:
                    handleSetServerTimer(message);
                    break;
                case Types.BroadcastMessage:
                    handleBroadcastMessage(message);
                    break;
            }

        });
    });
});

In the connection event I will receive an instance of the socket object, so I will hold onto it and use closure to allow everything else have access to it.  Likewise I will add it to a running list of open connections that I have going so I can send any messages of a certain type to all active sockets.  

In the close event I will set the socket to null, and remove it from the active socket array.  Nothing crazy here.

The message event is where we will do the bulk of the work.  You can make this part as complicated or simple as possible.  I recommend that you keep it really simple for a few reasons.  First, you typically don’t really care about the data being sent back or any actual saving of the data.  Your front end typically cares that the message got sent correctly, and any responses that happen, do happen.  Second, you want to be able to change it easily.  The more you add to it the more complicated it will get.  You should be able to change the parameters or the message sequences easily, or it defeats the purpose of a quick and dirty server.  Finally, your end product is most likely just the front end.  Since that is the case you don’t want to spend more time than you need to, when you just want to test the scenarios or demo it to your client or PM.  

Finally we need to actually call our mock server.  That is where the specific version of gunt-contrib-connect comes in.  In the options for the server there is a callback function you can hook into when the server starts called onCreateServer.  That callback will pass you the server among with other information so you can hook into it.  In our case we just want to call our socketServer.js class and pass in the server.

      connect: {
      options: {
        port: 9000,
        // Change this to '0.0.0.0' to access the server from outside.
        hostname: 'localhost',
        livereload: 35729,
        onCreateServer: function(server, connect, options){
            var socketServer = require('./server/socketServer.js');
            
            socketServer.startServer(server, connect, options);
        }
      },

Before I ended this post, I wanted to just reiterate that the point of this is not to recreate all of the logic of the server, but to make one simple enough that it is easy to change and just big enough to work.

The code is available on Github. Feel free to download it and start playing. Just npm install, bower install, and grunt server and you are good to go.

Making Some Progress

I just started on a new project recently and one of the things on the page I was working on was displaying progress bars in different colors.  The progress bars need to display an indeterminate state and after talking with a co-worker we thought making it look like KITT from Night Rider was the best (and most awesome) way to display that.

Is that you Mr. Feeny?

Because of a lot of my work being done on the canvas, and to have communication being done with web sockets, I am only supporting modern browsers for this app.  I knew there are a ton of new elements in HTML 5 and one of my personal goals is to become better at writing semantically correct html.  A quick look through the elements, and I found the <progress> tag.  Even better, it comes with an indeterminate state and is supported in the browsers I care about.  My next requirement was just to style the progress elements different colors based on their states.  I thought this is going to be easy.  All I need to do is add some classes to set the color and I'm done!

So after a quick search I find out you pretty much can't do anything to style a progress element.  Which made me think...  how do you expect people to use this element if they can't make it look how they need it to?

I was determined to use CSS only to get this to work and make my page semantically awesome.  I found some sites like this that show you how to clear out the native browser appearance and add your own.  So I started with that and then added my own classes to style them how I needed.  A lot of CSS later, I came up with the following:

.progressContainer{
    position:relative;
    width: 100px;
}

progress{
  height:5px;
  border:0;
  appearance: none;
  -moz-appearance: none;
  -webkit-appearance: none;
  width:100% !important;
 
}

progress.connected::-webkit-progress-value{
  background: green;
} 
progress.connected::-moz-progress-value{
  background: green;
}
progress.connected::progress-value{
  background: green;
}
progress.connected::-webkit-progress-bar{
  background: green;
} 
progress.connected::-moz-progress-bar{
  background: green;
}
progress.connected::progress-bar{
  background: green;
}
progress.connected:not([value])::-webkit-progress-bar{
  background:green;
}
progress.connected:not([value])::-moz-progress-bar{
  background:green;
}
progress.connected:not([value])::progress-bar{
  background:green;
}

With the lower half repeated for each class that I wanted, I was almost done. While verbose, I was able to style the progress as necessary. The only thing left was to get KITT to work. After some quick thinking I figured I would use a ::before pseudo element and animate it.

@-webkit-keyframes knightRider {
   0% {
 left: 0%;
      }
 
  50% {
 left:100%;
        margin-left:-10px;
      }
  100%{
 left:0%;
      }
}
@keyframes knightRider {
   0% {
 left: 0%;
      }
 
  50% {
   left:100%;
        margin-left:-10px;
      }
  100%{
 left:0%;
      }
}

progress.searching::before{
  content:"";
  background-color:blue;
  width:10px;
  height:5px;
  position: absolute;
  animation: knightRider 2s infinite linear;
  -webkit-animation: knightRider 2s infinite linear;
}

A quick refresh of my fiddle and BOOM it was working.  Now I just need to make sure it is working in Firefox which should be just a formality, right?  Wrong!  Only webkit browsers support ::before and ::after on progress for some bizzaro reason.  So I ended up adding a hidden div next to the progress bar that would use the animation from above and only show up when a specific class is added to the progress bar.

See the Pen Progress by Andy (@andyjmeyers) on CodePen.

Finally some progress...

HTML 5 Canvas Zoom

Pretty much since my first day at Motorola I have been working with HTML 5 canvases.  I have been mostly working on creating a charting framework from scratch because the existing frameworks don't perform well enough.

Without going into too much detail, one of the newer pieces of functionality that I added was the ability to zoom into a section of the chart by dragging your mouse over a given area.  JS Fiddle Link.  

The first challenge was to figure out where on the canvas the mouse event is happening.  All of the mouse events (move, down, etc) use the same event object so I created a helper function that would get the current coordinates from the current event object.

function getCursorPosition(e) {
    var totalOffsetX = 0;
    var totalOffsetY = 0;
    var canvasX = 0;
    var canvasY = 0;
    var currentElement = e.target;
    do {
        totalOffsetX += currentElement.offsetLeft - currentElement.scrollLeft;
        totalOffsetY += currentElement.offsetTop - currentElement.scrollTop;
    }
    while (currentElement = currentElement.offsetParent)
    canvasX = e.pageX - totalOffsetX;
    canvasY = e.pageY - totalOffsetY;
    return {
        x: canvasX,
        y: canvasY
    }
}

The crux here is to make sure to keep getting the offset of the each element because the parent might not be the window.  Another thing I ran into with the mouse events was since I was handling the mousedown and mouseup events I had to add a flag on mousemove to indicate whether or not if the mouse action was a click event or not.

The next piece of logic is the actual zoom function.

function zoom(point1, point2) {
    var canvas = document.getElementById('myCanvas');
    var translation = {
        x: point2.x,
        y: point2.y
    };
    var canvasMax = {
        x: point1.x,
        y: point1.y
    };
    if (point1.x < point2.x) {
        translation.x = point1.x;
        canvasMax.x = point2.x;
    }
    if (point1.y < point2.y) {
        translation.y = point1.y;
        canvasMax.y = point2.y;
    }
    currentTranslation.x += 1 / canvasScale.width * translation.x;
    currentTranslation.y += 1 / canvasScale.height * translation.y;
    canvasScale.width = canvasScale.width * canvas.width / (canvasMax.x - translation.x);
    canvasScale.height = canvasScale.height * canvas.height / (canvasMax.y - translation.y);
    drawPicture();
}

In he function I choose the upper left hand point of the rectangle as the points of the translation.  I then figure out what the position should be based on the current zoom.  Next we have to figure out the new scale based on the current scale, and the size of the rectangle that we selected.  Once we have generated the scale and translation we can just plug them in our paint function.

While this code gave me a good start on what I needed, the actual implementation for the charting functionality is a bit more complicated due to having to scale the numbers on the graph without actually scaling the axis canvas.  

Writing Elegant HTML

The other day I was thinking to myself, "Self, I wonder if there are any books on writing elegant HTML and CSS."  I'm sure there are I thought, and spent some time looking on Amazon for a book.  I was hoping for something that would essentially teach me the correct way to do things since I am pretty much self taught in the area of web development and would like to know what I am doing right and what I am doing wrong.  Unfortunately I didn't find a book that was suitable for what I was looking for.  However I did find a site that I found just as good, if not better. The website, Webplatform.org, has a lot of nice tutorials on it, with explanations why to do it that way.

So far I have only gone through the HTML and part of the CSS tutorials, but I have already found plenty of flaws in the HTML I used to write and definitely plan on changing the way I write it to become more semantic of the page I am writing.  By writing html that is more semantic it makes everything clearer to other developers what the element does, and it also helps out those with disabilities that need to use things like screen readers.