Category: JavaScript

I Made a Scatter Plot Data Visualization with AngularJS and SVG. Here’s How I Did It.

Posted on by Rob Larsen

This post is a longer examination of the new scatter plot visualization I added to my upcoming data visualization presentation for jQuery Conference. As with the other Angular examples I’ve posted recently, this one is based on the data I keep about the world’s most valuable comics. This particular visualization lives on the page I keep detailing record comic book sales. Take a look. It’s pretty cool, especially if you’re fascinated by comic books that have sold for more than $100,000.

For reference, it looks like the following. It’s 20 years of comics sales plotted horizontally with their sale prices (ranging from $100,000 to $2,500,000) plotted vertically. You can see a couple of clusters of activity based on different events in the marketplace.

chart

So, how did I do it? And why did I do it this way? D3 actually has a few scatter plot examples out there, so I could have used one of those, but as I thought about it I decided I wanted to do something hand-rolled. A scatter plot is actually pretty easy to map out and, as an added bonus, I could lay out the basics of the chart in Illustrator, which allowed me to demo the fact that SVG can be authored like any image by a graphic designer and then manipulated with JavaScript and CSS.

So let’s take a look at the particulars.

First we’ll start with the markup. This is Angular, after all. I’ve simplified the markup a bit, taking out some of the elements representing grid lines. Check the source on Github for all the details.

The interesting bits, from the Angular perspective, happen around line 30. The core of the visualization is in the circle element. It’s an ng-repeat that goes through the items collection in the controller’s scope. It’s filtered by the venues property.

Take note of the prefixes on the next set of attributes. For starters, as I normally do, I prepend data-* to Angular attributes as that’s the way custom attributes are defined in the spec. I like to play by the rules. You’ve probably seen that before. What’s probably not so familiar is the use of the additional ng-attr-* prefix. If you’re familiar with Angular you know you should just be able to do fill="{{colorPicker(it.venue)}}" and Angular should evaluate the results of the colorPicker method and insert it in as the value of the fill attribute. This doesn’t work with SVG elements. Some browsers will parse the entire SVG element before Angular has a chance to evaluate the SVG attributes. So if you put n Angular expression inside an SVG attribute it will throw an error. ng-attr solves this by binding the attribute solely to Angular’s discretion. Here, I’m using this pattern three times. The fill attribute, which provides the color coding for the different venues s these comic books have sold (with the handy colorPicker method that accepts the it.venue as an argument) , and the cx and cy attributes which indicate the center of the circle.

The cx and cy attributes are passed through custom filters which transform the data into usable x and y coordinates. I’ll talk about those in the section on the Angular filters. Following the filters there are two mouse events used to show and hide a simple (svg) tooltip.

The tooltip is in the next section. Doing boxes in SVG isn’t quite the same thing it is with HTML. You can’t just throw a box in there and have text flow inside the box like you can with a DIV or something, so this would have been easier with an HTML tooltip. That said, I wanted to do this in SVG so it’s all in SVG. In this case I’m doing it with a solid box with several text elements layered on top. All of these tooltip elements show based on the tooltip model on the scope. If the price is truthy, the tooltip shows. Otherwise, it’s hidden. For each text element and the rect element, I’m using x and y coordinates generated with the same Angular filters in use on the circles and using the transform attribute to move the element around based on that common x/y. Each of the text elements also expose the data from the model.

Following the tooltip code, in the ‘legend’ g element there are several rect elements which allow for quick filtering by sales venue. ng-mouseover and ng-mouseout are used to update the venues filter.

That’s the markup. SVG isn’t that scary, is it?

 <div data-ng-app="comicsApp">
  <div data-ng-controller="chartCtrl"> 
    <svg version="1.1" xmlns="http://www.w3.org/2000/svg" 
    xmlns:xlink="http://www.w3.org/1999/xlink"
    width="980" height="600" viewBox="0 0 980 600" xml:space="preserve">
      <g id="lines">
        <line fill="none" stroke="#000000" stroke-miterlimit="10" 
         x1="90.828" y1="-8" x2="90.828" y2="579.297"/>
        <line fill="none" stroke="#B5B5B6" stroke-miterlimit="10" 
         x1="0" y1="579.297" x2="981.996" y2="579.297"/>
        <line fill="none" stroke="#595858" stroke-miterlimit="10" 
         x1="734.275" y1="49.246" x2="734.275" y2="579.161"/>
        <!--about 50 lines cut-->
      </g>
      <g id="text">
        <text x="10" y="16">price in USD</text>
        <text x="71" y="574">$0</text>
        <text x="480" y="595">2003</text>
        <text x="78" y="595">1993</text>
        <text x="881" y="595">2013</text>
        <text x="25" y="553">$100,000</text>
        <text x="25" y="468">$500,000</text>
        <text x="16" y="363">$1,000,000</text>
        <text x="16" y="256">$1,500,000</text>
        <text x="16" y="151">$2,000,000</text>
        <text x="16" y="45" >$2,500,000</text>
      </g>
      <g id="dynamic">
        <circle  data-ng-repeat="it in items | filter : venues" 
         opacity="0.8" 
         data-ng-attr-fill="{{colorPicker(it.venue)}}" 
         data-ng-attr-cx="{{it.date | xDate}}"
         data-ng-attr-cy="{{it.price | yPrice}}" 
         data-ng-mouseover="updateTooltip(it)" 
         data-ng-mouseout="updateTooltip(0)" r="7" />
        <rect data-ng-show="tooltip.price" data-ng-model="tooltip" 
         transform="translate(-55,-100)" 
         data-ng-attr-x="{{tooltip.date | xDate}}" 
         data-ng-attr-y="{{tooltip.price | yPrice}}" 
         width="150" height="80" 
         fill="white" style="filter:url(#drop-shadow)" />
        <text data-ng-show="tooltip.price" data-ng-model="tooltip" 
         transform="translate(-45,-90)" 
         data-ng-attr-x="{{tooltip.date | xDate}}" 
         data-ng-attr-y="{{tooltip.price | yPrice}}">
         {{tooltip.title}} {{tooltip.issue}}</text>
        <text data-ng-show="tooltip.price" data-ng-model="tooltip" 
         transform="translate(-45,-75)" 
         data-ng-attr-x="{{tooltip.date | xDate}}" 
         data-ng-attr-y="{{tooltip.price | yPrice}}" >
         {{tooltip.pedigree}}{{tooltip.collection}}{{tooltip.provenance}} {{tooltip.grade_src | srcFilter}} {{tooltip.grade}}</text>
        <text data-ng-show="tooltip.price" data-ng-model="tooltip" 
         transform="translate(-45,-60)" 
         data-ng-attr-x="{{tooltip.date | xDate}}" 
         data-ng-attr-y="{{tooltip.price | yPrice}}" >
         {{tooltip.price|currency}}</text>
        <text data-ng-show="tooltip.price" data-ng-model="tooltip" 
         transform="translate(-45,-45)" 
         data-ng-attr-x="{{tooltip.date | xDate}}" 
         data-ng-attr-y="{{tooltip.price | yPrice}}" >
         {{tooltip.date}}</text>
        <text data-ng-show="tooltip.price" data-ng-model="tooltip" 
         transform="translate(-45,-30)" 
         data-ng-attr-x="{{tooltip.date | xDate}}" 
         data-ng-attr-y="{{tooltip.price | yPrice}}" >
         {{tooltip.venue}}</text>
      </g>
      <g id="legend">
        <rect x="125" y="5" width="15" height="15" fill="#D95B43" 
         data-ng-mouseover="venues='comic connect'" 
         data-ng-mouseout="venues=''"></rect>
        <text x="145" y="15" class="legend">Comic Connect</text>
        <rect x="245" y="5" width="15" height="15" fill="#ECD078"  
         data-ng-mouseover="venues='heritage'" 
         data-ng-mouseout="venues=''"></rect>
        <text x="265" y="15" class="legend">Heritage</text>
        <rect x="325" y="5" width="15" height="15" 
         fill="#C02942" data-ng-mouseover="venues='comiclink'" 
         data-ng-mouseout="venues=''"></rect>
        <text x="345" y="15" class="legend">ComicLink</text>
        <rect x="405" y="5" width="15" height="15" fill="#542437" 
         data-ng-mouseover="venues='pedigree'" 
         data-ng-mouseout="venues=''"></rect>
        <text x="425" y="15" class="legend">Pedigree</text>
        <rect x="485" y="5" width="15" height="15" fill="#53777A" 
         data-ng-mouseover="venues='metropolis'" 
         data-ng-mouseout="venues=''"></rect>
        <text x="505" y="15" class="legend">Metropolis</text>
        <rect x="565" y="5" width="15" height="15" fill="#69D2E7" 
         data-ng-mouseover="venues='jp'" 
         data-ng-mouseout="venues=''"></rect>
        <text x="585" y="15" class="legend">JP/Mint</text>
        <rect x="645" y="5" width="15" height="15" fill="#FA6900" 
         data-ng-mouseover="venues='mastronet'"  
         data-ng-mouseout="venues=''"></rect>
        <text x="665" y="15" class="legend">Mastronet</text>
        <rect x="725" y="5" width="15" height="15" fill="#FE4365" 
         data-ng-mouseover="venues='pgc'" 
         data-ng-mouseout="venues=''"></rect>
        <text x="745" y="15" class="legend">PGCMint</text>
        <rect x="805" y="5" width="15" height="15" fill="#666666" 
         data-ng-mouseover="venues='unknown'" 
         data-ng-mouseout="venues=''"></rect>
        <text x="825" y="15" class="legend">Other/Unkown</text>
      </g>
    </svg> 
  </div>
</div>

Now let’s take a look at the JavaScript. First up is the controller. This is pretty simple as far as these things go. $scope.items is first populated with data provided by an Angular factory. The factory is a simple $http.get so I’m not showing it here. I just organized the data request into a factory so two controllers could share the same code for getting comics data. Following that, the $scope.tooltip model is set with a single property, price set to zero. This falsy value ensures that the tooltip we previously defined in the HTML isn’t shown by default. Following that is a small method designed to update the tooltip model with data representing the currently selected comic book. Finally, there’s a small method and related array used to choose colors for the different venues. The method accepts a string argument and matches against a list of known venues, returning the proper color.

That’s the whole controller. 

angular.module('comicsApp.controllers', ['comicFilters','comicsFactories']). 
  controller('chartCtrl', ["$scope",'dataService',function( $scope, dataService )  {
    $scope.items = dataService;
    $scope.tooltip = {
      price:0
    }
    $scope.updateTooltip = function(it) {
      $scope.tooltip = {
        price:it.price || 0,
        venue:it.venue,
        date:it.date,
    	 	title:it.title,
    		issue:it.issue,
    		pedigree:it.pedigree,
    		collection:it.collection,
    		provenance:it.provenance,
    		grade_src: it.grade_src,
    		grade : it.grade
      } 
    }
    $scope.colorPicker= function( venue ){
      switch (venue) {
        case "Heritage":
          return $scope.colors[0];
        case  "Comic Connect":
          return $scope.colors[1];
        case "Comiclink":
          return $scope.colors[2];
        case  "Pedigree":
          return $scope.colors[3];
        case "Metropolis":
          return $scope.colors[4];
        case  "JP The Mint":
          return $scope.colors[5];
        case "Mastronet":
          return $scope.colors[6];
        case  "PGCMint":
          return $scope.colors[7];
        default:
          return $scope.colors[8];
       }
    }
    $scope.colors = ["#ECD078","#D95B43","#C02942","#542437","#53777A","#69D2E7","#FA6900", "#FE4365","#666666"];
  }
]);

Now let’s take a look at the filters. As I mentioned earlier, this particular visualization isn’t generic, so these are only useful in the context of this particular implementation. Which is fine. I’m not doing a million of these.

The first, xDate, defines the x axis. It takes the date (in the form yyyy-mm-dd) and splits it into and array containing the years, months and days. I calculate the total number of months from 1993 by subtracting 1993 from the year and then multiplying that number by 12. I then adding that number to the number of months from the original date array. After that the numbers of months is fudged into a pixel measurement as the offset of the left of the chart, 90, is added to the number of months multiplied by 3.3333, which is a rough “month” unit on this particular chart.

The y axis filter is easier. It calculates a pixel representation of the book’s value (the value divided by 4700, the number of dollars per pixel) and that number is subtracted from 579, which is the offset from the top of the SVG element to the baseline of the chart.

If you were doing a generic chart that could take any similar data and produce a scatter plot, you would have to do these calculations on the fly. Getting the minimum and maximum values out of the data and creating a range and scales depending on the specific data. Since I defined the data and parameters for this before I wrote a line of code.

Yay for lazy.

angular.module('comicFilters', []).filter('xDate', function () {
    "use strict";
    return function (input) {
        if (input !== undefined) {
          var date = input.split("-");
          var years = date[0] - 1993;
          var months = (years * 12) + parseInt(date[1]);
          return 90 + months * 3.3333;
          
        }
    };
}).filter('yPrice', function () {
    "use strict";
    return function (input) {
      if (input !== undefined){
        return 579 - (input/4700);
      }
    };
})

And there you have it. Check out the $100,000 club repo for all the source code from this visualization and keep your eyes peeled for my jQuery Conference presentation where I’ll be talking about this as well as D3, the canvas element (with the web audio API) and Google Maps.

I’m Going to be Presenting at jQuery Conference. It’s in San Diego. You Should Go, Too.

Posted on by Rob Larsen

It was announced last week, so I can now share the fact that I’m going to presenting on Data Visualization in the Browser at jQuery Conference in February. Because it’s a new year and because the presentation time is so tight (30 minutes!) I’m going to redo the existing dataviz presentation I’ve been giving entirely. I’ve got a new branch and everything.

I’m thinking of using World Bank data. At minimum, I’ll be doing something interesting and meaningful and, at best, I might end up on the World Bank data visualization tumblr.

I’m also super happy to be going to San Diego in February.

The Project (soon to be) Formerly Known as ng-grid

Posted on by Rob Larsen

I’ve go a bunch of small updates to share (and one big one.) I’m starting with this one since it fits with the Angular kick I’ve been on.

I’ve recently gotten involved with the ng-grid project housed under the Angular UI banner. We’re using it at work, so it makes sense for me to pitch in to help keep the project humming along as best it can.

We had a meeting last week and hatched plans to rewrite the core of the project in order to juice performance and ease development going forward. Working in financial services, grids are a big deal for me and I’ve never been a huge fan of any of the ones out there. I’m hoping this project will end up being the quality grid I’ve always been looking for as a consumer.

On the name

While it’s called ng-grid at the moment, it’s going to get a name change for 3.0, in order to better fit with the rest of the Angular UI stable. I’m not 100% sure of this, since we haven’t followed up since the meeting, but it will likely be UI Grid.

So, yeah. If you’re an Angular type and are looking for a data grid, I’ve got your back.

Easy Autocomplete with the datalist Element, the list Attribute and AngularJS’s ng-repeat Directive

Posted on by Rob Larsen

Continuing to write about some Angular features I’ve been working with recently, I thought it would be interesting to show off one of the little conveniences I was able to put together using just a couple of Angular’s core features. This example leverages basically two basic features of Angular and two HTML5 features to create an easy, live autocomplete widget.

The pieces we’re going to use are:

  • Angular models This demo shows how easy it is to use Angular’s models. While they can get more complicated, this example shows the base case of simply binding to a variable in scope, creating dynamic links that update the UI when the data is updated with minimal or no JavaScript interaction.
  • ng-repeat Obviously if you’ve worked with Angular for any length of time you’ll be familiar with this directive. Here, it’s used here to iterate over a collection and create a dynamic datalist element.
  • datalist and list The datalist element and the list attribute are the two new HTML features that bind up the autocomplete functionality. In short, the datalist element defines a list of autocomplete entries. The list attribute binds the datalist to an input using an id reference. This combination is available in Chrome, Opera and Firefox with buggy support in IE10+

Let’s see how it all goes together.

First up, since this is Angular, let’s look at the markup.

To get started you need to define an Angular controller. This one is named MainCtrl. Since this is just a simplified demo to illustrate the autocomplete widget, it doesn’t do much, but the code it’s pulled from is a form used to add information about individual comic books to a simple database. Many of those comics are from the same titles (Amazing Spider-Man, Action Comics, etc.) so it’s handy to have an autocmplete list in order to improve data entry efficiency.

The input itself has two pieces to note, the data-ng-model attribute and the list attribute.

Note: As always, I append data- to Angular attributes in order to follow the HTML spec’s directions on custom attributes. That’s just me. You might want to let it all hang out. Fair play.

The data-ng-model attribute defines the model for the input as the title property of variable named item bound to the controller’s scope. Interestingly, this variable doesn’t exist at run-time. Instead of erroring with a warning about item being undefined Angular handles the creation of it once the form is submitted. This is a great example of how casual Angular is with their models. There’s no ceremony here. Add the data-ng-model attribute and the binding is done for you. If the variable exists in scope, the input is bound to it. If it doesn’t exist, Angular creates it. That’s pretty sweet.

The list attribute references the datalist element that the input should look for in order to build value hints. The value of the attribute is the id of the datalist you want to use.

The button at the end of the form is the next interesting piece of markup. On that input we’re using data-ng-click in order to bind the addItem function to click events on the form. Angular will look for addItem in the global namespace and in the controller’s scope. As you’ll see in the JavaScript example, we’re safely binding the method to the controller’s scope. If you click this button after adding a new title the title will be immediately added to the autocomplete list.

Finally, there’s the datalist element which uses data-ng-repeat to build out the list of autocomplete options. This directive is bound to the titles property of the controller scope. Since Angular models are bidirectional, once you update $scope.titles this list will be updated with new values. Pretty sweet. As you can see, this datalist element has the id we referenced earlier in the list attribute of the input. 

<body data-ng-controller="MainCtrl">
    <form>
      <div>
        <label>title</label>
        <input type="text" data-ng-model="item.title" list="comicstitle">
      </div>
      <div>
        <input type="Button" value="Add" data-ng-click="addItem(item)">
      </div>
    </form>
  </div>
  <datalist id="comicstitle">
    <option  data-ng-repeat="ttl in titles" value="{{ttl}}"> 
  </datalist>
  </body>

Now that you’ve seen the markup, let’s look at the JavaScript that binds it all together. It’s actually pretty simple. Most of the wiring is done in the markup so the JavaScript is pretty sparse. That said, there are some interesting things so let’s look at it in-depth.

For starters we’re defining the starter list of titles as a variable on the controller’s scope. It’s a simple array bound to $scope.titles.

Have I mentioned that data binding is one of my favorite things about Angular? There’s absolutely no ceremony with creating a model. It’s just a variable on the controller’s scope (or even the global scope if you’re super lazy.)

After the model, the only other thing on the controller is a function called addItem.

This function is the method referenced in the data-ng-click attribute in the markup. It does a couple of things. The first is that is pushes the new value to $scope.titles. This example uses lodash’s contains function to test whether or not the title already exists in the array before it’s added. That keeps the list unique.

The next bit is another neat illustration of Angular models. Since the item we created when we added a title, setting it the object to an empty object is a shortcut to clearing the values out of the form. In this case there’s just the one form element, but if there were fifty or five hundred form elements with their data-ng-model attributes bound to properties of item this single line would clear all the values out of every bound form element. 

app.controller( 'MainCtrl' , function( $scope ) {
  $scope.titles = [ "Action Comics" , "Detective Comics" , "Superman" , "Fantastic Four" , "Amazing Spider-Man" ];
  $scope.addItem = function( item ) { 
    if ( !_.contains( $scope.titles , item.title ) ){
      $scope.titles.push( item.title );
    } 
    $scope.item = {};
  }
});

Check it out, in action, with this plunker.

And there you have it. Let me know if you have any questions, comments or corrections in the comments.

Sorting by Multiple Fields with Different Sort Orders Using the AngularJS orderby Filter

Posted on by Rob Larsen

I’m working on a new, interactive, interface to the $100,000 Club dataset I keep on itsalljustcomics.com. The data deals with the world’s most valuable comic books (those worth more than $100,000) and it’s always just been a static list. Since I’ve started keeping track of that data, the number of entries has steadily grown to include hundreds of books. Because of that growth and in concert with the relaunch of that site, I’ve started to put together an AngularJS based interface to the data. One of the things I need to do is sort the displayed data by three different fields and with two different sorts. This is very easy to do in Angular, you just might not know it since the documentation for orderBy is crummy. The one example that’s there is more confusing than it needs to be and only shows one variation.

Here’s how you do it.

ng-repeat

For starters, this example uses the ng-repeat directive. ng-repeat makes it trivial to loop through a collection and apply an HTML template to every member of the collection. In this example items is a collection of JavaScript objects stored in an array on the controller’s scope. 

app.controller('MainCtrl', function($scope) {
  $scope.items = [{
    "title":"Action Comics",
    "issue":"1",
    "publisher":"DC Comics",
    "cover_date":"1938-06-01",
    "pedigree":"Church",
    "grade_src":"Anecdote",
    "grade":"9.2",
    "general_commentary":"Best Existing",
    "uid":0}]
});

Every time through the loop the value of that particular item in the array is exposed to the template as it and any properties of that object are available using familiar dot notation. Those values are included in the template using a familiar bracket pattern and Angular recognizes them, interpolating the values into the page.


<tr data-ng-repeat="it in items">
  <td>{{it.title}}</td>
  <td>{{it.issue}}</td>
  <td>{{it.pedigree}}{{it.collection}}{{it.provenance}}</td>
  <td>{{it.grade_src}}</td>
  <td>{{it.grade}}</td>
  <td>{{it.general_commentary}}</td>
</tr>

You’ll notice I don’t just throw ng-repeat in there. While ng-repeat will work as an attribute, I prepend data- to the attribute name. While this doesn’t matter to Angular, it’s the way the HTML specification defines adding custom attributes, so I use that pattern for consistency across all my code.

This initial version would output something like the following:

out-of-order

orderBy

While the previous example works fine, the rows are simply presented in the order of the members of the array. That might be okay for you if you’re getting a presorted collection, but if you want to change the order, you have the power to adjust it using Angular’s orderBy filter.

In this example we want to sort by the title of the comic (ascending, from A to Z), then by the issue number (ascending, from 1-∞) and finally by the grade (descending, from 10.0 to 0.5.) While this is dead easy to do using Angular, it’s not clearly documented.

This is where the superheroics happen. Inside the ng-repeat you invoke the orderBy filter on your data. This is done with the pipe “|” character. This will be a familiar pattern for you if you’ve hacked around on the Unix command line. Everything on the left of the pipe is passed to the filter referenced on the right of the pipe. In this case we’re using orderBy but it could be any of the built-in filters or a custom filter of your own devising. Commonly, orderBy is shown accepting a single variable (technically an Angular expression) representing one of the available properties of the members of the collection. What’s not clear from the documentation or common examples is that is that it can also accept an array of properties with which to order the collection.1 So passing ['title','issue','-grade'] to orderBy tells Angular to sort first by the title, then by the issue and finally by the grade properties of the it object. Notice something interesting about the way grade is passed into orderBy? Yes, it’s that simple to reverse the sort in this syntax- simply negate the variable with “-” and the sort on that property will be reversed. Pretty sweet. 


<tr data-ng-repeat="it in items | orderBy:['title','issue','-grade']"> 
  <td>{{it.title}}</td>
  <td>{{it.issue}}</td>
  <td>{{it.pedigree}}{{it.collection}}{{it.provenance}}</td>
  <td>{{it.grade_src}}</td>
  <td>{{it.grade}}</td>
  <td>{{it.general_commentary}}</td>
</tr>

Here’s a plunker with a working demo. This example is basically Exhibit A of what I like about Angular and what drives me crazy about it at the same time. I love the elegant, declarative syntax that really does feel like HTML rewritten for web apps. I’m giddy over how powerful some of the pieces are- like the directives and filters we’ve seen here. The out of the box stuff is great and custom filters and directives are just such a powerful option.

And then there’s the documentation. While the documentation is mostly correct, and they provide some form of documentation for pretty much everything in the library, none of the documentation is great. It often reads like it’s written by an Angular engineer who’s been tasked with writing documentation- technically correct, but often bare-bones and written from an insider’s or experts perspective. For an example of the latter see the usage of the word “predicate” in the documentation of orderBy. Quick show of hands- how many of you know what that means? Obviously some of you will, but many of the self-taught hackers that populate the web developer sphere won’t and the documentation suffers because it often errs on the side of the experts.2

1. It can also accept a more complicated Angular expression or a function reference.
2. They promise it’s going to get better, so I’ll keep my fingers crossed, but I’ve been tempted to start an unofficial AngularJS documentation project on Github to do an end-around the 808 issues and counting in their issue queue (many of which are documentation PRs).