My New Book, The Uncertain Web, is Available in Early Release

rc_lrg

The raw stuff, available for your reading pleasure. I’m really excited to see what people think.

The best way to approach the web today is to forgo hard and fast rules and design for uncertainty. Embracing uncertainty as a core tenet of web development and scrapping the rules we’ve relied on in the past few years is the best bet for creating future proof web solutions. By combining web standards, progressive enhancement, a full technical toolbox, an iterative hybrid approach to design and development and embracing a desire to question the status quo and perceived wisdom, teams can create dynamic web sites and applications that should perform admirably in future devices, with unknown capabilities. By focusing on optimal solutions with intelligent fallbacks and forgoing the desire for absolute solutions design and development can work together to create the web of today and tomorrow.

Posted in Web | Tagged , , , | Leave a comment

Introducing My New Book: The Uncertain Web

Hey everybody, I’ve got a new book coming out. It’s called The Uncertain Web and it’s being published by O’Reilly. Right now we’re targeting a November release.

Just in time for Christmas. Buy a dozen and hand them out like candy canes.

If that seems like too long to wait, never fear. I’m about 50% finished and once that chunk has been hammered on by a hand-picked team of geniuses, it will be available as an Early Release.

So, what’s it all about?

Here’s my original pitch:


The Uncertain Web

The best way to approach the web today is to forgo hard and fast rules and design for uncertainty. Embracing uncertainty as a core tenet of web development and scrapping the rules we’ve relied on in the past few years is the best bet for creating future proof web solutions.

In the early 2000s, there was basically one browser (Internet Explorer 6,) one platform (Windows XP) and one screen resolution (1024 by 768) that mattered. With that set up you could design, develop and test against the vast majority of web users with one desktop computer. The biggest question on the horizon, it seemed, was when it would be viable to design for 1280 pixel screens

This limited field of play meant that there was an expectation that sites and applications would look the same everywhere for everyone. Best practices were honed and codified into hard and fast rules which drove design and development. Major choices, such as the size of the basic grid to design for, were no longer choices. You started with a static, 960 pixel grid and sliced and diced it as needed.

Today, things couldn’t be more different. With the launch of the iPhone and the iPad, the rise of Android and the growth of not just one, but two real contenders to Microsoft’s position as the dominant web browser (Firefox and Chrome), developers and designers have an ocean of variables to navigate. Every question around a site design is now pregnant with options.

Initially, developers and designers tried to navigate this new reality by creating new rules.

The problem was, the goalposts kept moving. As soon as a new hard and fast rule was created, some new wrinkle would render it impotent.
People designed and built “iPhone” sites, assuming that Apple’s dominance in the smartphone market was somehow a permanent condition. They tested for touch capabilities and assumed that touch users would never have a mouse.

As Android’s huge growth over the past few years, and the presence of devices like the Chromebooks and Windows 8 laptops with both mouse and touch capabilities have proved that those new rules have a short shelf life.

Even patterns like Responsive Web Design, which some saw as a single solution for design and development moving forward fall apart when applied against complicated application patterns and the vagaries of bandwidth and mobile performance.

By combining web standards, progressive enhancement, an iterative approach to design and development and embracing a desire to question the status quo and perceived wisdom; teams can create dynamic web sites and applications that should perform admirably in future devices, with unknown capabilities. By focusing on optimal solutions with intelligent fallbacks and forgoing the desire for absolute solutions design and development can work together to create the web of today and tomorrow.

This book will outline both the concept and underlying principles of the Uncertain Web and introduce some of the techniques necessary to make the successful transition.


So, that’s the thing.

I’m really excited about this one. I hope you enjoy it.

Posted in Web | Tagged , , , | 3 Comments

You’re So Smart You Turned JavaScript into xHTML

Have you ever seen an error message like this one? XML parsers are designed to call it quits as soon as they encounter an error. XML grammar is strict, so the parsers need to be strict too. This particular error is from an & in a URL. &s in URLs are super common occurrence, but if you throw one into a plain XML document, without inserting it as a character entity (&) or numerical character reference (&) you get an error.

Sweet.

xml-error

To be honest, I’m not a huge XML hater. I’m a technical guy so ensuring that some of the documents I create adhere to a technical standard doesn’t kill me.

I mean, Ant.

That said, I don’t like the way XML’s strict error handling inserted itself into the web with xHTML. A good part of the reason the web blossomed the way it did was because the language underpinning it, HTML, it was so forgiving people could get by without knowing what they were doing. My initial HTML was so bad, I’m surprised I ever did anything.

But yet, I did.

And I’ve been building sites since. 17 years and counting! All because I could create a tag-soup mess that, kinda/sorta did what I wanted it to do and the browsers at the time did their best to try to make sense of it. That was great.

XML/xHTML took that possibility away. Which is why, while many people (including myself) wrote xHTML documents, we didn’t actually serve them as XML. The danger of having a catastrophic error was too great, so all but the hardiest developers basically treated xHTML like HTML 4.0, just with XML style syntax.

Which is kinda’ goofy.


Remember when they said “Users should not be exposed to authoring errors?”

The group that went on to found the WHATWG (the group responsible for the current HTML specification) recognized that this was a serious flaw and put a specific reference to this issue with xHTML in their proposal for the future of web application development. It was one of their seven guiding design principles. In case you’re unaware of the history, that paper is the foundation of the modern HTML specification. One piece of that foundation is as follows:

Users should not be exposed to authoring errors

Specifications must specify exact error recovery behaviour for each possible error scenario. Error handling should for the most part be defined in terms of graceful error recovery (as in CSS), rather than obvious and catastrophic failure (as in XML).

These are concepts embedded in the very heart of modern web development.

  • Avoid catastrophic errors.
  • Allow for graceful error recovery.

Why, then, do so many people create web applications that rely so much on JavaScript that they catastrophically fail? The web, at its best, should be resilient. Nothing warms my heart more than a 20 year old page that’s still kicking, a 10 year old link that redirects properly onto a completely new domain or platform or a modern site that can serve something useful to a 15 year old browser. To me, that’s the web at it’s best.

The opposite end of that spectrum, from my perspective at least, are the sites that do nothing at all without a relatively modern (or, in some places, completely modern) browser with JavaScript turned on and all dependencies (many of which are on third-party domains) loaded. While you can make an argument that JavaScript should be a hard requirement for certain kinds of applications (I’ll give you games written in the Canvas 2d API, for example,) I don’t think it’s the way of the web to have your site fail and show a blank screen because some third-party dependency doesn’t load, JavaScript is turned off or because your developer left a trailing comma in a JavaScript object and didn’t test in Internet Explorer.

I understand that creating a completely static version of your site or application is impractical. Granted. Still, showing nothing if there’s a problem is just terrible. But yet, people are happy to do it.

Client side JavaScript all the way

Nicholas Zakas has talked about this a little bit with his presentation, Enough With the JavaScript already. We’ve moved so far towards JavaScript in some circles that some people are starting to use tools like Backbone even in use cases that don’t require complicated interactions, like a static content site. Whether it’s a case of being obsessed with the new-shiny or simply not knowing any better, people are parsing and rendering entire pages in the client, even when there’s no real dynamic content. Beyond the fact that that’s going to be slower (why would you serve 1MB of JavaScript/JSON to render 100kb of HTML when the server can just serve the parsed/rendered 100kb to begin with?) you’re just setting yourself up for catastrophic failure if, for example, your static content server goes down and all that fancy Backbone code disappears.

What’s wrong with progressive enhancement?

Nothing, that’s what. Just because we can do everything in JavaScript doesn’t mean we should. In fact, the more complexity and dependencies we bring into the front end, perversely, makes the danger of something catastrophic happening even more likely. Third party scripts, web service calls, browser bugs, users with JavaScript disabled and your own application errors can all cause catastrophic failures if you’re not careful. That’s why, in my mind at least, the principles of progressive enhancement are more important now than they’ve ever been. Building solutions that are robust and resistant to changes in the environment (including many that are going to be forever beyond your control) is going to ensure that you can always reach your audience.

Your users don’t care how fancy your stack is. They care that they’re getting their content, that they’re getting it as fast as possible and it appears consistently from visit to visit. If your stack is getting in the way of any of those goals then you need to revisit the way you’ve designed your web solutions.

Posted in Web | Tagged , , | 4 Comments

Let’s Talk About your Billable Rate

I’ve got a few topics relating to my experience at agencies that I’ve been looking to share for some time. Now that I’m going my own way as a consultant with the goal of maybe expanding on that as well as somehow helping out at an agency again; I’m going to start to work through different agency related subjects as time allows. Hopefully you’ll enjoy this stuff.

I’ve related the following (albeit in much less detail) to people who work at agencies and a lot of them really hadn’t thought about the way that their employer makes money off of their services.

It can be an interesting revelation.

When you work for an agency your talents are for sale to clients. While the relationship is marketed in another way (agencies bid for projects and/or try to gain permanent relationships with clients) this is the basic financial currency of agency life- providing bodies to clients either directly or hidden within the completion of a project. There are basically three ways that your clients pay for your services. One of them ties directly to the number of hours you work. Another feeds into the internal agency accounting in ways that can be detrimental to your health. The third can just be boring. I’ll define them and then I’ll let you in on a little secret about the way that the agency views your relationship with each of those models.

  • Time and Materials

    In this model, the agency bills the client for your time and expenses when working on a project. For example, if you’re, say, an Art Director at an agency[1], the company might charge something like $200/hour for your services. If you then worked 40 hours in a given week, the company would charge $8,000 for your time.

    This model is useful because, generally[2], the agency gets paid for the work they’ve done. There will be an estimate done at the outset of the project, but if the project goes, say, 10-20% over budget the client can still be charged for those hours.

  • Fixed Cost or Fixed Price

    In this model, the agency provides a fixed estimate of the cost of the project. Working through a gigantic spreadsheet (or something web-based that does the same thing as Excel) with limited information, the agency team will put together a proposal that’s supposed to predict, with pinpoint accuracy, how much the project will cost- including enough margin left over to keep the lights on.

    This model is great for the client, since they know how much the final bill will be. This model can be great for the agency, especially if the project comes in under estimate. The thing is, that almost never happens and when the reverse is true and the project spirals out of control it can quickly turn into an emergency situation. Just like a startup working through their pot of money, projects have a burn rate, which indicates the daily/weekly/monthly resource usage on a project. When the burn rate indicates that a project is going to bust the budget, step back, grab some popcorn and watch as executives go into disaster recovery mode.

  • Staff Augmentation/Dedicated Teams

    In this model the client buys your exclusive services for a period of time. This is not project based and can last for many years[3]. This can be on or off-site. If it’s on-site it’s basically like you’re an employee of the client company (except you have to kiss everyone’s ass and get none of the perks of the client company.) If it’s off-site it’s basically an arrangement that means that important staff can’t be pulled away from one client to serve another need within the agency.

    This model can work for both sides as it represents a steady stream of income for the agency (albeit potentially a discounted one) and the client can flexibly fill positions with skilled staff.

Now it’s time for the secret. While your boss, their boss or even their boss’s boss (all the way up to the guy or gal that runs the place) might care about your well-being, the Agency, as a capitalist organism, does not. The Agency as a capitalist organism doesn’t believe that you’re a special little snowflake. The Agency thinks you’re a replaceable cog and doesn’t care about how many awards you’ve won or how clever your code is. With that in mind, with any of the models the agency is almost always healthier in the short term if you work 50-80 (or more) hours a week[4], every week until you quit, die or retire.

In reading this, if you’re unfamiliar with the way that agencies work, you should keep in mind that agency deadlines are pretty psychotic.

The folks that interface with clients would rather chew broken glass than tell a client that a deadline is going to be missed. Missing deadlines hits the wallet and (worse) the reputation and relationship with the client. It’s not something you want to do ever. So whenever a deadline is approaching the project team goes into “whatever it takes” mode to get the work out the door.

So, with each of the models, you being willing to put in 80 hours a week, sometimes for months at a time, is vital to the success of the organization. This is both down to the danger of missed deadlines and also down to the pure math of being an agency employee.

Lets look at how that math works with each of the models.

  • Time and Materials

    Cutting through any potential confusion, if you’re working 80 hours a week on a time and materials project you are getting royally screwed. Let’s take the example of the $200 an hour Art Director. To keep the math round, assume the Art Director makes $125,000 a year plus $25,000 in various other compensation (health insurance, 401k match, etc.) That works out to about $72 an hour[5] for a 40 hour workweek. That adds up to $2880 a week. So, how much do you get paid if you work 80 hours that week? $2880.

    What’s your effective hourly rate? $36 an hour. But yet, the agency is going to bill $16,000[6] for the full 80 hours. You’ve increased the company’s margins by a wide margin with your dedication.

    You win free pizza at your desk for dinner, stress related hives, and an angry spouse.

    There’s also a morale cost with this model. If you’ve got pure freelancers working alongside agency employees and both sides are working late nights, the freelancers are invariably going to be happier about the situation since they’re actually getting paid extra for their effort. Trust me, it sucks when you’re stuck in the office at 9:30 and the freelancer next to you is whistling while he works thinking about the new addition he’s going to put on his house.

  • Fixed Cost/Fixed Price

    Fixed cost projects only go awry, from an accounting perspective, when the agency needs to add more bodies to a project. Otherwise, with the planned number and makeup of the team, the only thing that matters to the Agency is whether or not the deadline is hit. To that end, if you need to sleep at your desk for two weeks straight and work weekends for six months, then it’s all good.

  • Staff Augmentation/Dedicated Teams

    Your number one job in this kind of role is making sure the client is happy. Depending on the engagement this can, of course, include long hours. That said, I’ve seen examples where these can be the healthiest situations in terms of the number of hours you have to put in every week and the pace of your day.

    There are other downsides to being in one of these roles, but generally it‘s not the hours or the pace. The biggest is the common refrain of, “If I wanted to work at Client Company for the next five years I would have applied for a job at Client Company.”

    That said, in terms of hours you can get lucky in these kind of engagements.


[1]
The weight of titles can vary, as can bill rates across agencies so don’t linger too long on the price quoted here. I’m sure there are folks with the title Art Director that bill out over four figures and others that bill out much less than $200. It’s just a nice round number.

[2]
generally.

Projects can be 50-100% (or more) underestimated, and in those situations it’s tough to keep asking for more.

Imagine a project budgeted for three years of development at, say, $1,000,000 a year. Then imagine that $2,000,000 worth of work is done in the first 6 months… That’s capital T trouble and there will be some awkward C-suite type conversations.

[3]
I’ve seen people who were embedded for more than five years at client organizations. Something like this: “I got into the office for my first day. they told me to go to client x later that afternoon and I’ve been here ever since. That was in 2007. I got back to the agency office 2 or 3 times a year.”

[4]
Some agencies have slightly different DNA that values talent as talent and wants to cultivate it and keep it from burning out. If you’re at one of those places you’re winning.

[5]
Keep in mind that the gap between $200 an hour charged out to the client and the $72 in total compensation per hour isn’t the entire story on margins in the industry. There are a lot of non-billable people in an agency and there are always billable people who aren’t billable at any given moment. There are a lot of slices to be taken out of that $128 difference. So many, in fact, that margins can be down in the single digits at some shops. I don’t have encyclopedic knowledge of margins across the breadth of traditional and digital agencies, but I don’t think they get much higher than maybe 15-20% anywhere

[6]
No, this doesn’t always happen. Sometimes an agency will eat the extra hours to maintain the client relationship or because it was your, or the agencies’ fault that you had to work 80 hours in the first place. That said, many times, if the numbers are okay, then $16,000 it is.

Posted in Web | Tagged , , , | Leave a comment

Notes from the Interview Front

Interrogation Room #5

Sadly, I won’t be sharing a bunch of funny anecdotes from my time on the interview trail. While that might be fun, there are some bigger trends that I wanted to talk about after having reviewed dozens of job descriptions and talking to dozens of people at one level or another over the month of March.

AngularJS is the new boss

At least in terms of green field development it seems like no one is talking about anything other than AngularJS right now. My year and a half’s worth of Angular experience set off alarm bells for many recruiters. Backbone comes up, but not with as much frequency. Several people were moving from Backbone to Angular. Nothing else in the MV* space was really in the running.

Lots of smart people (including many that I know) are still pouring a lot of time and effort into Backbone, so it’s clearly not going anywhere. It is weird, however, as Angular seems to be winning developer mind share no matter what the regular front end engineering taste-makers might think about it. I’ve had a lot of conversations with people (including some big names) where they’re completely flabbergasted that anyone is even using Angular, forget about the fact that it’s storming the castle.

It’s definitely storming the castle.

In the course of writing this post, I saw a bell clear sign of Angular’s rise. I saw an “Angular expert” job posting from a company who had spent a ridiculous amount of time and money (hundreds of thousands of dollars worth of developer hours) crafting a “perfect” Backbone based application development stack. I’m sure they’re still quite fond of Backbone, but it was interesting to see them hedging their bets, trying to get some Angular expertise on staff.

Start-ups are spreading like wildfire.

The Boston start-up scene is pretty nuts. I’m not generally interested in start-ups, but as part of my “talking to everyone” thing I ended up talking to a few of them. I remain mostly uninterested from a full-time perspective, although I did get up to the brink with one group that seemed pretty cool.

There is a lot of opportunity out there for someone like me to help them out on a short-term basis, however, so in my new life as an indie consultant, I might go back into the fray.

Start-up Geography

At this point start-ups are no longer crowding around South Station. They’re in South Boston proper. It’s a walk to get over there these days.

Bocoup might be leading a smart charge over to North Station.

Agency vs. Startup/Product

When I interview at agencies I don’t do tech screens. No one has asked me to do anything like that in 4 years. The shared experience of the agency existence and the number of people I usually have in common with agency folks means that the “tech” portion of interviews usually doesn’t go beyond philosophy and general experience. For most agency people knowing the level I’ve been at the places I’ve worked is a pretty good indicator that I know what I’m doing.

Product people, on the other hand, are super wary of my experience. They seem to think that because I’ve been a manager and been in charge of teams that I don’t know how to code. Some of them have even made the assumption that I don’t want to code (yes, I do open source software to torture myself.)

Interviewer: “So you’ve been a manager, running teams for a while. You haven’t really been coding since 2007 then?”

Rob: “No, unless today is opposite day. I’ve worked on probably 20 sites or applications in that time. Some with thousands of lines of JavaScript and incredibly complex UIs.”

Let me set the record straight on that assumption. Being an engineering manager at some companies might include no coding/all meetings, but at an agency the pendulum usually swings way too far in the other direction.

A technology manager at an agency will probably be doing coding/architecture most of the time and will not have any time officially allotted to do anything strategic or to work with their team on anything other than project work. .

Even if their target allocation is something like 60% billable work, they will probably be fully allocated to client work. Occasionally, they will be over-allocated to a client (or two.) Oftentimes, soft-skills “manager stuff” happens in hours 40 and above (or 80 and above.)

Personally, at one point a couple of years ago I was allocated 100% to two separate divisions at the same investment bank for six weeks. Not a lot of time to get soft when you’re scheduled to work 80 hours a week :)

This is actually a systemic problem at agencies. There’s not a lot of room for people at even the director level and above to really nurture staff. There are exceptions, of course. If you look around and spot stable teams at different agencies you can usually spot a guy or gal at the top who makes a point of keeping their staff happy, but the pressure to do that is not coming from up on high. The pressure from up on high is to keep your billable rate as high as you possibly can in order to justify your continued presence at the agency.

To that end, senior people at agencies are constantly getting their hands dirty. No one is hiding. I know start-up people think their experience is special, and in a lot of ways it is, but the experience on the agency side isn’t any easier.

To be perfectly honest, looking at someone from an agency with a good engineering culture and thinking they’re going to be soft is insane. That person has probably built up and torn down more architectures than most people do in a career in the product world.

There is a reverse bias

Agency people don’t want to hire people who don’t have agency experience. This is undeniably true. There’s a fear that you will hire someone, they will walk in the door, see the maelstrom and walk back out the door.

That happens, so it’s a valid concern, even if I don’t put as much weight into it as some people do. For my money, I’d rather have people get the shot and fill the role than play gatekeeper just because someone is coming from another industry. Sometimes it turns out really well.

Front-end engineering isn’t, universally, a first class discipline (although it’s getting closer to being one)

Something similar to the following happened twice when interviewing for very senior architect positions (including one that was titled something like “Front End Architect.”)

Hiring manager: “We want an architect whose focus is on the front end. You’ll lead projects, working with other developers and vendors, but we really want someone who can make a difference with the user interface layer. We see it as a game changer for us. ”

Me: “Sure, great. I’d love to come in”

Rob interviews

Feedback: “Rob was really strong on the front end, but he wasn’t as strong as we want him to be with Java/C#”

(So you actually want a Java guy? You want a Java guy who’s also an expert with JavaScript? Does such an unholy chimera even exist? )

The thing that drives me crazy about this scenario (beyond the fact that it’s a waste of my time) is that I’ve worked with architects who knew so little about JavaScript that they didn’t know enough to leave the JavaScript alone. I’ve seen very smart people completely break the UI because they were completely clueless about JS and didn’t even realize the depth of their cluelessness.

Up until they start breaking things, I’m cool with this. As long as folks delegate I don’t want or expect someone to be an expert across the whole system or to be able to code the whole system. I wouldn’t trust many of the absolute geniuses I’ve worked with to code the front end of any system. Just like they wouldn’t trust me to code up the services layer or come up with a database schema. But yet, in many organizations, you are simply not allowed to be in charge if you don’t have significant back-end experience. It doesn’t matter that the heavy lifting is happening in the browser now. It doesn’t matter that the potential advantage is in the user interface because there’s still an incredible shortage of strong front end engineers. It doesn’t matter than getting people to be effective at their jobs is a skill that stretches across technology domains. If you’re not a Java or C# expert, in some places at least, you still can’t straddle domains and run the show.

It is getting better

The good news is, the previous used to be universal. It’s no longer. I did interview for two general technology leadership roles (Director/VP) and those were judged on their own merits. No one once gave me the stink eye because I’m not a reformed Java developer.

Random notes

The Interrogation

  • I don’t like people who do tech screens like it’s an affront to their honor that you would even think you are good enough to work at their company. This doesn’t happen too often, but it does happen and I fill up with quiet rage pretty quickly when I encounter one of these people. When I run tech screens, I want to fill the role. I’m not happy when people aren’t up to snuff. I’m truly disappointed. It seems to me like these people would be happier to keep the role unfilled. That’s the absolute wrong way to approach the interview process. My reaction is invariably: “You don’t want me here? Cool, I won’t work here.”
  • I’ve been contacted by 28 separate recruiters in six weeks for various jobs at one company. I want to opt-out permanently please? I wish there was a central registry to opt out of contact by company.
  • I’ve got a hint for people who have been trying to fill roles for a six months or a year and can’t find anyone. Pay more money. Instead of losing out on some opportunity waiting to get someone at a 50th percentile wage and fitting it into some budget, push the pay envelope by 10% or more and people will be a lot more receptive. There are very few job descriptions I’ve seen in the Boston area, at any level, that had an aggressive (for the level) salary attached. There are some, but not many (maybe 10%.) Everyone else seems to think that being able to use new technology and getting free soda and coffee should be enough of a selling point to get you to choose them in the middle of one of the hottest skill sectors in the job market. The salaries seem to slot into very clear strata and there are few outliers. I know people are afraid because money is money and front-end developers should be worth $xxx. But if you have a need now, paying aggressively is one way to fill it that’s within your control. Everything else is outside of your control. Pay more than your competitors. Everyone is using Angular or other modern architecture. Everyone gives out free snacks and coffee and all that crap. You can’t win with that stuff.


And that’s it for now. I may add onto this as other trends percolate around in my brain. For the most part I enjoyed the conversations I had even if nothing turned out to be a perfect fit for me and I hope that people on the other side of the desk felt the same.

Posted in Web | Tagged , , | 1 Comment

Current Status: Independent Consultant

As many of you know, my run at Wellington ended at the end of February. My contract wasn’t renewed for reasons beyond my control and that was that.

No more suits, sadly.*

One of the big problems with that contract ending was that the timing of it was a little abrupt. I found out just a couple of hours before I got on stage to speak at jQuery conference which was, in turn, just over two weeks away from what would turn out to be my end date. Since the time frame was compressed and I wasn’t really in the “new job” mindset to begin with, I ended up going into the first month of my job search with the wrong attitude. That wrong attitude was that it was okay to talk to people because, at the end of the day, I wasn’t working anyway, so why not hear people out?

It turns out that talking to as many people as I talked to in March is actually a pretty crummy idea. While I had good conversations with a few companies and actually got pretty close on a couple of roles that made some sense for me at this stage of my career I ended up burning myself out on the interview process before the month was over.**

In hindsight, I should have chosen my conversations more wisely, written a bunch and done a ton more open source work. Oops.

Anyway, in the back of my mind, as I was doing all of this interviewing and listening to everyone who wanted a moment of my time I was nurturing the thought that I would be happier just doing my own thing as an independent consultant for a while. I’ve always threatened to start my own agency and while this isn’t the way I imagined doing it, it is a way to do it, so the option was one that I’d thought about for a long time. It might not be my own version of Sterling Cooper Draper Pryce (even if I’ve been know to dress like a modern Don Draper), but it’s still an opportunity to help clients with tough problems and do so under my own banner. That’s pretty cool by itself and if it turns into something bigger down the road, then even better. And if it doesn’t work out? That’s cool too. At least I gave it a shot.

So, to make a long story short, unless someone blows me away with a great role I’m going to be a sword-for-hire for the foreseeable future. Sure, I will listen to people that have really great engineering leadership roles, but I’m going to focus more on doing consulting and training on my own. To start, as I’ve already got an engagement that will keep me busy for the next few weeks, I’ve got a small page outlining the services I provide. I’ll be creating a more official presence as time allows, but for now that page is good enough to give you a sense of what I’m available to do.

So, hire me if you’ve got the need.


* Stated without irony. I like the suits.

** I actually had some help on that, but more on that in a subsequent post. I’ve got a couple of anecdotes to share about the interview process as I’ve witnessed it, but that will come in the next couple of weeks.

Posted in Web | Tagged , | Leave a comment

I Made a Scatter Plot Data Visualization with AngularJS and SVG. Here’s How I Did It.

This post is a longer examination of the new scatter plot visualization I added to my upcoming data visualization presentation for jQuery Conference. As with the other Angular examples I’ve posted recently, this one is based on the data I keep about the world’s most valuable comics. This particular visualization lives on the page I keep detailing record comic book sales. Take a look. It’s pretty cool, especially if you’re fascinated by comic books that have sold for more than $100,000.

For reference, it looks like the following. It’s 20 years of comics sales plotted horizontally with their sale prices (ranging from $100,000 to $2,500,000) plotted vertically. You can see a couple of clusters of activity based on different events in the marketplace.

chart

So, how did I do it? And why did I do it this way? D3 actually has a few scatter plot examples out there, so I could have used one of those, but as I thought about it I decided I wanted to do something hand-rolled. A scatter plot is actually pretty easy to map out and, as an added bonus, I could lay out the basics of the chart in Illustrator, which allowed me to demo the fact that SVG can be authored like any image by a graphic designer and then manipulated with JavaScript and CSS.

So let’s take a look at the particulars.

First we’ll start with the markup. This is Angular, after all. I’ve simplified the markup a bit, taking out some of the elements representing grid lines. Check the source on Github for all the details.

The interesting bits, from the Angular perspective, happen around line 30. The core of the visualization is in the circle element. It’s an ng-repeat that goes through the items collection in the controller’s scope. It’s filtered by the venues property.

Take note of the prefixes on the next set of attributes. For starters, as I normally do, I prepend data-* to Angular attributes as that’s the way custom attributes are defined in the spec. I like to play by the rules. You’ve probably seen that before. What’s probably not so familiar is the use of the additional ng-attr-* prefix. If you’re familiar with Angular you know you should just be able to do fill="{{colorPicker(it.venue)}}" and Angular should evaluate the results of the colorPicker method and insert it in as the value of the fill attribute. This doesn’t work with SVG elements. Some browsers will parse the entire SVG element before Angular has a chance to evaluate the SVG attributes. So if you put n Angular expression inside an SVG attribute it will throw an error. ng-attr solves this by binding the attribute solely to Angular’s discretion. Here, I’m using this pattern three times. The fill attribute, which provides the color coding for the different venues s these comic books have sold (with the handy colorPicker method that accepts the it.venue as an argument) , and the cx and cy attributes which indicate the center of the circle.

The cx and cy attributes are passed through custom filters which transform the data into usable x and y coordinates. I’ll talk about those in the section on the Angular filters. Following the filters there are two mouse events used to show and hide a simple (svg) tooltip.

The tooltip is in the next section. Doing boxes in SVG isn’t quite the same thing it is with HTML. You can’t just throw a box in there and have text flow inside the box like you can with a DIV or something, so this would have been easier with an HTML tooltip. That said, I wanted to do this in SVG so it’s all in SVG. In this case I’m doing it with a solid box with several text elements layered on top. All of these tooltip elements show based on the tooltip model on the scope. If the price is truthy, the tooltip shows. Otherwise, it's hidden. For each text element and the rect element, I'm using x and y coordinates generated with the same Angular filters in use on the circles and using the transform attribute to move the element around based on that common x/y. Each of the text elements also expose the data from the model.

Following the tooltip code, in the 'legend' g element there are several rect elements which allow for quick filtering by sales venue. ng-mouseover and ng-mouseout are used to update the venues filter.

That's the markup. SVG isn't that scary, is it?

 <div data-ng-app="comicsApp">
  <div data-ng-controller="chartCtrl">
    <svg version="1.1" xmlns="http://www.w3.org/2000/svg"
    xmlns:xlink="http://www.w3.org/1999/xlink"
    width="980" height="600" viewBox="0 0 980 600" xml:space="preserve">
      <g id="lines">
        <line fill="none" stroke="#000000" stroke-miterlimit="10"
         x1="90.828" y1="-8" x2="90.828" y2="579.297"/>
        <line fill="none" stroke="#B5B5B6" stroke-miterlimit="10"
         x1="0" y1="579.297" x2="981.996" y2="579.297"/>
        <line fill="none" stroke="#595858" stroke-miterlimit="10"
         x1="734.275" y1="49.246" x2="734.275" y2="579.161"/>
        <!--about 50 lines cut-->
      </g>
      <g id="text">
        <text x="10" y="16">price in USD</text>
        <text x="71" y="574">$0</text>
        <text x="480" y="595">2003</text>
        <text x="78" y="595">1993</text>
        <text x="881" y="595">2013</text>
        <text x="25" y="553">$100,000</text>
        <text x="25" y="468">$500,000</text>
        <text x="16" y="363">$1,000,000</text>
        <text x="16" y="256">$1,500,000</text>
        <text x="16" y="151">$2,000,000</text>
        <text x="16" y="45" >$2,500,000</text>
      </g>
      <g id="dynamic">
        <circle  data-ng-repeat="it in items | filter : venues"
         opacity="0.8"
         data-ng-attr-fill="{{colorPicker(it.venue)}}"
         data-ng-attr-cx="{{it.date | xDate}}"
         data-ng-attr-cy="{{it.price | yPrice}}"
         data-ng-mouseover="updateTooltip(it)"
         data-ng-mouseout="updateTooltip(0)" r="7" />
        <rect data-ng-show="tooltip.price" data-ng-model="tooltip"
         transform="translate(-55,-100)"
         data-ng-attr-x="{{tooltip.date | xDate}}"
         data-ng-attr-y="{{tooltip.price | yPrice}}"
         width="150" height="80"
         fill="white" style="filter:url(#drop-shadow)" />
        <text data-ng-show="tooltip.price" data-ng-model="tooltip"
         transform="translate(-45,-90)"
         data-ng-attr-x="{{tooltip.date | xDate}}"
         data-ng-attr-y="{{tooltip.price | yPrice}}">
         {{tooltip.title}} {{tooltip.issue}}</text>
        <text data-ng-show="tooltip.price" data-ng-model="tooltip"
         transform="translate(-45,-75)"
         data-ng-attr-x="{{tooltip.date | xDate}}"
         data-ng-attr-y="{{tooltip.price | yPrice}}" >
         {{tooltip.pedigree}}{{tooltip.collection}}{{tooltip.provenance}} {{tooltip.grade_src | srcFilter}} {{tooltip.grade}}</text>
        <text data-ng-show="tooltip.price" data-ng-model="tooltip"
         transform="translate(-45,-60)"
         data-ng-attr-x="{{tooltip.date | xDate}}"
         data-ng-attr-y="{{tooltip.price | yPrice}}" >
         {{tooltip.price|currency}}</text>
        <text data-ng-show="tooltip.price" data-ng-model="tooltip"
         transform="translate(-45,-45)"
         data-ng-attr-x="{{tooltip.date | xDate}}"
         data-ng-attr-y="{{tooltip.price | yPrice}}" >
         {{tooltip.date}}</text>
        <text data-ng-show="tooltip.price" data-ng-model="tooltip"
         transform="translate(-45,-30)"
         data-ng-attr-x="{{tooltip.date | xDate}}"
         data-ng-attr-y="{{tooltip.price | yPrice}}" >
         {{tooltip.venue}}</text>
      </g>
      <g id="legend">
        <rect x="125" y="5" width="15" height="15" fill="#D95B43"
         data-ng-mouseover="venues='comic connect'"
         data-ng-mouseout="venues=''"></rect>
        <text x="145" y="15" class="legend">Comic Connect</text>
        <rect x="245" y="5" width="15" height="15" fill="#ECD078"
         data-ng-mouseover="venues='heritage'"
         data-ng-mouseout="venues=''"></rect>
        <text x="265" y="15" class="legend">Heritage</text>
        <rect x="325" y="5" width="15" height="15"
         fill="#C02942" data-ng-mouseover="venues='comiclink'"
         data-ng-mouseout="venues=''"></rect>
        <text x="345" y="15" class="legend">ComicLink</text>
        <rect x="405" y="5" width="15" height="15" fill="#542437"
         data-ng-mouseover="venues='pedigree'"
         data-ng-mouseout="venues=''"></rect>
        <text x="425" y="15" class="legend">Pedigree</text>
        <rect x="485" y="5" width="15" height="15" fill="#53777A"
         data-ng-mouseover="venues='metropolis'"
         data-ng-mouseout="venues=''"></rect>
        <text x="505" y="15" class="legend">Metropolis</text>
        <rect x="565" y="5" width="15" height="15" fill="#69D2E7"
         data-ng-mouseover="venues='jp'"
         data-ng-mouseout="venues=''"></rect>
        <text x="585" y="15" class="legend">JP/Mint</text>
        <rect x="645" y="5" width="15" height="15" fill="#FA6900"
         data-ng-mouseover="venues='mastronet'"
         data-ng-mouseout="venues=''"></rect>
        <text x="665" y="15" class="legend">Mastronet</text>
        <rect x="725" y="5" width="15" height="15" fill="#FE4365"
         data-ng-mouseover="venues='pgc'"
         data-ng-mouseout="venues=''"></rect>
        <text x="745" y="15" class="legend">PGCMint</text>
        <rect x="805" y="5" width="15" height="15" fill="#666666"
         data-ng-mouseover="venues='unknown'"
         data-ng-mouseout="venues=''"></rect>
        <text x="825" y="15" class="legend">Other/Unkown</text>
      </g>
    </svg>
  </div>
</div>

Now let's take a look at the JavaScript. First up is the controller. This is pretty simple as far as these things go. $scope.items is first populated with data provided by an Angular factory. The factory is a simple $http.get so I'm not showing it here. I just organized the data request into a factory so two controllers could share the same code for getting comics data. Following that, the $scope.tooltip model is set with a single property, price set to zero. This falsy value ensures that the tooltip we previously defined in the HTML isn't shown by default. Following that is a small method designed to update the tooltip model with data representing the currently selected comic book. Finally, there's a small method and related array used to choose colors for the different venues. The method accepts a string argument and matches against a list of known venues, returning the proper color.

That's the whole controller.

angular.module('comicsApp.controllers', ['comicFilters','comicsFactories']).
  controller('chartCtrl', ["$scope",'dataService',function( $scope, dataService )  {
    $scope.items = dataService;
    $scope.tooltip = {
      price:0
    }
    $scope.updateTooltip = function(it) {
      $scope.tooltip = {
        price:it.price || 0,
        venue:it.venue,
        date:it.date,
    	 	title:it.title,
    		issue:it.issue,
    		pedigree:it.pedigree,
    		collection:it.collection,
    		provenance:it.provenance,
    		grade_src: it.grade_src,
    		grade : it.grade
      }
    }
    $scope.colorPicker= function( venue ){
      switch (venue) {
        case "Heritage":
          return $scope.colors[0];
        case  "Comic Connect":
          return $scope.colors[1];
        case "Comiclink":
          return $scope.colors[2];
        case  "Pedigree":
          return $scope.colors[3];
        case "Metropolis":
          return $scope.colors[4];
        case  "JP The Mint":
          return $scope.colors[5];
        case "Mastronet":
          return $scope.colors[6];
        case  "PGCMint":
          return $scope.colors[7];
        default:
          return $scope.colors[8];
       }
    }
    $scope.colors = ["#ECD078","#D95B43","#C02942","#542437","#53777A","#69D2E7","#FA6900", "#FE4365","#666666"];
  }
]);

Now let's take a look at the filters. As I mentioned earlier, this particular visualization isn't generic, so these are only useful in the context of this particular implementation. Which is fine. I'm not doing a million of these.

The first, xDate, defines the x axis. It takes the date (in the form yyyy-mm-dd) and splits it into and array containing the years, months and days. I calculate the total number of months from 1993 by subtracting 1993 from the year and then multiplying that number by 12. I then adding that number to the number of months from the original date array. After that the numbers of months is fudged into a pixel measurement as the offset of the left of the chart, 90, is added to the number of months multiplied by 3.3333, which is a rough "month" unit on this particular chart.

The y axis filter is easier. It calculates a pixel representation of the book's value (the value divided by 4700, the number of dollars per pixel) and that number is subtracted from 579, which is the offset from the top of the SVG element to the baseline of the chart.

If you were doing a generic chart that could take any similar data and produce a scatter plot, you would have to do these calculations on the fly. Getting the minimum and maximum values out of the data and creating a range and scales depending on the specific data. Since I defined the data and parameters for this before I wrote a line of code.

Yay for lazy.

angular.module('comicFilters', []).filter('xDate', function () {
    "use strict";
    return function (input) {
        if (input !== undefined) {
          var date = input.split("-");
          var years = date[0] - 1993;
          var months = (years * 12) + parseInt(date[1]);
          return 90 + months * 3.3333;
        }
    };
}).filter('yPrice', function () {
    "use strict";
    return function (input) {
      if (input !== undefined){
        return 579 - (input/4700);
      }
    };
})

And there you have it. Check out the $100,000 club repo for all the source code from this visualization and keep your eyes peeled for my jQuery Conference presentation where I'll be talking about this as well as D3, the canvas element (with the web audio API) and Google Maps.

Posted in JavaScript | Tagged , , , | Leave a comment

I’m Going to be Presenting at jQuery Conference. It’s in San Diego. You Should Go, Too.

It was announced last week, so I can now share the fact that I’m going to presenting on Data Visualization in the Browser at jQuery Conference in February. Because it’s a new year and because the presentation time is so tight (30 minutes!) I’m going to redo the existing dataviz presentation I’ve been giving entirely. I’ve got a new branch and everything.

I’m thinking of using World Bank data. At minimum, I’ll be doing something interesting and meaningful and, at best, I might end up on the World Bank data visualization tumblr.

I’m also super happy to be going to San Diego in February.

Posted in JavaScript | Tagged , , | Leave a comment

Late Notice: I’m Presenting on Canvas Tomorrow at HTML5 Boston

I tweeted about it but I never added it my public calendar and… never posted about it here. Anyway, I’m going to present on canvas tomorrow night at HTML5 Boston. It will be great.

It’s also at the Arsenal complex (at Harvard Business Review), which means I’ll be having Isobar flashbacks. It also means I’ll be running the western end of the Charles River for the first time in a while. Super fun.

Here are the slides.

Posted in HTML | Tagged , , , | Leave a comment

I’m Giving Away a Copy of My Book, Beginning HTML and CSS

beginning-html-css

Want a free copy of my book? Sure you do. This month, I’m giving away a copy of my book right here on the blog (on this very post.)

The rules of this contest are simple, simply share an interesting/funny anecdote about your experience interviewing for a technology or agency position (no names or companies needed, blind items are fine.) At the end of October, I’ll pick the best one and you get a free copy of my book, Beginning HTML and CSS. I can’t tell you what kind of story will win, but if you’ve got an interview story that’s funny, creative, cool, inspirational, or whatever share it and you might get some free stuff. This contest is open to anyone in the world with an address. If I can send you a copy of my book using the US Postal Service, you’re eligible.

And heck, even if you’re already an expert and don’t have need of a beginner book, you can certainly hand it off to someone looking to make their own site for the first time or to an engineer from another technology discipline looking to get introduced to the joys of web technology.

Posted in HTML | Tagged | 4 Comments