My New Book, The Uncertain Web, is Available in Early Release

Posted on by Rob Larsen

rc_lrg

The raw stuff, available for your reading pleasure. I’m really excited to see what people think.

The best way to approach the web today is to forgo hard and fast rules and design for uncertainty. Embracing uncertainty as a core tenet of web development and scrapping the rules we’ve relied on in the past few years is the best bet for creating future proof web solutions. By combining web standards, progressive enhancement, a full technical toolbox, an iterative hybrid approach to design and development and embracing a desire to question the status quo and perceived wisdom, teams can create dynamic web sites and applications that should perform admirably in future devices, with unknown capabilities. By focusing on optimal solutions with intelligent fallbacks and forgoing the desire for absolute solutions design and development can work together to create the web of today and tomorrow.

Introducing My New Book: The Uncertain Web

Posted on by Rob Larsen

Hey everybody, I’ve got a new book coming out. It’s called The Uncertain Web and it’s being published by O’Reilly. Right now we’re targeting a November release.

Just in time for Christmas. Buy a dozen and hand them out like candy canes.

If that seems like too long to wait, never fear. I’m about 50% finished and once that chunk has been hammered on by a hand-picked team of geniuses, it will be available as an Early Release.

So, what’s it all about?

Here’s my original pitch:

The Uncertain Web

The best way to approach the web today is to forgo hard and fast rules and design for uncertainty. Embracing uncertainty as a core tenet of web development and scrapping the rules we’ve relied on in the past few years is the best bet for creating future proof web solutions.

In the early 2000s, there was basically one browser (Internet Explorer 6,) one platform (Windows XP) and one screen resolution (1024 by 768) that mattered. With that set up you could design, develop and test against the vast majority of web users with one desktop computer. The biggest question on the horizon, it seemed, was when it would be viable to design for 1280 pixel screens

This limited field of play meant that there was an expectation that sites and applications would look the same everywhere for everyone. Best practices were honed and codified into hard and fast rules which drove design and development. Major choices, such as the size of the basic grid to design for, were no longer choices. You started with a static, 960 pixel grid and sliced and diced it as needed.

Today, things couldn’t be more different. With the launch of the iPhone and the iPad, the rise of Android and the growth of not just one, but two real contenders to Microsoft’s position as the dominant web browser (Firefox and Chrome), developers and designers have an ocean of variables to navigate. Every question around a site design is now pregnant with options.

Initially, developers and designers tried to navigate this new reality by creating new rules.

The problem was, the goalposts kept moving. As soon as a new hard and fast rule was created, some new wrinkle would render it impotent.
People designed and built “iPhone” sites, assuming that Apple’s dominance in the smartphone market was somehow a permanent condition. They tested for touch capabilities and assumed that touch users would never have a mouse.

As Android’s huge growth over the past few years, and the presence of devices like the Chromebooks and Windows 8 laptops with both mouse and touch capabilities have proved that those new rules have a short shelf life.

Even patterns like Responsive Web Design, which some saw as a single solution for design and development moving forward fall apart when applied against complicated application patterns and the vagaries of bandwidth and mobile performance.

By combining web standards, progressive enhancement, an iterative approach to design and development and embracing a desire to question the status quo and perceived wisdom; teams can create dynamic web sites and applications that should perform admirably in future devices, with unknown capabilities. By focusing on optimal solutions with intelligent fallbacks and forgoing the desire for absolute solutions design and development can work together to create the web of today and tomorrow.

This book will outline both the concept and underlying principles of the Uncertain Web and introduce some of the techniques necessary to make the successful transition.

So, that’s the thing.

I’m really excited about this one. I hope you enjoy it.

You’re So Smart You Turned JavaScript into xHTML

Posted on by Rob Larsen

Have you ever seen an error message like this one? XML parsers are designed to call it quits as soon as they encounter an error. XML grammar is strict, so the parsers need to be strict too. This particular error is from an & in a URL. &s in URLs are super common occurrence, but if you throw one into a plain XML document, without inserting it as a character entity (&) or numerical character reference (&) you get an error.

Sweet.

xml-error

To be honest, I’m not a huge XML hater. I’m a technical guy so ensuring that some of the documents I create adhere to a technical standard doesn’t kill me.

I mean, Ant.

That said, I don’t like the way XML’s strict error handling inserted itself into the web with xHTML. A good part of the reason the web blossomed the way it did was because the language underpinning it, HTML, it was so forgiving people could get by without knowing what they were doing. My initial HTML was so bad, I’m surprised I ever did anything.

But yet, I did.

And I’ve been building sites since. 17 years and counting! All because I could create a tag-soup mess that, kinda/sorta did what I wanted it to do and the browsers at the time did their best to try to make sense of it. That was great.

XML/xHTML took that possibility away. Which is why, while many people (including myself) wrote xHTML documents, we didn’t actually serve them as XML. The danger of having a catastrophic error was too great, so all but the hardiest developers basically treated xHTML like HTML 4.0, just with XML style syntax.

Which is kinda’ goofy.

Remember when they said “Users should not be exposed to authoring errors?”

The group that went on to found the WHATWG (the group responsible for the current HTML specification) recognized that this was a serious flaw and put a specific reference to this issue with xHTML in their proposal for the future of web application development. It was one of their seven guiding design principles. In case you’re unaware of the history, that paper is the foundation of the modern HTML specification. One piece of that foundation is as follows:

Users should not be exposed to authoring errors

Specifications must specify exact error recovery behaviour for each possible error scenario. Error handling should for the most part be defined in terms of graceful error recovery (as in CSS), rather than obvious and catastrophic failure (as in XML).

These are concepts embedded in the very heart of modern web development.

  • Avoid catastrophic errors.
  • Allow for graceful error recovery.

Why, then, do so many people create web applications that rely so much on JavaScript that they catastrophically fail? The web, at its best, should be resilient. Nothing warms my heart more than a 20 year old page that’s still kicking, a 10 year old link that redirects properly onto a completely new domain or platform or a modern site that can serve something useful to a 15 year old browser. To me, that’s the web at its best.

The opposite end of that spectrum, from my perspective at least, are the sites that do nothing at all without a relatively modern (or, in some places, completely modern) browser with JavaScript turned on and all dependencies (many of which are on third-party domains) loaded. While you can make an argument that JavaScript should be a hard requirement for certain kinds of applications (I’ll give you games written in the Canvas 2d API, for example,) I don’t think it’s the way of the web to have your site fail and show a blank screen because some third-party dependency doesn’t load, JavaScript is turned off or because your developer left a trailing comma in a JavaScript object and didn’t test in Internet Explorer.

I understand that creating a completely static version of your site or application is impractical. Granted. Still, showing nothing if there’s a problem is just terrible. But yet, people are happy to do it.

Client side JavaScript all the way

Nicholas Zakas has talked about this a little bit with his presentation, Enough With the JavaScript already. We’ve moved so far towards JavaScript in some circles that some people are starting to use tools like Backbone even in use cases that don’t require complicated interactions, like a static content site. Whether it’s a case of being obsessed with the new-shiny or simply not knowing any better, people are parsing and rendering entire pages in the client, even when there’s no real dynamic content. Beyond the fact that that’s going to be slower (why would you serve 1MB of JavaScript/JSON to render 100kb of HTML when the server can just serve the parsed/rendered 100kb to begin with?) you’re just setting yourself up for catastrophic failure if, for example, your static content server goes down and all that fancy Backbone code disappears.

What’s wrong with progressive enhancement?

Nothing, that’s what. Just because we can do everything in JavaScript doesn’t mean we should. In fact, the more complexity and dependencies we bring into the front end, perversely, makes the danger of something catastrophic happening even more likely. Third party scripts, web service calls, browser bugs, users with JavaScript disabled and your own application errors can all cause catastrophic failures if you’re not careful. That’s why, in my mind at least, the principles of progressive enhancement are more important now than they’ve ever been. Building solutions that are robust and resistant to changes in the environment (including many that are going to be forever beyond your control) is going to ensure that you can always reach your audience.

Your users don’t care how fancy your stack is. They care that they’re getting their content, that they’re getting it as fast as possible and it appears consistently from visit to visit. If your stack is getting in the way of any of those goals then you need to revisit the way you’ve designed your web solutions.

Let’s Talk About your Billable Rate

Posted on by Rob Larsen

I’ve got a few topics relating to my experience at agencies that I’ve been looking to share for some time. Now that I’m going my own way as a consultant with the goal of maybe expanding on that as well as somehow helping out at an agency again; I’m going to start to work through different agency related subjects as time allows. Hopefully you’ll enjoy this stuff.

I’ve related the following (albeit in much less detail) to people who work at agencies and a lot of them really hadn’t thought about the way that their employer makes money off of their services.

It can be an interesting revelation.

When you work for an agency your talents are for sale to clients. While the relationship is marketed in another way (agencies bid for projects and/or try to gain permanent relationships with clients) this is the basic financial currency of agency life- providing bodies to clients either directly or hidden within the completion of a project. There are basically three ways that your clients pay for your services. One of them ties directly to the number of hours you work. Another feeds into the internal agency accounting in ways that can be detrimental to your health. The third can just be boring. I’ll define them and then I’ll let you in on a little secret about the way that the agency views your relationship with each of those models.

  • Time and Materials

    In this model, the agency bills the client for your time and expenses when working on a project. For example, if you’re, say, an Art Director at an agency[1], the company might charge something like $200/hour for your services. If you then worked 40 hours in a given week, the company would charge $8,000 for your time.

    This model is useful because, generally[2], the agency gets paid for the work they’ve done. There will be an estimate done at the outset of the project, but if the project goes, say, 10-20% over budget the client can still be charged for those hours.

  • Fixed Cost or Fixed Price

    In this model, the agency provides a fixed estimate of the cost of the project. Working through a gigantic spreadsheet (or something web-based that does the same thing as Excel) with limited information, the agency team will put together a proposal that’s supposed to predict, with pinpoint accuracy, how much the project will cost- including enough margin left over to keep the lights on.

    This model is great for the client, since they know how much the final bill will be. This model can be great for the agency, especially if the project comes in under estimate. The thing is, that almost never happens and when the reverse is true and the project spirals out of control it can quickly turn into an emergency situation. Just like a startup working through their pot of money, projects have a burn rate, which indicates the daily/weekly/monthly resource usage on a project. When the burn rate indicates that a project is going to bust the budget, step back, grab some popcorn and watch as executives go into disaster recovery mode.

  • Staff Augmentation/Dedicated Teams

    In this model the client buys your exclusive services for a period of time. This is not project based and can last for many years[3]. This can be on or off-site. If it’s on-site it’s basically like you’re an employee of the client company (except you have to kiss everyone’s ass and get none of the perks of the client company.) If it’s off-site it’s basically an arrangement that means that important staff can’t be pulled away from one client to serve another need within the agency.

    This model can work for both sides as it represents a steady stream of income for the agency (albeit potentially a discounted one) and the client can flexibly fill positions with skilled staff.

Now it’s time for the secret. While your boss, their boss or even their boss’s boss (all the way up to the guy or gal that runs the place) might care about your well-being, the Agency, as a capitalist organism, does not. The Agency as a capitalist organism doesn’t believe that you’re a special little snowflake. The Agency thinks you’re a replaceable cog and doesn’t care about how many awards you’ve won or how clever your code is. With that in mind, with any of the models the agency is almost always healthier in the short term if you work 50-80 (or more) hours a week[4], every week until you quit, die or retire.

In reading this, if you’re unfamiliar with the way that agencies work, you should keep in mind that agency deadlines are pretty psychotic.

The folks that interface with clients would rather chew broken glass than tell a client that a deadline is going to be missed. Missing deadlines hits the wallet and (worse) the reputation and relationship with the client. It’s not something you want to do ever. So whenever a deadline is approaching the project team goes into “whatever it takes” mode to get the work out the door.

So, with each of the models, you being willing to put in 80 hours a week, sometimes for months at a time, is vital to the success of the organization. This is both down to the danger of missed deadlines and also down to the pure math of being an agency employee.

Lets look at how that math works with each of the models.

  • Time and Materials

    Cutting through any potential confusion, if you’re working 80 hours a week on a time and materials project you are getting royally screwed. Let’s take the example of the $200 an hour Art Director. To keep the math round, assume the Art Director makes $125,000 a year plus $25,000 in various other compensation (health insurance, 401k match, etc.) That works out to about $72 an hour[5] for a 40 hour workweek. That adds up to $2880 a week. So, how much do you get paid if you work 80 hours that week? $2880.

    What’s your effective hourly rate? $36 an hour. But yet, the agency is going to bill $16,000[6] for the full 80 hours. You’ve increased the company’s margins by a wide margin with your dedication.

    You win free pizza at your desk for dinner, stress related hives, and an angry spouse.

    There’s also a morale cost with this model. If you’ve got pure freelancers working alongside agency employees and both sides are working late nights, the freelancers are invariably going to be happier about the situation since they’re actually getting paid extra for their effort. Trust me, it sucks when you’re stuck in the office at 9:30 and the freelancer next to you is whistling while he works thinking about the new addition he’s going to put on his house.

  • Fixed Cost/Fixed Price

    Fixed cost projects only go awry, from an accounting perspective, when the agency needs to add more bodies to a project. Otherwise, with the planned number and makeup of the team, the only thing that matters to the Agency is whether or not the deadline is hit. To that end, if you need to sleep at your desk for two weeks straight and work weekends for six months, then it’s all good.

  • Staff Augmentation/Dedicated Teams

    Your number one job in this kind of role is making sure the client is happy. Depending on the engagement this can, of course, include long hours. That said, I’ve seen examples where these can be the healthiest situations in terms of the number of hours you have to put in every week and the pace of your day.

    There are other downsides to being in one of these roles, but generally it‘s not the hours or the pace. The biggest is the common refrain of, “If I wanted to work at Client Company for the next five years I would have applied for a job at Client Company.”

    That said, in terms of hours you can get lucky in these kind of engagements.

[1]
The weight of titles can vary, as can bill rates across agencies so don’t linger too long on the price quoted here. I’m sure there are folks with the title Art Director that bill out over four figures and others that bill out much less than $200. It’s just a nice round number.

[2]
generally.

Projects can be 50-100% (or more) underestimated, and in those situations it’s tough to keep asking for more.

Imagine a project budgeted for three years of development at, say, $1,000,000 a year. Then imagine that $2,000,000 worth of work is done in the first 6 months… That’s capital T trouble and there will be some awkward C-suite type conversations.

[3]
I’ve seen people who were embedded for more than five years at client organizations. Something like this: “I got into the office for my first day. they told me to go to client x later that afternoon and I’ve been here ever since. That was in 2007. I got back to the agency office 2 or 3 times a year.”

[4]
Some agencies have slightly different DNA that values talent as talent and wants to cultivate it and keep it from burning out. If you’re at one of those places you’re winning.

[5]
Keep in mind that the gap between $200 an hour charged out to the client and the $72 in total compensation per hour isn’t the entire story on margins in the industry. There are a lot of non-billable people in an agency and there are always billable people who aren’t billable at any given moment. There are a lot of slices to be taken out of that $128 difference. So many, in fact, that margins can be down in the single digits at some shops. I don’t have encyclopedic knowledge of margins across the breadth of traditional and digital agencies, but I don’t think they get much higher than maybe 15-20% anywhere

[6]
No, this doesn’t always happen. Sometimes an agency will eat the extra hours to maintain the client relationship or because it was your, or the agencies’ fault that you had to work 80 hours in the first place. That said, many times, if the numbers are okay, then $16,000 it is.

Notes from the Interview Front

Posted on by Rob Larsen

Interrogation Room #5

Sadly, I won’t be sharing a bunch of funny anecdotes from my time on the interview trail. While that might be fun, there are some bigger trends that I wanted to talk about after having reviewed dozens of job descriptions and talking to dozens of people at one level or another over the month of March.

AngularJS is the new boss

At least in terms of green field development it seems like no one is talking about anything other than AngularJS right now. My year and a half’s worth of Angular experience set off alarm bells for many recruiters. Backbone comes up, but not with as much frequency. Several people were moving from Backbone to Angular. Nothing else in the MV* space was really in the running.

Lots of smart people (including many that I know) are still pouring a lot of time and effort into Backbone, so it’s clearly not going anywhere. It is weird, however, as Angular seems to be winning developer mind share no matter what the regular front end engineering taste-makers might think about it. I’ve had a lot of conversations with people (including some big names) where they’re completely flabbergasted that anyone is even using Angular, forget about the fact that it’s storming the castle.

It’s definitely storming the castle.

In the course of writing this post, I saw a bell clear sign of Angular’s rise. I saw an “Angular expert” job posting from a company who had spent a ridiculous amount of time and money (hundreds of thousands of dollars worth of developer hours) crafting a “perfect” Backbone based application development stack. I’m sure they’re still quite fond of Backbone, but it was interesting to see them hedging their bets, trying to get some Angular expertise on staff.

Start-ups are spreading like wildfire.

The Boston start-up scene is pretty nuts. I’m not generally interested in start-ups, but as part of my “talking to everyone” thing I ended up talking to a few of them. I remain mostly uninterested from a full-time perspective, although I did get up to the brink with one group that seemed pretty cool.

There is a lot of opportunity out there for someone like me to help them out on a short-term basis, however, so in my new life as an indie consultant, I might go back into the fray.

Start-up Geography

At this point start-ups are no longer crowding around South Station. They’re in South Boston proper. It’s a walk to get over there these days.

Bocoup might be leading a smart charge over to North Station.

Agency vs. Startup/Product

When I interview at agencies I don’t do tech screens. No one has asked me to do anything like that in 4 years. The shared experience of the agency existence and the number of people I usually have in common with agency folks means that the “tech” portion of interviews usually doesn’t go beyond philosophy and general experience. For most agency people knowing the level I’ve been at the places I’ve worked is a pretty good indicator that I know what I’m doing.

Product people, on the other hand, are super wary of my experience. They seem to think that because I’ve been a manager and been in charge of teams that I don’t know how to code. Some of them have even made the assumption that I don’t want to code (yes, I do open source software to torture myself.)

Interviewer: “So you’ve been a manager, running teams for a while. You haven’t really been coding since 2007 then?”

Rob: “No, unless today is opposite day. I’ve worked on probably 20 sites or applications in that time. Some with thousands of lines of JavaScript and incredibly complex UIs.”

Let me set the record straight on that assumption. Being an engineering manager at some companies might include no coding/all meetings, but at an agency the pendulum usually swings way too far in the other direction.

A technology manager at an agency will probably be doing coding/architecture most of the time and will not have any time officially allotted to do anything strategic or to work with their team on anything other than project work. .

Even if their target allocation is something like 60% billable work, they will probably be fully allocated to client work. Occasionally, they will be over-allocated to a client (or two.) Oftentimes, soft-skills “manager stuff” happens in hours 40 and above (or 80 and above.)

Personally, at one point a couple of years ago I was allocated 100% to two separate divisions at the same investment bank for six weeks. Not a lot of time to get soft when you’re scheduled to work 80 hours a week 🙂

This is actually a systemic problem at agencies. There’s not a lot of room for people at even the director level and above to really nurture staff. There are exceptions, of course. If you look around and spot stable teams at different agencies you can usually spot a guy or gal at the top who makes a point of keeping their staff happy, but the pressure to do that is not coming from up on high. The pressure from up on high is to keep your billable rate as high as you possibly can in order to justify your continued presence at the agency.

To that end, senior people at agencies are constantly getting their hands dirty. No one is hiding. I know start-up people think their experience is special, and in a lot of ways it is, but the experience on the agency side isn’t any easier.

To be perfectly honest, looking at someone from an agency with a good engineering culture and thinking they’re going to be soft is insane. That person has probably built up and torn down more architectures than most people do in a career in the product world.

There is a reverse bias

Agency people don’t want to hire people who don’t have agency experience. This is undeniably true. There’s a fear that you will hire someone, they will walk in the door, see the maelstrom and walk back out the door.

That happens, so it’s a valid concern, even if I don’t put as much weight into it as some people do. For my money, I’d rather have people get the shot and fill the role than play gatekeeper just because someone is coming from another industry. Sometimes it turns out really well.

Front-end engineering isn’t, universally, a first class discipline (although it’s getting closer to being one)

Something similar to the following happened twice when interviewing for very senior architect positions (including one that was titled something like “Front End Architect.”)

Hiring manager: “We want an architect whose focus is on the front end. You’ll lead projects, working with other developers and vendors, but we really want someone who can make a difference with the user interface layer. We see it as a game changer for us. ”

Me: “Sure, great. I’d love to come in”

Rob interviews

Feedback: “Rob was really strong on the front end, but he wasn’t as strong as we want him to be with Java/C#”

(So you actually want a Java guy? You want a Java guy who’s also an expert with JavaScript? Does such an unholy chimera even exist? )

The thing that drives me crazy about this scenario (beyond the fact that it’s a waste of my time) is that I’ve worked with architects who knew so little about JavaScript that they didn’t know enough to leave the JavaScript alone. I’ve seen very smart people completely break the UI because they were completely clueless about JS and didn’t even realize the depth of their cluelessness.

Up until they start breaking things, I’m cool with this. As long as folks delegate I don’t want or expect someone to be an expert across the whole system or to be able to code the whole system. I wouldn’t trust many of the absolute geniuses I’ve worked with to code the front end of any system. Just like they wouldn’t trust me to code up the services layer or come up with a database schema. But yet, in many organizations, you are simply not allowed to be in charge if you don’t have significant back-end experience. It doesn’t matter that the heavy lifting is happening in the browser now. It doesn’t matter that the potential advantage is in the user interface because there’s still an incredible shortage of strong front end engineers. It doesn’t matter than getting people to be effective at their jobs is a skill that stretches across technology domains. If you’re not a Java or C# expert, in some places at least, you still can’t straddle domains and run the show.

It is getting better

The good news is, the previous used to be universal. It’s no longer. I did interview for two general technology leadership roles (Director/VP) and those were judged on their own merits. No one once gave me the stink eye because I’m not a reformed Java developer.

Random notes

The Interrogation

  • I don’t like people who do tech screens like it’s an affront to their honor that you would even think you are good enough to work at their company. This doesn’t happen too often, but it does happen and I fill up with quiet rage pretty quickly when I encounter one of these people. When I run tech screens, I want to fill the role. I’m not happy when people aren’t up to snuff. I’m truly disappointed. It seems to me like these people would be happier to keep the role unfilled. That’s the absolute wrong way to approach the interview process. My reaction is invariably: “You don’t want me here? Cool, I won’t work here.”
  • I’ve been contacted by 28 separate recruiters in six weeks for various jobs at one company. I want to opt-out permanently please? I wish there was a central registry to opt out of contact by company.

  • I’ve got a hint for people who have been trying to fill roles for a six months or a year and can’t find anyone. Pay more money. Instead of losing out on some opportunity waiting to get someone at a 50th percentile wage and fitting it into some budget, push the pay envelope by 10% or more and people will be a lot more receptive. There are very few job descriptions I’ve seen in the Boston area, at any level, that had an aggressive (for the level) salary attached. There are some, but not many (maybe 10%.) Everyone else seems to think that being able to use new technology and getting free soda and coffee should be enough of a selling point to get you to choose them in the middle of one of the hottest skill sectors in the job market. The salaries seem to slot into very clear strata and there are few outliers. I know people are afraid because money is money and front-end developers should be worth $xxx. But if you have a need now, paying aggressively is one way to fill it that’s within your control. Everything else is outside of your control. Pay more than your competitors. Everyone is using Angular or other modern architecture. Everyone gives out free snacks and coffee and all that crap. You can’t win with that stuff.

And that’s it for now. I may add onto this as other trends percolate around in my brain. For the most part I enjoyed the conversations I had even if nothing turned out to be a perfect fit for me and I hope that people on the other side of the desk felt the same.