agile – Page 3 – Amateur Coach

Power Models and Method Affinity

a.k.a. why are some people SAFe oriented, others DAD biased, yet others LeSS enthusiasts, and not forgetting the DSDM gang, the Nexus collective…and I’m running out of terms.

I’m sure there are a whole host of reasons, but during my time observing people, and knowing a bit about their past and what they’re like, it has led me to form an interesting (to me anyway) hypothesis. It’s to do with power.

A Brief History of Power

Without digging too far too much into Taylorism or others, there are a handful of basic power models that can theoretically exist in an organisation (heavily simplified for illustrative purposes, as real life is never this “simple”).

Power in the hierarchy, the higher up you go, the more power you have
Power in the middle management – sometimes disparagingly called the permafrost
Power on the edges – people on the ground in front of customers

These can manifest themselves onto a project context in a few ways (note for want of better options, I’m labelling these categories with overloaded terms)

Power lies with the “planners” – e.g. project managers, PMO
Power lies with the “architects” – EAs, Solution Architects
Power lies with the “developers” – developers, testers, BAs

Human nature is such that we flock towards things that are like us. Planners are more likely to favour other planners, and work using systems where the balance of power is in their direction. The same sort of thing is true for the Architects and the Developers.

Methods

And now we get to the Methods part of this blog. I’m going to focus on agile methods, and agile scaling frameworks. And in particular, how these methods and frameworks are perceived, at least initially. That bit is key. Most of this “natural affinity” stuff is emotional in nature, and not fundamentally driven by rational thinking (hint: there’s a lot of religion in this area). As there are lots of them out there, I’ll just pick the three major ones (based entirely on how often clients talk to me about agile at scale, and nothing remotely scientific).

SAFe

The overall guidance is dominated by the navigable map. It has several terms that will be comforting and reassuring to hierarchical type organisations with traditional reporting lines and financial controls – Programme / Portfolio Management, Enterprise Architect, as well as some guidance on mixing waterfall and agile deliveries. This looks to be solidly planted in the middle of the “Planners” camp.

Based on the hypothesis, likely proponents and allies are to be found within PMO, Project Governance,Configuration Managers, hierarchical organisations with a centralised power model, and organisations that perceive themselves to be traditional with a rich history / heritage.

DAD

The first thing that strikes you when you first look at DAD is that it’s rammed to the rafters with choices. It has a risk-value lifecycle (but you can choose others), many options on how to achieve pretty much any delivery related goal that you may have – from big ticket items such as considering the future – how much architectural insight do you need, to very focussed options like the right level of modelling to use. And that’s just part of the “Identify Initial Technical Strategy” goal. This resonates well with those with an architectural bias – architecture is mostly about decision making and communication.

Likely proponents and allies are to be found in technical leadership – Architects, DBAs, and organisations with a strong technical bias.

LeSS

The navigation map for LeSS in contrast to the previous two, looks relatively uncluttered. There are large concepts identified (such as Systems Thinking, Adoption) but these are all located around the periphery of the diagram. Slap bang in the middle is the engine, and those are feature teams. This puts the Developer at the centre of the universe (as it were).

Likely proponents and allies are to be found within teams and individuals using Scrum and XP on a regular / daily basis, and organisations that “have a small company vibe”, which may be startups on a growth spurt, or organisations in a highly fluid environment with significant localised decision making.

The Goldilocks Solution

As the heading suggests, I think the right mix for any given organisation is somewhere in the middle. Power isn’t solely contained within a single area (though granted, in many cases, the vast majority of the power is indeed concentrated that way), and any scaled agile adoption strategy will need to understand and accommodate that to increase the chances of tangible benefits being felt by the organisation.

Feedback

As this is just a hypothesis I’ve got, I’d love to hear what you think, whether you’ve observed things that support this theory, disprove it entirely, or somewhere in between.

Learning about estimation when you don’t care about estimates

This post follows on from the previous one, linked here – Help! How do we start estimating?

The problem with running a training course on estimation, is that there’s a danger that poor assumptions are made about the things being estimated – i.e. you could end up “assuming the problem away”. If that happens, the exercise becomes too sanitised and relatively meaningless other than in a theoretical and abstract sense. And that’s often hard to relate to.

Using analogies – e.g. the “throwing the cat game” [http://tastycupcakes.org/2016/05/throw-the-cat-and-other-objects/] can help. They show you the dangers of assumptions etc., but in all the cases I’ve seen, the variability is low-ish. You’re doing the same thing to all items.

But what if you’re doing potentially fundamentally different things on different tickets? For example, if writing a validation method with some clever logic is a “5”, what is “patching a Docker template”? With the industry using the term “DevOps” like there’s no tomorrow, the variability of work a single team will perform will inevitably rise.

We, as humans, are generally far better at comparison based estimation than comparing to an absolute measure (especially an abstract one like distance or time). We’re also very good at spotting patterns. Of if real patterns don’t exist, then inventing them (one of the reasons why the saying “correlation does not imply causation” exists – we need to be reminded!)

That innate ability to see patterns, regardless of whether or not they exist, can lead people to try and find a relationship or common attribute across entirely disparate work items. Some teams make a virtue of this, using terms like “complexity” or “relative effort”, which make reasonable sense – irrespective of work, it can be categorised as complex/simple or quick/time consuming. That allows a single team to use a single “scale” to estimate everything that they could do.

That single scale, is one of the reasons that things can get a bit confusing. With radically different types of work, one thing that can happen is that natural clusters may form. Infrastructure management tasks may hover around the 1-5 mark, development work might be 2-8, something architecturally significant might be 5-13 and spikes might be as high as a 20(*).

If that’s happening, one of a few things could be the cause

It could be true
There could be the hidden view that “development stuff” is harder than “infrastructure management” (or something along those lines) and that bias is gaming the numbers
Anything else I’ve not thought of right now – depends on the room, dynamics etc.

Relative estimation works well within a logical domain – there are sufficient overlaps and related attributes for a meaningful comparison. A comparison across fundamentally different domains makes far less sense. There’s even an old simile on the subject – “as different as chalk and cheese”.

To combat this, I usually recommend having a lot more than just a reference story. I suggest having a catalogue of items, from as many of the affected domains as possible, making sure we’ve got examples of a few of the magic numbers in each domain. When attempting to size an item, the first question to answer is which catalogue item is the closest match, and then go up or down however many sizes as is appropriate from there. It can take an awful lot of stress and confusion out of estimation as you’re no longer trying to shoehorn a square peg into a round hole.

There is a price to pay for this additional freedom – your velocity figures become less relevant, as you can’t really compare sprint against sprint as simply as before, in case the “mix of work” is changing. Your burnup charts may still look like they work, but scope changes are harder to visualise – some of your “scope changes” are likely to be technical debt that you’ve discovered as you could be making platform changes with no change in business vision or scope. It also takes a lot longer to create a useful catalogue, when compared to “just picking an average looking story and calling it a 5”

Teams that go through a learning process like this, usually end up realising that there isn’t a simple textbook answer, and their only viable option to cope is to be alert and have an open mind.

(*) All using the scale 0,0.5,1,2,3,5,8,13,20,40,100 for illustrative purposes only

Help! How do we start estimating?

…came the plea from the team.

Context: Here’s a team that’s primarily a support oriented team. As in, their stated purpose is “to keep the lights on while delivering improvements”. Recent history (as in several months) has had them in a firefighting mode where they just “did stuff”. Planning was iffy at best and they had lots of difficulty in gauging what was reasonable in a two-week sprint.

Hence the ask.

The first question to ask (like everywhere else IMHO) is “Why”. Why do you want to estimate?

Most of the answers I hear generally fall into a few areas:

We can use the points to work out our velocity
We can build information radiators with burndown charts etc.
We can tell if a story will fit into the sprint
We can have more credible estimates using relative sizing with points than if we had absolute estimates using hours or days
We can plan our releases – especially dates

Very occasionally, someone pipes up with something like:

We can work out if we’ve all understood the story well enough to deliver it in a sprint

For me, that last one’s easily the best reason. By coming up with a “magic number” estimate during the planning game, each person’s number represents all of the assumptions they’re making about the work. And during the process of playing the planning game, the most extreme sets of assumptions are surfaced and it gives the team as a whole the ability to learn more about the work from each other and gain some consensus. That shared understanding is the real prize. The number is just a side effect.

There’s a growing interest in “No Estimates” (#noestimates on Twitter). A highly simplified explanation is that estimation is a wasteful event, and you’re better off breaking your work down into small pieces and just work on them one at a time, while maintaining a smooth flow of work. For me the most interesting thing about this movement, is not the “inflammatory” stance on estimation, but about what a team gets by breaking down the work into small pieces and just working on them one at a time. They get work that naturally has very few assumptions inherent in it (smaller work = fewer places for assumptions to hide). And they also get very quick learning about the different assumptions that everyone in the team has about the work, because they’re working on one thing at a time. For me, that’s a similar sort of outcome to only using estimation as a means of flushing out the assumptions and gaining a shared understanding, and frankly, ignoring the magic number side effect. Breaking the work up into smaller pieces is just generally a good idea.

So, with all of that “intro stuff” out of the way, just how do we help a team learn how to get the benefits of estimation? Like most other attempts at teaching/learning, we need to find something that the team actually cares about, and help them learn how to solve the problem / make things better. In this case, we had to get to the root of what the team wanted to get out of an estimation-based process. That’s assuming that an estimation based process was in fact the right thing to adopt. We’d have to examine that too.

It looked like there were two main things that the team needed. The management elements of the team wanted to keep track of progress, detect when things were slowing down, and report upwards.

The rest of the team didn’t seem to care about that. They just wanted to make sure that they weren’t being overloaded with unmanageable levels of work. The reactive nature of a lot of their work was such that there was very little that could be predicted about a sizeable portion of their workload. They needed a mechanism that would let them work on enough planned work, while keeping sufficient capacity to cope with the unexpected.

I’ll write up how we ran the estimation training and practice/guided sessions later on.

The “Good Cop, Bad Cop” relationship with Tools and Automation

Context: This all started with a request to make the tool a team was using (JIRA) prevent a developer from moving a task that was “in progress” back to “not started”. The manager’s rationale was to forcibly highlight when the team started to work on too much stuff. The developer’s rationale was to fix an incorrectly started task.

It got me thinking about how we open up this discussion and make it inclusive. Some questions that seemed to work were:

Why do you need to be told how to “be good”?
When do you need the tool to “pick up after you”?
When do you need to be reminded about what your way of working is?
What mistakes do you need the tool to forgive or punish (or both)?
When do you need the tool to prevent all mistakes?

All of this stuff is enabled by automation. So just how much automation do you need, and what is it to be used for?

Automate or mechanise the “boring parts” of your job?
Validation for error prevention or error detection?
Automate all of the work variants? Or just the common ones?

As I like analogies, I thought I’d explore this using one, in this case, one of my favourite topics – “food”. For the purposes of this blog, assume time is magic and doesn’t cause any problems.

Making Soup

I want to eat something. Perhaps a bowl of soup.

The full process starts in the ground. I could start there with a fully manual solution:

Grow the raw ingredients from scratch. Prepare them as I want, cook them and voila, a delicious meal.

+ complete control of ingredients
– slowest method
– I could grow the wrong thing

I could “automate” that growing process, by buying the raw ingredients from a shop:

+ pretty good control
– may not have what I want in stock
– might not be able to manage the quality “precisely”

I could increase my levels of automation and also automate the preparation work by buying the pre-prepared stuff:

– only partial control
– may not have what I want in stock
– may not be prepared as I need – e.g. carrot sticks when I need grated carrot

Or I could be extreme and also automate the cooking process:

tin

+ fast
– limited control
– may not have what I want – e.g. I want “carrot and celery”, but all I can buy is “carrot and coriander”
– mass production, so probably very generic

Each of these levels of automation is accompanied by varying degrees of “policing”. If I’m a danger to myself when trying to chop vegetables with a knife, automating the preparation work is probably a good idea. But with that are constraints – I can only eat meals that can be made from pre-prepared ingredients.

Project Team Outcome

In the end, we left it such that you could move tickets back, and the policing aspect would be done by the humans in the team. Hopefully they chose that because it was the right thing to do, and not because they wanted my food analogy to stop…

The “Perfect” JIRA configuration

Well that got your attention. Aka sorry about the clickbait.

Context: A large-ish project is using JIRA(*) to manage their work, with varying degrees of success. And they’ve customised their instance heavily. No change was too small, or unreasonable. Over time, their configuration evolved as to be practically incoherent. There’s a parallel with teams maintaining software without paying any attention to technical debt. Things are built on top of other things. Workarounds are invented because it’s easier than refactoring, fixing deep seated faults and moving on. I’m sure most people have had an experience or two a bit like this.

Even your tools like Confluence and JIRA can have technical debt. Remember to pay that back regularly.

When I spent time with them, I came to realise that a significant underlying cause of their tool-based-pain seemed to be rather “simple”. They were not consciously aware of how they worked as a whole. They all knew what their roles were and how to do the day job. They even mostly understood how to work with their immediate neighbours (in a value stream sense). But they didn’t get the whole picture. No-one did. As a result, there was no way of telling whether a JIRA configuration change that would alleviate a local problem would have an adverse effect.

There’s a huge chasm between where they are now, and where they needed to be. Step one was to get back to basics – let them discover how they work (as opposed to how they use their tools) and then help them refine that.

Which brings us to another challenge. It was hard for this project to accept an OOTB configuration – “we do things differently here; we simply couldn’t work with a vanilla installation!”.

So I cheated.

They wanted to work sensibly, so they had great intentions. I tried to use that with a sequence of questions which allowed them to build up a tailored process for them:

Are you sure you want to use Sprints? Yes! Handy, it allows us to have backlogs, sprints, sprint planning, demos and retrospectives.
Do you want to visually track what you’re actively working on? Yes! That gave us an “In Progress” column and state
How do you tell if what you’ve worked on is good enough? We have testers in the team, they work with developers. Before we merge our code, we need to explicitly run tests. That gave us a “Being Verified” column and state.
When do you close off a story? When it’s accepted by the Product Owner. That gave us a “Done” column and state.
What happens when things get stuck? We have a blocked column and we put things in there so our scrummaster has a backlog of blockers. That’s another state & column (**).
When you run your “before-merge tests”, can you start that verification as soon as the developer is ready? No sometimes it takes us a bit of time or we batch up some related work and verify multiple changes at once. That gave us a “Ready to Verify” interim state that testers would pull from.
Is your product owner available at all times to accept stories as soon as they’re ready? No, our product owner has a day job so sometimes they have to batch accept stories. That gave us a “Ready to Accept” interim state that the product owner would pull from.

The final workflow they were happy with looked like this:

disciplined

The corresponding board looked like this:

disciplined_board

If you squint, that looks like Not Started > In Progress > In Test > Done. Which is pretty standard. But because they, as a team, worked through their process, it was their board. They also thought about a few useful elements, such as pulling work when you’re ready to do it, and trying to limit how much they had on the go at any one time.

Footnotes

(*) I don’t mind the Atlassian stack. It works well enough. But if you go well beyond OOTB then like a lot of products you’re flying without an electronic safety net.

(**) I’m not keen on having a discrete Blocked state as I’m a bit nervous of it being seen as a “rug to sweep things under and start something new”. But this team liked it, and it’s their way of working, so I accepted it.

Quis custodiet ipsos custodes?

Or in my case, who teaches the teachers?

I’m lucky at my current employer, in that part of my job is to deliver a load of courses to help my colleagues make some sense of words like “agile” and “mindset” and to learn about leading brands such as “Scrum” and “XP”. And more importantly, to help them realise that it’s OK to use your brain and common sense at work when trying to apply this stuff to really tough projects (my employer is relatively expensive, so we don’t get simple projects).

I’ve always enjoyed seeing students get a hold of a difficult concept – their eyes light up and you can see wondrous machinery whirring away at internalising their new-found knowledge. And for me, the ultimate in euphoric trip is when they think it’s an idea good enough for them to spend time and energy spreading (the payback usually takes longer as I don’t always find out, but that’s OK too).

This course is new for me though. It’s my first attempt at a train-the-trainer type of course. It forced me to do some soul searching and navel gazing.

Why do I like being a trainer?
What do I look for in a trainer?
What am I wary of in a trainer?
Why does this stuff matter?
What do I want from a trainer-production-system?
Who will the existence of all these trainers help?

Why do I like being a trainer?

I think it is just about seeing ideas enter people and change them (hopefully for the better). Their eyes light up, it’s brilliant. I think I’m just grateful that I get to see this a fair bit.

What do I look for in a trainer?

Someone who’s authentic. They bring their whole self to their course and they speak from their experience. Someone who’s not afraid of trying things out.

What am I wary of in a trainer?

Someone who’s voice and ego is bigger than the message. It’s about them, and how they do stuff.

Why does this stuff matter?

Project life is hard enough as it is. I want people to try and help make things better.

What do I want from a trainer-production-system?

Probably for it to not be a “mass production system” – if I wanted people who could parrot out the precise messages I wanted to give, then to be honest, I’d have recorded myself on YouTube and spent the budget buying Segways and IPads…

real_genius — Not a new idea. Scene from 1985 film called Real Genius

Who will the existence of all these trainers help?

The obvious (and stock) answer would be “future students”. But I think it’s the project teams that the new trainers will be in that will benefit the most. Not because they magically get someone who “knows all the answers”. I think it’s simpler, they get a “better team member”. By that I mean someone who doesn’t want to be the centre of attention, but is more interested in helping everyone else in the room (team) improve. Someone who feels learning is important enough to be willing to invest time and effort on it.

(Meta) Course Structure

On this train-the-trainer course, there are three distinct themes that need to come through:

The course content. So they can make the course flow “naturally”
The underlying material and knowledge. So they can speak from a position of knowledge and experience, as well as honesty and authenticity.
The confidence and ability to present / teach. Or even, the “permission” to make the course their own, and find their voice.

We mustn’t underestimate the third bullet, as I don’t think it’s usually something that we have in our minds as (potential) trainers. Especially when it’s someone else’s course. Terms like “Standards”, “Intellectual Property”, “Consistency” etc. all usually surface when broaching the subject of course content alteration for the first time. It’s not that those terms aren’t important – they’re very! It’s just that sometimes the underlying motivation for a course owner may not be the same as a trainer – especially the sort I’m looking to grow.

As a course designer and course owner, I feel successful when the courses that have been delivered/franchised/etc. are run well. The material was easy to understand, and people get what they were expecting. Course feedback is glowing. But that can’t be the only measure of success – a “successful” course that changes nothing in the lives of the students is a failed course IMHO.

So the future trainers of my courses need to be able to change how the courses function in order to give their students the best possible chance of getting something useful out of it. For that to happen, the trainers need to understand the course design and flow. Great courses have a logical flow to them, and there are a lot of hooks to connect concepts together, or to build on ideas. A course that is teeming with knowledge but is presented as a large list of independent facts will be dry, boring, and mostly pointless as the students won’t really learn much (other than how good/bad their cramming skill is).

With these hooks come many opportunities to connect ideas. For that to sound natural and based on what is actually happening in the room, trainers need to have put in the hours – they just need to have lots and lots of facts to draw from, from a huge range of scenarios and contexts. They’ll need to read around the subject. Anything less, then the trainer will spend effort trying to steer the room to a point where one of their pre-canned examples becomes relevant.

But to make sense of all of this potential flexibility, trainers need one more thing. They need to feel empowered to adapt the course to meet the needs that they see there and then, as opposed to what we saw while designing the course in the first place. Usual rules apply here, they need some skin in this game, that it’s their course too and they need to look after it. What they bring to the party, is their voice. How they tell the story, how they weave the narrative through all of those hooks and potential connections that the course material gives them. Course material should never be a paint by numbers. It’s a canvass and a palette for the trainers to produce the right result for their audience.

That is scary for a new trainer. Positively terrifying. What we had to do, was help them get this fear under control.

Some things we did:

Used a variation of “mob programming” so the whole class presented a difficult section on “mindset”
- They were unfamiliar with the topic and unfamiliar with the training material, so the mobbing approach would help reduce the fear of the unknown (they’re not alone). One minute timer, so the speaker would change about once a minute, give or take (they all got a chance to be part of it).
- That seemed to work well. They were able to come up with lots of interesting observations. There was a mix of “verifying the points were being made, and supplementing as needed” as well as “learning from the speaker and internalising the new ideas”
- The proof was in the ability to recall the themes and present the topic confidently the day afterwards. To help this, we had some diffuse time – we let them sleep on it.
Repeated practice runs on “industry standard content” on Scrum and XP
- They knew the subject matter reasonably well, but they didn’t know the training material or the defined flow
- By splitting them into smaller teams, we took them one step closer to their “training reality” – they’d eventually be teaching in pairs
- Smaller teams meant more sessions of the same content, so they could learn by observing others try to deliver the material
- We kept things as fair as possible – the first team to try a section were at a disadvantage as the training material was unfamiliar. By the third team, the content was less scary. So we swapped the running order every time a new topic was ready to be practiced.
Multiple opportunities to handle “difficult questions from difficult people”
- One of the larger areas of concern for new trainers – what do I do when faced with difficult people?
- Ground rules – “ask difficult questions, but don’t be difficult”
- DBAD – Don’t be a <swear word>

After three days they were cooked. But we’d covered enough of the scary stuff to think that they could give it a pretty decent go. Not everyone in the class would likely keep training on a regular basis, but they were all keen to try it out, as even if they weren’t selected as trainers, they’d still be better off and so would their project teams. Even the losing side has a positive outcome.

A culture of learning

It’s no secret that teams that have a “culture of learning” outperform similar teams that don’t care about that sort of thing. The Internet’s full of articles extolling the virtues of such a culture, along countless articles offering suggestions on things to do that would help you develop your learning culture. There’s a lot of good sense out there, but in my experience, it’s not really possible to develop (or introduce) a “learning culture” as a discrete thing. There is no carrot & stick solution that’s resilient to real life.

The best I’ve been able to do is make it possible for my teams to believe that it’s worth learning new things, growing and sharing. Mostly because it’s more fun that way. As a species, we are if nothing else, driven by emotions. Hopefully positive ones.

Learning new things is a risky business. For a start, if it’s something difficult, or requires lots of practice to master, then you’re going to get things wrong. A lot. Making a mistake is often seen “in the corporate world” as a bad thing. With penalties like “poor performance ratings” if you’re an employee, and terminated contracts if you’re self employed. Ironic really, as we tend to learn the most valuable stuff while making mistakes. So another thing that I do to help my teams, is make it OK to make mistakes, to break things. On one project, that was a counter – everyone was allowed 3 epic fails. On another project, it was “as many as we needed to in order to do the right thing by our customers” (we were very user need driven so we had lots of empathy with our users).

On an explicit front, we introduced “Lunch and Learn sessions” – once a week, for 30 minutes. The list of topics was on a (Product) Backlog – well, a whiteboard with a load of post-it notes on it. Anyone could add things to the backlog, and re-order things as they saw fit, with a deadline of the morning of the lunch&learn day. It started off as me giving the talks, which after a couple of months, saw others offering themselves up as speakers for a subject they were keen on. It wasn’t limited to digital development either. One speaker wanted to talk about macro photography. We were able to round up guest speakers from other projects, and they were usually delighted at the chance, and in payment, we would give a talk to their team in exchange. One session was a live demo – our accessibility specialist brought in a blindfold and some headphones, and we were to try and use the website that we were building, but from the perspective of a blind user. Talk about an eye opener!

On a subtle angle, we used lots of pairing and collaboration techniques to spread the knowledge about, as part of “doing the day job”. We tried to get the default model for solving a complex problem into one where a person finds a teammate with some time and they work it together. So by necessity, we introduced a little slack time to help with this. Irrespective of role, seniority or time on project, we all tried to treat each other as equals (part of our team charter that we signed up to when joining). That made it “OK” for information to flow around, and made it part of being in this team.

If you think all of this sounds like hard work, well you’re spot on. We had to invest heavily into creating the conditions that would make a learning culture more likely to happen, and we had to do it in a way that didn’t completely alienate us from the rest of the world (though there was a rumour that we laughed more than everyone else in the building put together! And despite such frivolity, we were easily the most effective team in the building). We had to sustain the effort too, as we were looking to build a lot of good habits (once an activity becomes a habit, it becomes significantly easier to do as there’s no longer the need for conscious willpower).

Research Based Development

The following diagrams show how one of my projects applied User Research to drive the software development process. There were two distinct modes of operation – before we went live with Release 1, and afterwards.

Before Release 1

Research was lab based, and essentially theoretical – we would do the best we could to recruit actual and potential customers and find out what they would do if they were faced with our system, and these test subjects would do the best they could to pretend that they needed our services.

Here’s a simplified workflow: hypothesis_pre_r1

We would start off with a hypothesis, which we’d refine by thinking about it and working with the business. We would design how the lab would run, what experiments we’d perform, all to maximise the learning that we could get from the lab. We’d build multiple prototypes to test the variations on the theme and we’d conduct several lab sessions. At some point our analysis would show whether or not the concept we tested had legs, possibly requiring a few refinements and retests. If proven, we would then tease out what we needed to do to deliver a product increment that capitalised on what we’d learnt, build it and demonstrate it to our stakeholders.

At this point, we’d hope we’ve done the right thing. We’d only really find out when we deploy into production.

Production Based Research

Even after we’d gone live for the first time with our Release One, our reliance on User Research remained. We’d still come up with hypotheses and design suitable experiments to prove or disprove them. All that fundamentally changed though, was how and where we ran our experiments. hypothesis_live Instead of conducting the experiment with theoretical users and pretend demands like before, we’d employ A/B or multivariate testing, build alternative flows through our system and just measure what our actual customers did. To measure demand for a new feature, we’d just put a button on the screen and count the number of times it was clicked (users would be given a “we’re assessing the demand for this feature, your interest has been registered, in the meantime do this” message). The proven hypotheses were demonstrably proven as the outcomes were better (as opposed to before when they were still technically just theoretical). The cost was that disproved hypotheses needed additional work to remove the code from the live system. We thought it was a cost worth paying.

User research is a team sport. Why?

A quick trawl through the internet with this blog post as the search term throws up lots and lots of results that all assert that user research is indeed a team sport. However, far fewer seem to go into why it’s important. Sure, the “usual stuff” about collaboration leading to better solutions because the problem is being observed through different lenses is certainly true, but that’s really about collaboration, and not about user research.

So why then, is user research specifically a team sport?

One thing I’ve observed, is that participation in user research sessions tends to make a somewhat abstract notion of a “user” rather more tangible. Painfully so, in cases where your users are vulnerable segments of society. It builds empathy.

Empathy’s a powerful motivator. One common effect is it can instil the desire to “do the right thing” especially if it helps someone. An effective team is one that has a shared purpose. And worthwhile purposes (i.e. helping someone) are fantastic motivators. If only half the team are motivated in this way and want to “do the right thing” and the other half aren’t, then you’ve probably got a problem.

It’s more than the pithy slogan “user research is a team sport”. The whole team need to feel motivated to “do the right thing” and user research makes it possible to tell just what that is. Just one reason why research should be a team sport. Surely this one’s enough?

The Secret Life of a Requirement

There is a lot of guidance out on the Internet about how to manage your requirements. The brand leader vehicle for a requirement is the User Story, which is essentially a place-holder for a conversation between two parties – someone who understands the problem or pain, and someone who can do something about it. This project is no different, we generally use User Stories as our requirements vessel. However, we also use other variations, as needs dictate – for example, Technical Stories and Spikes.

This project is being run in a way that’s compliant with the GDS lifecycle – it’s driven by user needs. So, with this in mind, how do our User Stories spring into existence? Especially as User Needs can represent very large things (consider stuff like Maslow’s work). Well, we think that we can look at User Needs in order to form hypotheses on what can be done to meet the need / alleviate the pain. These hypotheses need to be tested, and we do that with a mix of Lab testing, pop-up testing and specialist user testing (for example, we have good working relationships with Action For Blind People and we regularly test with them). Theories that have been proven are now at the next stage, and are ready to be analysed and expressed as a set of stories. These stories are usually fairly large (Epic or Theme scale), though there may be a few small, focussed quick-win items that are easily identified. These items go onto a list that’s ordered by timeliness – starting from things that need to be looked at sooner, and ending with things that’ll get looked at later. We call this list our Product Roadmap.

step_1

We have a regular process of reviewing and refining our Product Roadmap. Due to the nature of roadmap items, this refinement process is only matched to sprint durations (two weeks right now). We take a roadmap item, and as a team spend some time trying to understand it, break it up into smaller chunks that we need to prioritise over the remainder, and tease out these high priority elements and articulate them as User Stories. These go onto the Product Backlog. What’s leftover from the large Epic/Theme/Feature goes back onto the Roadmap. The separation of the Roadmap from the Product Backlog allows us to more easily focus either on a more strategic longer term vision, or on more immediate “next release planning”.

step_2

Once a User Story has been added to the Product Backlog, it becomes ready for thinking/analysis effort in order to understand it well enough to be sized and planned in an iteration. We’ve had to articulate a Definition-Of-Ready to capture this, as early on we detected inconsistencies in our understanding of the User Stories, which was leading to poorer estimates and a less predictable flow of work. We’ve called this process the “Three Amigos” (it matches what other projects in the area call this aspect of backlog refinement).

step_3

When it’s ready for estimation, we use the Planning Game to size the Stories. As we have a variety of backlog items – requirements based stories, technical stories, we use a set of reference points – for example we have a requirement based reference story that’s a “5” as well as a DevOps environment configuration reference story that’s also a “5”. This makes it easier to estimate using relative sizing as we can always compare like with like.

Once a story is sized, it can then be prioritised to the top of the backlog, ready for Sprint Planning.