Modern Agile: How to always learn about your software process

This blogpost intends to be the first one in a series of four blogposts delving into the discussion about what is modern agile, how it relates to regular agile and how companies can start adapting to it. The original point of this piece was writing a blogpost about how we at Vinta created our own software process. It focused on the idea that companies shouldn't be implementing the full package of scrum, crystal or any other methodology. The idea was that even scrum is too much for some companies and that upholding the principles is more important than doing the practices out of the box. Then we caught wind of the growing discussions around Modern Agile.

We unknowingly already upheld the 4 principles here and think that it's really amazing that someone gave a name to it and formalized the knowledge. Joshua Kerievsky, one of the creators of this 4 principles of modern agile, gave an amazing keynote which you can check here where he amazingly explains that the agile principles are great, but historic, the bar has been raised and we need to strive for even higher goals. Then we at Vinta decided to make a blog series about each of the 4 modern agile principles: Make People Awesome, Deliver value continuously, Make Safety a Prerequisite and Experiment and Learn Rapidly. On this first one, we will talk about Experiment and Learn Rapidly, or how do we experiment and learn as fast as we can about how to manage the development of software.

Why not the full package

It's been sixteen years since the Agile Manifesto was released (feels like a couple of years ago, right?!). It was the first time a lot of people had heard about agile methodologies. Over the following years, names like Scrum, Kanban, and so many others were introduced to the software development community. People started talking about project management in a totally different way than they did before. Nowadays, the Waterfall model and its variations like the Spiral model are still around, and will still be for a long time, but they are far from being the norm. It was a big shift in mentality, people learned how to replace planning for learning and iterating. But now we have new challenges ahead of us. Now it’s not only about "responding to change over following a plan". It’s about becoming more and more comfortable with change. It’s about making sure we learn after every change. Making sure that this loop happens faster and faster, and that we feel empowered enough to change it whenever it’s going the wrong way. It’s all about Experiment and learn rapidly. Our process needs to not only allow that but incentivize it. We are also not the only ones advocating this. Spotify makes a great case about why experimentation is good and how can we iterate with our own process over time.

In that sense, neither Scrum or any other agile methodologies need to be a norm. As Mike Sutton states here: Scrum is, often, marketed as a whole package, a monolithic solution where you can't take stuff out or put stuff in. But that shouldn't be the case. Even Scrum, as simple as it is, has techniques to solve problems some companies don't have. That's why at Vinta we don’t say we use Scrum or any other specific methodology. We have our custom agile methodology. We created it by aggregating bits of other methodologies after every iteration. Always adapting them to our preferences. We call it Vinta's agile process (very creative, right? :P). I’ll describe some of it here, for a more detailed description, check our playbook!

alt text

Overview, principles and the beginning of experimentation

Both Modern Agile and normal Agile rely on principles that are the building blocks for practices and processes. When we created our process, we focused on the agile principles because they were all that was. But we decided not to use a whole package like scrum and decided to iterate until we grew our own process. This decision was mostly based on our knowledge on the Lean Startup cycle.

This is a very important point to empathize: It doesn’t take an expert agile manager to iterate and learn rapidly about your agile process. Some techniques might be too advanced or rely on experienced programmers to work, but the whole process isn’t. To build, measure and learn you only need enough knowledge about your goal to see if you are getting closer to it or not and a commitment to stay with the validated output, even though your ego was in another option.

Here is how we chose to implement the four principles and how some of our processes look today, after a few iterations: Sprints are 1-week size. We believe they are the building block of the agile process, the idea is that deliveries and deploys should be made in small packages and built upon week after week. In our experience, if they last for 2 weeks or more, the client can stay too long without a delivery. Also, it's easier to slack off a little in the first week and run like crazy in the second, that would difficult measurement and learning after every feature delivered. They are also a lightweight process aligned to "Responding to change over following a plan" Use of Trello as the main management tool and the single source of truth. There, we align expectations. We believe aligning expectations is one of the most important things in project management. Trello is a very good tool for that because non-technical people can communicate with us over cards, checklists, and documents. This is to emphasize the "Customer collaboration over contract negotiation" principle. Use of GitHub for Git repository hosting and code review, along with continuous integration tools, like CircleCI and Travis CI. We believe automated testing is essential to ensure the "Working software over comprehensive documentation" principle. This way we greatly reduce the amount of bugs that appear in production. Along with our PR process, it helps distribute code ownership and increase the number of eyeballs looking at each line of code. Use of Slack for real-time communication, since we consider "Individuals and interactions over processes and tools". In order to better align expectations, we value transparency so keeping everyone updated on the daily basis avoids surprises. Daily interaction with the client using IN/OUT(more on that later in this post) leaves everyone on the same page.

alt text

No Estimates

Before diving into our process, it's important to know that we don’t do estimates on projects. This doesn’t mean that our practices of techniques don’t work in other contexts. But it does mean that some of them were picked and built with a No Estimates model in mind and work better with it.

To us, #noestimates was a decision made early on, mostly because we believe estimating is bad for both sides (the contractor and the client). To better exemplify that, here is how people think a fixed bid project works:

‘’’Project estimate -> internal development of the estimated features -> delivery to the client’’’

This sounds good in theory, but none of the parts happen as they are supposed to. Starting from the beginning, even Psychology tells us that we are way less analytical than we think when it comes to prediction. We tend to overestimate our abilities to deliver and to see everything in hindsight as ‘perfectly predictable’. That might be ok for short estimations, but when we are talking about projects that last more than 3 months, it gets impossible to accurately predict everything that might happen to delay or speed-up the project. Then, when you are developing a feature, lots of things can happen. It usually takes some time to study, bugs might take longer than expected to be corrected and integrations are never as smooth as predicted. Some people say, as you can check here and here, that these problems can be corrected with a better risk management or an improved estimate method. But, even with those corrections, we believe that in environments with more uncertainty, these methods are not enough to fix the problem. The client can change his mind, or the customers could validate the feature out, or even a new and urgent feature can pop up. These are only some problems that might come up. And there is just no getting ready for it in advance, you have to answer to it and it’s way easier if you are not bound by contract to deliver something that is not in the best interest of your client anymore.

Instead, when we talk about no estimates, we charge and agree on contracts based on hours worked, not features delivered. That means a project is never finished until the client is satisfied or the budget for that project ends. Although scary at first (the client won’t know exactly when the project will end, or how much will it cost), there are a lot of benefits that come from it: we don’t need to estimate, since we saw it’s difficult and error prone, that’s a relief; everything takes the time it should take to be done, that means more testing, more study and more time invested in quality, which leads to better code and increased maintainability; the client doesn’t need to waste time thinking/predicting every single need she/he might have. Also, she/he is forced to stay engaged to the project and receives more deliveries according to what she/he wanted, instead of waiting a long period of time to see something that isn’t exactly right.

Want to talk to us more about #noestimates? Comment on this post or mail us. That’s a subject we love to talk about. :)

Relationship with Scrum

We chose agile and decided how we could uphold the 4 agile principles in the way it made the most sense to us. But we did get a lot of tools from the same place: Scrum. Even so, we knew every technique that comes with a process adds weight to the company, and we didn’t want to bring any more weight than necessary. Mostly because it gets in the way of good experimentation. As you risk that people find problems in a technique that doesn’t solve anything and blaming the whole methodology for it. Once new problems arise, we incorporate new pieces to the methodology. For instance, we didn't do sprint retrospectives until a short time ago. Our teams were very small, like 2 to 3 people, so every lesson and opportunity to improve was caught on the spot or settled in our weekly meeting (which focuses on sharing lessons learned as well as keeping everyone on the same page). This also made formal sprint retrospective meetings rather unnecessary. Now that we feel we need to do it, we found a lightweight method, called the DAKI matrix. It's an open chat between project members to fill the DAKI board with things that should be Dropped, Added, Kept and Improved. It gets even better aligned with a track record so that you can understand at a glance what's changed and what requires a different solution. We are still experimenting with it and every team is free to try out anything that best suits their project. We also don't do the traditional stand-up meeting. So far our IN/OUT methodology along with continuous Slack communication has suited us well. This way the client always knows what's being done every single day and can talk to us at any time. We also don't have a formal appointment of a Scrum Master (or a Technical Facilitator, as Crystal users would call it, or any other name for the role that solves all problems). Our project management process has been enough so far. Especially because appointing someone with the role of being responsible for all of the problems of a project would be too overloading, we prefer to spread it out. Aside from that, we kept a lot of things from regular scrum. Like the weekly meetings, that we mentioned before. In it, each team has a timeboxed space to explain what they did and what they learned during the week before. We use a concept similar to the product Backlog, as well. We manage the client to collect requirements and inputs to create a backlog and sort tasks. But, as said earlier about we not doing estimates, this Roadmap is non-binding and we only focus on the top of the list that fits in a Sprint. We use stories too, and we try to fit those in Trello cards. But we understand that every project has its peculiarities so teams are free to experiment and figure what's more efficient for the particular problems of a given project.

Elements of other Methodologies and Experiments

We get a lot of things from other processes like XP's pairing preference (if any dev can, she/he should pair), Kanban's pull orientation on Trello and a bunch other stuff. We are also constantly experimenting with our process. Just like we are experimenting DAKI on retrospectives. Our process changed considerably over the last year and will change even more in the years to come, as a result of what we learn. Maybe that's the most important principle of all: Keep Learning. ALWAYS. In order to do that: don’t just follow the methodology for the sake of saying you are agile. You can and should adjust and compose methodologies to better fit your scenario.

Want to start a discussion about what to take out or put in an agile process? Feel like modern agile is not that big of a deal? Interested in talking more about how this relates to what you do in your company? Reach out in the comments or mail us!

Rob Novelino

Creator of organizational tools, whether they have code lines or not. Likes to read about crazy things like futurism, psychology, education and new economic models. When coding, goes for Python/Django