A common code release looks like this:
System before change consists of three components
System after change: One component is replaced with an improved component.
If something goes wrong and and a rollback is possible, the rollback should replace the new production code with the old production code (replace C2’ with C2). The rollback plan usually means:
1. Revert the current release by committing the opposite diff to the project’s version control.
2. Deploy the above change to production.
3. Restart the project’s services to apply the new code.
4. Once the rollback process is completed, additional fixes may be needed such as repairing damaged data.
Every developer knows that it does not matter if you work in Agile, Scrum, Lean, Kanban or Waterfall methodologies, when it comes to software projects — something always gets out of control. Requirements tend to change frequently, time estimations are a wild guess, coordination with other teams is not perfect, and every person in the release chain can easily fail the deployment. How can software engineers deliver under these conditions?
During my work as a backend team leader, I got to manage a few of the complexed software projects of the company. Most projects were challenging technically since we had to replace backend components of a live system which handles millions of agents but they were no less challenging socially, since I needed the cooperation of many people in the organization in order to push the code to production.
Diving into new technologies is not easy, especially when you are already an expert in another technology. It’s like learning a new language: You are afraid to sound stupid speaking it, but it’s the only way to improve. What did I learn about approaching new technologies? Specifically, how I overcome the steep learning curve and expedite it?
A few years ago, I became responsible for a new system. The code was in Lua and used Nginx as the HTTP server. I was programming in Ruby and wasn’t familiar with these technologies. As a young programmer, the thought of diving into massive amounts of unfamiliar code written in an unfamiliar language with minimal guidance terrified me. Since there was no other option I just did it. I learned the code, performed occasional bug fixes and wrote new features. It was not easy at the beginning: Each debugging session took hours, and I had to test every new feature countless times. There were many hours staring blankly at a screen.