Add 02-ops-dev article

This commit is contained in:
Wilfried OLLIVIER 2019-06-21 16:46:35 +02:00
parent 3c19b16dd0
commit eb3244cf40

336
content/post/02-ops-dev.md Normal file
View file

@ -0,0 +1,336 @@
---
title: "Dear ops, it's 2019 and you need dev skills"
subtitle: "Evolve, or die in pain"
date: 2019-06-21
draft: false
tags: [ops, dev, devops, opinion]
---
# To infinity... and beyond!
DevOps, IaC, AGILE, SCRUM, Containers, SysOps, CI/CD, microservices, serverless, FAAS
Yes, it's a buzzwords **shitstorm** but all of these concepts summarize how
_software development_ and _deployment_ evolves. Do i think that sysadmins
time is over ? **No**.
But time as come, _we_, as sysadmins, have to adapt to this moving
industry.
Accept it or die, but [cloud
computing](https://en.wikipedia.org/wiki/Cloud_computing) is the norm now.
Small IT companies with "good old LAMP" as the only business model will die
soon.
Dont' be scared, take a deep breathe and let's dive into this !
# DevOps is the new sexy
## DevOps = "Dev" + "Ops"
_DevOps_, _SecDevOps_, _DevOps Enginner_ are the most used terms in job
requests. But, after all, what the fuck is that shit ?
In most cases, IT projets human resources can be split into two teams :
- [Dev](https://en.wikipedia.org/wiki/Programmer) : This team write
application code. Pushed by marketing teams, clients or users, the main
purpose of the dev team is **adding new feature** to the software.
- [Ops](https://en.wikipedia.org/wiki/System_administrator) : This team have
to maintain operational conditions and ensure that **prod is up, up to date
and stable**.
By means, this two job positions are in conflict. On one side, ops team wants
stable and predictable stuff, on the other side, dev team wants liberty to
move quickly in order to add new features and meet commercial goals and
objectives.
What could be done to satisfy both worlds ? **DevOps**
## DevOps culture
First of all, no, DevOps is not a job title _(even if everyone on LinkedIn
thinks so)_. It's a culture, a way of thinking and the more important point
of this is : it tells how to organize software development teams.
The purpose is to put ops and devs to an agreement. Afterall, everyone works for the
same goal : **build the best product, work less and make more money**.
DevOps presents a new way of thinking how teams collaborate. The main purpose
of DevOps is to open discussions between Ops and Dev. Dev have to be aware of
how the code is deployed and how the production systems are handled by the
ops team. On the other hand, ops needs some dev skills in order to understand
correcly what type of software is served in production.
To sum it up, DevOps could be presented using a loop of all these concepts and
actions :
- Define a task, or feature, focused on needs from customers or clients
- Implement this feature or execute the defined task
- Code review
- Tests (unit tests, integration tests, staging tests)
- Functionnal testing on a preproduction environment
- Push to production
- Handle and analyse feedback from production
In order to archive all this tasks, teams have to communicate clearly on what
should be done by each team and in some cases, there is an overlap between
dev tasks and ops tasks but the final goal is the same : get to the best
possible result.
## Devops methods and concepts
Now that the culture is presented, let's take a look at how this principles
can be applied to the real world.
### Contract
The top most important thing is what I call a **contract**. This is an
agreement that create a link between dev and ops. A contract needs to be as
descriptive as possible. On one hand developpers needs to tell to administrators
all of the project details, what needs to be run and what are the
dependencies or services required by the application. On the other hand,
operators have to understand dev needs and do all the plumbing to deploy dev
requests to production.
In most cases, this contract can be represented by
[containers](https://www.docker.com/resources/what-container) and a
[docker-compose](https://docs.docker.com/compose/) file. It's declarative,
easy to read, easy to understand and clear enough to know what needs to be
run and what kind of plumbing is needed to make all services works together
to create the whole application.
If this not clear enough, here is a generic example :
{{< highlight yaml "linenos=table" >}}
version: '3'
services:
web:
image: galaxies:version
volumes:
- ./src:/dest
depends_on:
- db
ports:
- 8080
label:
- "frontend.rule=Host:galaxies.rick"
db:
image: postgres
volumes:
- ./mounts/db_data:/var/lib/postgresql/data
environment:
POSTGRES_USER: RICK
POSTGRES_PASSWORD: C137
POSTGRES_DB: galaxies
{{< / highlight >}}
An ops, receiving this file, can extract a lot of information on how the
application works.
- A web app that :
- is accepting requests on port 8080
- is responding using the url _galaxies.rick_
- needs persitent storage
- needs a a postgres database
Now let's explain what should be done by ops to push this to production :
0. Ensure that _galaxies.rick_ DNS points to production environment
0. Pull galaxies:version image
0. Ensure an access to a database (can be a container, a cluster or a standalone pg)
0. Inject database variables into production environment
0. Start the galaxies:version image
0. Update HTTP reverse proxy rules to redirect _galaxies.rick_ to _galaxies:version_ on port _8080_
And **voilà**, now you have a clear line between what kind of stuff
developers will push to production and how the operators will plug the
project on the production environment.
### Automation
Okay, okay ! But as an ops, I don't want to take care of all this stuff everytime dev needs to push a new version of the sotfware !
Me too, and this is why the second top most important thing is **automation** !
Take a look at all the tasks described earlier, do you really want to make all
those changes by hand using vim on the production environment ?
Ten or even twenty years ago, the first automated things was machine boostrap
and basic configuration using scripts. Modern applications requirements means
more machines, more complexity. The easieast way to handle this new level is
to delegate some of the tasks to computers using **declarative** structures.
This is why tools like _Ansible_ are now popular and widely used. Today we
want to describe a state and let tools do the stuff needed to get to this
state. Why ? Because this is the simpliest way to normalize how things have
to be done[^1] and to get complex systems up and running. If there is a bug
or a missing feature in one of this tool, there is a good chance that you
will have to put your hands in the grease and code it.
This first step of automation gives ops opportunity to work on tools
associated to concepts like [Continuous
Integration](https://en.wikipedia.org/wiki/Continuous_integration) and
[Continuous Delivery](https://en.wikipedia.org/wiki/Continuous_delivery).
Do not forget that current apps do more things than serving a simple website.
By using small iterations, dev gain the ability to make small changes often
in contrast to doing major updates. This way of develivering software enhance
stability because most of the code base is not changed between deployments
to production. If something goes wrong it will be a lot easier to bisect to
the root causes of this bug or unexpected app response.
To give dev the power to deliver atomic changes to pre-production, staging
and production, they need to deploy all the stuff by themselves in a
**predictable and reproducible** and the answer is automation !
### Metrics & alerting
```
With great power comes great responsibility !
```
This is why metrics matters. If devs can deploy stuff to prod, they also need
to know if everything is working as expected and no, I will not give them
root access to production !
Enabling low level metrics ensure to ops that production is up and running
smoothly but this is not enough to apply DevOps principles. Devs also need
**visitiliy** on how the app is handling requests (success / error rates) or
some status about the queue system. Every team needs different sets of
metrics, specific to their missions.
With metrics, comes **alerting**. If metrics are well defined, alterting can
be routed to people who are aware of what can be done to resolve the problem.
For example, if the app goes crazy, a member of the dev team will be in the
right position to take action and fix the problem but if it's a proxy memory
leak, an ops will probably know what to do and to dig, to find and resolve
the issue.
Performance monitoring is as mandatory as alerts. Teams wants to be sure that
the newly pushed feature is working and that there is no performance drop
somewhere else in the application.
### Culture, sharing and empathy
Sharing is caring
Sharing is one of the key to apply DevOps concepts and culture. In this
situation, sharing is not restricted to **communication** but also
**responsabilities**
If dev teams **share responsabilities** with ops team, there is a all new
field of possibilities to collaborate and simplify deployment and
maintenance. For ops team, a better communication and shared responsabilities
with dev ensure that they have access to informations regarding businesses
goals, productions requirements.
If devs and ops are both responsible of failure, or sucess, of the product,
there is less possibility to fall in a blame counter-blame situation. With
more communication and a strong trust chain between members of ops and dev
teams, everyone gets autonomy and a voice in dev or deployment process.
Last but not least, a lots of **empathy** is needed. Failure is a success !
It's one of the best way to learn new things but this is only possible in a
safe and tolerant environment. The war between dev and ops is over. Listening
and talking to eveyrone will probably help every member of the teams.
## Limits
Of course, DevOps is not perfect. When misunderstood, this can be quite
catastrophic. Reducing DevOps to a buzzword is as silly as taking everything
as Truth in the holy Bible. DevOps is a mentality, you will have to create
your own DevOps culture.
A lots of managers and leaders thinks that DevOps means, fire half of the
devs , half of the admin, mixup, and voilà. Is this sharing ? No ! Doing
DevOps means more distributed responsability and less stress for everyone.
Another wrong idea is to think that more deployments means more features.
Integrating deployments automations and testing is only used to enhance
robusteness of the all system. The idea is not to make more feature, but more
small changes in order to increase poduction stability.
# A new open field
After all that, there is clearly an emerging need of programming skills
common to all the principles and methods presented.
## Why Industry needs (smart)ops ?
A smartops is someone who clearly understand that the IT industry is
changing. Everything is moving to the **cloud**, more and more services are
externalized and everything becomes more and more automated. All this stuff
creates a violent shift between two sets of methods.
- an old one
- launch command in a terminal using ssh
- bash scripts to setup things
- edit file directly on production using _vim_
- a new one
- pipelines
- automation
- services interactions between HTTP services
No I'm not saying that ssh is dead. I'm saying that methods evolves.
As more automation means less human action, there is clearly a move to
descriptive infrastructure deployments and internal services doing all the
plumbing stuff needed to get a stable and viable production.
In order to achieve all this new challenges, industry needs to delegate tasks
to smart programs write using code. This new services and automation programs
have to be **written by ops**, beceause they are the ones who trully knows
how to run production systems at scale. But, sorry to say that, perl and bash
scripts can do that kind of jobs. More automation of everything also means
automation of the most complex tasks in the stack and this is where scripting
langages are not enough.
For the ones who thinks I'm wrong, maybe. But here is my opinion based on a
lots of bash and perl script experiments. When things needs at least
parallelism, http requests and response manipulations, strong error handling
or ability to push stuff inside a monitoring stack, golang will be my choice
and I deeply think it should be yours too because this is the main purpose
of this kind of new languages, created specificaly to answer dev and ops
problems.
Moving from scripting to programming will also help smartops understand how
software they put in production works. By knowing how to construct a
software, ops will gain the ability to help devs, integrating every one in a
DevOps culture.
## New profiles, new horizons
Yes, ops **needs dev skills** in order to get a role in teams resolving new
challenges that comes with modern infrastructures and cloud infrastructures.
This changing ecosystem also gives evolution ability to ops and dev. With
efforts, everyone can, at least, take a look at how roles and interconnection
between ops and dev works. To be clear, i'm saying that dev also needs ops
skills ! But i keep that for another article, stay tuned.
If old ops don't want to make the effort, that's not a problem because new kind of
people get what is happening. Believe it or not but it's real. The smartops
community is **inclusive**. Even if this is not perfect yet, the _golang_ and
_k8s_ is clearly LGBT and women friendly !
I want to thanks all the _LBGT_||women gophers||ops[^2] I follow because they
are the roots of this wonderful and refreshing community[^3]. The best thing
I can do is to invite you to follow this people ! Here is the list :
- [Jessie Frazelle](https://twitter.com/jessfraz)
- [Ashley McNamara](https://twitter.com/ashleymcnamara)
- [Ellen Korbes](https://twitter.com/ellenkorbes)
- [Kris Nova](https://twitter.com/krisnova)
- [Jaana B. Dogan](https://twitter.com/rakyll)
- [Francesc Campoy](https://twitter.com/francesc)
- [Aditya Mukerjee](https://twitter.com/chimeracoder)
[^1]: Alice likes apt-get, Bob likes aptitude ? I don't care, I just want a standardized way to install a package
[^2]: This is an inclusive OR
[^3]: If you want to be removed or added to the list, just send me a tweet or whatever.