Add 02-ops-dev article
This commit is contained in:
parent
3c19b16dd0
commit
eb3244cf40
1 changed files with 336 additions and 0 deletions
336
content/post/02-ops-dev.md
Normal file
336
content/post/02-ops-dev.md
Normal file
|
|
@ -0,0 +1,336 @@
|
|||
---
|
||||
title: "Dear ops, it's 2019 and you need dev skills"
|
||||
subtitle: "Evolve, or die in pain"
|
||||
date: 2019-06-21
|
||||
draft: false
|
||||
tags: [ops, dev, devops, opinion]
|
||||
---
|
||||
|
||||
# To infinity... and beyond!
|
||||
|
||||
DevOps, IaC, AGILE, SCRUM, Containers, SysOps, CI/CD, microservices, serverless, FAAS
|
||||
|
||||
Yes, it's a buzzwords **shitstorm** but all of these concepts summarize how
|
||||
_software development_ and _deployment_ evolves. Do i think that sysadmins
|
||||
time is over ? **No**.
|
||||
|
||||
But time as come, _we_, as sysadmins, have to adapt to this moving
|
||||
industry.
|
||||
|
||||
Accept it or die, but [cloud
|
||||
computing](https://en.wikipedia.org/wiki/Cloud_computing) is the norm now.
|
||||
Small IT companies with "good old LAMP" as the only business model will die
|
||||
soon.
|
||||
|
||||
Dont' be scared, take a deep breathe and let's dive into this !
|
||||
|
||||
# DevOps is the new sexy
|
||||
|
||||
## DevOps = "Dev" + "Ops"
|
||||
|
||||
_DevOps_, _SecDevOps_, _DevOps Enginner_ are the most used terms in job
|
||||
requests. But, after all, what the fuck is that shit ?
|
||||
|
||||
In most cases, IT projets human resources can be split into two teams :
|
||||
|
||||
- [Dev](https://en.wikipedia.org/wiki/Programmer) : This team write
|
||||
application code. Pushed by marketing teams, clients or users, the main
|
||||
purpose of the dev team is **adding new feature** to the software.
|
||||
|
||||
- [Ops](https://en.wikipedia.org/wiki/System_administrator) : This team have
|
||||
to maintain operational conditions and ensure that **prod is up, up to date
|
||||
and stable**.
|
||||
|
||||
By means, this two job positions are in conflict. On one side, ops team wants
|
||||
stable and predictable stuff, on the other side, dev team wants liberty to
|
||||
move quickly in order to add new features and meet commercial goals and
|
||||
objectives.
|
||||
|
||||
What could be done to satisfy both worlds ? **DevOps**
|
||||
|
||||
## DevOps culture
|
||||
|
||||
First of all, no, DevOps is not a job title _(even if everyone on LinkedIn
|
||||
thinks so)_. It's a culture, a way of thinking and the more important point
|
||||
of this is : it tells how to organize software development teams.
|
||||
|
||||
The purpose is to put ops and devs to an agreement. Afterall, everyone works for the
|
||||
same goal : **build the best product, work less and make more money**.
|
||||
|
||||
DevOps presents a new way of thinking how teams collaborate. The main purpose
|
||||
of DevOps is to open discussions between Ops and Dev. Dev have to be aware of
|
||||
how the code is deployed and how the production systems are handled by the
|
||||
ops team. On the other hand, ops needs some dev skills in order to understand
|
||||
correcly what type of software is served in production.
|
||||
|
||||
To sum it up, DevOps could be presented using a loop of all these concepts and
|
||||
actions :
|
||||
|
||||
- Define a task, or feature, focused on needs from customers or clients
|
||||
- Implement this feature or execute the defined task
|
||||
- Code review
|
||||
- Tests (unit tests, integration tests, staging tests)
|
||||
- Functionnal testing on a preproduction environment
|
||||
- Push to production
|
||||
- Handle and analyse feedback from production
|
||||
|
||||
In order to archive all this tasks, teams have to communicate clearly on what
|
||||
should be done by each team and in some cases, there is an overlap between
|
||||
dev tasks and ops tasks but the final goal is the same : get to the best
|
||||
possible result.
|
||||
|
||||
## Devops methods and concepts
|
||||
|
||||
Now that the culture is presented, let's take a look at how this principles
|
||||
can be applied to the real world.
|
||||
|
||||
### Contract
|
||||
|
||||
The top most important thing is what I call a **contract**. This is an
|
||||
agreement that create a link between dev and ops. A contract needs to be as
|
||||
descriptive as possible. On one hand developpers needs to tell to administrators
|
||||
all of the project details, what needs to be run and what are the
|
||||
dependencies or services required by the application. On the other hand,
|
||||
operators have to understand dev needs and do all the plumbing to deploy dev
|
||||
requests to production.
|
||||
|
||||
In most cases, this contract can be represented by
|
||||
[containers](https://www.docker.com/resources/what-container) and a
|
||||
[docker-compose](https://docs.docker.com/compose/) file. It's declarative,
|
||||
easy to read, easy to understand and clear enough to know what needs to be
|
||||
run and what kind of plumbing is needed to make all services works together
|
||||
to create the whole application.
|
||||
|
||||
If this not clear enough, here is a generic example :
|
||||
|
||||
|
||||
{{< highlight yaml "linenos=table" >}}
|
||||
version: '3'
|
||||
|
||||
services:
|
||||
web:
|
||||
image: galaxies:version
|
||||
volumes:
|
||||
- ./src:/dest
|
||||
depends_on:
|
||||
- db
|
||||
ports:
|
||||
- 8080
|
||||
label:
|
||||
- "frontend.rule=Host:galaxies.rick"
|
||||
|
||||
db:
|
||||
image: postgres
|
||||
volumes:
|
||||
- ./mounts/db_data:/var/lib/postgresql/data
|
||||
environment:
|
||||
POSTGRES_USER: RICK
|
||||
POSTGRES_PASSWORD: C137
|
||||
POSTGRES_DB: galaxies
|
||||
{{< / highlight >}}
|
||||
|
||||
An ops, receiving this file, can extract a lot of information on how the
|
||||
application works.
|
||||
|
||||
- A web app that :
|
||||
- is accepting requests on port 8080
|
||||
- is responding using the url _galaxies.rick_
|
||||
- needs persitent storage
|
||||
- needs a a postgres database
|
||||
|
||||
Now let's explain what should be done by ops to push this to production :
|
||||
|
||||
0. Ensure that _galaxies.rick_ DNS points to production environment
|
||||
0. Pull galaxies:version image
|
||||
0. Ensure an access to a database (can be a container, a cluster or a standalone pg)
|
||||
0. Inject database variables into production environment
|
||||
0. Start the galaxies:version image
|
||||
0. Update HTTP reverse proxy rules to redirect _galaxies.rick_ to _galaxies:version_ on port _8080_
|
||||
|
||||
And **voilà**, now you have a clear line between what kind of stuff
|
||||
developers will push to production and how the operators will plug the
|
||||
project on the production environment.
|
||||
|
||||
### Automation
|
||||
|
||||
Okay, okay ! But as an ops, I don't want to take care of all this stuff everytime dev needs to push a new version of the sotfware !
|
||||
|
||||
Me too, and this is why the second top most important thing is **automation** !
|
||||
|
||||
Take a look at all the tasks described earlier, do you really want to make all
|
||||
those changes by hand using vim on the production environment ?
|
||||
|
||||
Ten or even twenty years ago, the first automated things was machine boostrap
|
||||
and basic configuration using scripts. Modern applications requirements means
|
||||
more machines, more complexity. The easieast way to handle this new level is
|
||||
to delegate some of the tasks to computers using **declarative** structures.
|
||||
|
||||
This is why tools like _Ansible_ are now popular and widely used. Today we
|
||||
want to describe a state and let tools do the stuff needed to get to this
|
||||
state. Why ? Because this is the simpliest way to normalize how things have
|
||||
to be done[^1] and to get complex systems up and running. If there is a bug
|
||||
or a missing feature in one of this tool, there is a good chance that you
|
||||
will have to put your hands in the grease and code it.
|
||||
|
||||
This first step of automation gives ops opportunity to work on tools
|
||||
associated to concepts like [Continuous
|
||||
Integration](https://en.wikipedia.org/wiki/Continuous_integration) and
|
||||
[Continuous Delivery](https://en.wikipedia.org/wiki/Continuous_delivery).
|
||||
|
||||
Do not forget that current apps do more things than serving a simple website.
|
||||
By using small iterations, dev gain the ability to make small changes often
|
||||
in contrast to doing major updates. This way of develivering software enhance
|
||||
stability because most of the code base is not changed between deployments
|
||||
to production. If something goes wrong it will be a lot easier to bisect to
|
||||
the root causes of this bug or unexpected app response.
|
||||
|
||||
To give dev the power to deliver atomic changes to pre-production, staging
|
||||
and production, they need to deploy all the stuff by themselves in a
|
||||
**predictable and reproducible** and the answer is automation !
|
||||
|
||||
### Metrics & alerting
|
||||
|
||||
```
|
||||
With great power comes great responsibility !
|
||||
```
|
||||
|
||||
This is why metrics matters. If devs can deploy stuff to prod, they also need
|
||||
to know if everything is working as expected and no, I will not give them
|
||||
root access to production !
|
||||
|
||||
Enabling low level metrics ensure to ops that production is up and running
|
||||
smoothly but this is not enough to apply DevOps principles. Devs also need
|
||||
**visitiliy** on how the app is handling requests (success / error rates) or
|
||||
some status about the queue system. Every team needs different sets of
|
||||
metrics, specific to their missions.
|
||||
|
||||
With metrics, comes **alerting**. If metrics are well defined, alterting can
|
||||
be routed to people who are aware of what can be done to resolve the problem.
|
||||
For example, if the app goes crazy, a member of the dev team will be in the
|
||||
right position to take action and fix the problem but if it's a proxy memory
|
||||
leak, an ops will probably know what to do and to dig, to find and resolve
|
||||
the issue.
|
||||
|
||||
Performance monitoring is as mandatory as alerts. Teams wants to be sure that
|
||||
the newly pushed feature is working and that there is no performance drop
|
||||
somewhere else in the application.
|
||||
|
||||
### Culture, sharing and empathy
|
||||
|
||||
Sharing is caring
|
||||
|
||||
Sharing is one of the key to apply DevOps concepts and culture. In this
|
||||
situation, sharing is not restricted to **communication** but also
|
||||
**responsabilities**
|
||||
|
||||
If dev teams **share responsabilities** with ops team, there is a all new
|
||||
field of possibilities to collaborate and simplify deployment and
|
||||
maintenance. For ops team, a better communication and shared responsabilities
|
||||
with dev ensure that they have access to informations regarding businesses
|
||||
goals, productions requirements.
|
||||
|
||||
If devs and ops are both responsible of failure, or sucess, of the product,
|
||||
there is less possibility to fall in a blame counter-blame situation. With
|
||||
more communication and a strong trust chain between members of ops and dev
|
||||
teams, everyone gets autonomy and a voice in dev or deployment process.
|
||||
|
||||
Last but not least, a lots of **empathy** is needed. Failure is a success !
|
||||
It's one of the best way to learn new things but this is only possible in a
|
||||
safe and tolerant environment. The war between dev and ops is over. Listening
|
||||
and talking to eveyrone will probably help every member of the teams.
|
||||
|
||||
## Limits
|
||||
|
||||
Of course, DevOps is not perfect. When misunderstood, this can be quite
|
||||
catastrophic. Reducing DevOps to a buzzword is as silly as taking everything
|
||||
as Truth in the holy Bible. DevOps is a mentality, you will have to create
|
||||
your own DevOps culture.
|
||||
|
||||
A lots of managers and leaders thinks that DevOps means, fire half of the
|
||||
devs , half of the admin, mixup, and voilà. Is this sharing ? No ! Doing
|
||||
DevOps means more distributed responsability and less stress for everyone.
|
||||
|
||||
Another wrong idea is to think that more deployments means more features.
|
||||
Integrating deployments automations and testing is only used to enhance
|
||||
robusteness of the all system. The idea is not to make more feature, but more
|
||||
small changes in order to increase poduction stability.
|
||||
|
||||
# A new open field
|
||||
|
||||
After all that, there is clearly an emerging need of programming skills
|
||||
common to all the principles and methods presented.
|
||||
|
||||
## Why Industry needs (smart)ops ?
|
||||
|
||||
A smartops is someone who clearly understand that the IT industry is
|
||||
changing. Everything is moving to the **cloud**, more and more services are
|
||||
externalized and everything becomes more and more automated. All this stuff
|
||||
creates a violent shift between two sets of methods.
|
||||
|
||||
- an old one
|
||||
- launch command in a terminal using ssh
|
||||
- bash scripts to setup things
|
||||
- edit file directly on production using _vim_
|
||||
|
||||
- a new one
|
||||
- pipelines
|
||||
- automation
|
||||
- services interactions between HTTP services
|
||||
|
||||
No I'm not saying that ssh is dead. I'm saying that methods evolves.
|
||||
|
||||
As more automation means less human action, there is clearly a move to
|
||||
descriptive infrastructure deployments and internal services doing all the
|
||||
plumbing stuff needed to get a stable and viable production.
|
||||
|
||||
In order to achieve all this new challenges, industry needs to delegate tasks
|
||||
to smart programs write using code. This new services and automation programs
|
||||
have to be **written by ops**, beceause they are the ones who trully knows
|
||||
how to run production systems at scale. But, sorry to say that, perl and bash
|
||||
scripts can do that kind of jobs. More automation of everything also means
|
||||
automation of the most complex tasks in the stack and this is where scripting
|
||||
langages are not enough.
|
||||
|
||||
For the ones who thinks I'm wrong, maybe. But here is my opinion based on a
|
||||
lots of bash and perl script experiments. When things needs at least
|
||||
parallelism, http requests and response manipulations, strong error handling
|
||||
or ability to push stuff inside a monitoring stack, golang will be my choice
|
||||
and I deeply think it should be yours too because this is the main purpose
|
||||
of this kind of new languages, created specificaly to answer dev and ops
|
||||
problems.
|
||||
|
||||
Moving from scripting to programming will also help smartops understand how
|
||||
software they put in production works. By knowing how to construct a
|
||||
software, ops will gain the ability to help devs, integrating every one in a
|
||||
DevOps culture.
|
||||
|
||||
## New profiles, new horizons
|
||||
|
||||
Yes, ops **needs dev skills** in order to get a role in teams resolving new
|
||||
challenges that comes with modern infrastructures and cloud infrastructures.
|
||||
|
||||
This changing ecosystem also gives evolution ability to ops and dev. With
|
||||
efforts, everyone can, at least, take a look at how roles and interconnection
|
||||
between ops and dev works. To be clear, i'm saying that dev also needs ops
|
||||
skills ! But i keep that for another article, stay tuned.
|
||||
|
||||
If old ops don't want to make the effort, that's not a problem because new kind of
|
||||
people get what is happening. Believe it or not but it's real. The smartops
|
||||
community is **inclusive**. Even if this is not perfect yet, the _golang_ and
|
||||
_k8s_ is clearly LGBT and women friendly !
|
||||
|
||||
I want to thanks all the _LBGT_||women gophers||ops[^2] I follow because they
|
||||
are the roots of this wonderful and refreshing community[^3]. The best thing
|
||||
I can do is to invite you to follow this people ! Here is the list :
|
||||
|
||||
- [Jessie Frazelle](https://twitter.com/jessfraz)
|
||||
- [Ashley McNamara](https://twitter.com/ashleymcnamara)
|
||||
- [Ellen Korbes](https://twitter.com/ellenkorbes)
|
||||
- [Kris Nova](https://twitter.com/krisnova)
|
||||
- [Jaana B. Dogan](https://twitter.com/rakyll)
|
||||
- [Francesc Campoy](https://twitter.com/francesc)
|
||||
- [Aditya Mukerjee](https://twitter.com/chimeracoder)
|
||||
|
||||
[^1]: Alice likes apt-get, Bob likes aptitude ? I don't care, I just want a standardized way to install a package
|
||||
[^2]: This is an inclusive OR
|
||||
[^3]: If you want to be removed or added to the list, just send me a tweet or whatever.
|
||||
Loading…
Reference in a new issue