Jenkins: Jenkins Unchained! Deploy Jenkins to Cloud: 3-in-1
Jenkins is one of the most popular Continuous Integration servers on the market today. It is designed to maintain, secure, communicate, test, build, and improve the software development process. Setting up Jenkins and running build jobs is not enough for a production infrastructure.
For optimal performance and results, architecting, designing, and implementing a production-grade Jenkins deployment is essential. In agile development practices, developers need to integrate their work to fix bugs or to create a new feature or functionality. Jenkins is used for continuous integration, helping to enforce the principles of agile development.
This comprehensive 3-in-1 course is a step-by-step approach to get you up-and-running with Jenkins and then quickly moves on to implement a scalable and production-grade infrastructure. Implement a scalable, stable, and robust production grade infrastructure. Combine multiple services to create optimal code delivery workflows and pipelines. Discover the process of using Jenkins to build, test, and package Java applications. Create a Git repository and Docker images using a local development environment, and other DevOps configuration tools. Get equipped to deploy a modern Jenkins System by provisioning slave nodes to the cloud with Ansible and Docker.
Contents and Overview
This training program includes 3 complete courses, carefully chosen to give you the most comprehensive training possible.
The first course, Practical Jenkins, covers automation of Jenkins deployment in no time!. Get up and running with Jenkins and deliver an optimal Jenkins deployment. Explore and configure features such as high availability, security, monitoring, and backing up/restoring data, which are basically all of the things you need to implementing a scalable and production grade infrastructure. Learn how to implement distributed builds, automate build pipelines, and integrate your Jenkins deployment with external services, thus showing you how to increase your team’s productivity with pipeline as a code building advanced pipelines faster and easier.
By the end of this video course, you will be able to automate, implement, secure, and manage your Jenkins deployment in no time.
The second course, Hands-On Continuous Integration and Automation with Jenkins, covers building, testing, and packaging applications with Jenkins in this hands-on video course supported by practical real-world examples. Delve into the installation of the required software dependencies and libraries and demonstrates the workflow you’ll need to follow to perform continuous integration for a sample application. Learn how to integrate code repositories and build tools in order to build code pipelines to implement both continuous integration and continuous delivery. Finally, you will also learn to automate deployment to a cloud platform such as AWS.
The third course, Deploying Jenkins to the Cloud with DevOps Tools, covers professional practices, techniques, and solutions for using Jenkins in cloud environments. Explore different AWS services and use them for Continuous Integration. Customize and configure your Jenkins master automatically on boot-up with Groovy code in order to achieve the build process we want. This includes adding secrets to the credentials store, installing and configuring plugins, and setting some basic values within the Jenkins main configuration. Once we are able to interact with Gitlab, we will then configure a way to run Jenkins nodes on Kubernetes so that we can actually build our software.
When everything looks good in our development environment, we will then see what an almost production-ready CI system in the cloud looks like, especially with regard to security aspects.
By the end of the course, you’ll implement and manage your production – grade Jenkins deployments to build, test, and package applications, supported by practical real-world examples.
About the Authors
- Anirban Saha is an infrastructure professional with more than seven and half years’ experience in infrastructure management at various industries and organizations ranging from early startups to corporate environments. He has worked extensively with configuration management and automation tools including Puppet, Chef, Ansible, Saltstack, and Terraform, to name a few. He has extensive experience in architecting, deploying, and managing large infrastructures and speaks at various conferences on the latest technologies.
- Sandro Cirulli is a certified Jenkins engineer, co-maintainer of XSpec, an open source unit testing framework for XML technologies, and co-organizer of DevOps Oxford Meetup. Sandro currently works as Lead Language Technologist in the Dictionaries department of Oxford University Press (OUP) where he’s in charge of system administration, cloud, and DevOps. Sandro holds an MS degree in Computer Science from Oxford Brookes University and blogs at sandrocirulli. net.
- Martin Reinhardt has around 10 years’ experience in research, enterprise solutions, consulting, portal development, JEE solutions, application and infrastructure integration, test automation, virtualization, and Continuous Integration and Delivery. Specialties: Model driven architecture (MDA), Java Enterprise solutions, test automation with Selenium, Continuous Integration
Practical Jenkins
This video will give you an overview about the course.
To work with any technology, it is crucial and mandatory to have a detailed understanding of the concepts and terminologies associated with it.
- Learn about Jenkins and it’s features
- Understand the development stages and Jenkins’ role in supporting them
- Learn about advanced features of Jenkins
Jenkins is a release engineering tool and there are few terminologies which need to be clarified with respect to Jenkins and release engineering.
- Explore Continuous Integration in detail
- Explore Continuous Delivery in detail
- Learn about Continuous Deployment in detail
To carry out even the most basic development support tasks with Jenkins, some essential tools need to be prepared such as softwares, repositories, storage and others.
- Install git and the Java runtime environment
- Create a Github account and add basic data to use repositories
- Set up dedicated storage for Jenkins data
To use Jenkins, an installation is required first. Jenkins is not bound to a specific method and can be installed using different procedures and on different infrastructure platforms.
- Install Jenkins via packages from a Jenkins managed repository
- Install Jenkins by deploying the Jenkins WAR file in Apache Tomcat
- Install Jenkins in a Docker container
After installation, Jenkins is accessible in form which is not very user-friendly. Some configurations and steps are needed to make it available for first time use and to users.
- Install nginx and add reverse proxy configuration
- Access Jenkins user interface and explore the setup wizard
- Configure SSH user with keys and git data
Preparation, installation and configuration of Jenkins is a long process and even more tedious if needed to perform repeatedly for different environments and in case of failures. It needs automation.
- Disable setup wizard and automate the steps using scripts
- Create Puppet module for Jenkins installation and configuration
- Apply Puppet module to node and automate the process
The primary function of Jenkins is to let users configure jobs and run builds from them. Although they can be done from the user interface, for more large and complex deployments, they need to be done from the command line or by using scripts.
- Create a small program and write tests for it
- Create a Jenkins job from the user interface and run builds
- Learn about Jenkins API and create jobs using API
With the growing number of projects in a Jenkins deployment, it is crucial to make sure that it withstands failures without any downtime. A highly available infrastructure tries to achieve this objective.
- Explore high availability support in Jenkins ecosystem
- Create multiple Jenkins master nodes with shared storage
- Use HAProxy as load balancer and test by failing nodes
Jenkins data, be it build history, information or configuration files, it is a good production practice to back them up so that they can be restored in event of a failure.
- Explore methods to backup and restore Jenkins data
- Install Periodic Backup plugin, configure and run backup of Jenkins data
- Simulate data loss and restore from backup
Without proper monitoring of the Jenkins infrastructure and its various components, it would be impossible to know about arising problems and fix them in time.
- Monitor Jenkins deployment using the monitoring plugin
- Monitor Jenkins metrics using Graphite reporting plugin and a graphite server
- Monitor Jenkins components using "monit"
With bigger deployments, comes bigger risk of data and permission abuse including accidental actions causing widespread damage. The only way to check these situations is by implementing correct security measures and role-based access control.
- Explore best practices in security and advanced authentication mechanisms
- Configure LDAP based security realm for authentication
- Manage access control using role-based authorization strategy
The Jenkins user interface is quite good and very helpful in performing tasks. However, for the more automation savvy people, it is just not enough. They need solutions which can be scripted to achieve the same objectives but much faster.
- Use API token to make calls and explore API endpoints
- Use Jenkins-cli to perform tasks
- Manage plugins efficiently using Jenkins-cli
No matter how much we scale a Jenkins master vertically, it simply would not be enough to support an ever-growing deployment. At some point of time, a distributed approach will need to be adopted.
- Understand distributed architecture and slave concepts
- Explore agent launch methods
- Understand labels in details
An efficient way to run a Jenkins deployment is to offload build tasks to slave nodes while keeping the administrative tasks to the master. The most traditional way of running slaves is by adding dedicated Jenkins slave nodes.
- Set up slave nodes along with dependencies
- Configure SSH access and add slave node via user interface
- Configure labels and run builds on the slave
With cloud platforms running a huge part of the global infrastructure, testing is another function we can add to it. Dynamic EC2 instances are an excellent and cost-effective way of running Jenkins slaves.
- Configure AWS IAM user and credentials
- Install EC2 plugin and configure the data to connect to AWS
- Run builds on AWS EC2 slaves
Containers are the newest revolution to take over the infrastructure space. With launch times of few seconds and flexible architecture, they are a great choice for running disposable Jenkins slaves.
- Understand containers as Jenkins slaves and select an image for running slaves
- Install and configure the Docker plugin
- Run builds on remote Docker containers
Although standalone containers are quite good for running Jenkins slaves, an even more efficient workflow can be achieved by using container orchestration and clustering platform such as Kubernetes.
- Understand how Kubernetes runs Jenkins slaves
- Install and configure the Kubernetes plugin and dependencies
- Run builds on remote Kubernetes pods
To achieve continuous deployment, one of the most important methods of release engineering, an efficient code deployment design is required. With the numerous tools available for deployments, it is crucial that we choose the right one for the complexity and size involved.
- Understand code deployments and involved concepts
- Configure AWS EC2 and CodeDeploy to provide a platform for rapid and efficient deployment
- Install and configure the CodeDeploy plugin and deploy code from Jenkins
For complex integration and release scenarios, jobs are not just enough. Jobs need to work together to create a sequenced workflow called pipeline. In the new model, Jenkins pipelines are a game changer with all features of pipelines integrated in itself.
- Understand workflow of release engineering
- Understand legacy pipeline model using Jenkins jobs
- Understand new pipeline model and its features
The new pipeline model of Jenkins has its core in the Jenkisfile. It is the source of all configurations related to the pipeline and involves a domain specific language with its own syntax. With detailed documentation, it needs some exploring.
- Understand the Jenkinsfile and basic syntax
- Explore Jenkinsfile syntax documentation and reference materials
- Understand some important section and directives
With knowledge of the Jenkinsfile syntax, the next step is to create the pipelines following the different methods available. Also automating the creation and update of pipeline jobs are necessary.
- Create pipeline from Jenkinsfile in pipeline project
- Create pipeline from Jenkinsfile in the code repository
- Automate pipeline creation using scripts
Almost all projects have multiple branches for efficient integration practices and the general pipeline is only able to handle one branch. Multi-branch pipelines are essential for automating projects involving multiple branches.
- Understand concept of multi-branch pipeline
- Configure prerequisites for multi-branch pipelines
- Create and explore new multi-branch pipelin0065
Although Jenkins has been solving release engineering problems for long, for some users, the learning curve is a bit high. With Blue Ocean, even the least technical users are able to view and use Jenkins pipelines with ease.
- Understand concepts and features of Blue Ocean
- Install and enable Blue Ocean
- Create pipelines using Blue Ocean
To achieve continuous integration, efficient branching and code commits are not enough. There needs to be a process to automatically trigger and run pipelines to process builds continuously. Github support is essential for this process.
- Explore required plugins
- Configuring Github and Jenkins for communication
- Configure pipeline for workflow and test scenarios
In modern day software development, its not just enough to write code. Efficient and optimized coding practices are essential and continuous code inspection and analysis becomes an integral part of the process.
- Install and configure Sonarqube plugin
- Add pipeline configuration for Sonarqube analysis
- Run pipeline and generate analysis report for project
With the numerous programming languages available in the software ecosystem, there arises the need for packaging of each type of software. It is critical that the software is packaged efficiently and made available to end users for use.
- Install and configure Artifactory plugin
- Add configuration for package creating and upload to Artifactory
- Run pipeline, upload to Artifactory and test on instance
With continuous integration, problems are identified readily and can be fixed early in the process. However, problems can differ in complexities and need efficient tracking throughout the process of fixing. JIRA integrates in an excellent manner and achieves the same objective.
- Install and configure the JIRA plugin
- Configure JIRA, Github and Jenkins for inter-communication
- Configure pipeline for JIRA and test pipeline
With large environments and numerous projects and pipelines, identifying problems and getting readily notified about them becomes essential. In addition to emails, instant notifications come handy when concerned people need to be updated.
- Install and configure Slack plugin in Jenkins
- Install and configure Jenkins app in Slack
- Add pipeline configuration for Slack and run pipelines
Hands-On Continuous Integration and Automation with Jenkins
This video will give you an overview about the course.
This video describes what Jenkins is and how it used for.
- Describe what is Jenkins
- Describe what Jenkins is used for
This video shows how to install Jenkins on Windows.
- Describe options to install Jenkins on Windows
- Download the Windows package installer
- Install from package installer
This video shows how to install Jenkins on macOS.
- Describe options to install Jenkins on macOS
- Install Jenkins using brew
This video shows how to install Jenkins on Linux.
- Describe options to install Jenkins on Linux distributions
- Install Jenkins on Ubuntu
This video shows how to install Jenkins with Docker.
- Describe what is Docker
- Install Jenkins with Docker
This video gives an overview of the Jenkins dashboard and admin console and introduces key concepts and terminology.
- Explore the Jenkins dashboard
- Explore the Jenkins admin console
- Learn key terminology
This video explores configuration options in a job.
- Configure a job
- Run a job
- Learn key terminology
This video is a step-by-step guide to create your first Jenkins job.
- Create your first job
- Configure your first job
- Run your first job
This video shows how to extend the job created in video 2.3.
- Pull code from GitHub
- Pass a parameter to a build
- Build the job periodically
This video explains what Continuous Integration, Continuous Delivery, and Continuous Deployment are, how they compare, and which benefits they bring
- Learn Continuous Integration
- Find out Continuous Delivery and Continuous Deployment
- Understand Compare and contrast
This video shows the CI workflow, introduces the Maven build tool, and implements the first phase of the CI workflow.
- See CI workflow
- Maven build tool
- Implement first phase of CI workflow
This video explains the different types of tests and implements the testing part of the CI workflow.
- Why testing
- Explain the types of tests
- Implement testing workflow
This video shows the different types of notifications, introduces the information radiator and extreme feedback, and implements the notification part of the CI workflow.
- Explain the types of notifications
- Information radiator and extreme feedback
- Implement notification workflow
This video explains what Pipeline as Code means, how pipelines are implemented in Jenkins 2, and what are the benefits of using Pipelines.
- Explain Pipeline as Code
- How pipelines are implemented in Jenkins 2
- Benefits of using pipelines
This video explains what a Jenkinsfile is, shows the differences between declarative and scripted pipelines, and creates and runs simple pipeline jobs.
- Learn what a Jenkinsfile is
- Declarative versus scripted pipelines
- Create and run pipeline jobs
This video gives an overview of the CD pipeline, shows the web application deployed via the CD pipeline, and introduces Docker.
- CD pipeline overview
- Web application used in the deployment
- Intro to Docker
This video shows the Dockerfile used by the CD pipeline, the Python code of the web application, and Docker commands to build an image and run a container.
- Learn what a Dockerfile is
- Python code of the web application
- Docker commands
This video shows how to create a pipeline job in Jenkinsfile and how to create a Jenkinsfile with steps to build a web application and de-ploy it to a development environment.
- Jenkinsfilewith steps to build and deploy to dev
- Create and run the pipeline job
- Access web application on dev environment
This video adds unit tests for the Python code, deploys the application to a staging environment, and tests the deployments with user acceptance tests.
- Add steps to the pipeline
- Analyze unit and user acceptance tests
- Deploy to a staging environment
This video completes the CD pipeline with a step to manually approve the deployment and deploy to the live environment and shows additional resources to write pipeline scripts.
- Add step to approve deployment
- Deploy and test the live environment
- Resources to write pipeline scripts
This video explains what is Blue Ocean, which advantages it brings, and its main features.
- What is Blue Ocean
- See the advantages of Blue Ocean
- Overview of Blue Ocean
This video installs Blue Ocean and shows how to access it.
- Install Blue Ocean
- Access Blue Ocean
- Switch between Blue Ocean and classic UI
This video shows how to create, configure, and run a multibranch pipeline using Blue Ocean’s visual editor.
- Create a multi branch pipeline
- Create a personal access token
- Run and visualize the pipeline
This video explains what a pull request is and shows how to create and debug a pull request in a multibranch pipeline.
- What is a pull request
- Create a pull request
- Debug a pull request in the multibranch pipeline
This video gives an introduction to AWS, explains the master/slave architecture on Jenkins, and illustrates the outcome of this section’s practical part.
- Introduce to AWS
- Understand the master/slave architecture in Jenkins
- Learn about Jenkins build farm
This video introduces AWS resources and jargon and shows how to create a VPC and security group and launch an EC2 instance.
- Introduce AWS jargon
- Create a VPC and a security group
- Launch an EC2 instance
This video shows how to assign an Elastic IP address to an EC2 instance, access an EC2 instance via ssh, and install Jenkins on AWS.
- Assign elastic IP address
- Access EC2 instance via ssh
- Install Jenkins on AWS
This video introduces AWS concepts like IAM policy, IAM user, and AMI, and shows how to add AWS credentials to Jenkins and install and configure the EC2 plugin.
- Create IAM policy and user
- Add AWS credentials in Jenkins
- Install and configure the EC2 plugin
This video shows how to configure and run a job on a slave node.
- Configure job on slave node
- Run job on slave node
- Stop and clean up AWS resources
Deploying Jenkins to the Cloud with DevOps Tools
This video provides an overview of the entire course.
In this video, we will prepare our demo setup and machine.
- Install dependencies
- Create Route 53 domain
This video will help you learn how to AWS works and what services we are going to use.
- Describe AWS architecture and services
- Identity and Access Management on Amazon Web Services
- Logging with CloudTrail and CloudWatch
In this video, you will see what how Ansible can be used on AWS and how they can help.
- Learn idea behind Ansible and automation
- Architecture of Ansible
- Setup Jenkins on AWS with Ansible
This video will show how easy it is to use Beanstalk to setup production ready application.
- Learn what Beanstalk is
- Deployment on Beanstalk
- Setup our sample applications on AWS with Ansible
In this video, you will see what how slaves can be used on AWS and how they can help.
- Learn idea behind Jenkins slaves
- See what types exists
- Using AWS to create ephemeral slaves on demand in Jenkins
In this video, you will learn how Kubernetes works.
- Background about containers and Docker in special
- Learn Kubernetes architectural concepts
- Deploy with the help of minikube
This video explores how Deployment takes place on Kubernetes and how we can use it within Jenkins.
- Describe basic needed terms and artifacts
- Explore basic ideas on top of application docker build with all needed plugins
- Show example for deploying with Jenkins on Kubernetes
In this video, you will see how slaves can be used on Kubernetes and how they can help.
- Learn idea behind Jenkins slaves
- See what types exists
- Use Kubernetes to create ephemeral slaves on demand in Jenkins
In this video, you will see how easy it is to move from private environment to public one.
- Explore the prerequisites for Google Cloud Platform
- Show how to setup cluster on Google Cloud Platform
- Show sample project in public cloud
This video explains Jenkins X and why it’s useful.
- Explore the basics of Jenkins X
- Understand the architecture and Concepts, but also the limits
- Learn about the future of Jenkins X
In this video, we will prepare our demo setup and machine.
- Explain setup and login credentials
- Show priniciples of our setup
This video explores how to GitFlow works and how it can be used in Jenkins.
- Describe Git and GitFlow
- Integration in Jenkins with Jenkins multiBranch jobs
- Show samples
In this video, you will see how multiple branches can be used with the Jenkins Pipeline plugin.
- Learn idea behind build pipelines
- Learn about build libraries
- Adopt on sample project
In this video, you will learn how important it use to think of security.
- Explore Security background
- Learn about the tools that can be used in development
- Explore Integration in Jenkins
