Implementing Continuous Integration/Continuous Delivery (CI/CD) practices in your organization

30 Apr, 2020

Automation is a major part of the process in all of the industries. Software development is not an exception. As programmers thrive to develop high quality, reliable applications it is important to maintain the integrity of the source code, especially when developers are working in the team and rely on each other work.

Integration guidelines

There are several important points and guidelines which can ensure the smooth integration process:

Source code must be managed by Control Version Services (CVS). Most popular ones are Git, Subversion, Mercurial. Those systems allow tracking all the changes and enable collaboration between team members
Source code submitted to the repository must be compilable on other machines without additional setup. Everyone should be able to pull the code, open it in the development environment (such as MS Visual Studio) ad compile it. Of course, program might not be able to run as the additional environment might need to be set up (such as SOLIDWORKS or other CAD application must be installed in case the software is an extension to this application), but nevertheless, code must compile even if this environment is not setup.
Code must be accompanied by unit tests to ensure the integrity of the functionality
There should be no binaries in the repository. Of course, some applications rely on 3rd party interops and binaries. For example, add-ins to CAD applications would require the corresponding COM interops to be added to the project. To satisfy this requirement package services should be employed to redistribute the dependencies, such as NuGet.
There should be no credentials, certificates, or other secrets included in the source code (e.g. database connection string with password, web API credentials, code signing certificates).

When it comes to delivery of the project to your QA team or a customer there are few more important guidelines:

Delivery guidelines

It is required to setup a robust versioning scheme for the files in your package (e.g. dll and exe files) as well as deployment package (e.g. msi installer). Inspecting the version number it should be evident what version and revision of the software is, what build has produced this package (i.e. which commit in the repository produced this package) and what stage of the package (e.g. test, pre-release or release)
Deployment package must be compiled by automation systems on the centralized server, not manually on a developer's computer.

Executing the above guidelines manually is either time consuming or even impossible and prone to errors. Avoiding those guidelines usually results into integrations errors, frustrated team members and QA team, missed deadlines, and unhappy customers.

In addition, modern programming encourages continuous code integration with frequent commits and validation to ensure the integrity of the code and identifying and fixing issues on early stages.

Implementing CI/CD practices in the organization

This is where Continuous Integration/Continuous Delivery (CI/CD) process comes to place. CI/CD practice is commonly implemented across the majority of software development companies and open source developers, however, it is usually skipped in the companies who are not recognized as pure software development rather have developed and maintained their own in-house built programs to automate their company's specific business process. Such organizations either have one or two dedicated developers or not having developers at all and engineers or other employees with software development skills are nominated to this role and doing it part-time. These programs in most cases become 'backbone' of the organization workflow and usually quite strong and reliable functionality wise. Downtimes related to software not being operated can be very costly to the organization.

If that sounds familiar to you and happens in your organization I have a challenge for you. Get a copy of the code from your central repository. You might have the code stored in Git, FTP, or just zipped somewhere on a shared drive. Now take this code and copy it to any other computer, but developer one. You can only install the development environment, such as MS Visual Studio. If the project built-in Visual Studio you might just use msbuild utility to compile it avoiding the installation the full version of Visual Studio. Does it compile? Or do you have errors like the ones below?

Now you need to consult the developer to find the missing pieces.

But what if...

What if the developer is on vacation or even left the company?
What if his/her computer has crashed and he/she is no longer have access to the workstation and the missing files are not restorable?
What if the failing portion of the code was developed a long time ago that employee cannot remember how to make it work

This situation may be much worse if you need to urgently do some fixes and you might be fortunate enough to have another employee with software development skills in the company who can look it for you, but not much developer can do if original code cannot be compiled. And now you have a pressure from your Engineering and Procurement department demanding the solution soon...

In this situation, it is much better to be proactive and eliminate potential issues by implementing the CI/CD practices in your organization. At Xarial we offer the service of implementing CI/CD with your software. Please send us an e-mail for a free assessment and the quote.

Nowadays, there multiple CI/CD providers which either provide free or low-cost hosting plans. This includes, but not limited to MS Azure DevOps, GitHub Actions, Bitbucket Pipelines, you can also set up your in-house server using open source Jenkins or Hudson

Practical demonstration of the benefits of CI/CD

The above video shows 2 practical demonstrations of CI/CD benefits of building the CAD+ Toolset application. This is an open-source project and source code is hosted on GitHub and CI/CD is setup on MS Azure DevOps

CI/CD pipeline will be triggered automatically once the code is pushed into the dev and master branches.

The pipeline has 3 stages

Build - triggered once code is pushed to the branch or pull request is created
- Compose the package version and assign it to all assemblies of the package and to the installer package. Follow Versioning scheme for the explanation of the scheme used in Xarial.
- Compile the code
- Run unit tests
- Digitally sign all .exe files in the build
- Build .msi installer
- Digitally sign .msi installer
Deploy Preview - deploys preview installer into the MS App Center under the preview group. This package can be used by QA tester to test the preview of the features. This stage is not triggered on pull request validation
Deploy Production - deploys pre-release installer into the MS App Center under the preview group and also initiates the deployment to the production group but awaits a manual approval. As this build suppose to be redistributed to customers it is important to validate the pre-release before pushing it into the production.

If any of the steps fails, the pipeline fails as well and notification is sent via e-mail.

As all of the above steps performed automatically on the remote server, developers will not hesitate to push new code and QA team can immediately access intermediate releases.

Versioning scheme

When you select the versioning scheme for your software, it should be able to provide important information to your users as well as QA team and developers. Xarial has adapted the following versioning scheme for the developed products

X.Y.ZZZ.AB

where

X - major version of the software. Incremented when a new major release is published. 0 is used for the products which are not yet completed and released to the community to deliver some early features and benefits. The completed product will have more functionality.
Y - minor version of the software. Incremented when intermediate release is published, this might include bug fixes or new minor features added.
ZZZ - build number which corresponds to the pipeline run this package was produced from
A - revision number. Incremented when minor changes were done to the product. This is usually minor bug fixes, such as typos.
B - index of the build type. The following build types are recognized
- 4 - Preview build compiled from the dev branch
- 8 - Pre-Release build compiled from the master branch before the production build is released
- 9 - Production build