Continuous Integration (or CI) lacks adoption in the NI ecosystem, as do many other advanced software processes and tools that are standard in other programming languages. Either due to actual technical reasons or because of the team size and proficiency, and because of the way a typical LabVIEW project is run. Why is that? And does it have to be that way?
This is the first blog post in a series of three, discussing Automation in general (this post), our Release Automation Tools (part 2), and our workflow and supporting tools (part 3).
What is Continuous Integration?
Continuous Integration (CI from now on) is the practice of merging code changes (integrating) into the main branch of code constantly, as soon as possible (continuous). This reduces the chances of merge conflicts and avoids the dreaded merge hell, the ruiner of many a calm Friday afternoon.
To ensure usable code, the process is automated. This includes the automation of the very essential tests. Otherwise, the entire system will fall apart. The high level of automation makes for a much better developer experience in exchange.
The developer experience will have the following improvements:
- Greatly improved collaboration by increasing a team’s transparency and accountability
- Fast issue detection provided by automatic tests
- Better use of resources takes the tedious tasks away from the developers, which avoids burn-out by raising morale
What do you need for Continuous Integration?
Appropriate infrastructure consists of a Source Code Control Server, Build Servers and Deployment Targets. The developer writes the code on their PC and pushes it to the SCC Server, which activates the CI workflow and the latest version is run and tested on the Build Server. Once a release is finalised, it gets deployed to the customer in a very short time.
You also need an appropriate workflow. This necessitates (re)training for your employees. At HSE we use GitFlow. Put very simply, a separate development branch is used for day-to-day changes. Only complete releases are merged into the main branch. This enables a very secure and robust code.
A CI provider is also a must. Fortunately, most if not all SCC providers also have CI support. At HSE we use GitLab for our internal projects. But for our customers, we also have provided solutions for GitHub, Azure DevOps and BitBucket with Jenkins.
A practical improvement is the virtualisation and containerisation of the Build Servers. Instead of using dedicated machines for building, it is much quicker to spin up a Docker container for example.
Continuous Integration in the LabVIEW world
Usually, LabVIEW devs aren’t computer scientists, but engineers. Even though LabVIEW is extremely user-friendly, complicated software engineering processes are not. LabVIEW projects are usually short-lived and designed for a sole single-use application. The graphical interface, while improving the developer experience, makes the use of out-of-the-box CI solutions much harder than with text-based languages. In addition, using LabVIEW with Docker is highly impractical.
People in our bubble also tend to use the term CI or CI/CD or Continuous Integration for anything that has to do with triggering some process by/through source code control. Even though that’s not technically correct, we’re aiming to speak the language of our ecosystem and use these terms in conjunction with automation or server-side automation.
What are your options?
Completely applying all CI tenets is an intimidating endeavour – but it’s not the lone way forward. You can reap the majority of its benefits even with LabVIEW. You simply need to select the appropriate “better practices” for your specific case.
At HSE, we stick to the general spirit and idea of continuous development. However, we do not do the actual integration continuously, as LabVIEW development usually has slower cycles than other code ecosystems – and none of our customers would like us to deploy 10 times a day to their production test farm 😇.
At the end of the day, it’s a combination of the right set of tools paired with matching workflows and processes that allows you to fully leverage the benefits of automation.
Despite the difficulties, properly using CI automation tools with LabVIEW is possible, and a huge improvement for basically any use case. We even created a product just for this, allowing you to leverage our battle-proven way of working! Read about our Release Automation Tools in the next article, or check out its page now!
Workflow and Way of Working
Our way of working is now ingrained so deeply into our culture, that we barely notice how great it is. Using Gitflow and its feature branches to parallelise development is a huge benefit nevertheless. Utilising unit testing ensures constant high-quality code for us. Code reviews and other similar merging best practices for merge requests not only improved code quality but also our own efficient communication and teamwork skills.
It’s pretty as well! See for yourself:
Source Code Management with Git
Visualisations of Git repos are similar to LabVIEW code: Clean and elegant when done correctly, a terrible spaghetti mess when done poorly. We use Git Fork and recommend it to all our customers.
To make sure that branches are not kept open longer than necessary, we created our own little helper tool. It parses repositories from your SCC server and visualises any open branches, so we can go through them together with the team and make informed decisions. Check out the Open Branches Tool in action here!
A GitLab pipeline running successfully, lighting up green for each stage and test is always satisfying. A little treat for a job well done.
Releasing code into the wild
And finally, when everything is done and dusted, time for the release itself! At the end of a successful build pipeline, our tools will copy the build results (Windows or Real Time executables, libraries, zip archives of source distributions, VI packages…) to a location on one of our servers which is also accessible through a web server.
Luckily, DokuWiki is easily modifiable and adaptable to our needs, so we were able to create a plugin that shows information and download links for all our releases. That way, we make it very easy for our customers to access the latest versions of our tools and libraries.
Where do you go from here?
If you’re patient, then check back regularly as we’ll publish parts 2 (our Release Automation Tools) and 3 (our workflows and supporting tools) soon.
For those who cannot wait until early next year, this button is for you:Contact us!