Source: devops.com
Enterprises are quickly becoming an intricate mesh of many applications. As companies create more and more microservices, their deployment environments are becoming increasingly elaborate. Without proper configurations, a microservices road map could quickly become unmaintainable.
According to DeployHub CEO Tracy Ragan, if development teams are pushing points to a cluster continuously, application development can suffer without high visibility into their dependencies and application interrelationships.
In her recent talk, “Evolving Your Pipeline for Microservices with DeployHub,” presented by the Continuous Delivery Foundation (CDF), Ragan, also Governing Board member of CDF, discussed ways in which teams can obtain improved insight into microservices delivery pipelines.
Microservices Tracking Issues
A 2018 study conducted by O’Reilly found that 86% of respondents reached mild success in adopting microservices in production. Only 15% reported “massive success.” These results demonstrate both the excitement surrounding microservices and the rarity of flawless execution.
“I’m a huge fan of microservices,” said Ragan. “This is an exciting time to be in software development.” Microservices break from the traditional software delivery model, she said, forcing teams to emphasize the runtime. However, with this shift, it becomes easy to lose the logical views between applications.
According to Tracy, dependency tracking and version control are top issues in maintaining microservices arrangements. With continuous updates, it can be challenging to keep track of which version a service is using. “Microservices development falls apart without a coherent management approach,” she said.
Losing Visibility Moving From CI to CD
Part of the loss of visibility stems from the move from CI to CD. Continuous integration (CI) involves routinely updating source code and starting the compile process. This is often done every hour, as opposed to every week or month. With traditional CI, the system created a package and scanned for dependencies at compile. This was the first step in the continuous journey.
However, nowadays, microservices are typically linked at runtime or in the Kubernetes deploy file as part of a continuous delivery (CD) processes. In doing so, we’ve lost a certain amount of visibility that CI granted. Ragan likened it to “hammering a wine glass”—we still have the constituent parts, but they lie shattered in pieces.
Large microservices suites of 50, 100 or even 300 are not uncommon these days. Such large ecosystems require logical tracking of versions and dependencies. Ragan argued that development teams need a workaround layer to replace the critical configuration management lost in CI compile reports. Visibility into interrelationships is necessary to trace how changes in one service will impact other applications.
Enter Ortelius
Ortelius is a package that helps with continuous microservices management. Ortelius, named after Abraham Ortelius, the inventor of the first atlas, helps trace the connections between services to improve an ecosystem’s visibility.
To get an idea of how it behaves, consider an e-commerce store. To reach an evolvable microservices pipeline, each microservice should be independently deployable with its own workflow. For example, an e-commerce store may require a product microservice, payment microservice and an authentication microservice. These reusable components are independently deployable but rely upon one another to function as a whole.
At a macro scale, an enterprise with thousands of inter-application dependencies would visually resemble a constellation of nodes. An extreme example of this may resemble the Netflix Death Star, a mass conglomeration of intersecting points. Ortelius grants transparency into an otherwise tangled mess as an “internal marketplace for microservices,” said Ragan.
Paring Ortelius With DeployHub
Ortelius is the open source core of DeployHub, an open source SaaS for sharing and releasing microservices. Ortelius, paired with DeployHub, enables teams to see the logical application map, exposing the complexity of microservices dependencies.
Developers add DeployHub to any existing microservice development workflow prior to release. Ortelius is then called to increment the service and record dependency information. DepoyHub then uses for Helm or Ansible for the actual code push, and adds the record metadata and adds the tracking and mapping layer. Having such information is still critical for reporting like dif reports or a bill of materials report, noted Ragan.
This sort of solution, Ragan proposed, is also vital for continuous configuration management—when a microservice changes, the team will always be aware of the new application version.
The New Microservice Pipeline Model
As noted above, in the traditional model, DevOps teams map dependencies in the CI stage. When an API developer publishes to the main catalog, they define specific attributes based on the microservices. A versioning engine will then typically map to visualize dependencies.
Ragan outlined steps for a new microservices pipeline:
- Build image: Compile and build the container image.
- Put into registry: Quay or Docker registry. Specific for container registries.
- Track service-application configurations: This is where we collect configuration data to map the structural relationships.
- Push to cluster: DeployHub helps to standardize this process across an organization.
- Collect feedback: Configuration reporting and impact analysis.
Core to this process is being able to independently deploy microservices and aggregate useful information. Of course, this is an incomplete model—a complete pipeline would utilize other steps including testing and automated security scanning as well.
Benefits
By streamlining the microservices journey in this way, repos will be broken out and teams no longer go through one big long workflow. According to Ragan, the visibility garnered from this method could help organizations in multiple other ways:
- Reduce time: Ragan estimated automated application relationship mapping could replace one to two hours of manual work per week per development team.
- Reusability: Decomposing applications into their domains aids reusability across an organization.
- Avoid redundancy: Code sharing can significantly reduce duplicated code, thus avoiding redundancy and increasing flexibility.
- Visibility: Detailed application to service relational knowledge could empower SREs to make better data-driven decisions.
- Centralize: Helps standardize container deployment processes regardless of cluster type or collection, aiding both traditional and containerized microservices deployments.
- Versioning: Improved insight into versioning helps when you need to share services externally.
Lastly, what’s nice is Ortelius is open source on GitHub. Ortelius adopts an open community format to accept feedback. “Democratization is the core reason we have open source,” said Ragan. Ortelius can be used for free, or with added features from DeployHub pricing for enterprise cases.
A Confluence of Software Delivery Styles
Managing continuous software delivery across many microservices leads to unprecedented issues. In solving these issues, the industry appears to be fracturing slightly. For example, the model promoted by Ragan is in contrast with a recent presentation I covered from Codefresh.
Where Ragan suggested improving visibility into independent microservices deployment pipelines, Dan Garfield of CodeFresh suggested adopting a single CI/CD pipeline to operate all microservices from a single control plane. As it goes so often in software tooling, the use case will likely dictate adoption.
Whatever the software delivery process, when decomposing applications—or “taking the hammer to the wine glass,” as Ragan called it—we probably shouldn’t sacrifice visibility into the metadata associated with application versions and service to application interrelationships. If such data is lost or not managed correctly, we could lose a lot of potential value.
For more information on evolving DevOps pipelines, watch the CDF-presented webinar here. DeployHub also published an adjacent article on this subject here. You can also contribute to Ortelius on GitHub.
