Sapphire Venture’s Casber Wang: DevOps is Still in the Early Innings

ABSTRACT

KEY INSIGHTS FROM CASBER WANG'S POV:

Why is DevOps such an important category moving forward?

• DevOps implementation is a top priority among engineering leaders given its numerous advantages. “It is safe to say that from a technical and financial standpoint, adopting DevOps principles is a no-brainer,” says Wang. According to a recent Atlassian survey, 99% of companies in the early innings of DevOps implementation report high quality deliverables produced at faster speeds than with past approaches.
• The disconnect between interest in DevOps and successful implementation suggests we are relatively early in the evolution of this category. Bain & Company’s data reports that 90% of companies regard DevOps as a top priority, according to Wang, yet only 50% have implemented at scale, and only 12% consider their capabilities mature.
• The accelerated pace of modern software development. Modern requirements have pushed organizations to optimize all phases of the software development life-cycle or SLDC. “However, companies can only scale engineering headcount so fast, and the proliferation of new tools can sometimes contribute to complexity,” he says. New tools have emerged focused on supercharging developer productivity, responding to the need for autonomous optimization, that are capable of generating source code, optimizing test runs, remediating vulnerabilities, and rolling back code changes.

What are the key applications or use cases that might be attached to this category?

• Next-gen environments, like cloud-based Integrated Development Environments (IDEs), are emerging to extend developer speed, security, and capabilities. “The status quo for coding is via an IDE on a local machine. Cloud-based development environments are emerging to elevate the overall experience and provide teams flexible, secure, and consistently reproducible workspaces,” says Wang.
• Compared to local machines with built-in CPUs and GPUs, cloud-based computer power can be rapidly scaled as needed. Additionally, “as source code that is decoupled from machines, the endpoints are more secure.” Additionally, configuration drift and ramp time can be significantly minimized with more repeatable and consistent builds.
• Environments-as-a-Service (EaaS) vendors are becoming an increasingly popular replacement to local, "inner loop" environments as a form of infrastructure management. Environments are collections of both hardware and software that support engineers in four stages: development, testing, staging, and production. “While local, inner loop environments are relatively straightforward to set up, testing and staging can be complex,” he says. They require significant infrastructure and time to implement efficiently. “By adopting EaaS, companies can spin up ephemeral environments that are shareable, repeatable, and versioned with one click, enabling teams to validate changes and release to production quicker, and with more consistency.”
• Startups are aiming to secure various segments of DevOps processes as attack frequency continues to rise. “One rising attack vector is Continuous Integration/Continuous Deployment (CI/CD) infrastructure, and new tools are enabling security tactics like enforcing the lowest permission and privilege levels possible, multiplayer access controls, and picking up anomalous activity and threats.” Preemptive security postures are also gaining traction. “While many static scanning tools can discover where a vulnerable package version exists, they often struggle to identify the extent to which it is exploitable. Next-gen scanning tools are addressing this by embedding directly into CI/CD workflows and triaging the risk by assessing applications from attackers’ perspective.”

What are some of the potential roadblocks for this thesis?

• Transitions to DevOps are not without hiccups, and this friction can slow down adoption at the micro and macro level. “It’s important to remember that DevOps is not just about adopting innovative tools and platforms,” says Wang. There is a significant human element, some of which goes beyond pure technical expertise. For example, engineers who once focused on a specific aspect of software delivery must cultivate cross-disciplinary skills and communicate across teams and functions to succeed. Cultural cross-currents can create friction during adoption and implementation.
• The field is growing crowded and complex — making it harder to break through the noise to buyers and customers. “Incremental improvements, emerging practices and niche use cases have spawned a vast ecosystem of literally hundreds of DevOps tools,” says Wang. Meanwhile, platform players have expanded horizontally while other products have remained specialized and sought to become best-of-breed point solutions. “This proliferation of tools and vendors has increased complexity for modern development teams.”

VISUAL: DEVOPS ADOPTION STILL IN THE EARLY INNINGS

According to Bain & Company’s 2021 DevOps Pulse Survey, while 90% of companies consider DevOps a top strategic priority and are actively attempting to become proficient, only 50% of companies have implemented DevOps at scale and even less, 12%, consider their associated capabilities mature.

IN THE INVESTOR’S OWN WORDS

DevOps has emerged from several paradigm-shifting developments across broader tech and the SDLC. The explosion of software consumption across industries and new technologies like cloud platforms positioned SaaS as the de facto standard for delivering applications.

This expanded market and fast SaaS delivery altered the implicit contract between software developers and consumers. The latter were now demanding feature-rich applications with continuous improvements and dynamic scalability. This fueled an arms race to out-innovate  and out-perform competitors. Releasing new features quickly started to make financial sense.

However, this arms race drove dysfunction in engineering organizations. Developers were unable to scale communication and collaboration alongside other aspects of their work. Today, competitive software development requires visibility and communication across historically distinct disciplinary silos. And while Agile promoted integration within teams, practitioners have now turned to DevOps to unify all the relevant parties across the software development process.

To us, DevOps is a cultural shift — a set of practices and an enabling ecosystem of tooling — that connects cross-disciplinary teams and encourages increased collaboration. All of this matters when pursuing the goal of delivering high-quality and sustainable products quickly.

MORE Q&A

Q: What can you say more about the frictions being faced in the transition to DevOps?

A: "Legacy environments are the primary hurdle here. IT shops are increasingly standardizing on some combination of cloud, containers, functions, and micro-services. But many are still in a state of transition, supporting a hybrid of both legacy and modern tech stacks. Legacy or so-called brownfield apps can be particularly challenging to ‘DevOps-ify’ because they are often built upon monolithic architectures, leverage static physical infrastructure that can be difficult to automate, and utilize more traditional frameworks and languages."

Q: Can you dive into the main roles Machine Learning will play as DevOps evolves?

A: "There are roles across the cycle of design, testing, maintenance, and more. For example, in testing, it’s standard within the SDLC to write numerous tests that review functionality, stability, and system integration. Test suites are really effort-intensive and tend to balloon as new features are layered into any product or platform. ML-enabled tools are available to automate test generation. There are also ML techniques, including neural nets, which generate synthetic data mimicking production conditions, in order to supply tests with appropriate inputs."

Q: What other important trends are you following closely in this category?

A: "The “shift-left” principle implies that steps should be performed as early as possible in the cycle to avoid costly downstream redesigns. The issue is that staging environments rarely reflect the real world. With DevOps practices, many aspects of the cycle have actually begun to “shift-right,” meaning teams can address issues when they arise, and dispense with onerous early preemption. One example is feature flags, a safer way to ship code by hiding incomplete or untested features. Another is continuous verification, which ingests observability data and uses ML to assess performance and automate the rollback of any incidents, reducing manual checks.

A second trend is a growing focus on measuring and improving engineering productivity. Pockets of scarcity in engineering talent as well as new remote-work cultures have helped push forward new tools focused on code visibility. These aim to more accurately gauge productivity, pinpoint burnout, and onboard new hires faster."

WHAT ELSE TO WATCH FOR

Additional trends, including WebAssembly, meta frameworks, and Web3 dev tools could push the boundary on what is achievable in software delivery. “As application architectures and tech stacks continue to evolve, we have and will continue to see an explosion of new tools and capabilities in this space,” says Wang. These trends, as well as others, may support “the potential for numerous companies of consequence to be produced in the years to come.”

Watch for Machine Learning-driven capabilities that gradually augment the coding and building work. Most DevOps tools share the common goal of automating the tasks that distract engineers from writing code and delivering features. However, ML-driven capabilities will come to augment many coding tasks, as they have begun to do already with features like Github’s Copilot. With time, humans will focus on more high-order reasoning and problem-solving, while computers carry out the actual execution and implementation of code.