Build vs Buy: The Platform Engineer’s Guide

This article was originally published on The New Stack.

The question of whether to build or buy an Internal Developer Platform (IDP) is common for organizations starting their platform engineering journey. However, there is no one-size-fits-all answer. This article will explore the pros and cons of both options and provide insights on how to make the best decision for your organization.

Internal Developer Platform or Platform as a Service?

Before we dive in, let’s define what it means to build or buy. The CNCF Platforms White Paper uses Martin Fowler and Evan Bottcher’s definition of an Internal Developer Platform (or digital platform):

“A digital platform is a foundation of self-service APIs, tools, services, knowledge and support … arranged as a compelling internal product. Autonomous delivery teams can make use of the platform to deliver product features at a higher pace, with reduced coordination.”

Some argue that this definition means that IDPs must be built and cannot be bought. However, this distinction can be confusing as some vendors advertise their products as IDPs. Understanding what qualities distinguish built platforms from bought ones is more valuable.

An IDP that can be purchased is often called a Platform as a Service (PaaS). Whereas an IDP is a collection of different tech and tools glued together, a PaaS is one tool that covers some (but not necessarily all) of the same functionality.

Another key distinction is who decides how developers work. With a PaaS, the vendor defines the user experience and the platform’s limits. With a bespoke IDP, the company defines the user experience and the platform’s boundaries; the limitations arise from the platform team’s skills, budget and business context.

It’s important to understand that building an IDP in-house and from scratch or buying a PaaS are two ends of a spectrum of approaches. Build vs buy is not a binary choice.

Given the restrictive nature of PaaS offerings, you won’t see many organizations buying a PaaS and building the rest of the platform around it. However, they might buy an internal developer portal, platform builder, control plane, DevOps platform, application delivery platform or some other kind of platform tooling from a vendor and build in-house only what’s necessary. Most organizations leverage some combination of built, bought and open source tools for their platform.

With this spectrum of approaches in mind, let’s review their advantages and disadvantages.

Building an Internal Developer Platform: Pros and Cons

Pros of Building

Platform engineering is hard to get right, but it’s still gaining traction. For good reason, too. Building an IDP empowers organizations, particularly enterprises, to create something perfectly suited to their needs. Many organization build their IDP for greater:

• Customizability: Building an IDP gives organizations complete control over its specifications and feature roadmap. This autonomy enables platform teams to tailor the IDP to their developers’ specific needs and workflows without waiting on a vendor. Customizability is especially valuable for organizations in highly regulated industries with strict compliance requirements.
• Scalability: Organizations experiencing rapid growth find that vendor-provided options can’t keep up. Internal platform teams are better equipped to scale the platform alongside the business.
• Extensibility: Platform teams can architect their IDP to make its components easy to switch out when needed. This flexibility can be incredibly valuable as the platform matures.
• Integration with legacy systems: Most enterprises must build at least some of their IDP to guarantee full integration with existing built-in-house technologies. As CVS Health’s VP of engineering enablement, Jim Beyers, explains: “…once you get to a certain scale, I don’t know that there’s things out there that fit … probably we’re going to have to build it ourselves.”.

In short, custom-building an IDP offers unparalleled control and compatibility with existing systems for organizations with specific needs, strict compliance requirements or hyper growth. CVS Health’s experience emphasizes that PaaS solutions don’t make sense at a certain size.

Cons of Building

Building an IDP isn’t without its challenges. If you’ve been following the platform engineering conversation for any time, you’ve likely heard war stories from organizations that quickly got in over their head building their platform. Organizations pursuing this approach should consider the following:

• Upfront costs: Building an IDP in-house requires time and a lot of money. Platform teams should follow a platform-as-a-product approach to ensure that relevant stakeholders understand the investment required. Without sustained buy-in, platform initiatives can get axed before coming to fruition.
• Maintenance: As the platform expands, so does the responsibility of the platform team to keep up with new standards, technology updates and tooling. In Cloud Strategy author Gregor Hohpe’s PlatformCon 2022 talk, he explains how platform teams can understand and communicate how much of the platform they plan to maintain:
  “What will you do with your platform when the base platform grows? … You can leave your platform the same because you invested all this kind of money, and we call this a sinking platform as the water level rises, right; it might be justified from the investment, but you are kind of duplicating things that are now available in the base platform. … Or you build a ‘floating platform’ where, when the base platform gains the capabilities you have built, you say ‘Oh perfect! I don’t need that part anymore. I can let the base platform handle that, and I can innovate further on top.’”

Source: “The Magic of Platforms” by Gregor Hohpe delivered at PlatformCon 2022

• Dangers of copying other IDPs: Copying Google, Netflix or Amazon’s IDPs is a recipe for failure. There are more platform engineering examples and resources than ever, but platform teams still need to understand their organization’s unique needs.

The best way to mitigate these challenges is to follow a platform-as-a-product approach: conducting user research, creating a product roadmap, soliciting user feedback, continuously iterating and improving, and getting internal buy-in from all stakeholders. Not only will this approach help your platform team avoid common pitfalls, but it also helps ensure you build a platform developers actually want to use.

Buying a Platform as a Service: Pros and Cons

Pros of Buying

Some organizations lack the headcount, legacy systems or rapid growth that necessitate a custom IDP. For these organizations, a PaaS can be a wiser investment. The benefits of PaaS for these organizations include:

• Reduced resource burden: Buying a PaaS offloads the heavy lifting of development and maintenance from your internal teams. Buying frees up valuable internal resources, enabling them to focus on organization-specific problems.
• Vendor support: PaaS providers have a vested interest in keeping their platform competitive. Vendors can devote more resources to ongoing improvements, feature additions, security updates and support.
• Fastest time to value: The timeline to implement a PaaS is much shorter than building an IDP from scratch. With a PaaS, you can develop and push new applications to production much faster.
• Basic security: Most PaaS offer some built-in and up-to-date security features. While these may be insufficient for industries with specific compliance requirements, they can ease the burden on internal teams in less regulated spaces.
• Low upfront cost: Buying a PaaS comes with a more predictable and often lower upfront cost, making it a more cost-effective option for smaller organizations.

In sum, PaaS empowers smaller, greenfield organizations to unlock the power of an IDP without the monumental effort and expense of building from scratch. For those without specific needs or extensive legacy systems that necessitate a custom platform, PaaS offers a cost-effective and supported path to improving developer experience.

Cons of Buying

As alluded to earlier, PaaS is not suitable for all types of organizations due to some significant drawbacks:

• Limited flexibility. PaaS can be too opinionated or rigid for mature setups. As Camille Fournier explained: “It tends to become obvious when you need to build your own platform. If you are using Heroku, you will hit scaling limits and see teams peel off and do their own thing.”
• Potential vendor lock-in: PaaS customers should buy with an exit strategy in mind. Otherwise, they risk getting locked into their vendor’s approaches and workflows.
• Incompatibility with legacy systems. A PaaS is unlikely to be fully compatible with extensive legacy systems. Incompatibilities cause serious integration and security challenges for brownfield enterprises.

Given these limitations, organizations that expect to outgrow their PaaS in the short term should consider building an IDP instead. Otherwise, you’ll force your developers into one vendor’s workflows and approaches just to switch them to something different soon after. Frequent major changes to the platform can increase cognitive load and worsen the developer experience.

Build and Buy?

Most organizations build some parts of their IDP and buy others, leveraging a combination of open source, commercial and built-in-house tools in their platform. This build-and-buy approach allows them to enjoy more benefits while mitigating disadvantages.

However, this approach comes with its challenges. The CNCF Landscape, pictured below, is as vast as it is confusing. Platform teams often need help understanding how to choose the right tools and how best to glue them together. However, the Platform Engineering Community has done great work by creating a new and improved platform tooling landscape.

Source: CNCF

In general, platform engineering experts recommend avoiding reinventing the wheel wherever possible. Syntasso’s Abby Bangser writes:

“In general, as platform components become more of a commodity in the industry, it becomes more likely that you should purchase a solution to benefit from the investment other organizations are putting in to maintain their product in a competitive landscape.”

This advice applies to both the start and continuation of your platform engineering journey. Therefore, as you make your build or buy decision, it is important to understand how your platform might evolve and what tools your organization can leverage along the way.

Note also that the build-and-buy approach still runs the risk of vendor lock-in and high subscription fees. In theory, built-and-bought IDPs are more extensible but must be architected well to be extensible in practice. The cost of third-party platform components can quickly increase and become unmanageable if platform teams aren’t careful. Organizations should do their due diligence, whether buying a full PaaS or a platform component.

Key Considerations for Building or Buying an Internal Developer Platform

There is no one-size-fits-all answer to whether to build or buy an IDP. By carefully considering the pros and cons of each approach and getting feedback from users and other stakeholders, you can make the best decision for your organization.

In this article, the author Aeris Ransom has identified several key factors to consider when deciding whether to build or buy your IDP, including your:

• Existing setup. Do you have a greenfield or brownfield setup? What integration needs should your platform team be mindful of?
• Size: How many developers are in your organization? Is that number expected to grow quickly?
• Long-term goals: What does your product roadmap look like? What third-party solutions can support your platform’s evolution?

Most importantly, neither approach (building, buying, or both) can guarantee your platform’s success. There is so much more to sound platform engineering.

In the meantime, check out this free guide on how to evolve into a platform company.