What is an Internal Developer Platform

Software production has become exponentially more complex over the last few years as containerization and microservice architectures have exploded. These design decisions are rooted in aspirations of scalability and speed, but left unchecked, have devolved into data model mayhem, development silos, and environment inconsistencies.

While DevOps practices have been around since before cloud-native computing, these principles have re-emerged with new directives and even [some say] a new name—to tackle newfound complexity of the last five years. Platform engineering brings with it a renewed focus on reducing cognitive load for developers to do their best work, which has opened the door for new technology to abstract away complexity: Internal Developer Platforms and Portals.

What is an Internal Developer Platform?

An internal development platform is the totality of the tools and technology proactively assembled in order to provide structure to a company's software development and software delivery process. This includes infrastructure and access control provisioning, orchestration of CI/CD pipelines, and deployment best practices. Platforms aren’t complete without the idea of golden paths — approved methodologies and tooling for software or application development and software or application configuration.

To qualify as a platform, these should be consciously built to streamline processes, rather than emerging from individual decisions that lack a cohesive strategy. They can be bought as SaaS products, adjusted by operations teams or built from first principles by platform teams using open-source technology such as Spotify’s Backstage.

Platform providers, like Humanitec, should empower developers and reduce their cognitive load in the first instance. Coders code, and increasing the share of their time spent doing the deep work required to write software makes them more valuable employees who are happier at work.

One driver of this is autonomy: platforms provide self-service solutions built on automation that strip back complexity and reduce any bottlenecks deriving from an over-reliance on input from DevOps and third parties. They also reduce complexity by providing orchestration for infrastructure, and integrating tools and services such as source control systems, CI/CD and observability tools.

What is an internal developer portal?

Companies that wish to double down on stripping back cognitive load and make best practice more prominent can turn to internal developer portals (IDP).

Gartner defines IDPs as "the interface through which developers can discover and access internal developer platform capabilities", but this conception of portals as a layer of abstraction sitting on top of the platform's “engine room” is becoming outdated. This boundary has shifted in recent years, as more functionality sits portal-side, from plugins and extensibility to scorecards and active improvement based on defined best practice.

Portals represent the step from optimizing software architecture to optimizing choice architecture. They provide an environment where the right decision to make aligns with the intuitive thing to do at every step of the software development process. This starts with a catalog that provides immediate visibility of all software entities, includes a bring-your-own-data model that draws from ecosystem-wide integrations to provide live data for systems of record, and emphasizes a focus on self-service that incorporates scaffolding and resource provisioning in line with best practice.

Portals are loosely coupled with underlying infrastructure to allow for modular changes, and emphasize developer autonomy and self-service to the greatest possible degree. Portals also go beyond platforms in providing this self-service by creating similar experiences for developers and ops colleagues alike, decentralizing decisions while creating a shared perspective. This democratizes decision-making and allows everyone to self-serve. Rather than sidelining ops colleagues this evens the playing field, and allows both teams to specialize according to their strengths.

Role of the software catalog

Large software teams operate using a range of documents, runbooks and programming languages, before we factor in information about microservices and their dependencies. These point solutions can run from the sole job of archiving a PDF to more complex tasks such as managing security, authentication, and access control.

The number of disparate software in play tends only to increase, and the value of a software catalog to centralize information about the ownership and dependencies associated with each entity becomes self-evident with scale. Catalogs increase accountability by making ownership clear, which also improves security standards by ensuring that developers have skin in the game when it comes to identifying and addressing vulnerabilities in their code.

Platform as a product

Developers engage with portals mostly through a graphical interface, as opposed to using APIs and CLI commands in the case of platforms. By providing a product-like experience that simplifies choice architecture, portals bring best practice more to the fore. This prioritizes and clarifies golden paths, which clearly indicate best practice use of tooling, infrastructure, standards and related aspects of software development, all while emphasizing developer self-service.

Portals are inherently dynamic: improving engineering quality is a moving target, and no static approach can deal with changing best practice, shifting development requirements and evolving programming norms and philosophies. More detailed examples of recent use cases for portals can be found in this webinar.

What is the difference between an internal developer platform and an internal developer portal?

Platforms and portals differ based on ownership and maintenance, configuration management and permissions, and the prominence of best practice through scorecards and templates. The primary difference comes down to developer experience, particularly regarding complexity. Tesler’s Law states that for any system there is a certain amount of complexity that cannot be reduced. In platforms this complexity sits with the developer, whereas in a portal it is assumed by the system. This means that portals provide clearer golden paths by offering live data and embedding best practice throughout the software development lifecycle, while platforms allow more operations work to sit with developers.

Internal developer portals offer an extension of the functionality of internal developer platforms. They provide development teams with golden paths that incorporate easier access to internal best practice and improved orchestration. Rather than acting just as the frontend, they provide an enhanced, hands-on user experience rooted in nudging developers and ops towards best practice as standard.

Are Platforms the Engine?

It should not come as a surprise that internal developer platforms are central to the practice of platform engineering. But this obscures the full potential of platform engineering when it comes to optimizing software development and improving the developer experience at scale.

While platforms act as the backend engine for software development workflows, depending on your ultimate aims they are either a mid-level stage of maturity, or a conscious decision about locating complexity with the end users; in this case the developers. For some organizations platforms are a stepping stone towards implementing a portal, and for others they are the optimal fit for their engineering culture and philosophy.

Who uses and maintains an internal developer platform?

Platforms are commissioned, built and maintained by platform engineers, sitting within DevOps. These colleagues are responsible for providing the initial configuration and maintaining the associated technology and practices.

One approach to doing this is to build a platform as an internal product, with associated product managers and roadmaps. Companies with plenty of internal capacity, product expertise and a culture of building from first principles can find great success with this approach, but it is not for everyone.

Primary users of platforms

Primary users of platforms are developers, and having devs assume this role can be a useful experience for companies building internal platforms along product lines. Going from builder to end-user provides a new perspective, and platform engineers can make the most of this by looping in the wider product team.

Some developers enjoy the mix of self-service user experience and API-driven technical input from a platform, as it means more decision-making rests with the individual. For others, this decision-making amounts to cognitive load and acts as a distraction from their core competency of writing code.

What are the key components of an internal developer platform?

Infrastructure building and management tools provide a unified infrastructure to ensure all developers are working within the same architectural framework, improving consistency and compatibility across projects. This is an important first step in providing standardization across the organization.

Security and compliance are emergent concerns that come with growth and scale. Attempts to bolt these on retrospectively end in bureaucracy and frustration, so tools have emerged to enforce the relevant standards in a way that aligns these functions more closely within the software development lifecycle. Platforms allow you to incorporate these tools in wider best practice, and Portals allow the creation of scorecards off the back of catalogs and integrations that can fully embed in golden paths.

Deployment tools streamline and automate repetitive tasks and manage processes like code deployments, reducing the risk of human error. CI/CD tools fall under this category and have exploded in popularity in recent years, but without a platform or portal, these can make deployment faster without enforcing standards.

Tools for production application monitoring allow organizations to track progress towards technical and managerial goals. By focusing on specific metrics such as response time, throughput and error rates, these offer data that can be of use to developers, operations, product managers and even the finance team. Integrating these into a platform or portal adds further context to the performance data, particularly when used with a portal’s live data.

What are the benefits of an internal developer platform?

Productivity

The primary benefit of internal developer platforms is increased productivity at the level of individuals and the organization as a whole. By enabling self-service and golden paths, platforms increase the autonomy of every developer, which in turn increases production speed and benefits time-to-market.

Cognitive load

Platforms abstract away enough complexity for developers to get on with coding, while still ensuring they play an important role in ops. Reduced complexity means less cognitive load for each developer, which allows them to add more value by prioritizing writing code.

Best practice

By mapping out best practice, platforms give new starters clear direction on how to get started, which improves onboarding and reduces reliance on internal headcount to own this. Best practice is subjective to each organization and team, so platforms can reduce the time needed for new hires to internalize the systems and philosophies that dictate programming and add value from the start.

Where do internal developer platforms fall short?

Opting for a platform means organizations are choosing to position some of the underlying complexity of software development within their developer team. This is often appropriate for the skillsets available, but does mean that a portion of the developer team’s capacity remains committed to ops.

Platforms reduce cognitive load considerably, but do not minimize it to the extent that is technically possible. This distribution of resources isn’t appropriate for every team, as certain responsibilities continue to sit with developers.

What do developers continue to own?

Operational maturity is a measure of the extent to which best practice is defined and applied within an organization. It runs from the more basic metrics such as the presence of README files to automated testing and code coverage that can uncover bugs. Organizations using platforms will make the developer team responsible for operational maturity, which can amount to a burden that increases cognitive load.

Portals, by contrast, provide integrations and plugins that locate operational maturity in decentralized scorecards and make developers responsible for meeting these standards rather than monitoring them. While aligning on best practice is a benefit that platforms can support, the responsibility for keeping best practice current and prominent continues to rest with developers. Platforms lack a centralized repository of live data that can inform and enforce best practice standards, meaning best practice rests with developers as a group rather than being externally defined and abstracted.

Is an IDP right for my business?

All software development teams benefit from identifying and codifying best practice, from using tools that support consistent and coherent development and from reducing cognitive load for developers. At a small scale, this can be handled by spreadsheets and rigorous onboarding, but as companies scale the value of an engineered platform or a formalized portal increases.

Companies growing quickly can benefit from starting to build processes and automation early on, setting the groundwork for a future platform or portal. The benefits of adopting platforms and portals become more evident when velocity and efficiency plateau and drop, as toil creeps into developers’ workloads. Companies fighting back against this entropy will typically impose DevOps KPIs and targets, identify unnecessary dependencies and bottlenecks, and rationalize their org charts to keep things lean.

When a company is at the scale and maturity to benefit from this approach to management, they will benefit from extending it to tooling and infrastructure through a platform or portal. Larger organizations develop more bureaucracy and bottlenecks as a function of their size, as well as collecting more tools in line with the idiosyncratic needs of a large number of developers. This can cause cognitive load to build up at a greater rate, like technical debt in the minds of developers.

While diversity of thought and creativity is crucial for developer productivity, these can and should be enabled based on the shared language of best practice and the golden paths provided by platforms and portals. You can learn more about the difference between platforms and portals here.

How can Cortex help?

Cortex is an Internal Developer Portal that comes out of the box with 50+ SDLC integrations that you can use to build your engineering system of record, set standards of development maturity, security, and compliance, and drive progress to goals. Cortex’s top features include:

• More-than-Services Catalogs: IDPs must have the ability to catalog software in a way that makes ownership, components, language, changes over time, and documentation obvious. Cortex helps you auto-populate catalogs of any type (services, infrastructure, APIs, data pipelines, etc) using data from across your SDLC.

• Ecosystem-Spanning Integrations: Catalogs should be fueled by integrations that span your entire developer ecosystem, so that existing data dependencies and relationships can be preserved without needing to rebuild your data model from scratch. Cortex enables you to import data from all of your tools including observability, monitoring, security, CI/CD, git (GitHub, GitLab), gitops, and infra like AWS and Kubernetes.

• Custom Data: Data from internal systems, home-grown tools, or third-party applications can’t be overlooked when building a central system of record. Cortex enables users to add data from anywhere, and use that data in Scorecards and Initiatives to drive timely action.

• Scorecards: Create custom rubrics for any short or long-term use case using data from across your ecosystem. Cortex Scorecards can include rules related to maximum allowable vulnerabilities, integration with other security solutions, and minimum code coverage—or can be as simple as requiring Helm charts to be attached to each service as part of a one-time migration to Kubernetes.

• Initiatives: Initiatives add a temporal element to your projects, making them increasingly popular with organizations that want to track meaningful progress towards short term initiatives. Initiatives elevate Cortex’s platform from a simple wish list of improvement areas into the realm of real accountability and measurement.

• Scaffolding: IDPs now must have the ability to scaffold new services and resources according to defined best practices. Cortex’s Scaffolding enables users to templatize best practices to bootstrap new software quickly.

• Actions: IDPs should also enable resource provisioning. Cortex Actions enable users to send payloads out of the platform for execution elsewhere.

• Plugins: Popularized by open source Developer Platforms and since adopted by Developer Portals like Cortex, plugins enable users to inject data from anywhere into their daily workflows.

• Customization: As engineering teams work to blend new tooling into already crowded developer workflows, customization of the data model, UI, and workflows have become a non-negotiable for Developer Portals seeking to centralize tooling and activities. Cortex enables users to build and arrange the experience that works best for the way they work.

Interested in learning more about Cortex? Take a self-guided tour of the Cortex platform, or get in touch with our team to learn how Cortex can help support your journey to engineering excellence.