How to use Ephemeral Environments to support your Backstage development

‍

Introduction

An overview of Backstage

Backstage is an open-source developer portal designed to streamline and automate the software development process. Developed by Spotify, Backstage provides a unified interface for managing software projects, tools, and infrastructure. By consolidating various development tasks and services into a single platform, Backstage helps developers efficiently navigate and interact with their software ecosystem, reducing context-switching and improving productivity. The platform supports a plugin-based architecture, allowing developers to extend its functionality and customize it to fit their organization's specific needs. We will look at how B Backstage development can be sped up with the help of ephemeral environments.

‍

Ephemeral environments and their role in the Backstage development

‍

What are Ephemeral Environments?

Ephemeral environments are temporary, on-demand instances of an application or service that can be spun up quickly for development, testing, and collaboration purposes. They play a crucial role in the Backstage development process by enabling developers to work in isolated environments, minimizing the risk of conflicts and ensuring a consistent development experience.

‍

How can Ephemeral Environments be leveraged to speed up Backstage development?

In the context of Backstage, ephemeral environments can be used to develop and test plugins, as well as the platform itself, without affecting the main development or production instances. These environments allow developers to experiment with new features and validate their changes before merging them into the main codebase. Additionally, they facilitate collaboration by providing shareable instances for team members to review, debug, and test together, ensuring a smooth integration process.The following are a few specific ways of using the ephemeral environments:

‍

• Checking ingestion of new Entity Providers : At its core, Backstage is a catalog for managing softwares for an enterrise. Entity providers bring in the raw data to Backstage and help in the ingestion and then seeding the data to the database. To test new entity providers before putting them in production Backstage, ephemeral environments can be used to spin up temporary backstage instances with these providers installed and ready to test.

‍

You can learn more about the life of an entity here and about Backstage’s incremental entity provider over here.

‍

• Dry running software templates : Backstage doesn’t just catalog software but also allows developers to kickstart development. Templates empower developers to start building while complying with company requirements. Organizations can create these templates with important predefined logic or configurations. This logic or configurations are usually mandatory in some enterprises where they might have an internal CI/CD system or processes which need to be adhered to when onboarding a new project. Dry running and testing these templates will free the developer from doing so and keep developers away from bugs in these templates.

You can learn more about software templates here.

‍

Now that we understand how ephemeral environments can be helpful, let’s see how we can get started with building them for your Backstage development.

‍

Understanding how Uffizzi ephemeral environments are configured and hosted

Uffizzi runs on top of Kubernetes but the end user doesn’t have to bother with all that. If the application has a preconfigured Dockerfile and docker-compose ready to go with it then configuring Uffizzi is going to be straightforward. If your application doesn’t already have a Dockerfile and/or a docker-compose config we will be discussing a little bit about that in this blog, but these two configurations are prerequisites for Uffizzi.

‍

To learn more about Uffizzi ephemeral environments check out the documentation here.

‍

Setting up Uffizzi for Backstage development

We are going to use this Backstage sample repository to understand what it means to develop a sample Backstage application.The repository contains two different ways of spinning up a Uffizzi instance for a backstage application:

‍

• Backstage created with @backstage/create-app: Follow this if you have created your Backstage application with the`npx @backstage/create-app@latest` command.
• Developing on a fork of Backstage: Enterprises like Roadie and Frontside, fork and work on their version of Backstage for their customer. This is because of Backstage’s permissive licensing. If you own a similar setup of Backstage and would like to use Uffizzi ephemeral environments with your setup, this is for you.

Let’s explore each of the two ways of setting up Uffizzi for your Backstage development method.

‍

Uffizzi setup for Backstage created from @backstage/create-app

The following is the directory structure of this sample application with the relevant files and directories for this guide:

‍

Understanding the the sample repository structure

/.github		
  /workflows                   # github actions workflows
    uffizzi-build.yml	        # builds images for the ephemeral env
    uffizzi-preview.yml        # takes built images and creates preview
	  …	
/uffizzi-backstage	        # source code for the backstage app
  /packages/backend/Dockerfile	# dockerfile to build backstage backend
    …
docker-compose.yml             # docker compose for local development
docker-compose.uffizzi.yml     # uffizzi configuration
app-production-config.yml      # uffizzi specific backstage config
app-config.yml			# backstage configu for local dev
…

Project directory structure

‍

The backstage backend service will be built into a container image which is used to create the backstage container. The builds for both these images need to be mentioned in the uffizzi-build.yml configuration file which can be seen in the build-backstage job. The image tag is built and pushed from here are then referenced in the Uffizzi Compose using the ${BACKSTAGE_IMAGE} placeholder. To understand more on how this can be done, check out this blog which will guide you through extending Github Actions with Uffizzi. We will not be going through how to set up the uffizzi-build.yml and uffizzi-preview.yml in this blogpost.

‍

A backstage application created from the `npm @backstage/create-app` initially needs a more straightforward setup. We will be discussing what the setup for such a backstage application looks like in this case. We will discuss the architecture and the workflow definition to create the ephemeral environments below.

‍

Overview of the Architecture

‍

‍

Github Actions workflow definition

We are extending the reusable Github Action workflow for Uffizzi. The reusable workflow here, can be extended to work in a two stage mode to allow contributors to create pull requests and consume ephemeral environments without needing credentials or accounts on Uffizzi if they are new uses on Uffizzi itself.

‍

The workflow in this case is divided between the build and deploy part where the build part is executed in context of the pull request branch to build the relevant image. The deploy part of the repo runs in context of the main repo. It is triggered when the build is completed. If you are interested in the motivation and working of the two stage workflow, this blog post dives deeper in the same. Let’s look at the triggers for each to understand their relationship, let’s take meilisearch as an example where this has been implemented:

‍

name: Preview (build)
on:  
  pull_request:    
    types: [opened,synchronize,reopened,closed]

https://github.com/UffizziCloud/backstage-ephemeral-environments

‍

name: Preview (deploy)
  on:  
    workflow_run:    
      workflows:      
        - "Preview (build)"  
    types:    
      - completed

https://github.com/UffizziCloud/backstage-ephemeral-environments  

‍

In the above excerpts from the two stage workflow files we can see that there is a natural trigger for the uffizzi-build.yml. The build workflow is triggered when a pull request is opened or updated. In the uffizzi-preview-deploy.yml we can see that the workflow is triggered only when the first workflow completes.

‍

The deploy phase of the workflow works as follows :

‍

deploy-uffizzi-preview:
  name: Use Remote Workflow to Preview on Uffizzi
  needs:  
  - cache-compose-file
  if: ${{ github.event.workflow_run.conclusion == 'success' }}
  uses: UffizziCloud/preview-action/.github/workflows/reusable.yaml@v2
  with:  
    # If this workflow was triggered by a PR close event, cache-key will be an empty string
    # and this reusable workflow will delete the preview deployment
    compose-file-cache-key: ${{ needs.cache-compose-file.outputs.compose-file-cache-key }}  
    compose-file-cache-path: docker-compose.rendered.yml  
    git-ref: ${{ needs.cache-compose-file.outputs.git-ref }}  
    pr-number: ${{ needs.cache-compose-file.outputs.pr-number }}  
    server: https://app.uffizzi.com
  permissions:  
    contents: read  
    pull-requests: write  
    id-token: write

https://github.com/UffizziCloud/backstage-ephemeral-environments  

‍

Here we are using OIDC tokens provided by Github to secure the identity of the contributors. Hence, providing ease of use and security in one package. Meanwhile, workflows scoped to the default branch of the base repository are the only ones able to authenticate with Uffizzi. To learn more about the two stage workflow, check this blog post here.

‍

Uffizzi setup for a fork of Backstage

The Backstage project itself uses Uffizzi for speeding up its development with Ephemeral instances. The team primarily uses the ephemeral instances as preview environments. Considering that we have already seen how the two stage workflow works above, we are going to see how we can set up a Backstage fork for an enterprise with a one stage workflow to restrict the ephemeral environment creation access to internal developers.

‍

Using the single-stage Github Actions workflow for your fork of Backstage

‍

Let’s start with an example fork of Backstage. The only difference between the original Backstage and this fork is the Uffizzi workflow being used. The origin backstage repo is using Uffizzi’s two stage workflow, if you want to learn more about the different Uffizzi github action workflows, check out this blog. Considering the nature of the backstage project, they need contributors to be able to see previews of Backstage as well. The two stage workflow helps well with that. But if you are developing on Backstage directly for your internal needs, you should use Uffizzi’s one stage github actions workflow. This workflow makes sure that a contributor without the right permissions won’t be able to get Uffizzi previews.

‍

The difference between the two different workflows is that there is no context change between the build stage and the deploy stage. The build and deploy steps happen in the same stage, hence it is called the one stage workflow. You can see the excerpt definition of the workflow below :

‍

name: Preview (build and deploy)
on:  
  pull_request:    
    types: [opened, synchronize, reopened, closed]    
    paths-ignore:      
      - 'microsite/**'
jobs:  
  build-backstage:    
    env:      
      NODE_OPTIONS: --max-old-space-size=4096
      UFFIZZI_URL: https://uffizzi.com    
    name: Build PR image    
    runs-on: ubuntu-latest    
    if: ${{ github.event_name != 'pull_request' || github.event.action != 'closed' }}    
    outputs:      
      tags: ${{ steps.meta.outputs.tags }}    
    steps:      
      - name: checkout        
        uses: actions/checkout@v3‍      
      
      - name: setup-node        
        uses: actions/setup-node@v3        
        with:          
          node-version: 16.x          
          registry-url: https://registry.npmjs.org/‍   
          
     - name: yarn install        
       uses: backstage/actions/yarn-install@v0.6.3        
       with:          
         cache-prefix: linux-v16‍      
         
     - name: Use Uffizzi's backstage app config        
       run: |          
         cp -f ./.github/uffizzi/uffizzi.production.app-config.yaml ./app-config.yaml‍      
         
     - name: typescript build        
       run: |          
         yarn tsc‍      
         
     - name: backstage build        
       run: |          
         yarn workspace example-backend build‍      
     
     - name: Set up Docker Buildx        
       uses: docker/setup-buildx-action@v2‍      
       
     - name: Generate UUID image name        
       id: uuid        
       run: echo "UUID_TAG_APP=$(uuidgen)" >> $GITHUB_ENV‍      
       
     - name: Docker metadata        
       id: meta        
       uses: docker/metadata-action@v4        
       with:          
         images: registry.uffizzi.com/${{ env.UUID_TAG_APP }}          
         tags: type=raw,value=60d‍      
       
     - name: Build Image        
       uses: docker/build-push-action@v4        
       with:          
         context: .          
         file: packages/backend/Dockerfile          
         tags: ${{ steps.meta.outputs.tags }}          
         labels: ${{ steps.meta.outputs.labels }}          
         push: true    
         
  render-compose-file:	
    name: Preview on Uffizzi	
    if: github.event_name == 'pull_request'	
    runs-on: ubuntu-latest	
    needs:  	
      - build-backstage	
      steps:  	
      - name: Checkout git repo    	  
        uses: actions/checkout@v3  	
      - name: Render Compose File    	  
        run: |      	  
        BACKSTAGE_IMAGE=$(echo ${{ needs.build-backstage.outputs.tags }})      	  
        export BACKSTAGE_IMAGE      	  
        # Render simple template from environment variables.      	  
        envsubst '$BACKSTAGE_IMAGE' < .github/uffizzi/docker-compose.uffizzi.yml > docker-compose.rendered.yml      	  
        cat docker-compose.rendered.yml  	
      - name: Hash Rendered Compose File    	  
        id: hash    	  
        run: echo "COMPOSE_FILE_HASH=$(md5sum docker-compose.rendered.yml | awk '{ print $1 }')" >> $GITHUB_ENV  	
      - name: Cache Rendered Compose File    	  
        uses: actions/cache@v3    	  
        with:      	    
          path: docker-compose.rendered.yml           
          key: ${{ env.COMPOSE_FILE_HASH }}‍  
          
  deploy-uffizzi-preview:	
    name: Use Remote Workflow to Preview on Uffizzi	
    needs: render-compose-file	
    uses: UffizziCloud/preview-action/.github/workflows/reusable.yaml@v2	
    with:  	  
      compose-file-cache-key: ${{ needs.render-compose-file.outputs.compose-file-cache-key }}  	  
      compose-file-cache-path: ${{ needs.render-compose-file.outputs.compose-file-cache-path }}  	  
      server: https://app.uffizzi.com	
    permissions:  	  
      contents: read  	  
      pull-requests: write  	  
      id-token: write 	   
    
  delete-uffizzi-preview:	
    name: Use Remote Workflow to Delete an Existing Preview	
    uses: UffizziCloud/preview-action/.github/workflows/reusable.yaml@v2	
    if: ${{ github.event_name == 'pull_request' && github.event.action == 'closed' }}	
    with:  	  
      compose-file-cache-key: ''  	  
      compose-file-cache-path: docker-compose.rendered.yml  	  
      server: https://app.uffizzi.com	
    permissions:  	  
      contents: read  	  
      pull-requests: write  	  
      id-token: write

‍https://github.com/UffizziCloud/backstage-fork  

‍

In the above workflow the following two jobs are running:

‍

1. build-backstage :  Builds and pushes the image for backstage which will be used to deploy the backstage instance.
2. deploy-uffizzi-preview : Uses the image from the previous build-backstage job to deploy to Uffizzi. The workflow uses the OIDC flow which will allow users to use Uffizzi without necessarily logging into the Uffizzi account. A new account is also created if the user doesn’t already have an Uffizzi account.

To set up your backstage fork with Uffizzi, you have to copy the above workflow to your repo and that’s how straightforward it is to get started with Uffizzi.

‍

Different ways of using the Uffizzi URL created for your Backstage ephemeral environments

‍

‍

‍

After doing the basic setup of creating a workflow for Ephemeral Environments for your Backstage, you can think about how you can start sharing or leveraging the Uffizzi URL for your Backstage instance in the following ways:  

‍

• Sharing environments with stakeholders: Once the environments are created, they can be shared with other developers, project managers or owners who are stakeholders in the project. This can be done to get feedback on certain bug fixes or features being developed allowing the developers to get valuable feedback.
• Running integration tests against Ephemeral Environments: Once the ephemeral environments are up, and if the project uses playwright or selenium for end to end or integration testing, the final UFFIZZI_URL can be passed to the test suite to run tests against a running ephemeral environment instance. This would allow the project to have integration tests to run on every PR.

‍

Productive customizations for your ephemeral environments of Backstage

Once you have figured out how to share the Uffizzi URL for the ephemeral environment instance for your Backstage, you can go further and improve your ephemeral environmet experience by adding  customizations like the one below:

‍

• Using a load balancer for Backstage and its analogous services : We can point a Backstage instance to a loadbalancer so that we can route traffic to services other than Backstage. This is especially useful when texting to check the state of analogous services which backstage is dependent on.
• Interacting with your Ephemeral Environment using Webterminal : If the application has a CLI environment which would be useful to have access to, the user can build their application with a webterminal installed and then access every ephemeral environment instance using a webterminal.

‍

Backstage ephemeral environment success story

Uffizzi environments are used by the core Backstage team. Considering that it is currently being used by Spotify for Backstage we keep improving the Uffizzi experience for all Backstage contributors and see to it that we are able to provide the best experience for Backstage developers. Check out the blogpost Substantial increase in Development Velocity because of Uffizzi which has statistics and analysis on Backstage’s increased productivity after integrating Uffizzi into their workflows and how adopting Uffizzi increased Backstage’s development velocity by more than 20%.

‍

Next Steps

If you have a Backstage project for which you would like to build ephemeral environments for and discuss the different ways you can use them to better fit your needs, we look forward to speaking with you. You can get in touch with us by going to https://www.uffizzi.com/contact and we will reach out to you !

‍

Visit docs.uffizzi.com for more information on ephemeral environments. Visit backstage.io for further guides on Backstage development.