Amazon CodeCatalyst - Is it ready for the enterprise?

I’m very interested in recent advancements in cloud-hosted development environments. GitHub Codespaces is the option I have the most experience with and the one I use more generally. With cloud-hosted development environments, your local machine becomes more of a thin client that facilitates access to the internet and the development environment. That is a considerable step toward enabling better education in programming and potentially even progress in equity. With cloud-hosted development environments, you no longer need expensive computers for programming, allowing greater access. Many American school systems give all students Chromebooks. With them, students can run an entire development environment entirely in a browser. These machines can be bought for a few hundred dollars, much less than many smartphones. Hosted development environments also potentially lower an organization’s security risk, which is often a concern in American public schools.

In my day job, I’m a consultant and see how many enterprises approach problems and handle their IT equipment and software development practices. At this point, I have clients who are increasingly using GitHub Codespaces as the default way to do development in their organization, and it’s been highly effective. When set up appropriately, it’s a move that helps reduce the “it runs on my machine” issues. By using a consistent configuration for development environments, one that automatically sets up everything, new developers can start writing code much sooner than when they had to set up their own machine based on a Confluence page.

As a consultant, I have to be very aware of overhead costs associated with internal projects. Additionally, consultants on the bench and assigned to internal projects can be moved to billable client work as new projects begin. That makes services with subscription costs a hard sell; pay-as-you-go is a much better approach. That’s where Amazon CodeCatalyst caught my eye. With CodeCatalyst, you create a space that gets connected to one or more AWS accounts. Within the space, you can define multiple projects, and each project has its own repositories, issues, CI/CD workflows, and more. I appreciate how CodeCatalyst can encapsulate the different aspects of a project into one area. If you go beyond CodeCatalyst’s free tier, the standard tier price is based on the active users each month.

Until recently, CodeCatalyst wasn’t a realistic option for use with enterprise work. Access was handled through AWS Builder IDs. That required space administrators to manually manage people’s access. When you have to wear multiple hats, that becomes a difficult task. It also tied access to personal accounts, which tends to be an issue with infosec. AWS recently added support to CodeCatalyst for single sign-on through IAM Identity Center, which opens the door to more enterprise-based usage. At AWS re:Invent, CodeCatalyst gained a new enterprise pricing tier with much higher resource limits and pricing based on the number of users assigned to a given space.

With these changes, I decided I needed to give CodeCatalyst another try.

I initially tried to set up SSO for a CodeCatalyst space without referring to the documentation. There is a wizard that walks you through the process, but I quickly ran into issues and had to pull up the documentation page for setting up identity federation. Unfortunately, that didn’t clear up all my problems, and it took a lot of trial and error to get everything set up.

The configuration I ended up with was the CodeCatalyst space defined in my AWS Organizations management account, the same AWS account as my IAM Identity Center (AWS SSO) configuration. I then had to add the space to the different AWS accounts I wanted it to have access to.

The initial page of the wizard asks you to choose the AWS region where you have AWS Identity Center configured. Whether you are in the AWS Organizations management account or a child account, it determines the identity store ID and fills the value in. That led me to believe configuring SSO for CodeCatalyst could be done on a per-account basis, something that is attractive if you want to keep projects between different departments or business units separated. That isn’t actually supported.

When configuring the IAM Identity Center application name, the wizard says the application name must be unique. The wizard doesn’t explicitly state that the value must be globally unique, like an S3 bucket name. The value must be unique across all of AWS. Additionally, the wizard doesn’t appear to check the uniqueness of the name until the very end when you try to finish your configuration. I ran into issues with that when I didn’t understand it needed to be a globally unique name.

Creating or selecting a space to associate with IAM Identity Center also presented some confusion. I received an error if I tried to set up a new space in an AWS account that already had one. I hadn’t previously realized that each AWS account can be the billing account for only one CodeCatalyst space.

Specifying the groups and admins in the wizard was straightforward enough; I did that without issue.

When it came time to review the configuration details and complete the setup process, I ran into strange errors. One of the most confounding was an error that declared I had invalid parameters but didn’t inform me which parameters were invalid. I figured out the “ConflictException: Tenant already exists” error; the IAM Identity Center app was created but not fully configured. The orchestration behind the scenes doesn’t seem to have robust rollback logic. I still don’t fully understand the “ValidationException: The customer making the request must be the billing account associated with the space” error I received, and the only way I didn’t get that was to configure the space in my AWS Organization management account. Something I later determined was a prerequisite.

After much searching, I eventually found a few lines that clarified that the CodeCatalyst space must be set up in your AWS Organizations management account. One was under the “Prerequisite 1: Setting up an organization in AWS Organizations” heading:

The AWS Organizations administrator sets up the organization and AWS accounts for your company. The management account for the organization will be specified as the billing account for the space in CodeCatalyst. For more information, see What is AWS Organizations?.

The other is under the “Step 1: Create a space” heading in the documentation:

Sign in to the Amazon CodeCatalyst page in the AWS Management Console with the AWS account that you have set up as the management account for your organization in AWS Organizations. That’s what I get for not reading the directions clearly.

I wish those details had been called out much more clearly. It would have saved me a lot of frustration.

Once you get IAM Identity Center configured, it clearly informs you that “Only one CodeCatalyst space can be enabled with IAM Identity Center at a time.” With these restrictions, any AWS Organization can only have one CodeCatalyst space that uses SSO. That means CodeCatalyst’s concept of teams, when paired with SSO groups, is your only way to segment project access. I expect that will be an issue for some larger companies. I haven’t tested out any CodeCatalyst paid spaces yet, but I’m curious if there is a way to easily split out the cost for users to enable billback to different departments.

I’m excited about the potential of CodeCatalyst. It has increasing potential in the enterprise. Right now, startups, smaller companies, or companies that need a platform for more minor projects will find it the most useful. With a few more iterations, I see it becoming a more useful platform for larger enterprises.