OpenStack Summit Boston Recap

OpenStack Summit Boston Recap

Outside of being home to amazing clam chowder and lobster rolls, Boston was the location of the OpenStack Summit this spring. Here are my take-aways and highlights of the trip.

Baremetal/VM Working Group

This is a new approach that focuses on fixing larger cross-project issues. John Garbutt is one of the main drivers of the original proposal. The goal of the group is to locate cross-project issues when providing baremetal/virtual machine infrastructure and fixing them. Ideally there should be representation from each affected project. We had a total of three sessions at the forum to determine what pain points exist, their importance, and next steps. The three big topics that affected keystone were unified limits, policy, and need for API keys, which are documented respectively in the sections below.

Quotas & Limits

Several projects in OpenStack provide quotas and limits, but having each project roll their own quota and limit APIs/logic is time-consuming and prone to inconsistencies. A takeaway from the PTG in Atlanta was a proposal for keystone to implement unified limits. The long-term goal is to provide other services with the ability to query keystone for the limit on a project, and use that limit to enforce resource quotas or allocation. Sean Dague addressed the current approach and what lies ahead in this forum session

The idea of implementing unified limits in keystone was generally accepted. The session evolved to address enforcement models (i.e. defaulting to open limits or closed limits) and what to do in "overcommit" scenarios. The biggest hurdle for keystone is to validate limit updates within a project tree. Another concern is the complexity and performance of complex project tree structures and their limits. Our next step in keystone is to solidify the unified limits specification, which includes the performance concerns, and to merge it so we can get the implementation into Pike. Projects can then build on it in Queens and work towards a more consistent quota implementation across OpenStack.

Policy & RBAC

Policy issues across OpenStack were mentioned in several sessions, and it was one of the highest priority items for the Baremetal/VM working group. It became apparent, at least to me, that we could improve policy by working fixes in parallel. There is the admin-ness issue that we've been battling across OpenStack for some time. But there is also the lack of sane default roles. I don't think there is anything that prevents projects (including keystone) from working to define and move towards better default policies. At the same time, we should be able to introduce mechanisms that would allow operators to assign roles in the global context. In addition to this, I proposed we register policy in code as a community goal for Queens. To summarize my action items:

  • Propose a Queens community goal to move policy into code across OpenStack projects
  • Propose a specification to implement global role assignments (which contrasts the admin project approach)
  • Follow-up with the oslo.policy community to develop some sort of deprecation warning when a default policy is used
  • Follow-up with oslo.context to see what it would take to get global scoping honored through oslo.context

[update] In addition to the policy work with the Baremetal/VM group, Adam and Kristi presented their RBAC-in-middleware approach. The discussion generated a lot of dialog after the session, but in case you missed it, check out the recording below:


API Keys

In February, we discussed how to implement API keys into keystone. During the forum, this was brought up again during the Baremetal/VM working group session. The session was designed for developers writing applications on top of OpenStack. The whole group (which consisted of operators and developers across various projects) agreed that this would be useful. Keystone has a specification proposed, but it hasn't been accepted yet. We didn't plan to implement this in Pike because keystone is currently strapped for resources (especially after the OSIC fallout), but the feedback from the session ultimately raised its importance. As a result, I took an action item to email the Product Working Group to see if we could get any help staffing resources to implement API keys. Otherwise, we have some developers (Colleen Murphy and Gage Hugo) who are trying to pick up the work in their spare time.

Project Update

Typically this was always done in a recorded fashion with the Foundation and published afterward. This was the first summit where the session was held in person. We didn't use the entire 40-minute slot but we generated a lot of questions after the recording. In case you missed it, you can watch it below:


Project On-Boarding

The foundation organized project on-boarding for most of the projects. I wasn't sure what to expect because the on-boardings I've done in the past have had two types of attendees. The first being very technical individuals who want to spend as much time as possible digging into the guts of keystone code learning how things work. These folks already have Gerrit and Git worked into their daily development flow and understand most OpenStack processes. The second type are those completely new to open-source development, command-line tools and editors, and sometimes Git. By a show of hands, we had more people familiar with OpenStack development, and wanted to dig into specific keystone details.

We had a good show of keystone-core team members in the room, so our on-boarding quickly turned into a collaborative discussion and eventually moved to whiteboards. During the first half hour, we reviewed basic processes that explained what OpenStack Identity is and what projects are responsible for it (slides are available). After this, we spent the remainder of our time talking about what our new contributors wanted to learn. The most value came from our most experienced keystone developers who helped share some of keystone's tribal knowledge. For example, we explained why keystone has two port configurations, walked through the anatomy of scope, and explained in detail how federated authentication works. Overall I thought the session was super productive and well received. I look forward to doing them in the future and would like to have them recorded. 

Operator Feedback

Whether it's the Project Team Gathering or the forum, my favorite session is operator feedback. In keystone, we have some operators that continually close the feedback loop and engage us on how we're doing. The following were the main themes from Boston's operator feedback session:


  • Fix the admin-ness bug across OpenStack
  • Provide richer roles in policy (i.e. project_manager, helpdesk_support, global_admin, and reader roles were the most common roles operators have to maintain today)
  • Associate multiple domains to a single identity provider
  • Debug issues in federation
  • Split federated documentation into two categories: operator and user docs
  • Enhance the keystone-manage documentation
  • Improve support for Kerberos
  • Implement support for Native SAML
  • Implement API keys

The good thing is that we have several efforts underway to achieve or improve most of the items on this list. Otherwise, most of the work consists of taking an honest look at the current state of our docs and revamping them.


Samuel likes to live dangerously, which is why he decided to do presentation on all of keystone's PCI-DSS work, including a live demo. Attendance was high given it was one of the last talks of the entire week. Sam provided a great overview into what PCI-DSS requirements keystone support and then ran a user through a gamut of PCI scenarios. In case you missed it, checkout the recording:


Lobster Rolls & Clam Chowder

Yes, this warrants its own section. Throughout the course of the week, Luke's Lobster was the go-to place. It was only a couple blocks from the convention center and the food was amazing. The staff was super helpful, friendly, and knew us by name before Thursday (don't judge).


Photo Credit: My lunch at Luke's Lobster

Migrating token formats without downtime

Migrating token formats without downtime

Keystone Pike PTG Summary

Keystone Pike PTG Summary