OpenStack Keystone Mitaka Summit Summary

OpenStack Keystone Mitaka Summit Summary

Here are some of my general notes on the outcomes of the keystone specific OpenStack Mitaka Design sessions.

Tokens and Authentication

The keystone team will continue working towards making Fernet tokens the default. We have patches in flights to Tempest and Devstack to move in that direction. There also seems to be quite a bit of confusion around Fernet in general, so improvements to the documentation would be helpful. I have some scratch paper laying around with FAQs from the summit that I plan to add to the official docs. Currently, there is a wealth of information scattered across several blogs. This information would be good to sift through and propose an official set of Fernet documents to keystone. More insight into the discussions around the future of Fernet can be found in a previous post. The keystone team also wants to take strides to get token-less authentication with X.509 certificates to be the default in Devstack, specifically for service users. The full etherpad from the session can be found here.


Performance has been somewhat of an issue with Fernet tokens. One of reasons is because the revocation events tree has to be built and traversed in order to determine if a particular token has been revoked. The current revocation API allows revocations based on project and domain. With Fernet, we technically get this without having to store a revocation event. If a user has a project scoped token and the assignment is removed from the user, the token will no longer be valid since that user won’t have any role assignments on that project. Same holds true for domains. If Fernet ends up being the default token provider and we deprecate the other token formats, keystone no longer needs to persist revocation events for adding and removing users from projects or domains.

Hierarchical Multi-tenancy

Hierarchical Multi-tenancy (HMT) has been on it’s way into keystone for some time now (I think we started whiteboarding HMT in Atlanta?). It also requires a lot of changes and restructures how projects and domains work together. We took time in our HMT session to re-scope the priorities around HMT. The reseller feature won’t be pursued for this release and neither will the support for nested domains. Top level projects will still be considered domains, and projects will still be nest-able. Nesting domains within domains and the reseller case will be revisited once the base implementation has a little more time to bake. The full etherpad from the session can be found here.


We had some really good feedback from operators on policy file changes, and how they are working around the infamous admin-ness bug. A note was sent to the mailing list, asking for operators to record what they are changing and why. The development community is hoping to abstract some common patterns away from these changes and deliver them upstream. This will hopefully serve as a start in addressing keystone’s most notorious bug. There is also a detailed list of other options for addressing operational pain. The full etherpad from the session can be found here.

Mitaka Deprecations

The following bits of keystone are planning on being deprecated, or removed, in the Mitaka release:

  • LDAP assignment – this might actually be removed in Mitaka given that it was deprecated in the Kilo release.
  • LDAP writes support – this will require a change to rewrite ldap support in ldap3 instead of python-ldap. The result should be a much simpler LDAP code-base. Everything will be read-only as far as keystone is concerned, leaving only LDAP management tooling to handle writes to LDAP.
  • Eventlet – I believe this has been deprecated since the Kilo release, but there is a need to have better documentation around running keystone in Apache and nginx before ripping out eventlet completely. I have a TODO to collect that documentation from a few deployers and propose it to keystone.
  • v2.0 API – yep, this is hopefully going to happen, for real this time. The idea is to be able to remove all administrative CRUD from the v2.0 API. There is also focus on converting the v2.0 API to middleware that runs on v3. The middleware would just translate v3 calls to look and feel like v2.0, but give us the ability to drop more of the v2.0 codebase sooner. This also lead to discussions about the remaining gaps of running v3 exclusively in the current stack. The keystone team has an action item to add non-voting v3-only jenkins jobs to other projects.
  • PKI/PKIz token providers – we’re hoping to switch Fernet to be the default token provider in keystone. This would open us up to deprecate the PKI and PKIz token providers.
  • Inheritance model – the current inheritance model is undergoing some changes as it moves from an extension to keystone proper. The changes would be specific to roles and how they are applied through the hierarchy. The old model would be deprecated in favor of the new behaviors that will be a part of keystone core.
  • auth_token configurations – cleaning up old options and bump versions.

The full etherpad from the session can be found here.


We had lots of discussion around federation, specifically the seamless user-experience part. There was a lot of Q&A, especially around deployment configuration and troubleshooting, showing us where we need to improve our documentation. All the notes from the federation design sessions are available, as well as the cross project federation session.

Mix & Match Federation

We had the chance to get a demo from the Massachusetts Open Cloud (MOC) on their implementation of “Mix & Match” federation. They successfully attached cinder volumes from one cloud to nova instances in another. This was a similar demo to the one presented to the keystone team at the Boston mid-cycle meetup. Here, there was a lot more visibility and discussion involving projects outside of keystone. I think the folks from the MOC are working on getting a demo recorded and opening specs for other projects that require changes for mix and match federation to work.

Shadow Users

The keystone team had a few hallway tracks around persisting ephemeral users to keystone. This would be extremely helpful in certain federated cases. It would allow for better support of federated users, it would allow standard tokens to work for federated use-cases among other benefits. This is a very new idea but I believe dolphm is planning on writing a spec.


Do we need more fine-grained mapping control? We sat down and talked about the abilities of the current mapping engine and if it was addressing the things it needed to. There was quite a bit of interest in the ability to pull roles out of SAML directly. I think this is a discussion that will be revisited in the future, if and when the shadow user stuff happens.

Service Catalog

In Vancouver, there was a session that kick started the discussion on the current service catalog. One of the cross-project sessions in Tokyo was dedicated to continuing that conversation. The notes from Tokyo did a good job of highlighting the issues with the catalog as-is:

  • It’s project specific, meaning that you have to authenticate in order to get it.
  • There are no standard, dependable types.
  • There is logic built into keystone that works around size issues.
  • Endpoint differences, public vs internal vs admin.

The session determined what needs to change, what needs to stay, and why. There is also going to be a weekly status meeting that includes contacts for each project. I imagine we will have a similar session at the Austin summit in 2016.


Photo Credit: flodambricourt via pixabay 

Improving Auditing in Keystone

Improving Auditing in Keystone

The Future of Fernet Tokens

The Future of Fernet Tokens