Orange Interstellar Corporation Event WebApp

A WebApp for a Metaverse event built on Azure that combines Bring Your Own IDentity, Identity Proofing and Decentralised Identity with token binding using biometrics for MFA.

Comment

Orange Interstellar Corporation Event WebApp

Our team’s submission to the Microsoft Identity for All Hackathon is an Event Web Application solution for a Metaverse online event built entirely on the Azure Platform that combines Bring Your Own IDentity (BYOID), Identity Proofing and Decentralised Identity with token binding using biometrics for multifactor authentication.
We wanted to build a Microsoft identity based solution that would ensure that only the exclusive invitees to our launch event were able to register and enter the event. We did not want invitees sharing their invitation.

Inspiration

The inspiration for our solution was to leverage Decentralised Identity with the addition of biometric based multifactor authentication to provide a higher level of identity assurance whilst also being privacy preserving.

This essentially is also a form of token binding. You are issued a Verifiable Credential that also includes your facial image. In order to successfully enter the event, you need to have both a Verifiable Credential and perform facial biometric matching to the registered facial image associated with that Verifiable Credential.

Orange Interstellar Corp Announcement Event

Orange Interstellar Corporation is ready to unveil their latest Interstellar Sports Utility Vehicle. Rumour has it the Interstellar Sports Utility Vehicle can also come with an optional interstellar jetbike.

The event will be an online event in the Metaverse attended by industry A-Listers and interstellar transportation influencers who are the recipients of an exclusive and illustrious personalised platinum ticket invitation.

Key Solution Concepts

  1. Identity Proofing: Invitees are sent an exclusive invitation to the Orange Interstellar Corp Event.
    • Invitees are validated (via External Identities User Flow API Connector) to confirm they were sent an invite after they have validated their credentials at their home Azure AD Tenant (via AAD Federation).
  2. Verifiable Credentials and Token Binding: Enrolment for the Event takes Guest Azure AD User account metadata, takes a photo of the user via the web browser and their webcam to be used as a form of token binding and multifactor authentication with all information required stored in an Entra VerifiedID Verifiable Credential.
    • No biometric data is stored in Azure AD. The VerifiedID Verifiable Credential contains all the required information to access the event and validate the VC and verify the presenter of the VC is the same person that was issued it (using Facial Recognition and the Azure Cognitive Services Face API)
  3. Privacy Preserving: The solution whilst using biometric functionality does not store it at rest. The entire end to end flow from user redemption to event entry can be performed using Passwordless capabilities.

Demo

Demo Video

Brief Technical explanation on how the sign-up works

Technical Overview Video

Demo of someone NOT invited trying to sign-up

Demo Video

Brief Technical explanation on how non-invitees are denied sign-up

Technical Overview Video

Enrolment and Event Access Flows

The following graphic details the Azure services used in our submission and how they are utilised for both the Event Enrolment and Entry scenarios.

Challenges

AAD AuthN using JavaScript and SPA Example outdated

Example code uses MSAL.js 2.15.0. This does not include the MSAL Prompt=Create functionality required to create an AAD Account using Self-Service Sign-Up Flow

We updated the example code to use MSAL.JS 2.26.0 and updated the code to trigger Azure AD Guest Account Creation via Self-Service Sign-Up

AAD External Identities API Connector Python Function Example limited

The example code is just the scaffolding for an Azure Function to call and gives an indication as to what is passed during the Sign-Up flow. We had to extend it significantly to get the functionality we required. But it was a good starting point.

Azure Cognitive Services Face API Key Revoked

We had a working solution using the Cognitive Services Face API. Then API key was revoked (part of the responsible AI push?) and we received the following message. "UnsupportedFeature", "message":"Feature is not supported. Please apply for access at https://aka.ms/facerecognition“ We managed to dig up an API key from a previous demo project to use and complete our submission.

VerifiedID Verifiable Credential in Authenticator Wallet has limited customisation

We wanted to store the users registered facial image in the VC. We can, but just as Base64 encoded text. It would be great to have the VC in Authenticator Wallet support an Image Tag e.g. <img src="data:image/png;base64,iVBORw0KG…… For the purpose of this submission, we have stored the FaceID returned from Cognitive Services. We use this to do the compare when entering the event. See below for the limited usability of this feature and why we'd like to store the image on the VC (in a prettier format).

Azure Cognitive Services Face API has a retention period of 24hrs

As mentioned in the customisation of VCs in the Authenticator wallet we’d have like to incorporate the captured image. It just looks ugly having the Base64 encoding of the image in the VC. We used the DetectedFace capability of Face API. This does mean that the FaceID we stored on the VC to compare on redemption is only good if performed within 24hrs of capture.

We couldn't do full Passwordless

The External Identities Self-Sign-Up create guest user account flow does not allow the user to authenticate to their home Azure AD Tenant using advanced authentication methods such as Windows Hello or FIDO Tokens. It would have been awesome to make this a passwordless solution by allowing the Create User flow to authenticate the user to their home AAD with a token or Windows Hello.

Mix of Languages built by IT Pros

Our submission uses a mix of programming languages built by IT Pros (not developers). Our geographically dispersed team jumped between using Python for the Azure Function App for Self-Service Sign-Up User Flow, JavaScript for the Web Front-End and C# for the Verifiable Credential Issuance and Verification.

Solution Identity Elements (Built with)

Azure Identity services and capabilities utilised in our Orange Interstellar Corporation Event application are extensive. Below is a list of each Azure service or capability used in our solution along with a description of where and what the capability provides.

Azure AD

  • Azure AD used for Guest user objects created via External Identities Self-Service Sign-Up functionality
  • Azure AD used to register the Application used for the Single Page App for the Orange Interstellar Corp Event webpage for invitation redemption and event sign-in.

Azure AD B2B and External Identities

  • Orange Interstellar Corp Event invitees utilise Self Service Sign-Up to the Azure AD registered application using Identity Federation to users’ home Azure AD.
  • Validation of users’ validity to redeem an invite performed by a User Flow and an API Connector integrated to an Azure Function App.

Microsoft Entra VerifiedID Verifiable Credentials

  • Microsoft Entra VerifiedID used for both Verifiable Credential Enrolment and Presentation.
    • Following validation of RSVP redemption (see Azure AD B2B and External Identities and Azure Function App and Facial Recognition enrolment via Azure Cognitive Services) a Verifiable Credential is issued to the invitee that includes their Firstname, Lastname, DisplayName, Azure AD Guest User ObjectID and Facial Recognition registration ID.
    • On event day the event guest is authenticated by presenting their Verifiable Credential issued on enrolment and completing Facial Recognition to match the image taken at registration.

Azure Cognitive Services Face API

The Azure Cognitive Services Face API is used during both the invite redemption (enrolment) and event access (validation) scenarios.

  • During enrolment a Facial image is captured and presented to the Cognitive Services Face API. This is the registration reference image that is later used on event entry to validate the user presenting a VerifiedID Verifiable Credential is the same person that was issued the VerifiedID Verifiable Credential.
  • As mentioned above on Event Day and event entry another facial image is captured and compared to the registration facial image using the Azure Cognitive Services Face API to verify and provide assurance that the person presenting the VerifiedID Verifiable Credential is the same person that was issued the VerifiedID Verifiable Credential. This is also a form of token binding with Entra VerifiedID Verifiable Credential

Azure WebApp

The Orange Interstellar Corp Event website is an Azure WebApp that was built locally using Visual Studio Code. It utilises MSAL for JS as listed below under MSAL Libraries to enable the Sign-Up functionality of the WebApp to allow invitees to bring their own identity from their home Azure AD Tenant.

MSAL Libraries

  • MSAL for Python Azure Function for External Identities Sign-up User Flow API Connector
  • MSAL for JS Website front end for user authentication via Azure AD B2B External Identities

Managed Identity

Azure Function App for External Identities Sign-up User Flow API Connector. System Managed Identity is used to authorise the Azure Function App to connect to Azure KeyVault to retrieve the credentials used to validate against the authentication sent from the External Identities API Connector to secure the trigger of the Function App.

Microsoft Graph

Microsoft Graph is used to retrieve registered users Azure AD Guest user objects and the associated attributes populated via the Azure AD External Identities Self-Service Sign-Up enrolment process.

Azure KeyVault

Azure KeyVault used to store the credentials used by the Azure Function App to validate the API Connector is authorised to trigger it. Azure KeyVault is also used for Verifiable Credentials configuration.

What's next for the Event WebApp

This submission went from concept to working solution in two weeks by a small team geographically dispersed (Scotland, England and Australia). The solution was broken into elements that were predominantly built by a single individual in separated environments. Once all elements were built, they were pieced together to a working solution.

We see promise in combining biometrics with verifiable credentials. We plan on taking this hackathon proof of concept and enhancing it further by improving the flow (no more popup windows) and improving the security (remove the front end JavaScript) by implementing this as a server-side solution and the addition of additional features.

Solution Team

The Privacy Preserving Identity Proofing Event Web Application Solution was built over two weeks in a part time capacity by a geographically dispersed team consisting of Darren Robinson, Elias Ekonomou, Christian Chung-Tak-Man and Farzan Akhtar built specifically for the Microsoft Identity for All Hackathon.

Solution Repository

This GitHub Repo contains the Azure WebApp, Azure Function App and the VerifiedID Verifiable Credentials solution projects.

Built With

  • azure
  • azure-active-directory
  • azureapp
  • azureappservice
  • c#
  • decentralizedidentity
  • identity
  • python
  • verifiablecredentials
Share this project:

Updates