GIM - General Identity Management

The Future of Identity Verification with AI-Powered Data Processing

Comment

Inspiration

Creating something meaningful often starts with a solitary spark of inspiration. For me, that spark came from witnessing the challenges people face in proving their identity online. It was a problem begging for a solution, and I was determined to find one.

While there are existing commercial solutions for ID verification through government portals and private companies, the high cost of integration poses a significant barrier for startups and individuals. My mission is to develop an open-source alternative that empowers startups and individuals to conduct KYC (Know Your Customer) processes easily and affordably.

By creating an open-source solution, I aim to democratize access to ID verification tools, making them accessible to a broader range of businesses and individuals. This alternative will provide a cost-effective option for conducting KYC processes, removing the financial burden associated with traditional commercial solutions.

Through this initiative, I hope to support the growth of startups and businesses by providing them with the tools they need to comply with regulatory requirements and build trust with their customers. Additionally, individuals will benefit from a more accessible and streamlined KYC process, enabling them to verify their identity securely and conveniently.

The Solitary Journey

Working alone, I embarked on the journey of developing GIM – an open-source General Identity Management application. It was a daunting task, but I was driven by a deep-seated desire to make a positive impact on the world.

The decision to tackle identity verification on my own was fueled by a sense of personal responsibility. I saw the need for a better, more efficient solution, and I believed that I could be the one to provide it.

Building GIM single-handedly was no easy feat. I encountered countless obstacles along the way – from technical hurdles to moments of self-doubt. But with perseverance and determination, I pushed through, one step at a time.

Despite the challenges, there was a sense of joy and fulfillment in bringing GIM to life. Every line of code written, every feature implemented, was a testament to my dedication and passion for the project.

As I reflect on the journey behind GIM, I am filled with a sense of pride and accomplishment. Though I may have worked alone, the impact of GIM has the potential to reach far and wide, making a difference in the lives of countless individuals and businesses.

What it does

At its core, GIM is a powerful tool that simplifies and streamlines the identity verification process. By harnessing the latest advancements in artificial intelligence and data processing, GIM ensures accuracy, efficiency, and security in every verification.

GIM offers a comprehensive suite of features, each meticulously crafted to meet the diverse needs of users:

  1. AI-Powered Data Processing: GIM utilizes cutting-edge AI algorithms to process data from various sources, including facial recognition scans, ID card images, and personal information inputs. This advanced technology ensures precise and reliable identity verification.

  2. Scoring System: Leveraging AI-driven data matching and comparison techniques, GIM generates verification scores for each identity. This scoring system categorizes results into "Excellent" and "Average" categories based on the level of data consistency, providing valuable insights for further review.

  3. Comprehensive Data Collection: GIM facilitates the collection and secure storage of personal information, facial biometrics, government-issued ID card images, and digital signatures. This holistic approach to data collection ensures a thorough and accurate verification process.

  4. Advanced Verification: By integrating Convex for database management, GIM ensures secure storage and retrieval of sensitive user information, enhancing the integrity of Know Your Customer (KYC) processes. This advanced verification system offers heightened security measures, protecting against fraudulent activities and unauthorized access.

How we built it

The development of GIM involved a meticulous process, characterized by commitment to excellence. Key components of the development process include:

1. Architectural Design:

At the heart of GIM lies a robust architectural framework designed to optimize performance and scalability. The backend logic is powered by PHP and Convex, providing a solid foundation for data processing and business logic. For frontend interactions, a blend of TypeScript and Few was employed to create dynamic and responsive user interfaces. Convex was chosen for database management, ensuring secure storage and seamless integration with the rest of the application.

Most of the examples on the Convex documentations are on frameworks such as nodeJs, Python and others, however that did not limit me because I can literally do everything using PHP.

I first created a PHP SDK for Convex using the HTTP API Convex Docs for interacting with query, actions and mutations;

namespace Manomite\Services\Convex;
/**
 * Convex PHP SDK Implementations
 *
 * @link https://convex.dev/ 
 * @version 1.0.0
 */
use \Curl\Curl;

class Convex {
    protected $transport;
    protected $endpoint;

    public function __construct($token, $url){

        $this->endpoint = strtolower("{$url}/api");
        $this->transport = new Curl();
        $this->transport->setHeader('Authorization', "Convex {$token}");
        $this->transport->setHeader('Content-Type', 'application/json');
    }

    private function post($path, array $data){
        try {
            return $this->response($this->transport->post($this->endpoint.$path, json_encode($data)));
        } catch(\Throwable $e) {
            return $this->error($e->getMessage());
        }
    }

    private function get($path, array $data){
        try {
            return $this->response($this->transport->get($this->endpoint.$path, $data));
        } catch(\Throwable $e) {
            return $this->error($e->getMessage());
        }
    }

    public function mutation(string $path, object $args, string $format = 'json' ){ 
        $response = $this->post('/mutation', [
            'path' => $path,
            'args' => $args,
            'format' => $format,
        ]);
        return $response;
    }

    public function action(string $path, object $args, string $format = 'json' ){ 
        $response = $this->post('/action', [
            'path' => $path,
            'args' => $args,
            'format' => $format,
        ]);
        return $response;
    }

    public function query(string $path, object $args, string $format = 'json' ){ 
        $response = $this->post('/query', [
            'path' => $path,
            'args' => $args,
            'format' => $format,
        ]);
        return $response;
    }

    private function error($message){
        throw new \Exception($message);
    }

    private function response($response){
        return json_decode(json_encode($response), true);
    }
}

2. Implementation of Authentication and Registration: One of the foundational components of GIM is its authentication and registration system. Leveraging the power of PHP, a dedicated Auth class that talks to Convex was created to handle user authentication and registration processes. From validating user credentials to generating secure session tokens, every aspect of the authentication system was meticulously designed to ensure robust security and user privacy.

namespace Manomite\Mouth;
use \Manomite\{
    Protect\Secret,
    Protect\PostFilter as Provider,
    Engine\DateHelper,
    Route\Route,
    Engine\Fingerprint,
    Services\Convex\Convex
};
if (session_status() === PHP_SESSION_NONE) {
    session_start();
}

class Auth
{
    private $sec;
    private $date;
    private $route;
    private $turnel;
    private $filter;

    public function __construct()
    {
        $this->sec = new Provider;
        $this->date = new DateHelper;
        $this->route = new Route;
    }

    //This function always checks if a user still has an active session
    public function loggedin()
    {
        if (isset($_SESSION[LOGIN_SESSION]) and !empty($_SESSION[LOGIN_SESSION])) {

            $session =  $this->sec->strip($_SESSION[LOGIN_SESSION]);
            //Interacting with Convex SDK here by passing the convex token and url stored in config.
            $convex = new Convex(CONFIG->get('convex_token'), CONFIG->get('convex_url'));
            $obj = new \stdClass();
            $obj->sessionId = $session;
            //Check if user has active session
            $response = $convex->mutation('login:isLoggedIn', $obj);
            if (isset($response['status']) and isset($response['value'])) {
                if ($response['status'] === 'success' and $response['value'] !== false) {
                    //Get user informations
                    $responseData = $convex->mutation('user:getUserData', $obj);
                    if (isset($responseData['status']) and isset($responseData['value'])) {
                        $userData = $responseData['value'];
                        return array("status" => true, "userData" => $userData, "session" => $session);
                    }
                }
            }
        } else {
            session_destroy();
            unset($_SESSION[LOGIN_SESSION]);
            unset($_SESSION['request_generator']);
            unset($_SESSION['payload_date']);
        }
    }

    //The login function that the frontend sends login data to
    public function login($email, $password, array $device)
    {
        //Include rate limit to stop brute-force attacks
        $rateLimiter = new \Manomite\Engine\RateLimit;
        $status = $rateLimiter->limit();
        if ($status === true) {

            $email = $this->sec->strip($email);
            $password = $this->sec->strip($password);
            $device = $this->sec->filterArray($device);

            $fingerprint = $device['fingerprint'];
            $browser = $device['browser'];
            $os = $device['os'];
            $ip = $device['ip'];
            $dev = $device['device'];

             //Interacting with Convex SDK here by passing the convex token and url stored in config.
            $convex = new Convex(CONFIG->get('convex_token'), CONFIG->get('convex_url'));
            $obj = new \stdClass();
            $obj->email = $email;
            $obj->password = $password;
            $obj->fingerprint = $fingerprint;
            $obj->browser = $browser;
            $obj->os = $os;
            $obj->ip = $ip;
            $obj->device = $dev;

            //Send data to the convex login file in typescript
            $response = $convex->mutation('login:login', $obj);
            if (isset($response['status']) and isset($response['value'])) {
                if ($response['status'] === 'success' and $response['value'] !== false) {
                    //Always regenerate ID for new session
                    session_regenerate_id();
                    //set the session from here for high performance load
                    if (session_status() === PHP_SESSION_NONE) {
                        session_start();
                    }
                    $_SESSION[LOGIN_SESSION] = $response['value']['token'];
                    return array("status" => true, "user" => $this->sec->strip($response['value']['userData']['authToken']), "security" => false, "failed" => false);
                } else {
                    return array("status" => false, "failed" => false);
                }
            } else {
                if (intval($response['value']['userData']['status']) === 0) {
                    return array("status" => false, 'review' => true, "failed" => false);
                }
                return array("status" => false, 'locked' => true, "failed" => false);
            }
        }
        return array("status" => false, "failed" => false);
    }

    public function destroy_session()
    {
        unset($_SESSION[LOGIN_SESSION]);
        unset($_SESSION['request_generator']);
        unset($_SESSION['payload_date']);
        @session_destroy();
    }

    public function destroy_session_security($user)
    {
        (new Secret())->verify_session_setter('manomite_login_pass', $user, 300, true);
        (new Secret())->verify_session_setter('security_type', $user, 300, true);
    }

   //Register function that the frontend sends data to
    public function register($name, $email, $country, $pass, array $identity)
    {
        //Filter inputs
        $name = $this->sec->strip($name);
        $email = $this->sec->strip($email);
        $country = $this->sec->strip($country);
        $pass = $this->sec->strip($pass);
        $device   = $this->sec->filterArray($identity);

        //Send Details to model
        $convex = new Convex(CONFIG->get('convex_token'), CONFIG->get('convex_url'));
        $obj = new \stdClass();
        $obj->name = $name;
        $obj->email = $email;
        $obj->country = $country;
        $obj->fingerprint = $device['fingerprint'];
        $obj->password = $pass;
        $response = $convex->mutation('login:register', $obj);
        if (isset($response['status']) and isset($response['value'])) {
            if ($response['status'] === 'success' and $response['value'] !== false) {
                if(isset($response['value']['authToken']) and !empty($response['value']['authToken']) and isset($response['value']['code']) and !empty($response['value']['code'])){
                    return array( 'status' => true, 'key' => $response['value']['authToken'], 'code' => $response['value']['code'], 'userDoc' => $response['value']['userDocId'], 'veriDoc' => $response['value']['verifyId']);
                } else {
                    //Delete the registered data so the user can retry.
                    if(isset($response['value']['userDocId']) and !empty($response['value']['userDocId'])){
                        $obj = new \stdClass();
                        $obj->id = $response['value']['userDocId'];
                        $convex->mutation('login:deleteFromAuthUser', $obj);
                    }

                    if(isset($response['value']['verifyId']) and !empty($response['value']['verifyId'])){
                        $obj = new \stdClass();
                        $obj->id = $response['value']['verifyId'];
                        $convex->mutation('login:deleteFromVerifications', $obj);
                    }

                    return array( 'status' => false, 'error' => 'Token is null. Please try again later.');
                }
            }
        }
        $error = isset($response['errorMessage']) ? $response['errorMessage'] : (isset($response['errorData']) ? $response['errorData'] : LANG->get('TECHNICAL_ERROR'));
        return array( 'status' => false, 'error' => $error);
    }

}

3. Integration with Convex for Data Management: Central to the operation of GIM is its seamless integration with Convex for data management. By utilizing Convex powerful functions, database and storage capabilities, GIM ensures secure storage and retrieval of sensitive user information. From user profiles to authentication tokens, all data is managed with the utmost care and security, providing users with peace of mind and confidence in the platform.


import { mutation, query } from "./_generated/server";
import { v } from "convex/values";
import * as bcrypt from "bcryptjs";
import * as loginFunctions from './LoginFunctions';

//Login function
export const login = mutation({
  args: { email: v.string(), password: v.string(), fingerprint: v.string(), browser: v.string(), os: v.string(), ip: v.string(), device: v.string() },
  handler: async (ctx, args) => {
    if (!args.email && !args.password && !args.fingerprint) throw new Error('Invalid authorization values')

    const auth = await ctx.db
      .query('auth_user')
      .filter((q) => q.eq(q.field("email"), args.email))
      .unique();
    if (auth !== null) {
      //Check password match
      const isHash = bcrypt.compareSync(args.password, auth['password']);
      if (isHash) {
        //Check if user not blocked
        if (auth['status'] !== 'active') {
          return 'Sorry your account is not active.';
        }
        //create session
        const token = crypto.randomUUID();
        const now = new Date();
        now.setMinutes(now.getMinutes() + 30); // Session will only be available for 30 minutes
        const expire = parseInt(Math.floor(now.getTime() / 1000).toString()); //expires in 30 minutes time
        const timestamp = parseInt(Math.round(+new Date() / 1000).toString());//current time

        await ctx.db.insert("session", { stoken: token, authToken: auth['authToken'], fingerprint: args.fingerprint, browser: args.browser, os: args.os, ip: args.ip, device: args.device, expire: expire, dateUpdated: timestamp });
        return { token: token, userData: auth };
      }

    }
    return false;
  },
})

//register function
export const register = mutation({
  args: { name: v.string(), email: v.string(), country: v.string(), fingerprint: v.string(), password: v.string() },
  handler: async (ctx, args) => {
    try {
      //Check if this email already used
      const emailExists = await ctx.db
        .query('auth_user')
        .filter((q) => q.eq(q.field("email"), args.email))
        .unique();

      if (emailExists !== null) {
         return 'Sorry this email already exists';
      }

      //Hash the password - Never store plain passwords
      const salt = bcrypt.genSaltSync(10);
      const hash = bcrypt.hashSync(args.password, salt);

      //Get current time
      const timestamp = parseInt(Math.round(+new Date() / 1000).toString());

      const token = crypto.randomUUID(); //Generates safe ID
      const response = await ctx.db.insert("auth_user", { authToken: token, name: args.name, email: args.email, country: args.country, fingerprint: args.fingerprint, password: hash, salt: salt, status: 'pending', dateAdded: timestamp, dateUpdated: timestamp });

      if (response !== null) {
        const vtoken = crypto.randomUUID();
        //generate email verification code and return the code to the application for email sending.
        const vres = await loginFunctions.verifications(ctx, vtoken, token);
        return { authToken: token, code: vres.code, userDocId: response, verifyId: vres.docId };
      }
      return false;
    } catch (e) {
      return e;
    }
  },
})

//isLoggedIn function that  loggedIn function calls from the PHP Auth Class
export const isLoggedIn = mutation({
  args: { sessionId: v.string() },
  handler: async (ctx, args) => {
    if (!args.sessionId) return false;
    //Query this function if you want to check if user is loggedIn
    const response = await loginFunctions.isLoggedIn(ctx, args.sessionId);
    return response;
  },
});

//Delete data from verifications document
export const deleteFromVerifications = mutation({
  args: { id: v.id("verifications") },
  handler: async (ctx, args) => {
    await ctx.db.delete(args.id);
  }
});

//Function t activate account once email has been verified
export const activateUserAccount = mutation({
  args: { id: v.id("auth_user") },
  handler: async (ctx, args) => {
    const timestamp = parseInt(Math.round(+new Date() / 1000).toString());
    return await ctx.db.patch(args.id, { status: 'active', dateUpdated: timestamp });
  }
});

//Logout function to destroy session
export const sessionDestroy = mutation({
  args: { sessionId: v.string() },
  handler: async (ctx, args) => {
    if (!args.sessionId) return false;
    const timestamp = parseInt(Math.round(+new Date() / 1000).toString());
    const checkLoggedIn = await ctx.db
      .query('session')
      .filter((q) =>
        q.and(q.eq(q.field("stoken"), args.sessionId), q.gt(q.field("expire"), timestamp))
      )
      .unique();
    if (checkLoggedIn !== null) {
      await ctx.db.delete(checkLoggedIn._id);
    }
  }
});

4. Development of Dashboard and User Interface: In addition to backend functionality, GIM also features a sleek and intuitive dashboard interface. Designed with usability and aesthetics in mind, the dashboard provides users with easy access to essential features and functionalities. From viewing identity verification scores to managing personal information, the dashboard offers a seamless and intuitive user experience.

In the dashboard, there are five essential processes that users undergo:

  1. Agreement Section: Before entering any personal information, users must agree to the terms and conditions outlined in this section. It serves as a prerequisite for accessing the platform.

  2. Personal Information Collection: In this area, users input their personal details into a form and submit them. Each submission communicates with Convex for storage and retrieval, ensuring secure handling of sensitive data.

  3. Facial Recognition Data Collection: Users are prompted to capture facial recognition data using their device's camera. Clear instructions guide users on positioning their face correctly. Various facial data points and landmarks are extracted and stored on Convex, including picture, age, sex, and other relevant metrics.

  4. Government Issued ID Card Submission: Users are required to upload images of their government-issued ID cards. These images are securely stored on Convex storage, and relevant data from the ID cards, such as personal information, are extracted and stored for further processing and verification.

  5. Digital Signature Collection: In the final stage, users provide their digital signatures on the screen. These signatures are captured digitally and stored securely on Convex storage, ensuring authenticity and integrity.

After completing these stages, the collected information undergoes processing and analysis. The results are then sent to the application hoster via email or webhook URL, depending on their configured preferences for data collection and dissemination. This ensures that the hoster receives accurate and timely updates on user submissions and revisions, facilitating efficient management and decision-making processes.

Challenges we ran into

Firstly, am not familiar with using React and other frontend frameworks that convex provided. Most of the examples provided were on this frameworks so I had to spend many times figuring out how to use it with pure frontend languages.

Secondly, the backend language examples also provided are limited but I thank God that there was an HTTP API which assisted me in developing an SDK for the language am familiar with which is PHP.

Thirdly, some node packages were not working with convex functions. I wanted to use jsonwebtoken node package for session generation and verification but I could not. I tried all possible debugging combinations provided from the docs but none works. I used "use node" at the head of the script as said by the convex docs but still didn't work so I had to just use crypto.randomUUID() to generate a token for the login session with expiry time of 30 minutes. Then I created the session table on convex to store this information and query from using functions.

Accomplishments that we're proud of

I'm incredibly proud of developing GIM. It's been an exhilarating journey transforming my vision into reality. GIM's seamless integration with Convex, coupled with its innovative features like facial recognition and AI-powered data processing, make it a game-changer in the realm of identity management. The positive feedback and impact GIM is already generating are a testament to the hard work and dedication poured into its development. I'm thrilled to see how GIM is simplifying identity verification processes and making a tangible difference in users' lives."

What we learned

Throughout the development of GIM, I've learned invaluable lessons that have shaped not only the application itself but also my approach to software development. I've gained a deep understanding of the intricacies of backend architecture, particularly in leveraging technologies like PHP, TypeScript, and Convex for seamless data management. Additionally, I've honed my skills in integrating complex functionalities such as facial recognition and digital signatures into a cohesive platform. Moreover, I've learned the importance of user-centric design, ensuring that GIM offers an intuitive and engaging experience for its users. Also have started pushing my interest to Typescript. Convex has opened my eyes to it and it seems am already falling in love with Typescript. Thanks to Convex for that.

Overall, the journey of developing GIM has been a transformative learning experience, and I'm excited to apply these lessons to future projects.

What's next for GIM - General Identity Management

What's next for GIM is an exciting journey of continual improvement and expansion. Our focus is on breaking new ground in the field of identity management by pushing the boundaries of innovation and technology. Moving forward, we're committed to enhancing GIM's capabilities by incorporating advanced machine learning algorithms for even more accurate and efficient identity verification. We're also exploring new avenues for integration, such as biometric authentication and blockchain technology, to further enhance security and reliability. We're planning to introduce NFC chip scanning capabilities for ID verification. This cutting-edge technology will enable users to verify their identity using NFC-enabled devices, adding an extra layer of security and convenience.

Furthermore, we're exploring the possibility of routing ID cards to different country portals based on the issuing country. This innovative approach will enable us to leverage the respective verification processes of different countries, ensuring comprehensive and accurate identity verification for users regardless of their location. Additionally, we're dedicated to expanding GIM's reach by forging partnerships with industry leaders and exploring new markets globally.

Our goal is to empower businesses and individuals alike with a cutting-edge solution that revolutionizes how identity is managed and verified. With GIM, the possibilities are endless, and we're excited to continue breaking barriers and shaping the future of identity management.

Share this project:

Updates