• 

  Poster Not a GAN DL Day

Title: Not a GAN

Link to Final Reflection/Writeup
Link to second check-in

Who:

• Joshua Kruzan (jkruzan)
• Helen Huang (hhuang65)
• Pavani Nerella(pnerella)

Introduction

Our goal is to train different models— a CNN and a (vision) transformer–- on the binary classification task of detecting signature forgeries. We will then compare and contrast the models and perform a comprehensive analysis on our results.

Related Work

Convolution Neural Networks (CNN) are a popular framework to use for image based recognition and quite beneficial in applications like signature detection/ verification. This method was proposed in the 2018 paper, Handwritten Signature Recognition: A Convolutional Neural Network Approach, which detects offline signatures using a CNN, which extracts features from genuine and forged signatures.

Coming to transformers, a recent paper published in August 2022, Offline Signature Verification with Transformers, proposed using a model based on a transformer framework. Signatures are encoded into patches, and the decoder learns contrast based discriminative features. By using Transformers, the model can represent and learn global information which can enhance signature detection. This paper was built off of the Vision Transformer (ViT) proposed in the paper, An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, which extends Transformers previously used in language modeling to images.

Data

CEDAR dataset: 55 individuals contributed 24 signatures thereby creating 1,320 genuine and forged signatures. BHsig260 - Signatures in the Indian languages - Bengali and Hindi with the number indicating the no of individuals they were collected from.

Methodology

A signature verifier can be efficiently modeled by a Siamese network which consists of twin convolutional networks accepting two distinct signature images coming from the tuples that are either similar or dissimilar. The parameters between these twin networks are shared, which in turns guarantees that two extremely similar images could not possibly be mapped by their respective networks to very different locations in feature space because each network computes the same function. For our architectures, we looked at two models: Convolutional Neural Networks and Vision Transformers.

Metrics

We evaluated our models just on the accuracy of the results.

While these metrics will help to train the model, We have tried to inspect on what parts of the signature are the features picked up by using a LIME explainer.

Ethics

What broader societal issues are relevant to your chosen problem space?

Signatures play an important role in human daily lives, and serve as a supplementary characteristic to facilitate identity verification in many scenarios, e.g., credit card fraud and criminal detections. This has also led to an increase in signature forgery which can be used to defraud others’ assets and bulk properties of large organizations such as insurance companies and banks. Thus, signature verification becomes a very important security aspect. Signature recognition is fundamental to forensic analysis and verification of documents.

Division of Labor

We will equally divide labor to implement the base models and then we will split up the datasets among group members to train different instances of the model.

Built With

• python