-
Final Project Poster
Vesuvius Challenge Grand Prize
Jason Tapper, Dan Hu, Eitan Zemel, Jake Stifelman
*Introduction: * The Vesuvius Challenge calls on archeologists and computer scientists to uncover, reveal, and read ancient papyrus scrolls that were buried in carbon ash in 79 CE in Herculaneum (a town near Pompeii). The problem is that the ink used on these scrolls was also carbon based, so the scans of the scrolls are incomprehensible to the human eye. The part of the paper that we are implementing succeeds at classifying each pixel as ink or not ink for a pretty decent accuracy. This is one of the only papers that has tackled the problem of deciphering the ink on the papyrus scrolls.
*Related Work: * link link link link link link link link link younader blogpost
The main process outlined in the EduceLab paper is unraveling the scrolls by training a model to recognize the outline of a given segment of the papyrus from a cross section of the 3d scan, and then taking that and constituting a segment from the bits that the imaging has identified. From there, they train the model on a set of fragments that have broken off from unraveled scrolls, giving these fragments a label of Greek text that matches the fragment, and then training the model on the fragments. The architecture is borrowed from that which is specified in another paper, which is a 3D CNN.
*Data: * link link The fragments are each a couple gigabytes. Theoretically the fragments come already unrolled and with publicly available labels and so there shouldn’t be that much in the way of preprocessing.
*Methodology: * We are going to try using CNNs, Res-net, Diffusion models, and we are looking into times-formers (see link) Our data is of 4 scroll segments that we will separate into 8 segments with their respective “ground truth” ink labels. These will train our models and then we will pass in unlabeled segments to get the results. Since we are working to implement results from a large challenge with multiple different approaches, we anticipate one challenge will be figuring out how to structure our model and comb through all of the prior results. We also expect that it will be very difficult to work with the data, both the labeled and unlabeled.
*Metrics: * We want to have legible papyri at the end with readable and recognizable sequences of letters. The original Vesuvius challenge uses a metric of how much of the characters are recoverable as a percentage as a success metric, which we will also use. We will use manual review and visual confirmation to derive this percentage. Our base goal is to implement an ink detection such that we can get at least one character out of the raw data. Our target goal is to reach 30% accuracy/readable characters on our extracted papyrus. Our stretch goal is to get comparable results to the original solutions 85% and also optimize it for speed and reducing human labor (less than the original’s 48 hours of human labor + training time for the fully implemented model).
*Ethics: * Why is Deep Learning a good approach to this problem? Deep learning is suited to this problem because the scrolls we are working with have the text hidden in data that is imperceptible to the human eye but could be detected by a trained model
What is your dataset? Are there any concerns about how it was collected, or labeled? Is it representative? What kind of underlying historical or societal biases might it contain? Our training dataset consists of 4 incredibly high-res labeled fragments. Although each fragment contains a lot of information, there is still a concern that the fragments are not sufficiently representative of the larger corpus.
*Division of labor: * Dan: reconstituting fragment label dataset Jason: designing model architecture for ink detection Eitan: designing model architecture for ink detection Jake: plans to be fluent in Homeric Greek by the end of the project to be our living loss function
Checkin 3 Reflection: https://docs.google.com/document/d/1OsgPv_1PIa6iCbNmhNnCLU6miSi76MjQGWrMFaWuSPY/edit?usp=sharing
Log in or sign up for Devpost to join the conversation.