Inspiration

Pokémon Go is a popular augmented reality mobile game (my friend Ursula's loves Pikachu). However it does not currently exist on the blockchain - where there is integration between real world, metaverse, and cloud. There are three main problems why:

  1. Computing infrastructure for an immersive Pokemon Go metaverse with many virtual worlds will be expensive and time consuming. Predictions using data on the edge as streaming to the cloud adds significant latency.
  2. Lack of portable computations and AI models infrastructure connecting agent and data with verifiable proofs.
  3. Pokemon Go Metaverse would require tested computations that needs to run on multiple devices (Pokemon Go is mobile).

What it does

PokeDEX metaverse uses AI and Blockchain technology to create generative pokemon simulations with a fully open decentralized convergence of virtually and physical reality in the PokeDEX metaverse. Landing page with domain name can be found on www.PokemonDEX.tech:

PokeDEX "Gotta generate them all!" features:

  • Generative Adversarial Network (GANs)
  • RUNE Edge AI multiple devices computations
  • Rare NFT Art Collection
  • $PKM PokeDEX Marketplace

How we built it

  • Ethereum's Layer1 Infrastructure with ERC20 Tokens for users ERC20 $PKM with Solidity smart contracts self-sovereign financial system. PokeDEX Zora erc-721 NFT Assets Marketplace to buy, sell, and trade NFTs off-chain metadata, assets stored on NFT Storage and IPFS

  • Generative adversarial network (GAN) with Rune Computer Vision for Pokemon GO on Pokemon dataset. Model trained on dataset of 800 Pokemon images from Kaggle represented as a N-dimensional input tensor. Generator creates a random vector and converts it to final realistic looking Pokemon. Unsupervised learning with a GAN model to create new kinds of rare with shapes and colors for Pokemon-like characters. Architecture implemented with DCGAN PyTorch framework with generative learning and personalizing with data on-edge and on-chain.

  • HOT-G's edge ML Rune MobileNet computer vision model with Google Cloud connects hardware, software, and AI, with verified proofs on the performed actions via edge oracles. This makes PokeDEX portable, includes our collectible NFTs, virtual identities, payments, and helps with interoperable gaming standards. Google Cloud project name was Lucy Cloud PyTorch, project number 886953643480, and project ID was lucy-cloud-pytorch.

Challenges we ran into

  • Due to the time limitations of a 36 hour hackathon, struggled with making it end-to-end with Google Cloud, RUNE ML, and the DCGAN model.
  • Had to add eyes to the generated Pokemons because the detail was lost due to the small 800 sample training size. If there was more time, would of added more Pokemon samples to the size of my training data

Accomplishments that we're proud of

  • The generative art featuring Pokemon-based game assets turned out better than expected
  • As a Pokemon fan - getting anything to work was an accomplishment

What we learned

  • Different kinds of GAN machine learning models like WGAN v.s. DCGAN
  • Blockchain infrastructure connecting agent and data with verifiable proofs on the Ethereum network
  • More about IOT and RUNE Edge AI running computations on multiple devices
  • Metaverse And The Edge Tinyverse intersects with Metaverse https://tinyverse.substack.com/p/metaverse-and-the-edge
  • Generative Adversarial Networks Paper by Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio https://arxiv.org/abs/1406.2661

What's next for PokeDEX

  • Better GAN model for the rare generated Pokemons
  • End-to-End on-chain connection between mobile hardware, software, AI, and Figma front-end mockups with verified proofs on the performed actions via Rune's edge oracles
  • Monetization of the NFT PokeDEX marketplace
  • Deploy on landing page with domain name www.PokemonDEX.tech

Discord name: ! Lowcy#5208

Built With

Share this project:

Updates