Pinecone doesn't work with all the values that are generated by some transformer models. This will take time to work through as well, so I ditched huggingface transformer models and used openai as they work out of the box and can insert to pinecone without issues. (they incurr cost but speed is important)