Cross posted from https://github.com/wesc/pathfinder-0/blob/main/constructive-criticism.md

MapsMap Hackathon Constructive Criticism

I'll break this criticism into two broad categories: of the prize funding mechanism, and of the technology tree idea itself.

Funding Mechanism

I have at this point participated in countless hackathons. I've slept with the dinosaurs at the American Museum of Natural History, won some bounties (Bluesky Satellite, runner up rebuttal to a Balaji 1729 essay), have a pending grant on improving IPFS use (Interplanetary Library Catalog), and even received seed funding from a hackathon idea. Point is, I'm no stranger to how these things work.

As someone who has been very interested in retroactive public goods funding, open source funding, and prize mechanisms, I thought it would be a good idea to dive into MapsMap to get a taste of what it would be like working towards a sizable prize ($30k at the time of announcement, later upped to $60k).

$60k is very generous. That said, it's in the unfortunate uncanny valley of seriousness.

• it's great for an ad-hoc team

• it's too small for a professional team if there's a risk they won't win

• it's too small to seek funding like teams do with the X-Prize

• it's too big to encourage small experiments

A winning project should require 100+ hours of work, and if there is only one winner then the majority of projects, including many that have sunk in 100 hours, net out zero. What's more, the likelihood a single developer wins is low, especially one who is employed full time and has family obligations (NB: that's me!).

If there are ten competitive entries (Devpost has over 300 registered participants as of this writing) and you need at least 100 hours of work, then that works out to an expected value of $60 / hour, and an upper end of $600 / hour. A professional developer may instead prefer to guarantee a dollar rate of $200.

A $60k prize also signals that participants should consider reorganizing their life. When the deadline was extended by two weeks, I was on the fence. I welcomed the extra time, but I had also already pushed myself for a March 1st deadline. A few participants angrily posted in Discord.

The vast majority of the hours across all submissions going into the competition is either developing overlapping ideas, or will eventually be thrown away.

In short, this competition structure is optimizing for ad hoc teams and produces a large amount of wasted effort, especially if there's little variation on the desired result: to have a system that edits a knowledge graph.

If the point in the MapsMap Hackathon is to encourage creativity, experimentation, and sprout ideas the organizers may not have considered, then there should be many small prizes, and one large grand prize.

If the point in the MapsMap Hackathon is to develop a well formed idea, then MapsMap should just have just hire a professional development team with the $60k.

Furthermore, it's generally the case that Hackathons are a "young person's game." If the MapsMap team wants experienced participants, providing a reasonable space with respect to time commitments is essential. An intellectually diverse crowd tends to generate better ideas. Instead, the structure of MapsMap encourages competition, not cooperation.

Technology Trees

It's a neat idea. I absolutely love it. However, the major problem is that the inspiration comes from Civ, a game that has the benefit of historical 20-20. As Kierkegaard says, life can only be understood looking backwards, but it must be lived forwards. I suspect that we can not come up with forward looking maps with a good degree of accuracy and specificity.

I think the best pro-MapsMap argument is the DARPA funding system, which relies on program managers with deep knowledge of academia and industry who can fund technological progress bigger than any particular research group or company. These plans, though, are unlikely nodes and edges. Rather, they're imaginative bluesky roadmaps tethered to existing capabilities and reasoning from first principles.

A popular example within the Foresight community might be Feynman's Plenty of Room at the Bottom, where he reasons his way to nanotechnology and presents blocking breakthrough challenges. There is a kind of map here, but it's a very sparsely populated. In fact, despite it's ambiguity, Feynman's argument is communicated quite clearly in a linearly spoken speech. Where are the nodes and edges? Rather, Plenty of Room is more like JFK pointing to the moon and saying "in principle we should be able to get there."

A major -- perhaps the major benefit to having a map is legibility. There is no compendium of DARPA roadmaps. There should be! The ability to open up a page and point to a roadmap enables funders and pioneers to be convinced that a thing might be possible.

Along with legibility also comes a Schelling point that pioneers can focus towards, like dogs to fire hydrants. Those that embrace focal points, like animals in nature, are more likely to cooperate and survive. But I need to ask -- does this necessarily mean these maps are describing the best solutions we have? Is it the case that the road to a deep solution can necessarily be written down, or do we risk derailing progress with overconfident maps?

More often than not, scientific breakthroughs arrive from unpredictable angles. mRNA vaccines were not on a well accepted map towards pandemic preparedness. The economic and theoretical underpinnings of cryptocurrency are decades old, but were not a part of the web's roadmap until recently (and some argue are still not on the map). The precision of a good mapping tool needs to be balanced by epistemic humility, and a structure that accepts exploration of unseeable territory.

Built With

• markdown