Granting access to desired callers and remaining undisturbed by unwanted callers is a problem which predates the invention of the telephone. From the dawn of civilization, people have protected the privacy of their homes using doors, fences, gates, drawbridges, doormen, secretaries, etc. The historic common law dictum that "an Englishman's home is his castle" enshrines the right of the homeowner to exclude anyone from his home. The telephone subscriber also has an absolute right to accept or reject incoming calls for any reason (or no reason at all) as he or she sees fit. But in order to do so, the telephone subscriber needs the ability to reliably determine who is actually calling. A subscriber with sufficient means could choose to have an employee answer and screen all calls. “The Phone in the Castle” is a technological framework for automating the functions of such an employee. By presenting challenges to unknown callers, “The Phone in the Castle” determines whether to connect the call to the subscriber or not. Access to a “proof-of-concept” implementation and demonstration is included with this submission. The proposed framework expands upon this simple challenge-response demonstration to plan for taking advantage of developments in the telephone network as well as countering robocallers attempts to build automated systems capable of responding correctly to the challenges.