This question seems to be a cause of major heartburn for many folks who come to the Anglo-Catholic tradition from "lower" Church traditions. Personally speaking, I have no problem at all with parishioners referring to me as "Mr." or "Rev." or "Vicar" or even "Thierry". But let’s take a look at just why priests are called "father".
To begin with, in the Bible, the concept of fatherhood is not restricted to just our earthly fathers and God. It is used to refer to people other than biological or legal fathers, and is used as a sign of respect to those with whom we have a special relationship. We see examples of this in figures such as Joseph, Job, and Eliakim, all of whom are referred to as "father" to their particular people. In fact, in Isaiah 22, God himself refers to Eliakim as the "Father of the inhabitants of Jerusalem." Many Protestants will often refer to the 23rd chapter of Matthew in which Jesus says to call no man "Father" except God, as an example of why we should not call priests "father." In doing this, however, they are taking the entire passage out of context which was meant only for those who were taking the "highest" places at feasts. This sort of interpretation would be akin to "plucking out our eye," which Jesus also commands, or literally "taking up serpents" as many sects do, an action also mentioned in scripture. If you refuse to call priests "father" then you must be ready to pluck out your eye and also take up serpents!
The point here is to stress the importance of sound biblical interpretation.
Referring to a priest as "father" is used as a sign of respect and to indicate the special spiritual relationship the priest has with his congregation. Priests, in turn, follow the apostles’ biblical example by referring to members of their congregation as "my son/my daughter" or "my child" as evident in the Books of Galatians, Phil, Tim, Peter, and John.
This is another reason why a woman can NOT be a priest. God had designed priesthood as a male institution or priesthood as an institution for male individuals only. What God had designed and instituted, let NO man change it!
Peace and blessings!
"For the vision is yet for the appointed time; It hastens toward the goal and it will not fail. Though it tarries, wait for it; For it will certainly come, it will not delay" (Habakkuk 2:3).