No. Distribution of prayer points comes before the distribution of permanent victory points, so in this case, the player sacrificing his last unit for five prayer points would lose control of the temple. This would cause him to lose both the temporary victory point and the opportunity to claim the permanent victory point.