Musar על ברכות 15:5

I have, therefore, undertaken to set forth the importance of congregational prayer. But first, let us explain the great obligation thereof. The formulation of our sages of blessed memory — "four mils to prayer" is well known. Many of the Rishonim explain this as meaning that if one is walking on the road and he knows that four mils further on he will find a minyan, he must wait [to pray] until he comes there. As far as going back, if he knows that he will find a minyan within the distance of a mil, he must go back. From this we can understand that certainly, if he can find a minyan in the city, even if it is almost a mil away, he must go there. And our sages of blessed memory have already said (Berachoth 8a): "Whoever has a synagogue in his city and does not frequent it for prayer is called 'a bad neighbor.'"