מדרש על סוכה 110:23

Whence is it derived that all the priestly watches share equally in the (devoted portions of the) offerings of the festival? From (6) "then he shall come in all the desire of his soul to the place that the L-rd shall choose." I might think that the same holds true for all the days of the year; it is, therefore, written (6) "from one of your gates" — when all of Israel are assembled within one gate, i.e., during the three festivals.