Musar על יומא 171:7

The summary of the matter: one who is hired out to his fellow for any kind of work, behold, all of his hours are sold to his employer for the workday as the Sages stated: "to hire oneself out is to sell oneself for the day" (Bava Metzia 56b). Whatever time he takes for his own pleasure, whatever it may be, is completely guilty of stealing. And if his employer does not forgive him, he is not forgiven. For the Sages already stated: "sins between man and his fellow are not atoned for on Yom Kippur until he has pacified his fellow"(Yoma 85b).

The fifth principle of repentance is worry. One should worry about the punishment of his sins, for there are sins where repentance holds the atonement in suspense and only afflictions cleanse away the wrong (Yoma 86a), as it is said, "For I do declare mine iniquity; I am full of care because of my sin" (Ps. 38:19). And what is the difference between sorrow and worry? Sorrow is for what has already taken place, while worry concerns the future. And a person who has sinned should always worry whether he has fallen short in the matter of repentance and has not completed the full measure of repentance. And he should worry lest his evil inclination overpower him. As our Sages, of blessed memory, said, "Do not trust in yourself until the day of your death" (Aboth 2:4).