A company can't legally pay money to shareholders if it will be unable to pay its creditors in full.
For example, here is the British Columbia statute:
Redemption prohibited when insolvent
79 (1) A company must not make a payment or provide any other consideration to redeem any of its shares if there are reasonable grounds for believing that
(a) the company is insolvent, or
(b) making the payment or providing the consideration would render the company insolvent.
(2) On the application of a director of a company, the court may declare whether a redemption of shares by the company would contravene subsection (1).
(3) A redemption of shares is not invalid merely because it is in contravention of subsection (1).