What happened to Oliver's money?

They make it seem like he is broke and yet still seems to be able to live without a normal job. I understand he lost control of Queen Consolidated. But, he should still be extremely wealthy..Especially after Moira died.

Is this ever explained??