What happens to mass and energy in a closed system?

In reality closed systems do not exist. In the theoretical case of a closed system the total energy of the system would stay the same (matter and particles are considered energy to). More important: the energy level throughout the whole system will eventually be evenly spread and uniform (entropy), therefore no energy differential is left to perform work.