The Earth is not a closed system but since it is spinning in a vacuum, the only way for it to significantly lose rotational energy is through radiation. Rotation of stable matter does not produce very much radiation at all.
Earth is slowing transferring some of the angular momentum of its rotation to the Moon because of tidal friction. This is causing the Earth's rotation so slow down and the Moon to speed up in its orbit, which causes it to move further away. This is a very slow process, though.
Since there's a much smaller component of tides caused by the Sun, this is probably affecting earth's orbit a little, also.
I suspect that interaction of the Earth's magnetic field with the solar wind and other magnetic fields and charged particles probably drags the Earth's rotation down a little, also, but don't know for sure or by how much. Maybe someone can look this up if they think it's an issue.
Essentially, though, earth and moon are a closed system for purposes of conservation of momentum since that's the biggest factor by far. For gross purposes, I think the Earth alone can be treated a closed system as far as conservation of momentum is concerned. Its rotation fluctuates very slightly and unpredictably due to motion of materials within and on it; otherwise it's steady day to day; the next factor is tidal braking already mentioned.
The biosphere, on the other hand, is powered by the Sun. Earth itself provides only a tiny amount of the energy required to sustain life here. All the rest comes from outside as an enormous amount of radiant energy from the Sun, so there's no way you can say life on earth is a closed system.
This may be where legion was getting confused where he brought the "not a closed system" and evolution thing into the discussion. Angular momentum and solar irradiation are totally different things.