Well, there's fluid that's bad for the pump when it's in it, and fluid that's bad for the pump when it's NOT in it, and no other fluid is either. I think that's the worse one and that's why im suggesting if someone starts their tractor and the FEL doesn't move AT ALL until it has warmed a little they should try a thinner fluid. If someone started their car and noticed 'huh the oil pressure light didn't go out' or they put the automatic transmission in gear and it didn't move, they would instinctively NOT pass it off as 'it's just cold, it'll be fine'. With the hydraulic system pressure going to the loader there isn't a warning light, but it's the same situation. A pump that can't suck in fluid. The only reason it's even ok to have an HST trans spinning in that same situation is because in 'neutral' position it has zero displacement and isn't trying to suck any fluid anyway.
I am not a hydraulics expert but im at least halfway there after my automotive career and the main downside i can think of from thin fluids in a circuit driving cylinders is simply that pump clearances might be too large, and you'd be getting a bunch of fluid shear and putting heat into the fluid while also not being able to make peak pressure. Which is not good, but I wasn't saying 'it WILL be fine', just i'd put decent odds on it being fine. Id definitely leave it in there until the weather is warm and actually see for myself if the fluid was aerating or hot or the pump was noisy and hot, couldn't hit relief valve etc, vs just swapping it out again preemptively because i was scared. But im a mechanic. To each their own.