Are Rusty Weights Bad For You