I'm not usually sorry I let the kids play with something in a way that society deems inappropriate. As my brother said yesterday about a play structure in a playground, "What do you mean, 'play with it properly?' It is just there, and however the kid decides to use it, that is "how" to play with it." (As I said, we grew up with this sort of attitude.)
But I do regret letting Elazar (when he was about 2) shred the styrofoam packers that came in the mail. Those were really tough to vacuum up. They had static electricity and kept dispelling... Though I guess we learned some science.