Do you think its okay to take children to races?

I have some family members that can't believe that I take my kid to the race...and I"m like shhh its a family sport....but I get grief because of the drinking that takes place..We go to Dover in June (without the kids) and September (with the kids)...The little group we go with does this too....I was just curious what others thought about chrildren at the race track...ty