How can I keep my 4 year old from hitting others?

My son, who is usually a sweet and well behaved boy, has recently been having a problem with hitting others. He only does this when he gets together with all of the neighbor kids, who he's friends with. A couple of the other kids have a problem with this too. I am friends with the other moms and we are all so frustrated!!! We have talked to them, given time outs, taken things away, grounded them, etc. Currently I am using a chart system. When my son behaves, he gets a smiley face but when he misbehaves he gets a sad face. If he gets 3 sad faces in a day, he loses his dessert after supper. It has been working really well up until yesterday when him and his friend were hitting the bigger boys with baseball bats and today him and his friend were hitting each other with sticks. His friend is a girl, so they aren't just being "boys". What else can I try??