"Sweetheart" Whats the Problem?

On your part you may be calling your colleagues in a friendly manner.But we cannot expect everbody to be broad minded person like you.Every body has their own way. S some may like that and some not.So you can use that word to the person who likes it and it would be better if you dont use to those who dont like it.