Yes, there are.
On the assumption that you are walking for cardiovascular benefits, you will walking at a pace to elevate your heart rate to the appropriate zone and maintain it there for a length of time (this also applies to interval training). This kind of walking requires the use of the arms to assist in forward motion.
Holding weights (or attaching wrist weights) when walking fast requires that the body decelerates additional load in the upper body extremities. Since walking is an activity done over an extended period of time, this may lead to injuries of the shoulder.
However, there are options: if you use the hand weights to increase your body weight, you may consider wearing a weighted vest where you can distribute the addtional weight close to your body. Another alternative is to use Nordic poles.