This year (in South Carolina - the bug capital of the world) I have not had a single tomato worm bother mine.
A nest of white faced hornets is growing beside my back door. Two others have also started on other parts of the house. These guys are busy carrying worms and other insects to feed their babies and they keep the area quite pest-free.
(Later note) I have walked right by them within 8 feet many times, taken photos from even closer. These guys can sting, so I've been wary with them so close to our back door. I have not waved my arms or ran past them.
There has only been one time when they've been the least bit aggressive; that was once when I inadvertantly let the storm door slam hard. The key to coexistance is to prevent vibrations and excessive or rapid movement around them.
More hornet nest photos
| These l'il gals are also helping. These semisocial
wasps build a small umbrella-shaped nest and use various pest insects to feed their young.
They are quite gentle, and highly unlikely to bother anyone unless you try to grab them,
or hammer on the soffit. Many folks are terrified of them and exterminate them, but then
wind up having to spray again for the pests in the garden.
Social wasps (yellow jackets) and hornets can be quite defensive of their nests, but these semisocial ones are not.
The rule of thumb I follow is to let these creatures be, whenever possible. I had a nest of red wasps in the door frame of my workhop a couple years ago, and I had to do them in, because they popped me every time I closed the door. But there were three other nests in the building, which I was glad to leave alone. Mud daubers and semisocial wasps are no real threat to humans. Social wasps and hornets are only a threat if their home is near to human activity. The white faced hornets pictured above are eight feet above and six feet sideways from my back door. So far they have not shown any interest in my passing back and forth. As long as they keep to their work, I will not bother them (except for photography).