Let me give you with a specific exercise. A quick "thanks, but no thanks" note is a bunch of better than no reply at mostly.
Let's begin with an analogy: When you're driving an auto at 100 miles per hour, a small thing as the bumblebee hitting the windshield may result in you shed control and crash.