Alright, maybe it's not quite as simple as the Hokey Pokey when you're first starting out, but as you get used it, you'll realize that with a strong delivery foundation comes muscular coordination, speed, accuracy and muscle memory. So let's get started.
Approaching the foul line, you will see two blue lines on the lane. Start on the second one up. Hold the ball to your chest and bring your elbow in. This will ensure that your arm will not swing to the side with the weight of the ball upon release. I was taught to hit myself in the ribs with my elbow, and eventually I found that it worked. I will assume that most readers are right-handed. If you're left handed, simply reverse direction.
You'll be delivering in four forward steps. Begin with your right leg. One step up with the right leg, then the left, and once again, the right. Huge steps are not required. Take strides that you're comfortable with. As you practice, you'll learn to adjust your stride to your height. At this point, the ball should be at your chest with your right elbow pressed to your ribs.
Step forward with your left leg as you bring the ball back straight. I find that kicking my right leg out behind my left gives me the freedom I need to work with the ball. Your left foot is now your lead foot, as this is the foot that you'll be throwing off of. The weight of the ball in your right arm and a favoring of your left leg will give you the balance you need to throw the ball straight.
Here's where it gets tricky.
For all practical purposes, it may be worthwhile practicing without a ball at first. The most important part of the delivery is the direction that the toes on your left foot are pointed in. You always want them facing the pin that you're aiming for. Next time you throw a gutter ball, dont take the weight off your left foot. Instead, look down and see which way your toes were pointed. Unless you've balked on the release, they'll likely be aimed directly at the gutter.