Love these. Keep up the good work.

To be able to drill straight into your spine pieces (I assume that's where the problem is) you're going to have to clamp it to a jig in your drill press. Use sharp drill bits. All your stock should probably be dead square as well.
If you still aren't drilling straight holes, then you may need to set the sweep on the press. It's easily done with a dial indicator and an "L" shaped piece of cold rolled steel mounted in the chuck. You will need to flatten the end and drill a hole to mount the dial indicator. Then adjust your table so that as the dial moves in the circle, there's no deflection of the needle. You can get very close to a 90 degree angle this way.