Working with small diameter drill bits is sometimes difficult and frustrating. Especially when they are mounted in a drill press. The small bits tend to deflect and walk before drilling into the material resulting in holes being misaligned and / or misplaced. To alleviate this, consider using centre drills to either drill the complete hole in thin material or to provide a starting hole. The benefit of centre drills is they are shorter and stiffer resulting in less deflection. The small diameter drill is machined in the end. The extra material in the centre provides greater stiffness resulting in much less deflection and minimal drill bit walking.
I've attached a picture of some centre drills which I have purchased, along with a chart of the standard dimensions which are available. They are readily available on the Internet and of course Busy Bee for those of you in Canada.