For some reason, the "cam" must not be tightening up against the bit adapter. With no bit and the bolt tightened, use a mirror to look into the hole. Does the cam protrude into the hole? Maybe take out the bolt and cam and have a look for damage. It is a simple mechanism, there could be a foreign object behind the cam?