Major budgetary issues.
Also to actually use them as a delivery system would require them to negate all their best features to eject/spurt/launch/drop ordnance. In other words they would have to slow down to a reasonable (translated as subsonic) speed to launch* a weapon which based on the meagre carrying capacity would undoubtedly be the size of a tactical nuke. Back in those days they didn't have the precision ordnance capability to put on such a small package. So to hit that kind of target an ICBM would do the job cheaper and far more effectively.
That said I wouldn't be surprised if the early A-12 version didn't have some capability. They did a lot of things back then just to see if it could be done. (C-130's on and off carriers, etc, atomic cannon, the Marines nuclear hand grenade, etc)
*The pilot would have to start to slowdown about halfway to the target, even then he would be guesstimating distances and times, and then opening the bay would cause a real degradation in stealth characteristics (yes they had some) and add major drag, drop a small package and hope the ballistics would get it there. Then clean up and start accelerating back to supersonic. An external mounted package would add a lot of drag and also have to withstand the heat loadings that it would encounter.