With a dumb charger you should be okay to charge em, as the charger just puts out the voltage needed to charge em. As the smaller gets close to charged its acceptance rate changes as the voltage of the battery goes up. Using a smart charger that changes voltage as it charges would over charge the smaller one.
Discharge if low enough draw they should be okay as well. The larger battery will be able to provide higher amperage longer, keeping its voltage higher therefore providing more power.
The issue is higher draw loads, anything over say 5c, or 1/5 of the smaller ampacity could be a problem. Meaning a 10 hr and a 15 ah battery, dont discharge more than 2a of load, or 24 watts at 12 volts.