Look the picture. There are two 74LS273s (D-Type flip-flop x 8) for a reason. Each holds 8 bits for one of the expanders. Each of the flip-flops will only respond to write or read from the primary slot they're connected to (ports from the 74LS32 make sure it goes that way).
