Keep read receipts for the other person only, and let photos, voice, and files stay cached until they are actually replaced.