The worst part of holding the memories is not the pain. It's the loneliness of it. Memories need to be shared.

This quote means that although recalling memories can be difficult because of the pain associated with them, the most difficult part is not being able to share them with other people. Memories are much more meaningful and valuable when they can be shared with other people who can relate and share their own stories.