This phrase is used to express the idea that if other people heard the conversations that two people have, they might think the conversations were so strange or unusual that they would put the two people in jail. The speaker is expressing concern that their conversations might be considered too strange or not socially acceptable by other people.