Reliable Reasoning Beyond Natural Language