How do language models handle Black Swans?

Most language models are trained on a large dataset (i.e. the pile). Because of the costs associated they are expensive to update. Figuring out how they handle an uncertain future (like War in Ukraine and other Black Swans) could therefore inform how reliable they are. Concretely, the problem will look at how they "predict" an uncertain variable (like the prize of oil) after their training period (versus after).