Exactly. But it is unclear if there is intrinsic language knowledge somehow encoded too (which is what the OP is implying: https://news.ycombinator.com/item?id=35590091)