It's notorious that that kind of system is bad at tabular data. It's a research topic to make similar systems that are good at it. I worked for a startup that was trying something like just around the time BERT came out and transformers were not quite ready.