For counting, there are specialized counting models, e.g. https://huggingface.co/spaces/MengqiLei/count-anything-demo
I tried to parse hand-drawn ER diagrams in the past and did not have much success with any model, frontier or otherwise. If you have annotated data, I'd recommend finetuning a recent (dense) VLM, but don't expect 100% accuracy. https://unsloth.ai/docs/basics/vision-fine-tuning
good advice; tbh I'm really trying to use counting as a proxy for "understand symbol, flag annotation next to it, associate annotation with symbol and store as object in location".
Ideally, look at second diagram and see something similar in the same location and understand it's the same physical object.
Can you point me at one that can understand a detailed block diagram? Frontier is fine, soliciting recommendations