Re:Ask HN: Vision Models to Parse UI
by throwaway2016a on Tuesday January 24, 2023 @12:08PM
(#34506531)
I too would like to see this! I'm pretty sure it can be accomplished by using screenshots as training data for an object recognition algorithm. As with a lot of machine learning, gathering and tagging the data would be tricky and overfitting to specific design systems could be a big problem.
But the hierarchy piece I think is a bit tricker.
I'm really curious to see what the comments come up with.
But the hierarchy piece I think is a bit tricker.
I'm really curious to see what the comments come up with.