There's definitely a middle-ground approach here, I just haven't found a good enough heuristic. It's true that images are part of the branding and they impact the color palette, like banners and logos, so they should be considered in some quantity. I thought of maybe averaging the results of image/no image but I don't have enough test data to validate it yet.
Let's chat and discuss more! I'll also add you on Twitter/X
One filter I added to make the palette a bit more relevant is that I try to find out which is the background color (the most used color) and only pick colors that are at least somewhat contrasting with the background (assuming that websites would follow somewhat basic color contrast rules).
When I tried extracting the palettes from CSS rules, what I ran into is that each website uses lots of vendors that render stuff (cookie consents, promotional pop-ups, etc.) and it's hard to distinguish what's rendered and what's not with pure CSS selectors, so I resorted to decomposing the website's theme from the perspective of a user. (through a screenshot)
It's a great idea nonetheless, I'm sure I would have some good results with it if I manage to filter out the noise, thanks a lot!
The crawler also blocks image loading (to not take image colors into account because it's screenshotting the entire page and creating a frequency of each color from the screenshot, pixel-wise) so it's possible that some fallback CSS was displayed instead of the images.
I definitely have to refine it more and it's far from perfect. There are only ~25k websites indexed and I used a list of 1M most popular websites created by the developers of a Chrome extension, based on what most people were visiting.
I tried my best to sort by popularity but I'm sure that the list is not perfect. Plus, some websites are really unfriendly to crawlers and, as much as you try to hide it (by faking user agents, using realistic window dimensions, proxies, etc.), some will still detect you.
I also have a lot more content (i.e.: the full HTML code of the website, the fonts and typography options) that I extracted for each website but it's tricky to come up with an easy to use UI/UX for this as a search filter, given that there's not many other websites that do this.
The first version of it, I didn't take any screenshots and it felt very empty. It's a bit hard to manage the images and deliver them rapidly, in an optimized form, but I think it's worth it.
I've also collected what fonts and typography styles different websites use so I'm looking to create a visual for that as well, thanks for the feedback!
I crawled lots of websites and extracted their color palettes (primary, secondary, etc) to build a searchable collection of website colors!It was a fun side-project and I learned a lot about color theory. Let me know what you think and what features you would like to see :)If you have any questions about the tech behind it, feel free to ask in the comments. Feedback is also greatly appreciated.