You can get ClamAV's database files. Though most AVs are cloud based now. They store the hash database online and the clients just query it. Most integrate sandboxes with heuristics for malicious behavior (for unseen samples) and machine learning on previously malicious samples to create rules for new detections. Most will also monitor network traffic and have rules for detection there too.