The context of an artifact is its position in the site relative to everything else, and it's incredibly important. A stone flake by itself tells you a little, but a stone flake next to a core by a fire with carbonized bones in it tells you a lot more about the people who lived there. This is why archaeological digs painstakingly record the position and orientation of everything they find, in addition to soil features, etc. that might reveal even more. If you see an artifact in a museum that is from an archaeological dig, there's a book in an archive somewhere that has a drawing of the level in which it was found that gives its orientation and position relative to a datum. You could return to the dig and place the artifact in precisely the same location that it was found in using this data.
Archaeology is an evolving discipline, and better ways of extracting additional information from digs with new methods are being found all the time. For this reason, most major digs do not cover 100% of the site. It's considered good practice to leave some of the site undisturbed so that future archaeologists can apply yet-to-be-invented techniques to the site to reveal more than is presently possible.
So, by scooping up artifacts indiana-jones style, you're destroying a huge part of the information that they might have revealed. Is it better than looters digging the place up? Perhaps, but only just barely. It's a highly destructive, unethical act. It's like justifying a smash-and-grab because there's a riot going on and you happen to be slightly more needy than most. There's a small chance that looters won't find the site or that a government will decide to protect it in time, and it's better to take that chance than it is to loot the place and ensure the site is destroyed.
> In 1856, General Alexander Cunningham, later director general of the archaeological survey of northern India, visited Harappa where the British engineers John and William Brunton were laying the East Indian Railway Company line connecting the cities of Karachi and Lahore. John wrote: "I was much exercised in my mind how we were to get ballast for the line of the railway". They were told of an ancient ruined city near the lines, called Brahminabad. Visiting the city, he found it full of hard well-burnt bricks, and, "convinced that there was a grand quarry for the ballast I wanted", the city of Brahminabad was reduced to ballast.
go take a look at any bigger western museum. in most, most of the artifacts are foreign and no way they are willingly giving them back. if that would be the case, for example maybe 80% of the Louvre in Paris would be plain empty
So, presuming you're an archaeologist and know how to quickly-and-efficiently extract and catalogue artifacts without damaging them; and presuming the country that the site belongs to has no governmental body or NGO willing to protect the site; then you may as well book the first plane ticket there, rush in, (carefully) pick up everything of value, box it all up and ship it to a museum—any museum—where it will be put under glass until the significance of the artifacts can be understood.
If the locals want their history, their museum can get it back from your museum. Until then, it'll at least be protected from ending up in some foreigner's Spanish class.