Git scraping: track changes over time by scraping to a Git repository

Re:Git scraping: track changes over time by scraping to a Gi... by mdaniel on Thursday August 10, 2023 @10:32PM (#37084411)

Thanks for linking to the topic, that was interesting

As a heads up to anyone trying this stunt, please be mindful that git-diff is ultimately a line oriented action (yeah, yeah, "git stores snapshots")

For example https://github.com/pmc-ss/mastodon-scraping/commit/2a15ce1b2... is all :fu: because git sees basically the "first line" changed

However, had the author normalized the instances.json with something like "jq -S" then one would end up with a more reasonable 1736 textual changes, which github would have almost certainly rendered

  diff -u \
    <(git ls-tree HEAD^1 -- instances.json | cut -d' ' -f3 | xargs git show --pretty=raw | jq -S) \
    <(git ls-tree HEAD   -- instances.json | cut -d' ' -f3 | xargs git show --pretty=raw | jq -S)
  --- /dev/fd/63 2023-08-10 19:31:03.000000000 -0700
  +++ /dev/fd/62 2023-08-10 19:31:03.000000000 -0700
  @@ -1,6 +1,6 @@
   [
     {
  -    "connections": 5088,
  +    "connections": 5089,

[ Parent | on HN ]

Re:Git scraping: track changes over time by scraping to a Gi... by WorldMaker on Friday August 11, 2023 @11:06AM (#37089535)

It doesn't help fix GitHub UI views, but you can use the --tool option to git diff and configure alternative diff tools in your git config, including something like piping through a pretty printer or using a (generally much slower) character-based diff tool rather than a line-based one.

[ Parent | on HN ]